python3 conversion.

[clegoues]

I have been trying to convert phys to python3. I have gotten stuck with converting libDAI to python3.

The branch is python3-conv

I ran everything in a docker container built on latest ubuntu stable release (instead of whatever version the phys docker image is).

Current failure/status

I did (I think) all of the appropriate conversion of the python2 code in the main src/ and src/pgm library besides the generated dai code. I removed all versions from requirements.txt while I was at it since the old versions of those libraries mostly didn't play nice with python 3; I assume we can pin them to actual current version numbers if/when everything works.

The commit where I pushed the new bindings/library for libdai is this one: 4d5dcaa

The part that interacts with libdai is in src/pgm. dai.py is the generated python code from swig, and _dai.so is the generated library it calls into.

This all runs (I've just been using the first example from the phys webpage) but chokes when it gets to the code that interacts with the factor graph library. My most recent attempt to fix it was this commit: 08ea824

What was happening is that the call on previous line 131 to parseNameProperties should have return a std::Pair, which swig should be able to handle natively. Like, swig has special handling for C++ pairs. But, the program was failing at the call to newInfAlg (deleted/replaced line 132), saying that methodname2props doesn't have a field called first. Instead it's of type (I think) SwigPyObject *, and AFAIK you can't get at field names under raw Swig object pointers like that. But in any case, it shouldn't be a SwigPyObject * at all, it should be a std::Pair, which Swig should be able to handle.

I tried to work around this by finding another way to call newInfAlg, which is the code you see inserted on lines 130 and 131 in the above-linked commit. Looking at the DAI documentation, this should be equivalent to what happened before. But now when I run it, I get an error later indicating an attempt to normalize a vector with an invalid norm (of 0). I don't remember exactly where, but the point is, something's going wrong with how the libdai code is being called/run.

I think the better solution is to revert the above change, and instead figure out what happened in pgm.py that the type returned from parseNameProperties is not what python expects, but that involves reading Swig's mind, which is hard.

The main reference for DAI documentation I've used is https://staff.fnwi.uva.nl/j.m.mooij/libDAI/doc/namespacedai.html

---

how did I make the new _dai.so and dai.py?

DAI is here: https://github.com/dbtsai/libDAI

I didn't fork it, just checked it out in the above mentioned Ubuntu container and built it there. I updated Makefile.LINUX so the INCLUDE_PYTHON path points to /usr/include/python3.10 and SWIG=swig. (which should default to swig4.0 if installed in a latest Ubuntu image. I had no notable trouble building libdai at the toplevel in the same docker container I was using for the rest of phys development.

Experimental support for swig bindings is in swig/. I modified the call to swig on line 15 of swig/Makefile to drop the classic flag (which is not supported in 4.0 IIRC). So the call now reads $(SWIG) -python -c++ dai.i Then I executed make in libdai/swig. _dai.so and dai.py were successfully built. The process died on dai.oct. I don't have a good reason we'd need octave bindings and it looked like phys only needed the _dai.so target.

I messed around the tiniest of bits with swig/dai.i to try to experiment with new/different pair type templates, but didn't learn much notably interesting in those experiments.

And that's where I left off.