How-To Guide: Building a python extension module with pyo3, setuptools-rust & cibuildwheel

[MusicalNinjaDad]

I've been playing around with pyo3 in a sandbox project for a while with a view to using it to create a fast, accurate math library. I walked through the front to back flow in a cross-department engineering demo at work and had really positive feedback.

So I decided to document and publish it: "Fizzbuzz: combining rust & python" on github.io

I'd be really interested in feedback, ideas, corrections and additional tips from the community. I'm just starting my journey with pyo3, many of you have been travelling for a long time already.

The guide focusses on all the "other" topics that come together to actually make an extension library rather than how pyo3 works:

• Setting up a development environment
• Structuring the codebase
• API design
• Testing
• IDE hints
• Packaging & wheel building

Along the way I created a few bits of "duck tape and string" to fill some gaps I found:

• pyo3-testing to make it easy to test the wrapped functions from within rust, rather than jumping eco-systems all the time
• cibuildwheel-rust with pre-built docker images for all architectures to drastically speed up and simplify wheel building
• pyo3-stubgen to auto-generate stub files for type & doc hinting
• pyo3-devcontainer with pre-built docker images (containing rust, python, headers, coverage and testing tools etc) and an example devcontainer.json for fast-start / reproducible development environments

These are all linked and detailed in the How-To.

[jessekrubin]

@MusicalNinjaDad This is REALLY and TRULY an amazingly well done and put together write up and repo!

[MusicalNinjaDad]

Thank you @jessekrubin - is there anything you would like to see added / recommend to add?

[jessekrubin]

@MusicalNinjaDad Ack! Sorry I did not see your response!!! I have been busy but I would like to give your set of workflows/cicd a try on one or both of my main pyo3 projects:

• misc-rust-crate-wrappers -> https://github.com/jessekrubin/ry
• web-map-tile stuff -> https://github.com/jessekrubin/utiles

If you have a min to scan over my projects, any recs on where I should start would be much appreciated!

utiles is MUCH more built out and I think might be more involved to migrate.