Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pin all dependencies in the technical reports #13

Closed
2 tasks done
redeboer opened this issue May 6, 2024 · 2 comments · Fixed by #23
Closed
2 tasks done

Pin all dependencies in the technical reports #13

redeboer opened this issue May 6, 2024 · 2 comments · Fixed by #23
Assignees
Labels
🖱️ DX Improvements to the Developer Experience

Comments

@redeboer
Copy link
Member

redeboer commented May 6, 2024

Currently, TRs have a pip install statement where the packages that are required for the notebook itself. The packages are pinned to a specific version in an attempt to make the notebook reproducible, but indirect dependencies are not pinned. This results in bugs like ComPWA/compwa.github.io#272.

Unfortunately, Jupyter notebooks do not support launching sub-kernels. That would be the ideal solution: you have a main developer environment for the repository that you can for instance use to start Jupyter Lab, and then each notebook launches a subkernel with it's own specific set of dependencies.

The closest we can get to an isolated kernel is by doing a pip install with --prefix and then using sys.path.insert to import from that directory. You can create a constraint file for all indirect dependencies with pip compile (or faster: uv pip compile). Providing a constraint file for each notebook then makes each technical report reproducible.

Disadvantages:

  • The notebook cannot be run on its own: you need the constraint file. That works fine on Binder, but Google Colab (and Deepnote?) only have access to the notebook, not the whole repo.
  • You need to pip compile the constraint files. Is that done through the notebook as well? How to provide a mechanism where the writer of the report can specify which direct dependencies are required and then compile a constraint file from that?
  • ...

Related PRs

  1. 10 of 10
    🔨 Maintenance 🖱️ DX
    redeboer
  2. 10 of 10
    🔨 Maintenance 🖱️ DX
    redeboer
@redeboer redeboer added the 🖱️ DX Improvements to the Developer Experience label May 6, 2024
@redeboer redeboer transferred this issue from ComPWA/compwa.github.io Oct 1, 2024
@redeboer
Copy link
Member Author

redeboer commented Oct 3, 2024

See also astral-sh/uv#6969

@redeboer
Copy link
Member Author

Perhaps pyproject-local-kernel is the solution?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🖱️ DX Improvements to the Developer Experience
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants