Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix slowness of loading large experiment lists due to pkg_resources->importlib change #658

Merged
merged 4 commits into from
Aug 24, 2023

Conversation

jbeilstenedmands
Copy link

Fixes #657

@codecov
Copy link

codecov bot commented Aug 22, 2023

Codecov Report

Merging #658 (cfb50e5) into main (9657a16) will increase coverage by 0.03%.
The diff coverage is 80.00%.

❗ Current head cfb50e5 differs from pull request most recent head 33fe178. Consider uploading reports for the commit 33fe178 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #658      +/-   ##
==========================================
+ Coverage   37.96%   37.99%   +0.03%     
==========================================
  Files         182      182              
  Lines       15817    15825       +8     
  Branches     3037     3037              
==========================================
+ Hits         6005     6013       +8     
  Misses       9244     9244              
  Partials      568      568              

@ndevenish
Copy link
Collaborator

So, after digging into this, this seems to be an issue to do with either:

  • Python 3.10 (goes away on 3.11)
  • tbx-distribution layout (goes away on 3.10 with CMake bootstrap, where entry_points are installed into site-packages)

There are change notes on python 3.11 that performance regressions were addressed; I don't know why the backport didn't fix these.

So, this seems pretty reasonable.

@ndevenish ndevenish merged commit 2fc8ffa into main Aug 24, 2023
4 of 17 checks passed
@ndevenish ndevenish deleted the fix_importlib_slowness branch August 24, 2023 09:20
toastisme pushed a commit to toastisme/dxtbx that referenced this pull request Jul 18, 2024
Regression observed in large experiment lists with scaling/profile models.

Only seems to affect Python 3.10, potentially stemming from:
    https://bugs.python.org/issue44246

Fixes cctbx#657
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Loading large experiment lists has become >> 10x slower in recent versions.
3 participants