-
Notifications
You must be signed in to change notification settings - Fork 79
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[WIP] Using the robust solver for pyMBAR - avoiding convergence Failu… #735
Changes from 12 commits
8896e63
d72fe8e
d3d211b
060d5c1
39b683b
50b058f
41ce632
0d733d2
c1ac13e
4916e4b
7d3dc30
fc0f4ad
f8458bf
a3dab11
51cc3d0
b1badbb
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -22,12 +22,14 @@ | |
import shutil | ||
import sys | ||
import tempfile | ||
import time | ||
from io import StringIO | ||
|
||
import numpy as np | ||
import yaml | ||
|
||
import pytest | ||
import requests | ||
|
||
try: | ||
import openmm | ||
|
@@ -306,6 +308,7 @@ def run(self, include_unsampled_states=False): | |
# Clean up. | ||
del simulation | ||
|
||
@pytest.mark.flaky(reruns=3) | ||
def test_with_unsampled_states(self): | ||
"""Test multistate sampler on a harmonic oscillator with unsampled endstates""" | ||
self.run(include_unsampled_states=True) | ||
|
@@ -1861,7 +1864,7 @@ def test_analysis_opens_without_checkpoint(self): | |
del reporter | ||
self.REPORTER(storage_path, checkpoint_storage=cp_file_mod, open_mode="r") | ||
|
||
@pytest.mark.flaky(reruns=3) | ||
@pytest.mark.skipif(sys.platform == "darwin", reason="seg faults on osx sometimes") | ||
def test_storage_reporter_and_string(self): | ||
"""Test that creating a MultiState by storage string and reporter is the same""" | ||
thermodynamic_states, sampler_states, unsampled_states = copy.deepcopy( | ||
|
@@ -2612,6 +2615,42 @@ def test_resume_velocities_from_legacy_storage(self): | |
state.velocities.value_in_unit_system(unit.md_unit_system) != 0 | ||
), "At least some velocity in sampler state from new checkpoint is expected to different from zero." | ||
|
||
@pytest.fixture | ||
def download_nc_file(tmpdir): | ||
FILE_URL = "https://github.com/user-attachments/files/17156868/ala-thr.zip" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I believe we cannot rely on this long term, since as far as I know the structure of these URLs can change without notice. Is there another way we can have a long-lived url? Not a big issue but probably something to keep in mind. |
||
MAX_RETRIES = 3 | ||
RETRY_DELAY = 2 # Delay between retries (in seconds) | ||
file_name = os.path.join(tmpdir, "ala-thr.nc") | ||
retries = 0 | ||
while retries < MAX_RETRIES: | ||
try: | ||
# Send GET request to download the file | ||
response = requests.get(FILE_URL, timeout=20) # Timeout to avoid hanging | ||
response.raise_for_status() # Raise HTTPError for bad responses (4xx/5xx) | ||
with open(file_name, "wb") as f: | ||
f.write(response.content) | ||
# File downloaded successfully, break out of retry loop | ||
break | ||
|
||
except (requests.exceptions.RequestException, requests.exceptions.HTTPError) as e: | ||
retries += 1 | ||
if retries >= MAX_RETRIES: | ||
pytest.fail(f"Failed to download file after {MAX_RETRIES} retries: {e}") | ||
else: | ||
print(f"Retrying download... ({retries}/{MAX_RETRIES})") | ||
time.sleep(RETRY_DELAY) # Wait before retrying | ||
yield file_name | ||
|
||
|
||
def test_pymbar_issue_419(download_nc_file): | ||
from openmmtools.multistate import MultiStateReporter, MultiStateSamplerAnalyzer | ||
|
||
n_iterations = 1000 | ||
reporter_file = download_nc_file | ||
reporter = MultiStateReporter(reporter_file) | ||
analyzer = MultiStateSamplerAnalyzer(reporter, max_n_iterations=n_iterations) | ||
f_ij, df_ij = analyzer.get_free_energy() | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would encourage doing a number regression check here rather than just a pure smoke test. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I thought about that but we already do that in other tests, happy to add it. What threshold do we want to use to check that it is close? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This shouldn't be sampling anything, so ideally we should always be converging to a very similar value. If it changes, something changed enough to affect all our free energies. I would suggest something like 1e-4 or 1e-5 precision. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'll be honest, I forgot we were not sampling something, so I didn't even consider that we will get the same result each time we run (or really close to the same result) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We probably need a docstring here telling what the test is doing and how the nc file was generated. For future reference. |
||
|
||
|
||
# ============================================================================== | ||
# MAIN AND TESTS | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would using something like pooch be better for this kind of thing?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought about that but it felt like a lot to add for a single test, if we have more to download then we can add it.