Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiprocessing issues with "differential evolution"? #435

Open
chris-simpson opened this issue Apr 28, 2023 · 0 comments
Open

Multiprocessing issues with "differential evolution"? #435

chris-simpson opened this issue Apr 28, 2023 · 0 comments

Comments

@chris-simpson
Copy link
Contributor

The use of "differential evolution" as the default optimizer in KDTreeFitter appears to have led to some issues due to its use of multithreading. These include:

  1. Large numbers of open "files" requiring ulimit -n 1024
  2. Inability to run python scripts that do not have the if __name__ == "__main__" idiom
  3. Issues running scripts in an interactive python environment (probably related to the previous item)

The second is well-understood here but may trip up people who don't code to a high standard. The other two are less clear, and it's also unclear whether any of this is due to the change from "fork" for "spawn" for python multiprocesses on OSX in python 3.8+.

Some more investigation/reports might be needed, as well as possibly exposing a parameter to use a different method of optimization.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant