Instllation tips using uv #3503
Conversation
| If you are not (yet) an expert in Python installations, a main difficulty is choosing the installation procedure. | ||
| The main ideas you need to know before starting: | ||
| * python itself can be distributed and installed many many ways. | ||
| * python itself do not contain so many features for scientifique computing you need to install "packages". |
There was a problem hiding this comment.
| * python itself do not contain so many features for scientifique computing you need to install "packages". | |
| * python itself does not contain so many features for scientific computing you need to install "packages". |
| The main ideas you need to know before starting: | ||
| * python itself can be distributed and installed many many ways. | ||
| * python itself do not contain so many features for scientifique computing you need to install "packages". | ||
| Numpy, scipy, matplotlib, spikeinterface, ... are python packages that have a complicated dependency graph between then. "uv" |
There was a problem hiding this comment.
| Numpy, scipy, matplotlib, spikeinterface, ... are python packages that have a complicated dependency graph between then. "uv" | |
| numpy, scipy, matplotlib, spikeinterface, ... are python packages that have a complicated dependency graph between then. "uv" |
There was a problem hiding this comment.
maybe 'that interact in complex ways'? instead of 'have a complicated dependency graph'? If targeted to newer uses, they might not be familiar with the concept of a dependency graph.
*typo: between them
EDIT:
or instead, because dependency graph is a useful concept to learn, have a brief example? e.g. something like "complicated dependency graph between them. For example, you maybe have two packages (such as scipy, numpy) that work together but both require a certain version of a third package, say matplotlib. If you try and upgrade matplotlib in this environment to use a fourth package (e.g. seaborn), you might break scipy and numpy."
There was a problem hiding this comment.
I think the "uv" can be deleted?
| * python itself do not contain so many features for scientifique computing you need to install "packages". | ||
| Numpy, scipy, matplotlib, spikeinterface, ... are python packages that have a complicated dependency graph between then. "uv" | ||
| * installing package can be distributed and installed several ways (pip, conda, uv, mamba, ...) | ||
| * installing many packages at once is challenging (because of the depenency graph) so you need to do it in an "isolated environement" |
There was a problem hiding this comment.
| * installing many packages at once is challenging (because of the depenency graph) so you need to do it in an "isolated environement" | |
| * installing many packages at once is challenging (because of their dependency graphs) so you need to do it in an "isolated environement" |
| Numpy, scipy, matplotlib, spikeinterface, ... are python packages that have a complicated dependency graph between then. "uv" | ||
| * installing package can be distributed and installed several ways (pip, conda, uv, mamba, ...) | ||
| * installing many packages at once is challenging (because of the depenency graph) so you need to do it in an "isolated environement" | ||
| to not destroy any previous installation. You need to see an "environement" as a sub installtion in dedicated folder. |
There was a problem hiding this comment.
| to not destroy any previous installation. You need to see an "environement" as a sub installtion in dedicated folder. | |
| to not destroy any previous installation. You need to see an "environment" as a sub installation in a dedicated folder. |
There was a problem hiding this comment.
For newer users, it might be worth explaining how this could destroy a previous installation, maybe a link to dependency hell.
| * installing many packages at once is challenging (because of the depenency graph) so you need to do it in an "isolated environement" | ||
| to not destroy any previous installation. You need to see an "environement" as a sub installtion in dedicated folder. | ||
|
|
||
| Choosing the installator + a environement manager + a package installer is a nightmare for beginners. |
There was a problem hiding this comment.
| Choosing the installator + a environement manager + a package installer is a nightmare for beginners. | |
| Choosing the installer + an environment manager + a package installer is a nightmare for beginners. |
| * use python from the system or python.org + venv + pip : good idea for linux users. | ||
| * use "uv" : a new, fast and simple. We recommand this for beginners on evry os. | ||
|
|
||
| Here we propose a steps by step recipe for beginers based on "uv". |
There was a problem hiding this comment.
| Here we propose a steps by step recipe for beginers based on "uv". | |
| Here we propose a step by step recipe for beginers based on "uv". |
| * use "uv" : a new, fast and simple. We recommand this for beginners on evry os. | ||
|
|
||
| Here we propose a steps by step recipe for beginers based on "uv". | ||
| We used to propose here a solution based on anaconda. It is kept here for a while but we do not recommand it anymore. |
There was a problem hiding this comment.
| We used to propose here a solution based on anaconda. It is kept here for a while but we do not recommand it anymore. | |
| We used to recommend installing with anaconda. It will be kept here for a while but we do not recommend it anymore. |
| Steps: | ||
| 1. On macOS and Linux. Open a terminal and do | ||
| `$ curl -LsSf https://astral.sh/uv/install.sh | sh` | ||
| 1. On windows. Open a powershell and do |
There was a problem hiding this comment.
Hey Sam we recommend users use Command Prompt since our shell scripts for sorters only work in command prompt. It might be better if they have a command prompt version of instructions.
| Note for **linux** users : this conda recipe should work but we recommend strongly to use **pip + virtualenv**. | ||
| Some tools in the spikeinteface ecosystem are getting regular bug fixes (spikeinterface, spikeinterface-gui, probeinterface, python-neo, sortingview). | ||
| We are making releases 2 to 4 times a year. In between releases if you want to install from source you can use the `requirements_rolling.txt` file to create the environment. This will install the packages of the ecosystem from source. | ||
| This is a good way to test if patch fix your issue. |
There was a problem hiding this comment.
| This is a good way to test if patch fix your issue. | |
| This is a good way to test if a patch fixes your issue. |
| @@ -62,6 +79,28 @@ This script tests the following: | |||
| * opening the spikeinterface-gui | |||
| * exporting to Phy | |||
|
|
|||
| ### Legacy installation using anaconda (not recomanded) | |||
There was a problem hiding this comment.
| ### Legacy installation using anaconda (not recomanded) | |
| ### Legacy installation using anaconda (not recommended) |
There was a problem hiding this comment.
Hey @samuelgarcia this is very nice, I actually havn't tried uv yet so will run the commands to test on windows later. I request changes for some minor typos but feel free to ignore the other suggestions. BTW the creator of uv also makes another tool called Ruff 😏 maybe we should try it!
| The main ideas you need to know before starting: | ||
| * python itself can be distributed and installed many many ways. | ||
| * python itself do not contain so many features for scientifique computing you need to install "packages". | ||
| Numpy, scipy, matplotlib, spikeinterface, ... are python packages that have a complicated dependency graph between then. "uv" |
There was a problem hiding this comment.
maybe 'that interact in complex ways'? instead of 'have a complicated dependency graph'? If targeted to newer uses, they might not be familiar with the concept of a dependency graph.
*typo: between them
EDIT:
or instead, because dependency graph is a useful concept to learn, have a brief example? e.g. something like "complicated dependency graph between them. For example, you maybe have two packages (such as scipy, numpy) that work together but both require a certain version of a third package, say matplotlib. If you try and upgrade matplotlib in this environment to use a fourth package (e.g. seaborn), you might break scipy and numpy."
| The main ideas you need to know before starting: | ||
| * python itself can be distributed and installed many many ways. | ||
| * python itself do not contain so many features for scientifique computing you need to install "packages". | ||
| Numpy, scipy, matplotlib, spikeinterface, ... are python packages that have a complicated dependency graph between then. "uv" |
There was a problem hiding this comment.
I think the "uv" can be deleted?
| * python itself can be distributed and installed many many ways. | ||
| * python itself do not contain so many features for scientifique computing you need to install "packages". | ||
| Numpy, scipy, matplotlib, spikeinterface, ... are python packages that have a complicated dependency graph between then. "uv" | ||
| * installing package can be distributed and installed several ways (pip, conda, uv, mamba, ...) |
There was a problem hiding this comment.
| * installing package can be distributed and installed several ways (pip, conda, uv, mamba, ...) | |
| * packages can be distributed and installed in several ways (pip, conda, uv, mamba, ...) |
| Numpy, scipy, matplotlib, spikeinterface, ... are python packages that have a complicated dependency graph between then. "uv" | ||
| * installing package can be distributed and installed several ways (pip, conda, uv, mamba, ...) | ||
| * installing many packages at once is challenging (because of the depenency graph) so you need to do it in an "isolated environement" | ||
| to not destroy any previous installation. You need to see an "environement" as a sub installtion in dedicated folder. |
There was a problem hiding this comment.
For newer users, it might be worth explaining how this could destroy a previous installation, maybe a link to dependency hell.
| 1. On windows. Open a powershell and do | ||
| `powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"` | ||
| 2. exit session and log again. | ||
| 3. Download with right click + save this file corresponding in "Documents" folder: |
There was a problem hiding this comment.
| 3. Download with right click + save this file corresponding in "Documents" folder: | |
| 3. Download with right click and save this file corresponding in "Documents" folder: |
|
|
||
| Note for **linux** users : this conda recipe should work but we recommend strongly to use **pip + virtualenv**. | ||
| Some tools in the spikeinteface ecosystem are getting regular bug fixes (spikeinterface, spikeinterface-gui, probeinterface, python-neo, sortingview). | ||
| We are making releases 2 to 4 times a year. In between releases if you want to install from source you can use the `requirements_rolling.txt` file to create the environment. This will install the packages of the ecosystem from source. |
|
|
||
| Choosing the installator + a environement manager + a package installer is a nightmare for beginners. | ||
| The main options are: | ||
| * use "anaconda" that do everything. The most popular but bad idea because : ultra slow and agressive licensing (not free anymore) |
There was a problem hiding this comment.
Maybe it is better to lead with the recommended one (i.e. uv first bullet point)?
On conda, is true in some cases but newer versions use mamba backend which is quite fast, and conda-forge channel does not have licensing issues. A problem is that HPC systems often use by default the older slow versions and the default channel is anaconda's.
No point going into too much detail but a suggestion to amend to say that that the default anaconda channel has agressive licensing and that it can be slow / is slower than uv. It is probably worth mentioning it can handle non-python dependencies while the other two can't, but this is not required for spikeinterface.
No description provided.