Adding Nim to nimble.lock makes nimble download and compile the compiler

[rockcavera]

I was installing nimlangserver with nimble and noticed that it always downloaded the Nim compiler, compiled csource, koch, compiler, tools... an unnecessary and slow process for those who have the compiler in the path.

langserver/nimble.lock

Line 235 in 3a537e5

"nim": {

[rockcavera]

Today, after recompiling from HEAD, I checked the files that were used to compile nimlangserver and it didn't even use anything from stdlib or the Nim compiler that was installed by nimble, that is, keeping Nim in .lock only delays the process of nimlangserver compilation, as it used the Nim compiler that I already had (devel) and my devel's stdlib and not the one installed by Nimble.

[arnetheduck]

This sounds more like a nim/nimble bug than one in langserver? ie the purpose of a lock file is to deterministically choose which libraries are used to compile the application - that quite obviously includes the standard library which makes up most of the code (and compiler, by extension).

[rockcavera]

This sounds more like a nim/nimble bug than one in langserver? ie the purpose of a lock file is to deterministically choose which libraries are used to compile the application - that quite obviously includes the standard library which makes up most of the code (and compiler, by extension).

Of course, there can be several interpretations of what was intended by adding Nim to .lock. I can see two:

1. Should the compilation be done by the Nim executable compiled by the Nimble installation when installing nimlangserver? In this case, stdlib and compiler of the Nim version informed in the .lock will be used. If this is the case, then we are facing a bug, in which Nimble should call "Nimble/pkgs2/nim-2.1.1-ee369ca2ef4fd48e55732c09c5f8b67a4a22daa4/bin/nim".

2. However, if the requirement is to use the Nim executable previously installed on the machine, nimlangserver should import pkg/compiler/pathutils or compiler/pathutils to use the compiler package installed by Nimble. Regarding stdlib, I have no idea how to import it other than from the Nim executable that will perform the compilation.

In my opinion, the second option seems more obvious, mainly because I don't know of any package manager that installs a specific interpreter or compiler for each package you intend to install. Generally, if the interpreter or compiler does not match the version required by the installed package, the package installation should fail.

[arnetheduck]

If this is the case, then we are facing a bug, in which Nimble should call "Nimble/pkgs2/nim-2.1.1-ee369ca2ef4fd48e55732c09c5f8b67a4a22daa4/bin/nim".

this is the case.

However, if the requirement is to use the Nim executable previously installed on the machine

This is also the case ;)

The key insight here is that in order to compile the application, you need to have matching versions of all dependencies - it's really that simple. If you have a system nim installation and it matches the required version, that should be used - if not, nimble should install a matching version just like it installs libraries. This same logic applies to other "tools" you might need to build the package (ie c2nim or some other custom binary of which nim is the most common example).

This enables a number of things:

• you chances of compiling the applications shoot up significantly
• you don't need to have a system nim at all - just a nimble (or atlas, which has similar functionality)
• for the vast majority of packages, you will only ever install one or two nims and the correct one will automatically be used when you work on a specific package.
• tooling like nimsuggest can also automatically pick the right compiler and std lib when you open a project in your editor of choice

Note that the above is not limited to lock files - the same logic applies if you have not locked your Nim but instead specified a version range with which it works.

don't know of any package manager

Modern package managers (ie cargo, npm etc) track the compiler/runtime version required to compile the application just like any other dependency. They also typically offer tooling to switch between versions (ie nvm, rustup, choosenim etc). The only difference you're observing is that nimble includes this functionality in the same program.

[arnetheduck]

Oh, and finally, in nimble there should obviously be room for local overrides based on flags (--override:nim=myversion or something like this) - this applies to nim but also to any other dependency - so as an option, you should be able to override which version of a dependency is going to be used, if you are in a position where you feel like dealing with random compile errors and runtime issues because the dependency changed.

[rockcavera]

Was removed in 8449dfb

[jmgomez]

For the record, I removed it by mistake. But I do see the pain on downloading it every time, so my vote is to keep it as is for the time being.

[arnetheduck]

But I do see the pain on downloading it every time

every time? Only once, it should be cached, no?

[jmgomez]

Added it back. It seems I need to learn more about Nimble. I just wanted to update some deps, turned out it somewhat dumped my global Nimble folder when doing the lock (and removed Nim from there as I didnt have it)

[melMass]

Not sure if this is the cause but I couldn't get nimble to install nimlangserver on Mac using a brew installed nim (2.2.1)

nimble install nimlangserver
Downloading https://github.com/nim-lang/langserver using git
Downloading https://github.com/nim-lang/Nim.git using git
       Tip: 7 messages have been suppressed, use --verbose to show them.
nimble.nim(703)          downloadDependency

    Error:  Downloaded package checksum does not correspond to that in the lock file:
        ...   Package:           [email protected]@r.5935c3bfa9fec6505394867b23510eb5cbab3dbf
        ...   Checksum:          efb469b3ca8c4ec85c5829fa18997508672f9740
        ...   Expected checksum: 46333e8f4bda41dd6d3852a3f5fa4975b96b66a2

Cloning this repo, removing this line AND the lock file made it work