WIP: wrap: Implemented wraplock #13784
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
To avoid raceconditions, where one instance of meson currently downloads a subproject defined in a wrapfile, while another either a. starts the download itself too b. attemts to evaluate the partially downloaded subproject wraplock introduces a lockfile, which should prevent simultaneous access of subprojects by wrap between different instances of meson.
This is used when fnctl is not found. In these cases there will be no guarding of subprojects and multiple meson instances may still run into eachother.
See mesonbuild/utils/platform.py and posix.py / win32.py -- we have an existing locker and it supports msvcrt. |
| return; | ||
| # Do not remove the lockfile: | ||
| # https://github.com/tox-dev/py-filelock/issues/31 | ||
| # https://stackoverflow.com/questions/17708885/flock-removing-locked-file-without-race-condition |
There was a problem hiding this comment.
Do you need to create a file? IIRC you can os.open(subdir_root, os.O_RDONLY) and lock it. Would that work?
There was a problem hiding this comment.
flock should work with directories, but for potability reasons (mostly orienting on what filelock did) files are better.
|
@eli-schwartz platform.py provides BuildDirLock, which is fairly specific. Should I rewrite BuildDirLock into a more generic lock and implement BuildDirLock then ontop of the generic one? |
Mostly a POC looking for feedback, but:
As I've described in #13758
When you start multiple meson setup instances at the same time, then during their attempted access of the same subproject, one instance manages to start the download, while the other attempts to evaluate a partially download subproject:
../meson.build:406:17: ERROR: Subproject exists but has no meson.build file.The current implementation is a heavily shortend/modified version of tox-dev:filelock/UnixFileLock. The issue with the current code is hower, that fcntl isn't cross-platform.
Unlike the filelock package, I was thinking of just providing the fnctl locking implementation and letting meson processes race into each other of systems that don't implement it.