fix: lock shared cache directory

[lengau]

Locks the shared cache directory to prevent concurrency issues.

Fixes #1845

CRAFT-3313

[lengau]

This stops using the fake filesystem and uses the real FS. I think it was a mistake for me to use pyfakefs for integration tests.

[lengau]

These changes are to deal with the change from using pyfakefs to using a real path.

[tigarmo]

I'm confused here - why isn't the cache visible? Did the call to _maybe_lock_cache() inside provider.instance() fall into the except OSError: codepath?

[lengau]

Yes, because we run _maybe_lock_cache on line 36. Looking back I realise I named the function for the success case and tested for the failure case. I'll fix that

[tigarmo]

this is surprising to me because I thought flock() was per-process, so a process trying to flock a file it already holds would be a noop
If this isn't the case, what happens if we do multiple consecutive managed builds? Like a build plan with multiple entries

[lengau]

You're right! I wasn't checking for that properly, either. However, it's not quite per-process either. It's per file descriptor, so the same process can't lock the same path in two places if it's got the file open separately twice. (This also requires making two separate Path objects, as pathlib will cache the file descriptor for the same object, as I learnt the hard way.)

I've updated it now with a much better lock and test.

[mattculler]

Looks good to me. If this is an issue that users see frequently then a nicer solution might be more fine-grained cache locking control, so multiple instances of charmcraft could run with the shared cache at once, but they couldn't both use the same package (or whatever the things inside this cache are) at the same time.