[no-release-notes] go/store/nbs: During a GC process, take dependencies on chunks that are read through the ChunkStore. #8760
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously GC was constructed to walk the transitive closure of reachable chunks from when it started, and to take additional dependencies on any chunks that were written to the ChunkStore during the GC process. This change makes it so that we can additionally take dependencies on any chunks that are read from the ChunkStore during the GC process.
For the
nbs
layer to signal a new dependency during a GC, it makes use of akeeperFunc
, which is passed into theBeginGC
call. The contract is that, ifkeeperFunc
is non-nil
and it returnsfalse
, the chunk address which was given to the function will make its way back to a call toSaveHashes
on someMarkAndSweeper
whose contents will end up in the final store once the GC is completed.keeperFunc
, however, is allowed to returntrue
, which signals to the block store that the in-progress GC has passed a certain point in time and can no longer make that guarantee. As a result, anytimekeeperFunc
returnstrue
, the block store implementation needs to block until the GC process is over, and then it should resume the operation from the beginning.That machinery already exists for the
Put()
andCommit()
use case. This PR adds machinery to do all of that on various read paths that explicitly touch chunks, such asHas
,HasMany
,Get
,GetMany
andGetManyCompressed
. In order to add dependency tracking on the read path, it's important for the GC process itself to not cause the chunks it has read to form additional dependencies. This PR makes a slight change tomarkAndSweeper
so that it can fetch chunks from the source store without recording unnecessary duplicate dependencies and potentially becoming mired in deadlocks.