👌 Improve performance of needs builders #1054
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
closes #957
This PR refactors the
needs
andneeds_id
builders, so that they can completely skip the "write" phase of the Sphinx build.The main aim of these builders is to output the final needs data, in a particular format.
The needs data is collected during the "read" phase of the Sphinx build, i.e. when all documents are read, then after all needs data has been collected, a number of "post-processing" functions are applied to the needs data.
This post-processing is independant of (a) the individual documents, (b) the output format, and so does not require the Sphinx "write" phase to run.
Here we combine all of the post-processing into a single
post_process_needs_data
function that can be called directly by these builders.Note, ideally this would be called in a Sphinx event occuring between the "read" and "write" events, however, this does not currently exist (the closest is
env-check-consistency
, but this is not triggered on "write-only" rebuilds).The only corner case for skipping the "write" phase, is when the
needs
builder is required to also export filters, due to the presence ofexport_id
options on certain directives.These filters are currently computed during the "write" phase, so here we do run that, but emit a warning, to let the user know of this degradation in performance.