Update hatch_build.py #305
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Depending on selective checks, but also on the job executed, we
choose whether to run job on public runners or self-hosted runners.
So far the set of labels to select the runners were passed in a bit
inconsistent way. Outputs of selective checks can only be strings
and the `run-as` accepts array of strings (labels) - so we were
using fromJSON to convert between the two. And we used runs-on
inputs on a number of our workflows to pass the selection.
However this meant that runs-on could be either string or array and
that sometimes we passed public/self-hosted labels as strings
directly and some of those were hard-coded.
This PR changes it consistently across the board to introduce
consistent approach:
* build info have no selective checks results yet, so for them
runs-on is hardcoded
* similarly for "windows" and release jobs that are manually run
without running selective checks
* selective checks will produce three outuputs - JSON stringiified
array of labels:
* default (one that is selected depending on who runs the build)
* public (for cases where we want to force the builds to use public
runners
* self-hosted (for cases where we want to force the builds to use
self-hosted runners
* all the outputs are named `<type>-runs-on-as-string` to make it clear
they are all strings
* all inputs of workflows expectings strings are named the same (with
as-string suffix and <type> prefix)
* whenever a job is run, we pass "runs-on" parameter to be `fromJSON`
with appropriate type we want to use passed as input
This will make it easier to reason on which job is using which type
of runner and it will make it easier in the future to make it more
flexible when we add ASF self-hosted runners and possibly our own
K8S runners, or when we would want to change labels for public runners
or self-hosted runners.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.