Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fine grained presubmit sample tests #1038

Open
dizcology opened this issue Apr 13, 2021 · 1 comment
Open

Fine grained presubmit sample tests #1038

dizcology opened this issue Apr 13, 2021 · 1 comment
Labels
lang: python Issues specific to Python. samples Issues that are directly related to samples. type: process A process-related concern. May include testing, release, or the like.

Comments

@dizcology
Copy link
Contributor

Is your feature request related to a problem? Please describe.

Some library+sample repos (such as python-aiplatform) has many samples and sample tests, which are frequently updated even if the library itself has not changed. Presubmit checks would run all the sample tests even if a PR modifies only a few samples. Some sample tests take a long time to run often causing the presubmit checks to take up to an hour.

Describe the solution you'd like

Ideally, PR presubmit sample test runs should run only the modified sample tests and the samples tests whose corresponding samples have been modified in the PR.

If a PR changes anything outside of the samples folder, all sample tests should run. (This is already the current behavior.)

Describe alternatives you've considered

Alternatively, the granularity could be at directory level: PR presubmit sample test runs should run only sample tests in subdirectories that have been changed in the PR whenever all changes in the PR are in the top level samples folder.

@product-auto-label product-auto-label bot added the samples Issues that are directly related to samples. label Apr 13, 2021
@yoshi-automation yoshi-automation added the triage me I really want to be triaged. label Apr 14, 2021
@dandhlee dandhlee added type: process A process-related concern. May include testing, release, or the like. and removed triage me I really want to be triaged. labels Apr 14, 2021
@chingor13 chingor13 added the lang: python Issues specific to Python. label Apr 14, 2021
@SurferJeffAtGoogle
Copy link
Contributor

@dizcology, I agree, but my first instinct is to update python-aiplatform so it behaves as you describe. What changes do you imagine in this repository?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lang: python Issues specific to Python. samples Issues that are directly related to samples. type: process A process-related concern. May include testing, release, or the like.
Projects
None yet
Development

No branches or pull requests

5 participants