Use torch generic workflow for CI, add ssh, artifacts #325
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
wconstab
added a commit
that referenced
this pull request
May 14, 2024
ghstack-source-id: fc0b11e214ee065e9843a926b6610d58c78dc989 Pull Request resolved: #325
facebook-github-bot
added
the
CLA Signed
wconstab
added a commit
that referenced
this pull request
May 14, 2024
ghstack-source-id: 3a4a6a3d7e557386bb78e9ad629bd9af429cade2 Pull Request resolved: #325
wanchaol
reviewed
May 14, 2024
wconstab
added a commit
that referenced
this pull request
May 14, 2024
ghstack-source-id: 6ba9a97f2012ff99ed334e4cbfc35804d7bb192b Pull Request resolved: #325
wconstab
added a commit
that referenced
this pull request
May 14, 2024
ghstack-source-id: b26e91b0d7b92cdb8d55859186777c1ff6669503 Pull Request resolved: #325
wconstab
added a commit
that referenced
this pull request
May 14, 2024
ghstack-source-id: 008612abbfe85656a1cae9ef590710687744bfd6 Pull Request resolved: #325
wconstab
added a commit
that referenced
this pull request
May 14, 2024
ghstack-source-id: 9f410e6b6b7d616c7beb044ac9382be3179df812 Pull Request resolved: #325
wconstab
added a commit
that referenced
this pull request
May 14, 2024
ghstack-source-id: 121fe0b027a4d20b9ec49da1bbf17d5d1cc1928a Pull Request resolved: #325
This was referenced May 14, 2024
Closed
Closed
Closed
This was referenced May 15, 2024
Closed
Closed
Closed
Closed
Closed
wconstab
changed the title
wanchaol
approved these changes
May 15, 2024
wconstab
added a commit
that referenced
this pull request
May 15, 2024
ghstack-source-id: b1fa8d8c1778ecb532ed71792ead9f4dbb067cf4 Pull Request resolved: #325
tianyu-l
added a commit
that referenced
this pull request
May 22, 2024
…rchdata import failure" 1. use the same generic torch CI workflow for periodic integration test, as in #325 for cpu/gpu unit tests. 2. `StatefulDataloader` is not in `torchdata` official release yet. Print helper message if user doesn't have a recent nightly installed. [ghstack-poisoned]
tianyu-l
pushed a commit
that referenced
this pull request
May 28, 2024
ghstack-source-id: b1fa8d8c1778ecb532ed71792ead9f4dbb067cf4 Pull Request resolved: #325
tianyu-l
pushed a commit
to tianyu-l/torchtitan_intern24
that referenced
this pull request
Aug 16, 2024
ghstack-source-id: b1fa8d8c1778ecb532ed71792ead9f4dbb067cf4 Pull Request resolved: pytorch#325
tianyu-l
pushed a commit
that referenced
this pull request
Aug 16, 2024
ghstack-source-id: 3569a5ed289a7645378983b6d18fb2b95126b34b Pull Request resolved: #325
tianyu-l
pushed a commit
that referenced
this pull request
Aug 16, 2024
ghstack-source-id: d27aa7d4a8fb93e8df750626e5e1c0d6db04c3d9 Pull Request resolved: #325
tianyu-l
pushed a commit
that referenced
this pull request
Aug 16, 2024
ghstack-source-id: b1fa8d8c1778ecb532ed71792ead9f4dbb067cf4 Pull Request resolved: #325
tianyu-l
pushed a commit
that referenced
this pull request
Aug 16, 2024
ghstack-source-id: 3569a5ed289a7645378983b6d18fb2b95126b34b Pull Request resolved: #325
tianyu-l
pushed a commit
that referenced
this pull request
Aug 16, 2024
ghstack-source-id: 3569a5ed289a7645378983b6d18fb2b95126b34b Pull Request resolved: #325
tianyu-l
pushed a commit
that referenced
this pull request
Aug 16, 2024
ghstack-source-id: b1fa8d8c1778ecb532ed71792ead9f4dbb067cf4 Pull Request resolved: #325
philippguevorguian
pushed a commit
to YerevaNN/YNNtitan
that referenced
this pull request
Aug 17, 2024
ghstack-source-id: b1fa8d8c1778ecb532ed71792ead9f4dbb067cf4 Pull Request resolved: pytorch#325
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stack from ghstack (oldest at bottom):
This moves over to using the standard pytorch CI job template. (doc).
The general advantages should be that we can more easily add features or options in a maintained way. A specific reason is becuase I was not able to ssh-debug on our old CI and @seemethere mentioned that the 'generic workflow' is where the CI SSH support lives.
SSH
Use ssh just like pytorch/pytorch CI:
Artifacts Uploading
The job.dump_folder for each test is uniquely named and bundled into an
outputs.zipwhich can be downloaded from github actions UI:To implement the artifacts upload, the following changes are made to test_runner.py
artifacts-to-be-uploaded