fix periodic integration test and add helper message on torchdata import failure #353
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…mport failure [ghstack-poisoned]
tianyu-l
added a commit
that referenced
this pull request
May 22, 2024
…mport failure ghstack-source-id: 1c2355c2d7d05f3118a8864ac996afe552fa3ff6 Pull Request resolved: #353
facebook-github-bot
added
the
CLA Signed
tianyu-l
changed the title
wconstab
reviewed
May 22, 2024
torchtitan/datasets/hf_datasets.py
Outdated
|
|
||
| try: | ||
| from torchdata.stateful_dataloader import StatefulDataLoader | ||
| except ModuleNotFoundError as e: |
There was a problem hiding this comment.
Nit: maybe also include import error here incase user did not install torch data at all? For my case I had tried to install but the version was wrong.
There was a problem hiding this comment.
It turns out ModuleNotFoundError is a subclass of ImportError, so changing to ImportError here.
…rchdata import failure" 1. use the same generic torch CI workflow for periodic integration test, as in #325 for cpu/gpu unit tests. 2. `StatefulDataloader` is not in `torchdata` official release yet. Print helper message if user doesn't have a recent nightly installed. [ghstack-poisoned]
tianyu-l
added a commit
that referenced
this pull request
May 22, 2024
…mport failure ghstack-source-id: 4db9ec111c83f7873253f19f0c95a997800e0f6b Pull Request resolved: #353
tianyu-l
added a commit
that referenced
this pull request
May 22, 2024
…mport failure ghstack-source-id: 4db9ec111c83f7873253f19f0c95a997800e0f6b Pull Request resolved: #353
tianyu-l
added a commit
that referenced
this pull request
May 28, 2024
…mport failure ghstack-source-id: 4db9ec111c83f7873253f19f0c95a997800e0f6b Pull Request resolved: #353
tianyu-l
added a commit
to tianyu-l/torchtitan_intern24
that referenced
this pull request
Aug 16, 2024
…mport failure ghstack-source-id: 4db9ec111c83f7873253f19f0c95a997800e0f6b Pull Request resolved: pytorch#353
tianyu-l
added a commit
that referenced
this pull request
Aug 16, 2024
…mport failure ghstack-source-id: 4db9ec111c83f7873253f19f0c95a997800e0f6b Pull Request resolved: #353
philippguevorguian
pushed a commit
to YerevaNN/YNNtitan
that referenced
this pull request
Aug 17, 2024
…mport failure ghstack-source-id: 4db9ec111c83f7873253f19f0c95a997800e0f6b Pull Request resolved: pytorch#353
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stack from ghstack (oldest at bottom):
StatefulDataloaderis not intorchdataofficial release yet. Print helper message if user doesn't have a recent nightly installed.