-
Notifications
You must be signed in to change notification settings - Fork 350
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
skip dummy inference and run_shape_analysis #3212
Open
lanluo-nvidia
wants to merge
22
commits into
main
Choose a base branch
from
lluo/save_remove_inputs
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
lanluo-nvidia
added
WIP
Work is in progress, pull request should not be merged yet
and removed
cla signed
labels
Oct 6, 2024
github-actions
bot
added
component: conversion
Issues re: Conversion stage
component: api [Python]
Issues re: Python API
component: runtime
component: dynamo
Issues relating to the `torch.compile` or `torch._dynamo.export` paths
labels
Oct 6, 2024
lanluo-nvidia
changed the title
skip run_shape_analysis
skip run_shape_analysis and get the output_shapes from compile stage saved in the TorchTensorRTModule
Oct 13, 2024
lanluo-nvidia
removed
the
WIP
Work is in progress, pull request should not be merged yet
label
Oct 13, 2024
narendasan
reviewed
Oct 14, 2024
narendasan
reviewed
Oct 14, 2024
narendasan
reviewed
Oct 14, 2024
narendasan
reviewed
Oct 14, 2024
narendasan
reviewed
Oct 14, 2024
narendasan
reviewed
Oct 14, 2024
narendasan
reviewed
Oct 14, 2024
narendasan
reviewed
Oct 14, 2024
Here's what I think could be a simpler way of doing this
Here the node corresponding to trt_module_node = [node for node in gm.graph.nodes if node.name == "_run_on_acc0"]
trt_module_node.meta["val"] - This should already have fake tensors which need to be used in the exporter.
trt_node.meta["val"] = trt_module_node.meta["val"]
Replacing the dummy inference will also need changes to our converter test suite. |
7 tasks
lanluo-nvidia
changed the title
skip run_shape_analysis and get the output_shapes from compile stage saved in the TorchTensorRTModule
skip dummy inference and run_shape_analysis
Oct 20, 2024
github-actions
bot
added
the
component: lowering
Issues re: The lowering / preprocessing passes
label
Oct 21, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
cla signed
component: api [Python]
Issues re: Python API
component: conversion
Issues re: Conversion stage
component: dynamo
Issues relating to the `torch.compile` or `torch._dynamo.export` paths
component: lowering
Issues re: The lowering / preprocessing passes
component: tests
Issues re: Tests
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
There is two changes introduced in this PR:
during the compile stage:
skipped dummy inference and use graph inspection instead to get the output_node.meta['val']
during the save stage:
skipped run_shape_analysis and use graph inspection instead to get the output_node.meta['val']
Fixes # (issue)
Type of change
Please delete options that are not relevant and/or add your own.
Checklist: