📖 [Story] Torch-TensorRT Coverage Support for AI Model Hubs

[chohk88]

TL;DR

• This story focuses on identifying and resolving errors or issues that arise during the Torch Compile (Torch-TRT) process for key AI models, addressing both model-specific and common challenges.

Goal(s)

• Identify and log errors or issues that occur during the Torch Compile (Torch-TRT) process for models from Timm, HuggingFace, and Clara/Monai.
• Resolve these issues one by one, ensuring full support and compatibility for these models with Torch-TensorRT.

Tasks

Tasks

Beta Give feedback

1. [Coverage] Value Error for aten._native_batch_norm_legit_no_training.default #3180
   +0
   
2. [Coverage] NAN output for BasicUNet and Qwen-7B-Chat #3182
   +0
3. [Coverage] Runtime error related linear, reshape or slice ops #3183
   +0
4. [Coverage] Value Error for aten_ops.scatter.src #3184
   +0
   
5. [Coverage] ValueError: The meta val for input node max_pool2d_default is of type : <class 'tuple'> #3185
   +0
6. [Coverage] torch.OutOfMemoryError: CUDA out of memory (GPTJForQuestionAnswering, falcon-7b etc.) #3186
   +0
7. [Coverage] Type Error for torch.ops.aten.cumsum.default #3187
   +0
   
8. [Coverage] TypeError: _ModuleStackTracer.__init__() missing 1 required positional argument: 'scope_root' #3188
   +0
9. Large output differences with facebook/bart-base model #3252
   +0
   
10. [Coverage] RuntimeError (torch.ops.aten.expand.default) during matrix multiplication in SwinTransformer block #3257
    +0

Additional context