Python Too Slow Inference Speed, Colab T4 Gpu

[emrahe82]

this is the Colab file link used for test

https://colab.research.google.com/drive/1E-gt9d88qhG8bmjtmA1ov6Ciz_eI1UCQ?usp=sharing

i had to play with requirements.txt, and removed all the versions

torchvision
tqdm
opencv_python
torchsummary
torch
numpy

after this modification, able to pip3 install the requirements

then running the below code on Colab T4 Gpu
!python3 test.py --data data/coco.data --weights modelzoo/coco2017-0.241078ap-model.pth --img img/000139.jpg

output is:

load param...
forward time:275.415182ms
/usr/local/lib/python3.10/dist-packages/torch/functional.py:507: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3549.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]

getting 275 ms inference time on this one, while getting results under 10 ms on optimized Yolov8 & Tensorrt Half Precision
im probably doing something wrong, could you please help on this?