bge-reranker-v2-m3 model throughput benchmark

[rere950303]

Hello.
I'm trying to estimate how much GPU equipment is needed to serve a bge-reranker-v2-m3 fp16 reranker model.
We plan to input about 23 documents with a chunk size of 512, and the required TPS (transactions per second) is around 37.5. Could you help me determine how many T4 or A100 GPU units would be needed to meet these conditions?
When we ran a load test after allocating 2 SM slices on an A100 device, we observed about 5.4 TPS. However, we're unable to check the GPU utilization, so I was wondering if you could provide some assistance with this.