Benchmarking the Performance (FPS) of Different Stages of the Pipeline

[vaghawan]

Hi,

Thanks a million for such a wonderful and seamless framework. If you've to deal with tools e.g deepstream and such, you know how thankful one can be of pipeless.

Now to my problem, I need to benchmark the performance at the each stage of the pipeline, to begin with I need to capture the average frame per second (FPS) performance of each stage. I've 6 stages, and my goal is to stream back the results in almost real time, but to even begin to assess it, I need a way to capture the FPS first.

Can somebody who may have the idea or who may have already done it give me some input, or share how can I possibly do it?

Thank you in advance.

[miguelaeh]

Hi @vaghawan ,
A way in which you can measure that is using the KV store to store the values you canculate on the stage. For example, you can calculate how long the code in your stage takes to run and then store the value or add it to a mean or whatever you need. Do you think that will help you obtain what you need?

[vaghawan]

Hi @miguelaeh , Thank you!

I'm referring to this page : https://www.pipeless.ai/docs/docs/v1/kvstore

I believe pipeless_kvs_set("my_key", "My value") , this is to be set only once, but I need to measure the time it took in the process in every single call of the hook to determine the average processing time.

   def hook(frame_data, _):
    start_time = time.time()
    frame = frame_data["original"].view()
    yolo_input_shape = (640, 640, 3) # h,w,c
    frame = resize_with_padding(frame, yolo_input_shape)
    frame = np.array(frame) / 255.0 # Normalize pixel values
    frame = np.transpose(frame, axes=(2,0,1)) # Convert to c,h,w
    inference_inputs = frame.astype("float32")
    frame_data['inference_input'] = inference_inputs
    end_time = time.time()
    # Write the time difference to the file
    with open(log_file, "a") as f:
        f.write(f"Time taken: {(end_time - start_time) * 1000} milliseconds\n")

Say if I were to use the KV, would I be setting the value in each hook call, wouldn't that replace the old value?

Thanks

[vaghawan]

Hi @miguelaeh , I've the average benchmark for some models on Jetson Orien device. How do you want me to forward it to you? I just have average and standard deviation of the inference time for Yolo TensorRT converted models and some other models, which may not so relevant for general usage.

[miguelaeh]

Hi @vaghawan ,
Regarding the timestamps you can just use the time method on the hook. That's the moment when it gets executed.

Thank you very much for the Orien benchmark!! You can open a PR on the docs repo directly to add them, I will be more than hsppy to review it

[vaghawan]

Thanks @miguelaeh , I'm going to do that very soon, I already have the numbers. By the way, one thing, is it possible to dump the kv storage (user data) when the whole pipeline is finished? I could record the individual timestamps that way too.
It appears writing the time for each frame by appending to a txt file seems to add latency in itself. I would like to export the frame_ids at the end too, so that I can keep track of them in some kind of database.

[miguelaeh]

I guess we could implement such dump on the core itself. Everything you add to the pipeline will add latency, so I guess the tolerance depends on the use case

[vaghawan]

Hi @miguelaeh , Thanks, sent in the PR for the benchmark.

I guess we could implement such dump on the core itself.

I think this will be useful.