video inference

[MELANCHOLY828]

I want to use it for video inference. Does the exported SAM2 encoder and decoder support it?

[ibaiGorordo]

Unfortunately no. It requires the memory part to be added. I think it is possible, but I gave up after trying for a while. The way the memory is recorded in the dictionary and passed away makes it a bit hard to decouple it from the rest of the code.

[bhack]

I think the only opportunity is to pass the state:
huggingface/transformers#32394 (comment)

[alawasoft]

Would be great if the memory attention added to the onnx stacks

[bhack]

If you are interested in video check facebookresearch/sam2#3 (comment)

[alawasoft]

If you are interested in video check facebookresearch/sam2#3 (comment)

any plans to update to Segment anything 2.1? great work btw.