Update: ONNX conversion doc

[b8zhong]

Update onnx + onnxruntime

Don't think quantize_onnx_model.py supports model name flag? Did not work w/ SPLADE++ Ensemble Distil earlier.

https://github.com/castorini/anserini/blob/master/src/main/python/onnx/quantize_onnx_model.py#L19-L24

[lintool]

Actually, @b8zhong you know what would be helpful? A snippet of code that compares the encoded output of the hgf model vs. the ONNX model - and take the two vectors and compute l1 norm or something - to confirm that the conversion worked.

would you might adding this?

[b8zhong]

@lintool any idea why such a large L1 diff?

L1 difference between PyTorch and ONNX outputs: 0.18276220560073853

Maybe AutoModel should actually be AutoModelForMaskedLM... ?

[lintool]

@b8zhong I ran into similar issues also. I believe you need to normalize the vector, see: #2645

The PR is defunct, but look at computeL2Norm.

[b8zhong]

Ty for the hint.. I don't think I would have ever thought of that 😛

There we go:
L1 difference between PyTorch and ONNX outputs: 0.009487475268542767

Acceptable I think?

[lintool]

Yup, that's good.

The page as written is specific to SPLADE++, can you make it generic, using SPLADE++ as an example?

[b8zhong]

Thoughts?

[lintool]

minor nits, I'll fix and then I'll merge.