Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

f16 support #172

Open
SludgePhD opened this issue Jun 4, 2023 · 2 comments
Open

f16 support #172

SludgePhD opened this issue Jun 4, 2023 · 2 comments

Comments

@SludgePhD
Copy link
Contributor

Network weights can be stored as f16 floats, halving the size of the network, which is often very desirable.

It would be nice if wonnx could support loading networks that do that. WebGPU has native support for f16 "half-precision" floats, so all GPU buffers could store them natively. Rust does not, however, so all network inputs and outputs would have to be converted.

@FL33TW00D
Copy link

Network weights can be stored as f16 floats, halving the size of the network, which is often very desirable.

It would be nice if wonnx could support loading networks that do that. WebGPU has native support for f16 "half-precision" floats, so all GPU buffers could store them natively. Rust does not, however, so all network inputs and outputs would have to be converted.

F16 is not supported in Naga yet: gfx-rs/wgpu#4384
Also not shipped in Chrome yet: https://bugs.chromium.org/p/dawn/issues/detail?id=1775&q=f16&can=2

@SludgePhD
Copy link
Contributor Author

Ah, that's unfortunate. In that case, wonnx could still upconvert these values to f32 when loading these models to make them work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants