v2.0.0-rc.1 #180
decahedron1
announced in
Announcements
v2.0.0-rc.1
#180
Replies: 1 comment
-
Amazing effort!! |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Value specialization
The
Value
struct has been refactored into multiple strongly-typed structs:Tensor<T>
,Map<K, V>
, andSequence<T>
, and their type-erased variants:DynTensor
,DynMap
, andDynSequence
.Values returned by session inference are now
DynValue
s, which behave exactly the same asValue
in previous versions.Tensors created from Rust, like via the new
Tensor::new
function, can be directly and infallibly extracted into its underlying data viaextract_tensor
(notry_
):You can still extract tensors, maps, or sequence values normally from a
DynValue
usingtry_extract_*
:DynValue
can beupcast()
ed to the more specialized types, likeDynMap
orTensor<T>
:Similarly, a strongly-typed value like
Tensor<T>
can be downcast back into aDynValue
orDynTensor
.Tensor extraction directly returns an
ArrayView
extract_tensor
(and nowtry_extract_tensor
) now return anndarray::ArrayView
directly, instead of putting it behind the oldort::Tensor<T>
type (not to be confused with the new specialized value type). This means you don't have to.view()
on the result:Full support for sequence & map values
You can now construct and extract
Sequence
/Map
values.Value views
You can now obtain a view of any
Value
via the newview()
andview_mut()
functions, which operate similar tondarray
's own view system. These views can also now be passed into session inputs.Mutable tensor extraction
You can extract a mutable
ArrayViewMut
or&mut [T]
from a mutable reference to a tensor.Device-allocated tensors
You can now create a tensor on device memory with
Tensor::new
& an allocator:The data will be allocated by the device specified by the allocator. You can then use the new mutable tensor extraction to modify the tensor's data.
What if custom operators were 🚀 blazingly 🔥 fast 🦀?
You can now write custom operator kernels in Rust. Check out the
custom-ops
example.Custom operator library feature change
Since custom operators can now be written completely in Rust, the old
custom-ops
feature, which enabled loading custom operators from an external dynamic library, has been renamed tooperator-libraries
.Additionally,
Session::with_custom_ops_lib
has been renamed toSession::with_operator_library
, and the confusingly namedSession::with_enable_custom_ops
(which does not enable custom operators in general, but rather attempts to loadonnxruntime-extensions
) has been updated toSession::with_extensions
to reflect its actual behavior.Asynchronous inference
Session
introduces a newrun_async
method which returns inference results via a future. It's also cancel-safe, so you can simply cancel inference with something liketokio::select!
ortokio::time::timeout
.If you have any questions about this release, we're here to help:
#💬|ort-discussions
Love
ort
? Consider supporting us on Open Collective 💖❤️💚💙💛
This discussion was created from the release v2.0.0-rc.1.
Beta Was this translation helpful? Give feedback.
All reactions