Roadmap for Open3d and future of Legacy vs Tensor api

[saurabheights]

Hi,

I wanted to get some idea on the future development of Open3d and was wondering if there is a roadmap set. This could be useful in directing open source development effort in a more meaningful direction.

Also, is there any plan in removing Legacy vs Tensor api, i.e. dual apis like o3d.geometry and o3d.t.geometry for a single api. Abstracting out the implementation details away and providing a single api to users will reduce the complexity. However, deprecating api would require a more concrete plan, which brings to my original question of any available roadmap/vision for open3d.

Thank you in advance.

[andy1122]

Hi @saurabheights

I was also looking into the code of the open3d, but i notice something interesting. Where the tensor implementations are based on the eigen based implementations. As you can see in this pull request:

#5330

Here the HiddenPointRemoval function is using the legacy(eigen) version to implement the tensor version, same goes for the other functions as per my understanding.

I am not sure, if they plan to remove one or other. Because from the way some of the above tensor api are implemented. They are completely dependent on the eigen versions.

Let me know, what do you think about this. This was just my observation. Btw, i am a complete beginner to open3d so my understanding can be wrong at places. Feel free to correct me at any place.

[saurabheights]

Hi @andy1122 , that is fine. Reimplementation efforts is also quite substantial, so using wrapper over legacy api is fine. However, IMO there should be a single api to same functionality. For e.g. o3d.t.geometry.Pointcloud() is giving the user same thing as o3d.geometry.Pointcloud(), aka a Pointcloud. Having double code, means twice replication, both for users and developers. P.S. Removal of one of these data structures, wont be so easy because the change will be backward incompatible.

Its questions and thoughts like this which made me ask the broader question - Whats the roadmap?

[andy1122]

I agree, atleast from the user POV it really makes no difference to get the same result from two different functions. However, there is going to be some difference in the performance for some specific function based on tensor api. Do you know, if someone has already done some benchmarking for the tensor vs eigen based functions. And if some of the functions are just wrapped over legacy api, then i am not sure if there is going to be some performance improvement or not. Which essentially kills the purpose of creating two different libraries.

Once we reach the point, where we see the clear difference from the performance POV, and check other tradeoffs as well. Then it will be possible to get rid off one of the libraries.

[ssheorey]

The broad roadmap is to replace eigen based functionality with Tensor based functionality throughout the library. This allows Open3D to take advantage of GPUs (and other future accelerators). The Tensor API is also more flexible in some ways (e.g. generic point cloud attributes). For CPU specific versions, some operations may be slower or faster compared to the Eigen API. As @saurabheights mentioned, Eigen is great well optimized library, so it's not trivial to improve upon it. GPU versions are significantly faster.

Our immediate goal is to provide feature parity between the tensor and legacy APIs so that the legacy API can be deprecated. Finally, (after deprecation notices for at least a couple of versions) o3d.t.geometry will simply become o3d.geometry. This is being accomplished either through re-implementation or through wrapping Eigen APIs depending upon complexity and potential for speedup when switching to the Tensor API. Afterward, we can continue optimizing the tensor functions after API is stabilized.

In parallel, we are continuing to add new functionality, bug fixes and enhancements. If progress seems slow, that is because we have a small team who only work on Open3D part-time. We would really encourage everyone to contribute to Open3D in any way (PRs, issues, discussions, help,....).