Plan for supporting data destinations #62
wangxiaoying
started this conversation in
Ideas
Replies: 1 comment
-
I think that using arrow as data destination gives you a lot of future destinations for free. I believe arrow has the fastest algorithm to ingest data into pandas without triggering block manager re-allocations. This would give you zero copy transfer to products build around apache arrow like Polars and Dremio. And probably fast ingestion to other products. I could help on the arrow interop. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Other than Pandas.DataFrame, we plan to support the following destinations:
Reply here if you have other ideas!
Beta Was this translation helpful? Give feedback.
All reactions