Polars extension #3356
Polars extension #3356
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
There was a problem hiding this comment.
Instead of adding a new namespace and requiring users to learn about it, I believe we can register custom formatters for Polars, in this file:
https://github.com/marimo-team/marimo/blob/main/marimo/_output/formatters/df_formatters.py
For example, we could monkey-patch show_graph to do the right thing in marimo.
I agree that the ideal solution is to get Polars to support marimo natively. They can detect if they are running in a marimo notebook by checking mo.running_in_notebook(), and then integrate with marimo in one of various ways (https://docs.marimo.io/guides/integrating_with_marimo/displaying_objects/).
@MarcoGorelli , if you don't mind, I would appreciate hearing your thoughts on whether implementing this upstream in Polars would be possible, or if we should plan to monkey-patch Polars in marimo instead.
| from marimo._polars import ( | ||
| lazyframe, # noqa: F401 # Import is required for lazyframe registration | ||
| ) |
There was a problem hiding this comment.
If registration happens on import of the polars module, then this can be removed.
https://github.com/marimo-team/marimo/blob/main/marimo/_output/formatters/df_formatters.py
As far as I can tell there's no need to add lazyframe to the marimo public API?
if the solution isn't too complex on the Polars side, then I think this would be welcome! |
|
I don't think we need to monkey-patch anything. showing mermaid should be the default (easier to navigate, adds no deps and not lossy) and instead we can add a formatted for LazyFrame in https://github.com/marimo-team/marimo/blob/main/marimo/_output/formatters/df_formatters.py If anything could be upstreamed, we could add |
Lazyframes already render fine in marimo according to #3355. I believe the core issue is modifying |
|
I was looking at this:
Which is possible to move to mermaid since marimo can change the formatting of the last line. You would still run the issue of explicitly calling But yea @akshayka you are right, without anything upstreamed w.r.t mermaid, we would need to monkeypatch to cover all cases. |
|
Looks like there was some appetite for mermaid formatting in polars: pola-rs/polars#12075 That part seems non-controversial, but could go as far as making the default |
|
Thank you all for the discussion on this. It sounds like there is already interest on the polars side for introducing support for mermaid, which is great to here. If polars does support mermaid and is ok with us inserting a simple snippet for detecting marimo then I think we could forgo the polars extension which would remove the need to add any imports to the marimo init file. I think then that the best way forward would be to:
If that sounds good to you all, I can close this PR and then make a new one to handle step 2 above. |
|
That sounds good to me, thank you! Would you like to work with Polars to lead 1 and 3 as well? If you opened an issue or PR with them, please feel free to tag me. |
|
Yes, I will open a PR in Polars for 1 to get started and tag you in it. |
📝 Summary
Fixes issue #3355 by adding LazyFrame extension that allows output of mermaid graphs for polars lazyframe query plans.
🔍 Description of Changes
In this pull request I have added a polars extension that adds a
monamespace to LazyFrames, this namespace includes one "public" methodshow_graphthat accepts the same arguments as polars LazyFrameshow_graphand returns anHtmlobject containing the mermaid chart.Issue #3355 shows side by side outputs.
Before merging it would be necessary to add documentation for this new polars namespace, but I wanted to request your feedback before I took that step.
📋 Checklist
📜 Reviewers
@akshayka