PMTiles: tooltip, layers, mapping data to aesthetics?

[cboettig]

Description

Leafmap has amazing support for pmtiles, allowing us to visualize very large vector datasets.

One limitation of the PMTiles implementation appears to be that while we can select a fixed color (hex code) to apply to every feature, I can't see how we can map feature aesthetics like fill color to the polygons. I think every other layer tool for vector objects supports the ability to map the fill color of a polygon based on a data value.

We can work around this limitation only crudely -- e.g. if the data value is categorical, we can split the data into multiple pmtiles files, and add each as a separate layer in a different color. This works reasonably well if there are a small number of classes we wish to color by (e.g. in this example). This creates more issues though -- for instance, the tooltip rendered is based only on the last layer added. To support a tooltip for all layers, we have to create a a single style that has multiple layers in it. This fixes the tooltip issue, but means that all the layers can only be added or removed simultaneously.

Source code

See source code at https://huggingface.co/spaces/boettiger-lab/pad-us/blob/ace796d8f9ddba2911d13550a1c2146230eb2176/app.py . The top map shows a single vector dataset that is split into 4 separate PMTiles files, layers added individually, and then a transparent (alpha=0 ) layer with all the polygons is added on top for a tooltip. This hack reduces performance by doubling the amount of tiles to be rendered.

[cboettig]

A bit out of my depth, but I see that some renders, like maplibre, seem to support a style syntax that permits mapping aesthetics like color to a feature property value instead of a literal color using this special get syntax:

  'paint': {
                'line-width': 3,
                // Use a get expression (https://maplibre.org/maplibre-style-spec/expressions/#get)
                // to set the line-color to a feature property value.
                'line-color': ['get', 'color']
            }

from: https://maplibre.org/maplibre-gl-js/docs/examples/data-driven-lines/. I see that the folium pmtiles driver includes an example that seems to support maplibre, https://github.com/jtmiclat/folium-pmtiles/blob/master/example/pmtiles_vector_maplibre.ipynb, haven't managed to test this syntax in leafmap yet but it may already do what I'm looking for if it's supported!

[giswqs]

Indeed, the leafmap pmtiles functionality originates from the folium-pmtiles pakcage. Here is an example using folium-pmtiles without leafmap. If you can make it work with folium-pmtiles, I can probably integrate it into leafmap. I don't know JavaScript well enough to fix these styling and tooltip issues on my own.

import folium

from folium_pmtiles.vector import PMTilesMapLibreLayer, PMTilesMapLibreTooltip

m = folium.Map(location=[43.7798, 11.24148], zoom_start=13, tiles="cartodb positron")
tooltip = PMTilesMapLibreTooltip()
pmtiles_url = "https://pmtiles.jtmiclat.me/protomaps(vector)ODbL_firenze.pmtiles"
pmtiles_layer = PMTilesMapLibreLayer(
    "folium_layer_name",
    style={
        "version": 8,
        "sources": {
            "example_source": {
                "type": "vector",
                "url": "pmtiles://" + pmtiles_url,
                "attribution": '<a href="https://protomaps.com">Protomaps</a> © <a href="https://openstreetmap.org/copyright">OpenStreetMap</a>',
            }
        },
        "layers": [
            {
                "id": "buildings",
                "source": "example_source",
                "source-layer": "landuse",
                "type": "fill",
                "paint": {"fill-color": "steelblue"},
            },
            {
                "id": "roads",
                "source": "example_source",
                "source-layer": "roads",
                "type": "line",
                "paint": {"line-color": "black"},
            },
        ],
    },
    tooltip=tooltip,
    overlay=True,
)
m.add_child(pmtiles_layer)
folium.LayerControl().add_to(m)
m

[cboettig]

ok, looks like the maplibre syntax for ['get', 'color'] in fact already works with leafmap as is:

url = "https://minio.carlboettiger.info/shared-data/pad/public.pmtiles"

style = {
    "version": 8,
    "sources": {
        "public": {
            "type": "vector",
            "url": "pmtiles://" + url,
            "attribution": "US PAD v3"}},
    "layers": [{
            "id": "public",
            "source": "public",
            "source-layer": "public",
            "type": "fill",
            "paint": {"fill-color": ["get", "color"],
                      "fill-opacity": 0.5}}]}


m = leafmap.Map(center=[35, -100], zoom=4)
m.add_basemap("Esri.WorldGrayCanvas")
m.add_pmtiles(url, name="Public", style=style, overlay=True, show=True, zoom_to_layer=False)
m

This is quite nice. This does require that we have literal colors in a column, but I guess that's a feature request I should make to maplibre.

I'm still stuck on being able to toggle layers off and on individually (e.g. the example you show above has a 'roads' layer and 'buildings' layer, but the corresponding Layer Control menu shows only a single PMTilesVector layer -- you can't display roads without buildings, say. I probably need to report that to Folium-pmtiles though? I think solving that upstream would also resolve the tooltip.

I don't see any remaining issues at the leafmap level, so I'll close this and try to request these features upstream.

[cboettig]

okay wow, digging a little deeper into the maplibre docs I realize these abstractions are indeed already possible. I had no idea how powerful the syntax options were in that paint block. In addition to get, we have all kinds of operators for doing color mapping to variables! Check out this example

 'paint': {
                'fill-color': [
                    'let',
                    'density',
                    ['/', ['get', 'population'], ['get', 'sq-km']],
                    [
                        'interpolate',
                        ['linear'],
                        ['zoom'],
                        8,
                        [
                            'interpolate',
                            ['linear'],
                            ['var', 'density'],
                            274,
                            ['to-color', '#edf8e9'],
                            1551,
                            ['to-color', '#006d2c']
                        ],
                        10,
                        [
                            'interpolate',
                            ['linear'],
                            ['var', 'density'],
                            274,
                            ['to-color', '#eff3ff'],
                            1551,
                            ['to-color', '#08519c']

I'll try to work out some useful but simple examples.

[lopezvoliver]

Moved this to a discussion, because it's really helpful. Thanks @cboettig for these tips, it helped me a lot to discover these functionalities for PMTiles!

I wonder if there is also filtering (based on attributes) or at least potentially map the "transparency" to a specific value that we want to filter out..

[cboettig]

@lopezvoliver 💯, I was wondering the same thing. It looks like there's quite a lot tooling in MapLibre to do filtering by transparency at least (e.g. mouse over a color causing other colors to go transparent, e.g. https://maplibre.org/maplibre-gl-js/docs/examples/hover-styles/) but it looks to me like folium-pmtiles doesn't have the handles we need to implement that yet?

I've been playing around with other approaches that let us hack around this a bit while staying in a (more familiar to me) python setting by just embedding my leafmap maps into a streamlit app, and using streamlit inputs to toggle different pre-baked MapLibre styles on and off. It's not quite as elegant, and indeed feels somewhat silly to fall back on a server-side solution when it can all be done on client-side javascript, but provides a short-term option at least.