Specifying requests to screen extent (bounding box) for Esri Feature and WFS requests

[BenVanzino]

The current Esri Feature and WFS datasource implementations are not using BBOX extent limits in their API interactions. This is resulting in an unfiltered extent for the web service being requested for every interaction. This is not best practice for working with these kinds of spatial web services and leads to overload and timeout, failure to draw or general poor performance for the user.

Could these datasource interactions with terria use the map state bounding box to filter the interactions with the generated service requests as a fundamental improvement to how they work?

This is how best practice implementations are designed in spatial web applications and allows large feature count API interactions to be robust and performant. These requests are also tiled (3x3 or 4x4) extending slightly beyond current screen extent in many "good" implementations that i've seen.

This issue is repeatedly coming up with client use cases and interactions for our product and jeopardises our decision and use of TerriaJS as the foundation.

[zoran995]

Hi @BenVanzino, thanks for raising the issue.
For large vector datasets, the recommended approach would be using vector tiles as they are primarily designed for that. In the long run, it would be definitely nice to explore this possibility but not sure how huge a priority it is to the team right now.
Currently, TerriaJS loads all data at once and then converts it to vector tiles on the fly, which should prevent performance issues with rendering. If you are experiencing any problems, please let us know.

Could you share some examples of the solutions you are referring to? If you're interested in exploring this further, we'd be happy to review a pull request.

[BenVanzino]

Hi @zoran995,

Here are some examples:

https://services-ap1.arcgis.com/sRWrfkKjSushj6ql/ArcGIS/rest/services/Road_Crash_TEST/FeatureServer/0

https://services-ap1.arcgis.com/sRWrfkKjSushj6ql/ArcGIS/rest/services/speed_zones_v2/FeatureServer/0

Happy to explain further if the issue isn't apparent.

[AnaBelgun]

hi @BenVanzino @zoran995 this is the ticket that we'll actively take on as a priority to address the Esri Feature Services (part of the problem). Add support for Esri Vector Tile Services #6909

[nf-s]

Hi @BenVanzino

I have a very rough first version of this working. There is no styling, and there are some issues alone the tile borders.

I have only tested the two examples you provided, but it seems to work pretty well so far

• Speed zones http://ci.terria.io/tile-arcgis-featureserver/#share=s-ukspzS1sMDiZCzKoawp4VDSdnal
• Road crash http://ci.terria.io/tile-arcgis-featureserver/#share=s-2x51VYwXng8uGbRiSlGwqGfkCMU

There are a lot of features in these, so it can be slow on first load - I have also capped the number of features to 100,000 per tile for the moment

Let me know what you think!

Thanks

[BenVanzino]

Awesome, thank you @nf-s I'll take a look and let you know

[BenVanzino]

That is working very well and is fast for the size and complexity of the data involved. Those data collections are many hundred thousand features (some will be greater than 600K eventually). I don't think it will be a blocker in terms of increasing the limit total.

One follow up question is whether you are considering allowing feature querying as an additional interaction for these data sources?

[nf-s]

Hi @BenVanzino that is great to hear!

The feature limit is per tile, so as you zoom in you will see all features. The limit will only be hit when you are zoomed out - at which point drawing 100,000 features in a 256px x 256px tile doesn't achieve much (in most cases).

I will continue to test this with different datasets and tweak as needed. This limit will be a configurable option, so you can always increase it if needed.

And yes I will add feature querying - I am aiming to have feature parity with existing ArcGis Feature Server support. So feature picking and styling will be the same.

[BenVanzino]

Sounds great, thanks @nf-s

Looking forwards to seeing how it progresses