UTFGrid parsing is not working

[tomasbedrich]

Groundspeak changed the dimensions of UTFGrid Blocks from 3 to 6. As a result, Block._get_corrected_limits doesn't work, therefore the whole search_quick doesn't work either.

See https://tiles01.geocaching.com/map.info?x=8800&y=5574&z=14

[tomasbedrich]

@mkouhia: Based on your previous great work, would you like to update the parsing code?

[mkouhia]

Unfortunately I cannot allocate time to work on this. Thanks for asking, though. 20.12.2016 4.08 ap. "Tomáš Bedřich" <[email protected]> kirjoitti: @mkouhia <https://github.com/mkouhia>: Based on your previous work, would you like to update the parsing code? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#75 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABZqpbca_BqrlW18_aaL44hPHNgIPCHRks5rJzipgaJpZM4LRY35> .

[twlare]

Tomas, is this functionality still relevant? Would you like someone to work on it?

[tomasbedrich]

It makes sense for me to fix this only if it takes less than 1-2 days. But for me it is not possible to fully understand the relevant code to fix so quickly. So if you can do better, go for it. ;)

But honestly, unless you use it, I doubt that there are many people waiting for this fix.. Personally I think, that solving #29 may be more interesting.

[twlare]

Hi Tomas, You're right - I'm not using the UTFGrid function in anything I do with the library. I was just looking at solving some of the problems. I don't understand what's left to do with problem #29. I see that some code has been written and tested for speed. What exactly needs to be done besides the refactoring that you said you would do? Tim

…

On Tue, Nov 6, 2018 at 11:51 AM Tomáš Bedřich ***@***.***> wrote: It makes sense for me to fix this only if it takes less than 1-2 days. But for me it is not possible to fully understand the relevant code to fix so quickly. So if you can do better, go for it. ;) But honestly, unless *you* use it, I doubt that there are many people waiting for this fix.. Personally I think, that solving #29 <#29> may be more interesting. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#75 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Ak4Ye3kVsoK3vsKO9OFIsxTLP76dL1jRks5usegjgaJpZM4LRY35> .

[pkubiak]

Hi all,
nowadays there is an API endpoint used on https://www.geocaching.com/play/map/ :

https://www.geocaching.com/api/proxy/web/search?box=50.256919%2C18.943434%2C50.205967%2C19.027462&take=500&asc=true&skip=0&sort=distance&origin=50.2314498%2C18.985447900000054

it returns JSON with a lot of cache information:

{
    "results": [
        {
            "id": 6358428,
            "name": "Ligota #5 Bolesław Leśmian",
            "code": "GC7A78G",
            "premiumOnly": false,
            "favoritePoints": 4,
            "geocacheType": 2,
            "containerType": 2,
            "difficulty": 1.5,
            "terrain": 1,
            "userFound": false,
            "userDidNotFind": false,
            "cacheStatus": 0,
            "postedCoordinates": {
                "latitude": 50.2316,
                "longitude": 18.983617
            },
            "detailsUrl": "/geocache/GC7A78G",
            "hasGeotour": false,
            "hasLogDraft": false,
            "placedDate": "2017-08-07T00:00:00",
            "owner": {
                "code": "PRP7QFB",
                "username": "vaski82"
            },
            "lastFoundDate": "2020-04-26T14:24:57"
        }, ...
    ],
    "total": 75
}

It could be probably used to reimplement Geocaching.search_quick method?

[FriedrichFroebel]

@pkubiak Feel free to send a PR for this - in fact it seems like this endpoint is returning much more information than the tile-based approach. I am not completely sure about using this alternative endpoint, though.

[pkubiak]

I am not completely sure about using this alternative endpoint, though.

In which sense? Are there some limitations / usage concerns (e.g. from Groundspeak) ?

[FriedrichFroebel]

I do not want to decide this on my own - let's wait for @tomasbedrich and his opinion on this.

[tomasbedrich]

Hi @pkubiak, thanks for discovering this interesting endpoint! I really like it. My only concern is that it may be more strictly rate-limited than normal search page. Do we miss something else @FriedrichFroebel? If testing proves it to be good, I would even consider replacing normal search using this endpoint.

[FriedrichFroebel]

@tomasbedrich As far as I have seen, the cache data used in the grid parser is reflected in the new endpoint as well, so there does not seem to be a problem from this perspective. The only difference I could spot is that the tiles endpoint did not require an active session, while the new proxy API does - but this should not be a problem as we have an active session anyway. Rate-limiting and level of detail for BM/PM should be evaluated as well to avoid unexpected problems.

From my perspective, I would probably tend to provide this as an additional search method for now. Depending on how this turns out and how this is handled regarding rate limits, we can always make this the default search endpoint later on.

[tomasbedrich]

Good point with the active session. This means that it is easier for Groundspeak to set and enforce some rate-limits for this endpoint.

Is there a different level of detail for BM/PMs? I haven't noticed.

@pkubiak Let's try to implement it as a new search_quick() for now. As @FriedrichFroebel says – we can make this default later.

[pkubiak]

Hi @tomasbedrich 👍 I'll do my bests!

I have stared developing this feature in https://github.com/pkubiak/pycaching/tree/feature-search-rect-via-api

As far as I have noticed, there is rate limiting after downloading 2000 cached, then you must wait for 30-60s and you can continue downloading another 2000 caches. During my experiment I was able to download 8600 cached in 3:40 min. Rate limit exceeding is signaled with HTTP Exception 429 Too many requests, I have handled it, so it is unnoticeable for users.

There is also small difference between BM/PMs. BM don't seepm_only cache coordinates, all other fields are the same for both.