Replies: 9 comments
-
|
Regarding the number of cached products which is interesting for the choice of db for web and probably for the whole app. Offline scanning (#18) got a bit out of sight but if we implement this feature (with sliced data sets) how many products can we expect? |
Beta Was this translation helpful? Give feedback.
-
@M123-dev That's 100% relevant: if we find out that we would be better off with another database system, it's better to acknowledge that before the app actually goes live.
Not really "at the moment": it's only in non-merged-yet #571, and the snackbar is not systematic - only when the staleness is longer than 5 minutes (which could be set longer or in the end-user preferences).
I already evoked this "freshness" trigger in #571 (comment). Assuming that we rely a lot on the backend, there's no reason why we should have a large local database. Let's say 1000 products is enough (about 10Mb). Which I think is more than generous.
Regarding the relevancy of the products, it's fair to apply a LRU algorithm for the garbage collecting if we go beyond the 1000 product limit. And that could also be part of the end-user settings, as I suspect that many people have faster smartphones that mine. |
Beta Was this translation helpful? Give feedback.
-
Yes I know, I had now described the behavior before the PR since it does not change the fundamental behavior that much.
Ohh that would of course be a reason against using the last edit time, but if we decide for such a kind of caching behavior then it probably won't be a problem to find / build an alternative for it.
LRU with maybe an exception for products which are stored in lists should do the job. But thats for another discussion. |
Beta Was this translation helpful? Give feedback.
-
I really think we should change that so that we always try to refresh the data that we show to users. e.g. when a user scans a product, we refresh it, when an user opens a product page, we refresh it, when an user accesses the history, we refresh it. The cache is just to show something faster (and then we update it if needed when we get the results back), or to offer a good experience when there is no internet connection. So there should not be a "refresh" button: we should refresh. Regarding the last edit / last modified timestamp: the current OFF API does not support returning 304 today, but that something we could explore. Instead of the last edit / last modified, we could also generate a fingerprint instead (similar to the HTTP ETag : https://developer.mozilla.org/fr/docs/Web/HTTP/Headers/ETag ). That would be much better to make sure we get the last version when and only when it has effectively changed. |
Beta Was this translation helpful? Give feedback.
-
|
@stephanegigandet That would mean that #571 should be dismissed, am I correct? (no problem, just checking) |
Beta Was this translation helpful? Give feedback.
-
|
@monsieurtanuki Right, I think it would make things simpler and provide a much better user experience. It's more like most other apps do things: they always refresh, and in the mean time they display what they had previously (if anything). |
Beta Was this translation helpful? Give feedback.
-
|
Okay that means we have a caching strategy. For now, I'd say go with the last edited time stack so we have the basic logic in place, but we should definitely switch to a ETag later. @stephane are you taking care of that server side? Now it's about how much we leave cached. |
Beta Was this translation helpful? Give feedback.
-
Eventually, yes, but as @monsieurtanuki said, if we always refresh when we show something, then proactively refreshing everything is not really needed. The ETag feature would save a few bytes though as we could just return a 304 or something if the product has not changed when we try to refresh. I filed an issue on the server to track it: openfoodfacts/openfoodfacts-server#5674 |
Beta Was this translation helpful? Give feedback.
-
Just for the record it's not what we do with osmdroid (openstreetmap library on Android): map tiles have an expiry date, and we refresh them only when they are accessed and expired. Typical freshness duration is one week. But here it's not osmdroid, I know ;) The ETag solution is elegant, and would save more than a few bytes as a product weights 10Kb. |
Beta Was this translation helpful? Give feedback.
-
A big question that is in the air right now is how we want to cache products and I think it would be good if we solve this question as early as possible. Especially for #470 (comment)
At the moment we cache the products in sqflite and show a snackbar on the product page with the time since caching and a refresh button. The only problem with the current implementation is that we also display outdated data teoretically forever.
It would be best if we find a way to get the best out of both of them, my suggestion would be that we request the last edited time from the backend as often as possible and when the data is outdated we either display a snack bar with "We have new data about this product, reload?" or even better refresh the product directly, of course we should then make sure that things like the scroll position don't change.
Since the offf-dart packge minifies the data (if we only need the last modified date, we only get that and not the whole product) we wouldn't overload the phone, the internet and the backend with unnecessary data.
What is your opinion on that: @monsieurtanuki @teolemon @stephanegigandet
Beta Was this translation helpful? Give feedback.
All reactions