There seems to be no protection against fetching again an again the same version of the catalog

[kelson42]

If I click twice on "fetch catalog" it takes many seconds each time to complete. Which tends to let me think there is no check if the version of the online catalogue is the same as the copy we have locally. If the version would be checked and proven to be the same (via Etag), then Kiwix should not redownload everything and it should be over in 0.1s.

See https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/If-None-Match

[kelson42]

@BPerlakiH Can you please confirm/explain how it works?

[BPerlakiH]

@kelson42 I can see in the traffic that the catalog is refetched on each button press:
https://library.kiwix.org/catalog/v2/entries?count=-1
It seems to have the correct headers though:

Cache-Control: max-age=0, must-revalidate
ETag: "1726580544242481858.2213904/z"

The amount of time it takes is divided though. From what I can see the request (on a good wifi) takes only 900ms, whereas parsing and (maybe inserting the data) takes longer, around 2 more seconds.

The libkiwix library is also attempting to download the favicons, which I think is not really needed as it just simply not working, so we download those "on demand", when displaying the list of categories. It is also creating a lot of log outputs:

Cannot download favicon from https://library.kiwix.org/catalog/v2/illustration/da7fb796-e4ee-641f-41ff-6d7da49a28cb/?size=48
Cannot download favicon from https://library.kiwix.org/catalog/v2/illustration/4f489a41-d2dd-8066-7136-d21277420e45/?size=48
Cannot download favicon from https://library.kiwix.org/catalog/v2/illustration/f73302be-60bc-b740-53ba-50b1624eb4b9/?size=48
Cannot download favicon from https://library.kiwix.org/catalog/v2/illustration/e2c51a75-6700-cb90-69bd-f22a9de13482/?size=48
Cannot download favicon from https://library.kiwix.org/catalog/v2/illustration/646f6c7f-60da-6a29-ebf6-47384265edce/?size=48
Cannot download favicon from https://library.kiwix.org/catalog/v2/illustration/c9470bd8-571d-4815-76d3-fb4086036613/?size=48
Cannot download favicon from https://library.kiwix.org/catalog/v2/illustration/9f6dcc7e-a1dc-4868-4a8f-30d01a38ba29/?size=48
Cannot download favicon from https://library.kiwix.org/catalog/v2/illustration/2460effe-5812-9c25-1cfa-1ea824d9a843/?size=48
Cannot download favicon from https://library.kiwix.org/catalog/v2/illustration/fca187d6-425a-2e11-7867-eb61e2a7916e/?size=48
....

over 3200 lines of those.

[kelson42]

@kelson42 I can see in the traffic that the catalog is refetched on each button press:

Thx for confirming the bug. Please read documentation I have given and fix both HTTP request and response handling.

[kelson42]

The libkiwix library is also attempting to download the favicons, which I think is not really needed

If you believe the libkiwix does not work properly, please open an issue there.

[BPerlakiH]

There's a side effect bug of this solution.
When the user has downloaded the library, that is parsed and saved to the DB.
So far so good, but the zim file categories to languages dictionary is saved only in the memory, and therefore on subsequent launches the user might be stuck with a single category:

For this reason, I am re-opening this issue.

[kelson42]

@BPerlakiH Related to #1023 which should be kept in mind... but here its looks like that the refershing of the category list is just branched at the wrong place.