Support for SkipHours and Cache-Control/Expires

[DamonHD]

Have you considered support for these (RSS-feed-specified skipHours tag, and server-supplied HTTP expiry time/date) to reduce bandwidth and CPU?

[spacecowboy]

regular http cache-control is already supported.

what's skiphours?

[DamonHD]

Hi, Thanks for getting back to me! From: https://www.rssboard.org/rss-specification skipHours A hint for aggregators telling them which hours they can skip. This element contains up to 24 <hour> sub-elements whose value is a number between 0 and 23, representing a time in GMT, when aggregators, if they support the feature, may not read the channel on hours listed in the <skipHours> element. The hour beginning at midnight is hour zero. My server is solar powered off-grid and would be better not polled at night, and I’m asleep and not making updates either! So I have set: <skipHours><hour>0</hour><hour>1</hour><hour>2</hour><hour>3</hour><hour>4</hour><hour>5</hour><hour>6</hour<hour>7</hour><hour>22</hour><hour>23</hour></skipHours> Rgds Damon

…

On 28 Mar 2024, at 17:37, Jonas Kalderstam ***@***.***> wrote: regular http cache-control is already supported. what's skiphours? — Reply to this email directly, view it on GitHub <#249 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAY5FYEBFA3R36GFAQRMBRLY2RBKZAVCNFSM6AAAAABFF7LQX2VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DSNBTG4ZDE>. You are receiving this because you authored the thread.

[DamonHD]

BTW, are you implementing If-Modified-Since (or If-None-Match)? I'm not seeing 304s for your client, even when RSS feed files are not changing.

Rgds

Damon

[spacecowboy]

Both.

They are handled by OkHttp

What Cache headers does your server return? Or what is your feed address?

Regarding skipHours, any implementation would result in stochastic behavior for users. The feature was designed for servers which can pick when they sync, but Feeder is not not in control of when its background sync runs. This is determined by Android.

Feeder only makes a request to sync every X minutes, but how often and when will be determined by the system's JobScheduler.

And it can be further complicated by users requiring background sync only to run while charging etc.

[DamonHD]

A thought: maybe during skipHours you could avoid actually doing a poll when woken when there have been no non-skipHours since your last poll. The source is telling you that you will not (likely) have missed any change in that time.

[DamonHD]

Hi,

Main RSS feed: https://www.earth.org.uk/rss/podcast.rss though there are others.

A client should see Cache-Control, Expires, Last-Modified and ETag at least.

If you were able to at least get If-Modified-Since to work that could help reduce CPU and bandwidth for both ends significantly!

Thank you!

Damon

[DamonHD]

FWIW, it Feeder definitely now seems to be paying attention to Cache-Control/Expires, which is excellent. Thanks!

However, it is still apparently doing unconditional GETs, ie fetching the whole feed file even though it has not changed (and getting a small 304 instead):

[03/Apr/2024:19:12:31 +0000] "GET /rss/podcast.rss HTTP/2.0" 200 10965 "-" "SpaceCowboys Android RSS Reader / 2.6.21(306)"
[04/Apr/2024:06:38:51 +0000] "GET /rss/podcast.rss HTTP/2.0" 200 10965 "-" "SpaceCowboys Android RSS Reader / 2.6.21(306)"
[04/Apr/2024:13:49:26 +0000] "GET /rss/podcast.rss HTTP/2.0" 200 10965 "-" "SpaceCowboys Android RSS Reader / 2.6.21(306)"

Rgds

Damon

[spacecowboy]

If it's getting 304s, then everything is working as intended?

[DamonHD]

It's not getting 304s, it's getting 200s, even when the feed has not changed!

Rgds

Damon

[DamonHD]

It is possible that this https://httpd.apache.org/docs/current/mod/mod_deflate.html#deflatealteretag is getting in the way of If-None-Match allowing 304s.

I have tried adding "Header unset ETag" for .rss files to see if that helps. This is effectively an Apache 2.4 bug I think, so likely widespread!

Rgds

Damon

[DamonHD]

Wooot! Feeder got its first ever 304 from me! B^>

Rgds

Damon

[DamonHD]

An update to say that Feeder seems to be playing very nicely with my site!

(Please do consider skipHours at some point, maybe as I formulated above to deal with your concern...)

Thank you!

Damon

[spacecowboy]

No it does parse it: in Rss mode.

But I don't use that. I use the general feed parser and skipHours had no equivalent in Atom or Jsonfeed.

[spacecowboy]

Listen, in 10 years of developing this reader I've never encountered skipHours before. It's a dead feature

[markcellus]

It's a dead feature

I suspect this is because there are better ways to convey this information in response headers.

"hours" has always seemed like an odd way to convey time frames. Smaller increments like seconds or milliseconds are easier to work with.

[DamonHD]

My aim is for skipHours at least to be revived to help with energy conservation and fixing the climate, particularly in the typical podcaster's night time, given the huge amounts of solar going into world power grids. Simply avoiding doing unnecessary work when energy would have to come from storage will reduce the need for storage. There is a chicken-and-egg problem that no one will want to work with that tag until enough other people already are. But clearly you don't have to somehow fix that single-handed, and getting RSS feeds fixed is only a tiny part of the wider energy problem. And your other fixes already put you streets ahead of the big boys, so thank you!

Rgds

Damon

[DamonHD]

@markcellus a problem is that Apple / Amazon / Spotify and others are using none of the available features such as Cache-Control max-age, nor even compression in some cases, and are using 2 or 3 orders of magnitude more resource than makes sense just on this one aspect of polling a feed. (There are other horrors too, of course!) I think that the simple declarative nature of skipHours (and skipDays for weekday-only shows) deserves a revival as part of wider attention to resource efficiency in a world increasingly powered by renewables, especially solar. Using an existing well-defined simple static upfront declaration of when there are unlikely to be updates, so when not to poll at all, would be a great start.