Proposal: <podcast:replaces> to fix broken episodes

[theDanielJLewis]

In thinking of another potential use for <podcast:expire> (#448), I realized a better way of implementing this other use as a separate feature.

<podcast:replaces date="[date of replacement]">[previous episode GUID]</podcast:replaces>

(I think replace could work, too, but I think replaces reads better.)

This would be used to indicate that the newly released episode should replace the other episode by GUID. Ideally, the apps should then delete the previously downloaded version and replace it with this new one, probably maintaining playback position (but that might not be good in all circumstances).

The problem

Podcast apps track consumption with the item GUID. You can replace everything about an episode (even the title and enclosure), but the apps should see it as the same episode as long as the GUID remains the same.

On the flip side, change nothing about the item except the GUID, and apps should see it as a new episode.

Here's where to use this. When an episode contains some bad edits (like maybe forgotten background music, extra trailing silence, or some neglected editing), but such mistakes don't destroy the consumption experience, a podcaster can re-upload the audio file and all new downloads will get the new file while everyone who already downloaded the episode will keep their file and not be forced to redownload.

When there is a catastrophic problem (like a cohost's audio track is missing or the wrong audio was published), a podcaster should change the GUID of the post in order to force a redownload. (I suggest appending "[FIXED]" to the title, for about a week, for clarity.) This forced method should force everyone to redownload the new episode. (Because most publishing systems rightly don't allow editing the GUID, the next thing to do is copy all the post contents to a new post and publish that while unpublishing the faulty post.)

In other words, the situation calls for a forced redownload if the episode is unlistenable, but not if the episode was merely missing some smaller edits.

How <podcast:replaces> would work

This replaces tag would look the same as the item <guid>, and it would include the GUID of a previous item, allowing that faulty item to be completely removed from the feed.

A podcast app would see replaces, delete the faulty replaced episode (by its GUID) and replace it with this new episode. If there is no already-download episode with that GUID, then the podcast app does nothing different. And if the replaced episode has been marked as played, then the app would not redownload.

As for playback position, a podcast app could work like the Amazon Kindle app works when reading e-books across different devices. Jumping back to the same playback position in a replaced file could put the audience in the wrong spot if the replaced version shifted anything before the current playback position. Thus, I think a prompt would help the audience, but only trigger the prompt if the episode lengths mismatch.

For example:

• Episode lengths match: no prompt, jump to previous playback position
• Episode lengths do not match: prompt "This episode has been replaced. Returning to your playback position might put you in the wrong spot. [Resume where I was] | [Start over]"

Extending the idea

While being required to change the GUID (or make a new post in order to get a new GUID) ensures backward compatibility with all podcast apps, I could foresee allowing <podcast:replaces> on an existing item. This would negate the need to change the GUID or create a new post but at the cost of backward compatibility. Nonetheless, this is where the date attribute would be crucial.

When the tag references its own GUID, then the podcast app would force a redownload only if the replaces date was not previously seen.

This would allow an episode to be replaced multiple times without having to change the GUID (or repost) every time.

[adamc199]

Regardless of how it works, I like his idea a lot.
When I make a 'catastrophic' mistake and it affects 900k listeners, the fallout it enormous.

This is the "break glass in case of emergency" to me.

[brianoflondon]

This later use <podcast:replaces> especially in the case of a mess up sounds like something we can sell to app devs.

My issue with the <podcast:expires> tag is that as ever it requires both the hosts and the apps to support it and we end up in the chicken and egg situation where nobody can rely on expire until lots of apps have implemented it.

the <podcast:replaces> option seems to me would fix the kind of thing that app devs (and I'm thinking about Marco as my model) get support calls: "why haven't I got No Agenda yet?". The answer "Because Adam messed up his feed" doesn't help... but if there's an automated way Adam can push out a fix, that's going to prevent Marco getting calls but only IF he implements watching for the tag.

This has the chance of being mutually beneficial.

[theDanielJLewis]

You inspired a thought: this is totally backward compatible if the podcaster changed the GUID (manually or by creating a new post) of the replacement. Legacy apps would still see the new GUID as a new episode and would redownload, but modern apps would see <podcast:replaces> and replace the old episode—thus preventing what might initially look like duplicate downloads (unless the audience can see the "[FIXED]" part of the title, if added by the podcaster).

[keunes]

Reading the thread I was thinking 'why not keep the same episode/GUID but apply a 'force redownload' tag or sth. But indeed, only this 'replaces' tag would be backwards compatible.

[Dwev]

Bumping this up after hearing @theDanielJLewis mention it on the Future of Podcasting this week (18 April 2024).

I think it's a great idea!

The main beneficiaries would be the creator who has a mechanism to "push" an episode update, then the audience who know they are listening to the latest information. Useful for story episodes that have an update after original publication for example.

[theDanielJLewis]

Thanks for the bump!

[keunes]

This replaces tag would look the same as the item <guid>, and it would include the GUID of a previous item, allowing that faulty item to be completely removed from the feed.

I'm sorry to ask only now, but doesn't the RSS standard already allow you to update items? Don't we already have http headers which can indicate if anything has changed since X? In other words: we can't you (as a podcaster/host) update the feed with correct episode information, maintaining the episode guid?

My colleague often complains that Podcast namespace is replicating already existing functionality, and I often try to find the thing that's extra, not possible with the existing method. (For example Funding: payment link, Chapters: Podlove simple chapters.) Not sure if I can make that case here.

More importantly, and I might be coming a bit strong in this one: duplicate episodes are already a major issue. Adding the proposed tag just makes the situation worse (rather than better) for any app that hasn't implemented this tag (yet).

Many apps including AntennaPod don't physically delete episodes that are no longer in the feed. That's done for several reasons, e.g. to not lose user data (such as favourites) when podcasters/hosts only include the latest X episodes in the feed (instead of keeping the full backlog with paged feeds).

Whenever a feed would take the proposed approach and this 'replaces' tag isn't supported/recognised (yet), the episode would get to exist twice in the database. Once with a working download URL, once with a broken download URL, if the podcaster/host changed the enclosure URL.

When there is a catastrophic problem (like a cohost's audio track is missing or the wrong audio was published), a podcaster should change the GUID of the post in order to force a redownload.

If that's the problem, then I would rather suggest adding a <podcast:forceRefresh> tag which contains the datetime at which the episode was corrected. Apps like AntennaPod can always check this at feed refresh or even before playback to force an audio re-download. And this would avoid making the episode duplication situation worse for most apps (see explanation above, if you haven't read it).

[theDanielJLewis]

This is not merely to update an RSS feed, but to trigger a replacement of a faulty episode. Such as if the episode published with the wrong audio (like happened with Podnews Daily just last week).

My colleague often complains that Podcast namespace is replicating already existing functionality, and I often try to find the thing that's extra, not possible with the existing method. (For example Funding: payment link, Chapters: Podlove simple chapters.) Not sure if I can make that case here.

Part of the purpose of the Podcast Namespace is to consolidate existing functionality into a single namespace that is not controlled by a corporation. And yes, while the current "cloud chapters" model came from Podlove, there's work to turn cloud chapters into "super chapters" to do much more.

More importantly, and I might be coming a bit strong in this one: duplicate episodes are already a major issue. Adding the proposed tag just makes the situation worse (rather than better) for any app that hasn't implemented this tag (yet).

This would not be to duplicate episodes, but to remove a broken episode and add a new one with the indication of what episode it replaces.

If that's the problem, then I would rather suggest adding a podcast:forceRefresh tag which contains the datetime at which the episode was corrected. Apps like AntennaPod can always check this at feed refresh or even before playback to force an audio re-download. And this would avoid making the episode duplication situation worse for most apps (see explanation above, if you haven't read it).

That's a good suggestion! Let's see what others thing.

[jamescridland]

This is not merely to update an RSS feed, but to trigger a replacement of a faulty episode. Such as if the episode published with the wrong audio (like happened with Podnews Daily just last week).

The correct way to fix that would be to change the episode GUID. Currently, I don’t have a way to do that, but a decent podcast host would produce a GUID using a hash of the audio file.

I’m not sold on the need for this feature, and share the concerns from @keunes - this is an issue for best-practice documentation, not a requirement for a new tag. (Also, see the generator tag, media:thumbnail, and plenty of other perfectly valid existing RSS features).

[jamescridland]

I would also add:

Part of the purpose of the Podcast Namespace is to consolidate existing functionality into a single namespace that is not controlled by a corporation.

This is not an agreed aim of the podcast namespace. I heartily disagree with it, too. I have no interest in reinventing wheels just so we can stick it to the man.

[Dwev]

a decent podcast host would produce a GUID using a hash of the audio file

Yes! Without thinking it through, I think this is a terrific idea. It also means that for a podcaster, their GUID is truly portable. I know some hosts use the url as an episode GUID, but if the domain name changes later, it means either new GUIDs for the same file, or having outdated data in the feed.

I have no interest in reinventing wheels just so we can stick it to the man.

I'm not sure it's about "sticking it to the man" as opposed to having an open equivalent to a proprietary namespace. So far it's mostly about iTunes: but if Google or Libsyn had a specific namespace tag that benefited the whole community, it would future-proof it more to have it "owned" by the community.

As to the original idea of this thread, if there is functionality in the RSS namespace already to indicate an episode metadata or media should be refreshed after a certain date, it makes sense to use that.

Maybe we could add a hash attribute to any external linked file like chapters or transcripts, so they can be reloaded if they change.

[theDanielJLewis]

The correct way to fix that would be to change the episode GUID. Currently, I don’t have a way to do that, but a decent podcast host would produce a GUID using a hash of the audio file.

That's the problem this tag is intended to solve. Podcasters generally can't change their GUIDs unless they duplicate the post and republish it. But even if changing the episode GUID was easier, that still doesn't solve the problem I invented this tag to solve: replacing the faulty file and preventing a duplicate download.

Right now, if you change the GUID for an episode your audience already downloaded, probably every podcast app would keep the faulty episode and redownload the replacement episode as a "new" episode. This would lead to duplicate episodes in libraries. The purpose of this tag is to replace an already downloaded episode with the newer, corrected version.

Yes! Without thinking it through, I think this is a terrific idea. It also means that for a podcaster, their GUID is truly portable. I know some hosts use the url as an episode GUID, but if the domain name changes later, it means either new GUIDs for the same file, or having outdated data in the feed.

Good hosting providers and migration tools will leave the item GUIDs unchanged. What's in them really doesn't matter, only that it's a globally unique identifier.

[Dwev]

This article that @johnspurlock posted on Mastodon seems relevant and is good reading anyway.

https://www.earth.org.uk/RSS-efficiency.html

(Via https://podcastindex.social/@js/112327292467529296)

[samsethi]

Trying to get my head around this. I can see the use case/problem. But if I upload a new episode (fixed) and republish using the same GUID. Will apps not get a podping to tell apps to refresh the feed? Isn't this the goal of podping to alert apps to ANY and all RSS feed changes? We use it this way for Live when the status changes from pending to live. We use Podping when a new episode is first published. I would love to see Podping used for more realtime RSS updates.

[theDanielJLewis]

No. Because apps use the item GUID to track items. So if that item GUID is unchanged, the app will not see it as a new episode and will not redownload it. And thus the audience would not get the corrected version.

If you do change the item GUID but leave everything else the same, the apps will think it's a completely new episode and download it as such.

There's another case to consider: "streaming." If someone starts streaming an episode with a catastrophic mistake, the app most likely predownloaded the full media file anyway on clicking Play. But even if the app didn't, it would probably stream the replaced file if the GUID was the same.

[theDanielJLewis]

Here's something else to consider for this, as probably either the length attribute or something for this specific situation. What if the length or order of the media significantly changes? Like what Sam or James forgot to insert an interview into Podnews Weekly Review and you hear simply, "I spoke with John Smith about it … And wasn't that a lovely conversation?" Then the replacement episode needs to indicate that the timing has changed, and that could trigger the app to prompt the user to start over or continue from where they were.

This shouldn't be gleaned from only the enclosure's length attribute because there could be cases where the length is actually the same, but that's because some silence was left in where audio was supposed to be, or one track didn't shift from editing the other track(s).

[jamescridland]

It seems to me that all of this is fixed by being clear how a GUID should be constructed. Just like the <generator> tag, podcasting has grown using 20 years of random shit being typed into RSS metadata, and hoping it works.

If:
a) the GUID is changed based on the audio file
b) podcast aggregators remove old episodes that no longer appear in the feed

...then this isn't required. Both are the right things to do. Podcast Index doesn't do b), and many podcasters don't do a).

[theDanielJLewis]

I don't think any podcast apps remove episodes you've already downloaded or started streaming if they're no longer in the feed. That would be super frustrating for me if any apps I use do that.

[jamescridland]

What is the UX you want?

Seems to me that if you replace "My Fourth Episode" with a new piece of audio also called "My Fourth Episode" but with a different GUID, any aggregator should switch the audio. Not sure how it would work with a podcast app if you're mid-playing.

[Dwev]

If the episode has already started being played, there should be some notice to the user to say "there is a newer version of this episode, what would you like to do?" If the episode is unplayed, the default in the player should be to download the new version automatically, but perhaps this can be controlled in a setting.

Not sure about episodes that are removed from the stream completely, as some feeds might do this automatically for old eps just to keep the feed size down. I don't think the player should then remove those.

I guess an ep. being replaced is different to an ep. being removed, and should be handled differently.

[theDanielJLewis]

Yes, I like @Dwev's suggestion: auto-replace if it's unplayed, or prompt the user if it's partially played. Ideally, the app should jump to the same time in the replaced version. But the prompt from the app should warn if the durations don't match and thus jumping to the same time may not be the same place in the episode.