Give up at some point

[Zash]

1. Hack around a bit on testxmpp
2. Wait 6 months
3. Hack around a bit on testxmpp
4. Observe that it's still trying a failed target from 6 months ago (IPv6-only targets #11)

Trying tests again is good, but unless it gives up at some point the number of failed and retrying tests will accumulate and likely run into some sort of limit eventually.

[guusdk]

Is a quick and dirty approach to a solve for this to check for 'creation_time' at or near https://github.com/xmpp-observatory/testxmpp/blob/master/testxmpp/coordinator/tasks.py#L72 ?

Alternatively, can we get away with simply never retrying a scan if it fails for whatever reason?

[root021]

It has been a while since I was hacking away at this, but I tried something similar, and it still seemed to retry failed tasks. I would need to recheck exactly what I have tried, though, as I took multiple different approaches to various limits.

[guusdk]

Is the retry functionality critical for operation? If this is difficult to get right, and also blocking us from moving forwards somewhat, maybe we should consider introducing a simple "sorry, something failed - please try again later" message.