Prevent duplication of unsent scheduled reminders

[webmaster-cses-org-uk]

Overview

This PR provides a fix for the issue identified and discussed in https://lab.civicrm.org/dev/core/-/issues/3824. In the original issue, a server misconfiguration caused the sending of scheduled reminders to fail partway through. This resulted in multiple copies of the same reminder being queued up for sending every time the job ran (even if the reminder was already queued to send), meaning that when the job eventually succeeded, thousands of duplicate reminders were sent to contacts in the database.

Before

When sending a scheduled reminder on a repetition schedule, there is no mechanism to check whether a previous instance of the reminder was already queued to send but never actually got sent. As such, every time the job runs, a new copy of the reminder is queued to send. If and when the emails are actually sent, you get duplicate reminders.

See issue linked above for further details and screenshots etc.

Also, as a side issue, if the repetition schedule is set to 'minutes', this is incorrectly interpreted as 'hours'.

After

When sending a scheduled reminder on a repetition schedule, we now check to make sure that no previous unsent copies exist. If there are, we leave those to be sent but do not queue up a new copy of the reminder to be sent, thus avoiding duplication.

Also, the code to interpret 'minutes' as a schedule interval option is added.

Technical Details

The linked issue includes details of the testing used to simulate the problem and prove that it can be contained by this fix, and to prove that the fix does not interfere with normal sending of reminders.

In addition to this, the proposed fix has been live for > 1 month on a production system running CiviCRM 5.78.3. Reminders have been observed to continue sending as normal.

Comments

This is a robustness improvement to prevent a highly undesirable cascade failure mode (sending many hundreds or thousands of emails to users) in the event of a fault.

[civibot]

🤖 Thank you for contributing to CiviCRM! ❤️ We will need to test and review this PR. 👷

Introduction for new contributors...

• If this is your first PR, an admin will greenlight automated testing with the command ok to test or add to whitelist.
• A series of tests will automatically run. You can see the results at the bottom of this page (if there are any problems, it will include a link to see what went wrong).
• A demo site will be built where anyone can try out a version of CiviCRM that includes your changes.
• If this process needs to be repeated, an admin will issue the command test this please to rerun tests and build a new demo site.
• Before this PR can be merged, it needs to be reviewed. Please keep in mind that reviewers are volunteers, and their response time can vary from a few hours to a few weeks depending on their availability and their knowledge of this particular part of CiviCRM.
• A great way to speed up this process is to "trade reviews" with someone - find an open PR that you feel able to review, and leave a comment like "I'm reviewing this now, could you please review mine?" (include a link to yours). You don't have to wait for a response to get started (and you don't have to stop at one!) the more you review, the faster this process goes for everyone 😄
• To ensure that you are credited properly in the final release notes, please add yourself to contributor-key.yml
• For more information about contributing, see CONTRIBUTING.md.

Quick links for reviewers...

• ➡️ PR Demo site (can be rebuilt with the command test this please).
• 📖 Review standards
• 🗒️ Review template (brief or verbose)

➡️ Online demo of this PR 🔗

[ufundo]

Thanks for this work @webmaster-cses-org-uk - it looks like a very valuable failsafe to have in place.

Code change make sense to me, I'm going to have to get creative to test it but will look at it in the coming days.

[ufundo]

@webmaster-cses-org-uk I have been trying to test this just now, and I'm afraid I'm struggling to reproduce the original error.

I was attempting to simulate failure of the sending job by commenting out

civicrm-core/CRM/Core/BAO/ActionSchedule.php

Line 504 in 993588d

CRM_Core_BAO_ActionSchedule::sendMailings($mappingID, $now);

and it creates reminders with NULL action_date_time - but I couldn't replicate the duplicate rows.

The code is very terse... but I thought it might be only when you have a particular configuration of relative dates for the repetition.

I tried this

and UNTIL = 1 hour after

Would you be able to share the original config that caused your issue?

[ufundo]

I also noticed a possible issue with the interpretation of 1 minute.

It looks to me like

civicrm-core/Civi/ActionSchedule/RecipientBuilder.php

Line 224 in 993588d

->having("TIMESTAMPDIFF(HOUR, MAX(reminder.action_date_time), CAST(!casNow AS datetime)) >= TIMESTAMPDIFF(HOUR, MAX(reminder.action_date_time), DATE_ADD(MAX(reminder.action_date_time), INTERVAL !casRepetitionInterval))")

could be effectively rounding the interval calculation to the nearest hour with the first param to each TIMESTAMPDIFF call, which I think would cancel out your correction of parseRepetitionInterval .
So I think you might be on the right track, but another step required?

[ufundo]

For the minute thing - I would prefer to remove that option from the config UI (sending emails to a contact every minute is.... not a nice thing to do!)

Could you perhaps split out that change for now? And then we can focus on merging the failsafe in this PR?

[ufundo]

(I should say - I haven't confirmed above theory on minute repetition and I could well be reading that SQL wrong. Were you able to get minute-wise reminders actually going out with this change @webmaster-cses-org-uk ? )