v3: fix the migration mess

[ewjoachim]

Closes #1189

Warning

the word "mess" is is solely directed at the mess I created trhough unclear rules and overly complex processes, the work done so far by everyone has been amazing

Introduce a new migration process: pre-migrations and post-migrations.

• They will be tied to the corresponding release
• Reworked v3 migrations with this new template in mind
• They allow us to provide a much saner path for blue-green deployment
• The process is currently not tested (but nor was the previous system)
• We will need to be careful when we do a release to rename the migrations in ongoing PRs (as today)
• We (hopefully) won't have to do multiple releases just for the sake of blue-green compatibility.
• I'm adding versioning in the name of all internal objects (functions, triggers, etc) (as mentionned here)

So the idea is that:

• Procrastinate v2 with the pre-migrations & without the post-migrations should work
• Procrastinate v3 with the pre-migrations & without the post-migrations should work
• Procrastinate v3 with all migrations should be what we currently have in v3

(I'm 80% sure that I have missed a few corner cases)

Is it going in the right direction according to you @medihack @onlyann ?

Successful PR Checklist:

• Tests
  • (not applicable?)
• Documentation
  • (not applicable?)

PR label(s):

• PR type: breaking 💥 Contains breaking changes
• PR type: feature ⭐️ Contains new features
• PR type: bugfix 🕵️ Contains bug fix
• PR type: miscellaneous 👾 Contains misc changes
• PR type: dependencies 🤖 Contains only dependencies updates
• PR type: documentation 📚 Contains documentation updates

[onlyann]

Let's see if I understand.

pre-script version N can be executed while the code version N-1 is running.
post-script version N is not compatible with code version N-1 and must be executed once code version N is running.

Is the idea that procrastinate semantic versioning is at the programmatic API level and that any new version can possibly introduce database breaking changes as long as there is the ability to split them in pre-scripts and post-scripts to maintain blue-green deployments of the codebase?

What are your thoughts about testing out that pre-migration script is backward compatible?

I like it.

[ewjoachim]

Is the idea that procrastinate semantic versioning is at the programmatic API level and that any new version can possibly introduce database breaking changes as long as there is the ability to split them in pre-scripts and post-scripts to maintain blue-green deployments of the codebase?

Yep. I think this change removes some hurdles that make keeping blue-green possible (and sane) while the current state of affairs made it more confusing than it needed to be and harder to achieve. We can iterate on the tooling to make this actually easy in the future.

What are your thoughts about testing out that pre-migration script is backward compatible?

My ideas:
1/ we could run the tests on n-1 & pre. (The n-1 tests supposedly should work, though it will be very annoying to fix if they don't for some reason)
2/ for n & pre, i'm tempted to run the normal tests with an escape hatch allowing to easily skip the tests that can't run yet because they don't have the DB ready
3/ an alternative could be to redo our end to end tests to be a bit more versatile: test a large part of the external facing API and not necessarily handle the setup of the database.
This way, we may just run the e2e tests on n-1&pre and n&pre I think this would cover what we actually want to check and would be unlikely to cause issues. I'm not entirely sure if the end to end tests that would run on n-1&pre would be the end to end tests as defined on the n codebase or on the n-1 codebase. Both have merits.

[medihack]

Must this not be created in pre? Some functions already use it there.

[medihack]

I still think this is true. I will move it over to procrastinate/sql/migrations/03.00.00_01_pre_cancel_notification.sql

[medihack]

I think I get the idea now and also like it 🙂. The separation between pre and post makes things clearer, as does the versioning of those functions. Should we also give the remaining functions and types a version to make it consistent (procrastinate_job_event_type, procrastinate_defer_job, procrastinate_defer_periodic_job, procrastinate_unlink_periodic_defers, procrastinate_trigger_delete_jobs?

[ewjoachim]

I'm delighted by your answers. You're most probably right that we'll need to continue adjusting a few things.

Given that we need v3+pre to work, we need v3's queries to work with pre, which means the new enum needs to be supported in pre.

Post can only be used to clean the mess.

[medihack]

@ewjoachim Do you need a helping hand here? I have some time left in the upcoming days and can take care of this issue (at least I could try).

[ewjoachim]

I probably won't have time to work on it Saturday, there might be a chance Sunday and then probably not for a few days, so I'd say: feel free to try something as long as we communicate where we're at.
Alternatively if you want, we can try to do it in pair programming on Sunday or on an evening next week :)

[medihack]

I probably won't have time to work on it Saturday, there might be a chance Sunday and then probably not for a few days, so I'd say: feel free to try something as long as we communicate where we're at. Alternatively if you want, we can try to do it in pair programming on Sunday or on an evening next week :)

Alright, let's see how far I get. I am probably not yet fully aware of the extent of this PR. And yes, pair programming sounds fun (I have never done anything like this before), but it sounds more like you will be tutoring me then ;-)

[medihack]

I don't think this is necessary when just changing the column type (it is compatible with the new type). We have also already created those functions in procrastinate/sql/migrations/03.00.00_01_pre_cancel_notification.sql.

[medihack]

Based on this one, I will work on a new PR (#1227). So we can throw mine away if I go in the wrong direction.

[ewjoachim]

(closed in favor of #1227 )