feat: Add prefixStagePaths parameter to aws-s3 deployments

[akash1810]

Note
Easiest to review commit by commit as this change extracts some existing behaviour into a trait - this specific change should be considered a no-op.

Background

In the aws-s3 deployment type, the path within S3 that we upload to is controlled by the following parameters:

• prefixPackage
• prefixStack
• prefixApp
• prefixStage

This results in the path being the same regardless of the stage being deployed.

For example, for the file image.jpg and a shared bucket the upload path will be something like <bucket>/<STAGE>/image.jpg. If the bucket was not shared across stages (as recommended) the upload path will be something like <bucket>/image.jpg.

What does this change?

Introduces a new prefixStagePaths parameter for the aws-s3 deployment type.

This is to support services such as https://github.com/guardian/interactive-atom-maker. interactive-atom-maker is configured in a slightly non-standard way - CODE files live in <bucket>/atoms-CODE/image.jpg, PROD files live in <bucket>/atoms/image.jpg¹. The behaviour described above doesn't quite work here.

This change allows us to do:

prefixStagePaths:
  - CODE: atoms-CODE
  - PROD: atoms

When set, prefixStagePaths takes precedence over all other prefix<App|Stage|etc...> options.

How to test

Unit tests have been added to check:

• When prefixStagePaths is not set, we maintain existing behaviours of prefixApp, prefixStage, etc.
• What happens when prefixStagePaths is set
• When prefixStagePaths is set, describing stages of CODE and PROD, what happens when deploying stage of UAT (it should fail)
• When prefixStagePaths and prefix* are both set, what wins

For a practical demonstration of this new parameter, see https://github.com/guardian/csti-interactives/pull/53. To test, deploy this branch to CODE, then deploy a build from https://github.com/guardian/csti-interactives/pull/53, and observe the paths which Riff-Raff uploads files to.

How can we measure success?

https://github.com/guardian/csti-interactives (specifically https://github.com/guardian/csti-interactives/pull/53) is able to use Riff-Raff for continuous delivery.

Footnotes

1. We could update how interactive-atom-maker works, however this would involve updating all the articles that reference the files in S3, which is non-trivial. ↩

[twrichards]

lol, I think I've just raised a very similar feature in #1033 without seeing this

EDIT: this is nicer, so closing #1033

[twrichards]

When might this PR get finished / merged? I'd like to use it for guardian/actions-static-site#19 (comment)

Happy to help. Is it just tests which need adding?

[kenoir]

It feels a little strange that some of the logic for generating a prefix is in S3Upload.prefixGenerator and some of it here, should this be consolidated?

[akash1810]

S3Upload.prefixGenerator is used in a few places. Suggest we further refactor/DRY out in a follow-up PR to keep the diff in this PR to a minimum. WDYT?

[kenoir]

👏 well tested, very nice.