feat(aws-tools)!: Update AWS CLI to v2

[akash1810]

What does this change?

Install the latest version of the AWS CLI (v2) using an AWS maintained mechanism.

It looks like installing the AWS CLI via pip3 isn't maintained by AWS as:

• It (still) installs v1
• It is not listed on https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html

Install the AWS CLI via an AWS provided zip file instead.

The package gets installed to /usr/local/bin/aws, replicating existing behaviour. This should make this change backwards compatible for places where we're using the CLI via its full path.

Why not snap?

I originally intended to install the CLI via the official snap package. However this'll install to /snap/bin/aws, and we'd have to symlink this to /usr/local/bin/aws for backwards compatibility. This was slightly less readable than installing from the zip file.

How to test

• Deploy to CODE (done)
• Bake a recipe that uses the aws-tools role, it should succeed (done)

What is the value of this?

• Using the latest version of the AWS CLI.
• This installation mechanism should allow AMIgo to provide better support for Ubuntu 24.04 as when installing via pip3 we're greeted with an error introduced in Python 3.12:

This environment is externally managed
╰─> To install Python packages system-wide, try apt install
python3-xyz, where xyz is the package you are trying to
install.

If you wish to install a non-Debian-packaged Python package,
create a virtual environment using python3 -m venv path/to/venv.
Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
sure you have python3-full installed.

If you wish to install a non-Debian packaged Python application,
it may be easiest to use pipx install xyz, which will manage a
virtual environment for you. Make sure you have pipx installed.

See /usr/share/doc/python3.12/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

Have we considered potential risks?

Version 2 brings some breaking changes - https://docs.aws.amazon.com/cli/latest/userguide/cliv2-migration-changes.html#cliv2-migration-changes-breaking. I can't see any immediate issues this would bring, however I could very well be wrong! If the breaking changes do impact us, I'd expect it to manifest in a failed deployment as the user-data script of an EC2 service would fail to run. That is, there wouldn't be any service disruption.

[adamnfish]

The "image bakery" model means we do this work once, properly. With that in mind, should we be verifying the signature as well? The verification mechanism uses pgp, so we'd need gpg installed on the box as well is the only real downside.

The instructions are fairly simple once gpg is installed

• include the AWS public key in this role
• make sure that the public key is imported into the poublic key ring
• also download the signature file for the AWS CLI
• Verify the signature of the downloaded zip file with gpg gpg --verify awscliv2.sig awscliv2.zip

I think generally in these image builds if we're downloading stuff off the internet we should be taking the extra step of verifying it, where possible.

Very happy to think about adding this separately though, we don't have those checks now and should focus on unblocking the 24.04 image creation!

[github-actions]

This PR is stale because it has been open 30 days with no activity. Unless a comment is added or the “stale” label removed, this will be closed in 3 days

[akash1810]

With that in mind, should we be verifying the signature as well?

Turns out this was simpler than anticipated. I've added this now.

Any advice on how to resolve the following warning welcomed:

[akash1810]

The contents of this file are from https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html.

[akash1810]

With that in mind, should we be verifying the signature as well?

Turns out this was simpler than anticipated. I've added this now.

Any advice on how to resolve the following warning welcomed:

Resolved in ffbb78f.

There's still more stdout detail than possibly desired, but we never really read the AMIgo build log so 🤷🏽:

[akash1810]

In this role, we're creating a gpg key for user AMIgo. This means on first run, it succeeds but on second it fails as the key we're trying to create already exists.

This register syntax was the only way I could see to create a variable in ansible. Ultimately, aws_cli_installed is used to detect if this role has already been executed or not.