From 276c916a3b8e7c4163280a77bd9319d35e3fc38f Mon Sep 17 00:00:00 2001 From: Khai Do <3697686+zaro0508@users.noreply.github.com> Date: Wed, 22 Jan 2025 07:16:29 -0800 Subject: [PATCH] [IT-3831] Update AMI and docs (#6) Update agora-data-manager-github-runner AMI and general project documentation. depends on https://github.com/Sage-Bionetworks/aws-infra/pull/421 --- README.md | 67 ++++++++++++++----- .../agora-data-manager-github-runner.yaml | 9 +-- 2 files changed, 54 insertions(+), 22 deletions(-) diff --git a/README.md b/README.md index c22953b..51819b3 100644 --- a/README.md +++ b/README.md @@ -7,21 +7,52 @@ Request using PRs provide history, gating, reviewing and an approval process. ### Add Images -Step by step instructions to add a new image to the image builder: - -1. Add image builder cloudformation templates should be added to -[Sage-Bionetworks/aws-infra](https://github.com/Sage-Bionetworks/aws-infra) -repo in the templates/ImageBuilder directory. -2. Add a new Sceptre config in `config/prod` directory referencing -a versioned (or tagged) instance of the cloudformation template. -3. Create a PR to merge the new config into the `main` branch. -4. Once the PR is merged, the image builder will be triggered to -to deploy to the AWS org-sagebase-imagecentral account which will then -trigger a build of the AMI. -5. The AMI will be shared to all accounts in our AWS organization which -means that it is searchable using the AWS console (EC2-Instances-AMIs or -Image Builder->Images) or the AWS CLI from any AWS account in our organization. - +Step by step instructions to create a new image: + +The first step is to create the definition template: +1. Create an Image Builder definition (cloudformation) template and add it to +the [Sage-Bionetworks/aws-infra](https://github.com/Sage-Bionetworks/aws-infra) +repo in the `templates/ImageBuilder` directory. +2. Create a PR with this change to aws-infra repo +3. Review, approve and Merge the PR +4. Tag the repo with a [version number](https://github.com/Sage-Bionetworks/aws-infra/tags) + +Next step is to use the definition template to provision an AMI: +1. In this repo, add a new Sceptre config in `config/prod` directory referencing +a versioned (or tagged) instance of the cloudformation template. Set the +the `ImageVersion` to the same number as the aws-infra repo tag (i.e. 0.9.2 without the `v`). +It must match a [semantic version](https://semver.org/) number. +2. Create a PR to merge the new config into the `main` branch. +3. Once the PR is merged, the cloudformation template will be deployed to the +AWS org-sagebase-imagecentral account which will in turn trigger a build +of the image which will generate an AMI. +4. The AMI will be shared to all accounts in our AWS organization which +means that it is searchable from any of our AWS accounts using either the +AWS console EC2 Instances->AMIs or Image Builder->Images pages. The AMI is +also searchable using the AWS CLI +[describe-images command](https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-images.html). + + +### Updating Images +To update an image the we must first update aws-infra then this repo. + +Update the cloudformation template in aws-infra: +1. Create a PR to update the ImageBuilder cloudformation template in the aws-infra repo +2. Review, approve and Merge the PR +3. Tag the repo with a version number + +Update the Sceptre config in this repo: +1. Create a PR to update the template `url` reference and `ImageVersion` number +in the Sceptre config file. +2. Review, approve and Merge the PR +3. Once merged the cloudformation template will be deployed which will trigger +an update to the AMI. AMIs are immutable therefore AWS will create a new AMI +on every change. The updated AMIs will retain the same name, only the version +number is updated. + +__Note__: An update to the image definition (or cloudformation template) requires +an `ImageVersion` update otherwise cloudformation may fail with a message similar to +`The following resource XXXXX already exists..`. ### Removing Images Important info when removing image builder resources and the generated @@ -37,14 +68,14 @@ situations where it is appropriate to delete AMIs, for example when testing AMIs To delete AMIs go into the AWS console EC2 -> Images -> AMIs, search for the AMIs then delete (or de-register) them. -__NOTE__: Steps above can also be done using the `sceptre delete` command or the AWS CLI. +__NOTE__: Step #2 above can also be done using the `sceptre delete` command or the AWS CLI. ### Testing Images To test an image we recommend that you manually provision an EC2 instance from the AMI image then connect to it using the [AWS session manager](https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager.html) or [SSH](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/connect-linux-inst-ssh.html). -Validate that the instance contains everything that was specified from the image builder and -that it's generally running as expected. +Validate that the image meets the specifications defined in the cloudformation template and +that image is generally running as expected. ## Contributions diff --git a/config/prod/agora-data-manager-github-runner.yaml b/config/prod/agora-data-manager-github-runner.yaml index 4c166e3..c6ef9a4 100644 --- a/config/prod/agora-data-manager-github-runner.yaml +++ b/config/prod/agora-data-manager-github-runner.yaml @@ -1,9 +1,10 @@ template: type: "http" - url: "https://raw.githubusercontent.com/Sage-Bionetworks/aws-infra/v0.9.3/templates/ImageBuilder/amazon-linux-2023-agora-bastian.yaml" + url: "https://raw.githubusercontent.com/Sage-Bionetworks/aws-infra/v0.9.4/templates/ImageBuilder/amazon-linux-2023-agora-bastian.yaml" stack_name: "agora-data-manager-github-runner" stack_tags: - OwnerEmail: "it@sagebase.org" - CostCenter: "Platform Infrastructure / 990300" + OwnerEmail: "jessica.britton@sagebase.org" + CostCenter: "Agora / 112300" parameters: - ImageVersion: "0.0.1" + ImageVersion: "0.9.4" + VolumeSize: "50"