Skip to content

Commit

Permalink
[IT-3831] Update AMI and docs (#6)
Browse files Browse the repository at this point in the history
Update agora-data-manager-github-runner AMI and general project
documentation.

depends on Sage-Bionetworks/aws-infra#421
  • Loading branch information
zaro0508 authored Jan 22, 2025
1 parent c3d011c commit 276c916
Show file tree
Hide file tree
Showing 2 changed files with 54 additions and 22 deletions.
67 changes: 49 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,21 +7,52 @@ Request using PRs provide history, gating, reviewing and an approval
process.

### Add Images
Step by step instructions to add a new image to the image builder:

1. Add image builder cloudformation templates should be added to
[Sage-Bionetworks/aws-infra](https://github.com/Sage-Bionetworks/aws-infra)
repo in the templates/ImageBuilder directory.
2. Add a new Sceptre config in `config/prod` directory referencing
a versioned (or tagged) instance of the cloudformation template.
3. Create a PR to merge the new config into the `main` branch.
4. Once the PR is merged, the image builder will be triggered to
to deploy to the AWS org-sagebase-imagecentral account which will then
trigger a build of the AMI.
5. The AMI will be shared to all accounts in our AWS organization which
means that it is searchable using the AWS console (EC2-Instances-AMIs or
Image Builder->Images) or the AWS CLI from any AWS account in our organization.

Step by step instructions to create a new image:

The first step is to create the definition template:
1. Create an Image Builder definition (cloudformation) template and add it to
the [Sage-Bionetworks/aws-infra](https://github.com/Sage-Bionetworks/aws-infra)
repo in the `templates/ImageBuilder` directory.
2. Create a PR with this change to aws-infra repo
3. Review, approve and Merge the PR
4. Tag the repo with a [version number](https://github.com/Sage-Bionetworks/aws-infra/tags)

Next step is to use the definition template to provision an AMI:
1. In this repo, add a new Sceptre config in `config/prod` directory referencing
a versioned (or tagged) instance of the cloudformation template. Set the
the `ImageVersion` to the same number as the aws-infra repo tag (i.e. 0.9.2 without the `v`).
It must match a [semantic version](https://semver.org/) number.
2. Create a PR to merge the new config into the `main` branch.
3. Once the PR is merged, the cloudformation template will be deployed to the
AWS org-sagebase-imagecentral account which will in turn trigger a build
of the image which will generate an AMI.
4. The AMI will be shared to all accounts in our AWS organization which
means that it is searchable from any of our AWS accounts using either the
AWS console EC2 Instances->AMIs or Image Builder->Images pages. The AMI is
also searchable using the AWS CLI
[describe-images command](https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-images.html).


### Updating Images
To update an image the we must first update aws-infra then this repo.

Update the cloudformation template in aws-infra:
1. Create a PR to update the ImageBuilder cloudformation template in the aws-infra repo
2. Review, approve and Merge the PR
3. Tag the repo with a version number

Update the Sceptre config in this repo:
1. Create a PR to update the template `url` reference and `ImageVersion` number
in the Sceptre config file.
2. Review, approve and Merge the PR
3. Once merged the cloudformation template will be deployed which will trigger
an update to the AMI. AMIs are immutable therefore AWS will create a new AMI
on every change. The updated AMIs will retain the same name, only the version
number is updated.

__Note__: An update to the image definition (or cloudformation template) requires
an `ImageVersion` update otherwise cloudformation may fail with a message similar to
`The following resource XXXXX already exists..`.

### Removing Images
Important info when removing image builder resources and the generated
Expand All @@ -37,14 +68,14 @@ situations where it is appropriate to delete AMIs, for example when testing AMIs
To delete AMIs go into the AWS console EC2 -> Images -> AMIs, search for the AMIs
then delete (or de-register) them.

__NOTE__: Steps above can also be done using the `sceptre delete` command or the AWS CLI.
__NOTE__: Step #2 above can also be done using the `sceptre delete` command or the AWS CLI.

### Testing Images
To test an image we recommend that you manually provision an EC2 instance from the
AMI image then connect to it using the [AWS session manager](https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager.html)
or [SSH](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/connect-linux-inst-ssh.html).
Validate that the instance contains everything that was specified from the image builder and
that it's generally running as expected.
Validate that the image meets the specifications defined in the cloudformation template and
that image is generally running as expected.


## Contributions
Expand Down
9 changes: 5 additions & 4 deletions config/prod/agora-data-manager-github-runner.yaml
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
template:
type: "http"
url: "https://raw.githubusercontent.com/Sage-Bionetworks/aws-infra/v0.9.3/templates/ImageBuilder/amazon-linux-2023-agora-bastian.yaml"
url: "https://raw.githubusercontent.com/Sage-Bionetworks/aws-infra/v0.9.4/templates/ImageBuilder/amazon-linux-2023-agora-bastian.yaml"
stack_name: "agora-data-manager-github-runner"
stack_tags:
OwnerEmail: "it@sagebase.org"
CostCenter: "Platform Infrastructure / 990300"
OwnerEmail: "jessica.britton@sagebase.org"
CostCenter: "Agora / 112300"
parameters:
ImageVersion: "0.0.1"
ImageVersion: "0.9.4"
VolumeSize: "50"

0 comments on commit 276c916

Please sign in to comment.