Queen's Awards for Enterprise

"QAE" is the application which powers the application process for the Queen's Awards for Enterprise.

Setup QAE Servers AWS Amazon Infrastructure Guide

This guide uses Terraform.

• First of all you need to setup necessary tools on local (Terraform, AWS CLI so on). Follow instructions in SETUP GUIDE below.

Setup Guide

STEP 1: Setup Terraform

STEP 2: Setup AWS CLI environment

Install packages:

$ sudo apt-get install awscli jq

• More Information

Setup AWS credentials:

$ aws configure
=>
AWS Access Key ID [None]: <AWS_ACCESS_ID>
AWS Secret Access Key [None]: <AWS_SECRET_ACCESS_KEY>
Default region name [None]: eu-west-1
Default output format [None]:

• Ask for Guys about AWS_ACCESS_ID and AWS_SECRET_ACCESS_KEY

STEP 3: Setup QAE Terraform app

$ terraform init [email protected]:bitzesty/qae-terraform.git

Provision AWS infrastructure from scratch

• Need to setup local env before you start SETUP GUIDE
• All operations with Terraform should be executed with providing AWS key pair, so that we need to generate AWS key pair at first.

STEP 1: Generate new AWS key pair

Generate AWS key pair via awscli (This command will generate and upload new key pair to AWS)

$ aws ec2 --region eu-west-1 create-key-pair --key-name qae_<ENVIRONMENT> | jq -r ".KeyMaterial" > ssh_keys/qae_<ENVIRONMENT>.pem

Then need to set proper permissions to generated .pem key:

$ chmod 400 ssh_keys/qae_<ENVIRONMENT>.pem

• Generated pem key would be saved to ssh_keys/qae_ENVIRONMENT.pem.

STEP 2: Go to target environment folder

$ cd staging
# OR
$ cd production

• All script actions are declarated on related ENVIRONMENT/main.tf file.

STEP 3: Setup variables

Setup variables.tf file:

List of possible variables and it's default values declarated in variables.tf. For example:

variable "aws_region" {
  description = "AWS region to launch servers."
  default = "eu-west-1" # Ireland is default
}

This example sets default region ("eu-west-1") and adds description for this variable. Default value of this variable can be overriden in terraform.tfvars file.

Most important variables:

aws_ami - setting AWS AMI ubuntu image id for QAE app EC-2 instances

# For example:
variable "aws_ami" {
  default = "ami-bb3a58cc"
}

ec2_instance_type - setting type of EC-2 instance for QAE app

# For example:
variable "ec2_instance_type" {
  default = "m3.large"
}

load_balancer_ssl_cert_id - setting SSL Certificate of Load Balancer for QAE app

# For example:
variable "load_balancer_ssl_cert_id" {}

* has no default value, so you need to specify it at terraform.tfvars file

virus_scanner_aws_ami - setting AWS AMI ubuntu image id for Virus Scanner Engine EC-2 instances

# For example:
variable "virus_scanner_aws_ami" {
  default = "ami-33254844"
}

virus_scanner_instance_type - setting type of EC-2 instance for Virus Scanner Engine

# For example:
variable "virus_scanner_instance_type" {
  default = "m1.small"
}

* It's worth to double check all used variables in order to make sure that you are going to provision right AWS infrastructure

Setup terraform.tfvars file

All private variables we store in terraform.tfvars file, which is in .gitignore You can use terraform.tfvars.example. It looks like this:

access_key = "<AWS_ACCESS_KEY>"
secret_key = "<AWS_SECRET_KEY>"
aws_region = "eu-west-1"
postgres_password = ""
load_balancer_ssl_cert_id = ""

• This file overrides default values in variables.tf

STEP 4: Make a Terraform Plan

• NOTE: This command will show you all planned actions. This command don't run provision scripts on your AWS infrastructure, it just displaying all planned actions. It's worth to review output of this command before you will run 'terraform apply'.

$ terraform plan -var 'key_name=qae_<ENVIRONMENT>' -var 'key_path=/<ABSOLUTE PATH TO ROOT OF THIS FOLDER>/ssh_keys/qae_<ENVIRONMENT>.pem'

Staging:

$ terraform plan -var 'key_name=qae_staging' -var 'key_path=./../ssh_keys/qae_staging.pem'

Production:

$ terraform plan -var 'key_name=qae_production_release' -var 'key_path=./../ssh_keys/qae_production_release.pem'

STEP 5: Provision AWS Infrastructure

• NOTE: This command run provision scripts. It's worth to review output of this command before you will run 'terraform apply'.

$ terraform apply -var 'key_name=qae_<ENVIRONMENT>' -var 'key_path=/<ABSOLUTE PATH TO ROOT OF THIS FOLDER>/ssh_keys/qae_<ENVIRONMENT>.pem'

Staging:

$ terraform apply -var 'key_name=qae_staging' -var 'key_path=./../ssh_keys/qae_staging.pem'

Production:

$ terraform apply -var 'key_name=qae_production_release' -var 'key_path=./../ssh_keys/qae_production_release.pem'

• Sometimes 'terraform apply' can take more time (for example on provision of RDS instance) and timing out - in this case best way is to wait a bit and run 'terraform apply' again

NOTES

* Terraform did a few things here:

• Add bunch of security groups
• RDS Postgresql instance
• Private S3 bucket
• Load Balancer (AWS LB), Launch Configuration and Auto-Scaling Group (AWS ASG) with 2 EC-2 instances from clean from the Ubuntu 14.10 AMI for QAE app
• Load Balancer (AWS LB), Launch Configuration and Auto-Scaling Group (AWS ASG) with 1 EC-2 instances from clean from the Ubuntu 14.10 AMI for Virus Scanner Engine

* Terraform saves the state of your infrastructure in a terraform.tfstate and terraform.tfstate.backup files (They are in .gitignore).

* It's always required to have latest version of terraform.tfstate and terraform.tfstate.backup files in folder (staging/ or production/) if you run provisioning of existing AWS infrastructure (not from scratch).

STEP 6: Review Infrastructure

$ terraform show -var 'key_name=qae_<ENVIRONMENT>' -var 'key_path=/<ABSOLUTE PATH TO ROOT OF THIS FOLDER>/ssh_keys/qae_<ENVIRONMENT>.pem'

• This command will display output with your AWS infrastructure, based on terraform.tfstate and terraform.tfstate.backup files.

If you want to refresh information about your Infrastructure, use:

$ terraform refresh -var 'key_name=qae_<ENVIRONMENT>' -var 'key_path=/<ABSOLUTE PATH TO ROOT OF THIS FOLDER>/ssh_keys/qae_<ENVIRONMENT>.pem'

STEP 7: Adding of other AWS Services

• Current version of Terraform doesn't allow to setup AWS SQS. Probably, it would be added in future. So, we need to add it manually.

Setup AWS SQS (Message Queue)

######* We user AWS SQS as a Message Queue for background jobs and delayed mailers More Information

1) Visit https://eu-west-1.console.aws.amazon.com/sqs/home?region=eu-west-1

2) Add queues

Add 2 queues per ENV:

staging_mailers
staging_default
production_mailers
production_default

Provision of existing AWS infrastructure

Need to setup local env before you start SETUP GUIDE

IMPORTANT NOTES:

* All private variables are located in terraform.tfvars file (which is in .gitignore).

* Terraform saves the state provisioned AWS infrastructure in a terraform.tfstate and terraform.tfstate.backup files (They are in .gitignore).

* It's always required to have latest version of terraform.tfvars, terraform.tfstate and terraform.tfstate.backup files in folder (staging/ or production/) if you run provisioning of existing AWS infrastructure (not from scratch).

* If latest Terraform provision of AWS infrastructure was runned by another person (not by you) - You should ask him to provide you following files and put them in related folder (staging/ or production/):

• terraform.tfstate
• terraform.tfstate.backup
• terraform.tfvars

* Double check all variables SETUP VARIABLES GUIDE

STEP 1: Make changes in terraform scripts (If needed)

• It's required to have latest terraform.tfstate and terraform.tfstate.backup before you go ahead!

All script actions are declarated on related ENVIRONMENT/main.tf file.

STEP 2: Update variables (If needed)

See SETUP VARIABLES GUIDE

STEP 3: Refresh information about AWS insfrastructure

$ terraform refresh -var 'key_name=qae_<ENVIRONMENT>' -var 'key_path=/<ABSOLUTE PATH TO ROOT OF THIS FOLDER>/ssh_keys/qae_<ENVIRONMENT>.pem'

STEP 4: Make a Terraform Plan

• NOTE: This command will show you all planned actions. This command don't run provision scripts on your AWS infrastructure, it just displaying all planned actions. It's worth to review output of this command before you will run 'terraform apply'.

$ terraform plan -var 'key_name=qae_<ENVIRONMENT>' -var 'key_path=/<ABSOLUTE PATH TO ROOT OF THIS FOLDER>/ssh_keys/qae_<ENVIRONMENT>.pem'

Staging:

$ terraform plan -var 'key_name=qae_staging' -var 'key_path=./../ssh_keys/qae_staging.pem'

Production:

$ terraform plan -var 'key_name=qae_production_release' -var 'key_path=./../ssh_keys/qae_production_release.pem'

STEP 5: Provision AWS Infrastructure

• NOTE: This command run provision scripts. It's worth to review output of this command before you will run 'terraform apply'.

$ terraform apply -var 'key_name=qae_<ENVIRONMENT>' -var 'key_path=/<ABSOLUTE PATH TO ROOT OF THIS FOLDER>/ssh_keys/qae_<ENVIRONMENT>.pem'

Staging:

$ terraform apply -var 'key_name=qae_staging' -var 'key_path=./../ssh_keys/qae_staging.pem'

Production:

$ terraform apply -var 'key_name=qae_production_release' -var 'key_path=./../ssh_keys/qae_production_release.pem'

• Sometimes 'terraform apply' can take more time (for example on provision of RDS instance) and timing out - in this case best way is to wait a bit and run 'terraform apply' again

Prepare new AWS AMI image from existing AWS EC-2 instance

STEP 1: Create new AWS AMI on aws.amazon.com, from existing EC-2 instance

AWS GUIDE

STEP 2: Update Terraform variables with new AWS AMI id

You can updated AWS AMI id for QAE app in ENVIRONMENT/variables.tf

# For example:
variable "aws_ami" {
  default = "ami-bb3a58cc"
}

STEP 3: Refresh information about AWS insfrastructure

STEP 4: Make a Terraform Plan

STEP 5: Provision AWS Infrastructure