Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cyhy-cvesync Lambda #25

Open
wants to merge 47 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
9a71ed2
Specify backend
dav3r Sep 5, 2024
00f90b7
Update variables from skeleton
dav3r Sep 5, 2024
5f6a052
Define default provider
dav3r Sep 5, 2024
6371941
Update versions
dav3r Sep 5, 2024
5d76754
Use correct default backend bucket
dav3r Sep 6, 2024
881e374
Remove an unused Terraform file
dav3r Sep 9, 2024
3eab4a1
Set up providers
dav3r Sep 9, 2024
f4a17b1
Replace default variables with more useful ones
dav3r Sep 13, 2024
ccf1b74
Create VPC and subnets
dav3r Sep 13, 2024
0e72e77
Remove example outputs inherited from skeleton
dav3r Sep 13, 2024
405c161
Define a DocumentDB cluster
dav3r Sep 13, 2024
fcc12d8
Add a required variable for ssh_public_key_path
dav3r Sep 17, 2024
6290d27
Add an EC2 instance
dav3r Sep 17, 2024
765d00a
Add SG rules to allow the EC2 instance to communicate with the Docume…
dav3r Sep 17, 2024
04104cb
Add a locals file and grab the CyHy AWS account ID
dav3r Sep 17, 2024
e2c3ac3
Add variables related to the cyhy-kevsync Lambda
dav3r Sep 17, 2024
22f10c1
Create a Lambda that runs the cyhy-kevsync-lambda code on a schedule
dav3r Sep 17, 2024
90594df
Initial README updates from skeleton template
dav3r Sep 17, 2024
02c6fcb
Mention cisagov/cool-accounts-cyhy in the pre-requisites section
dav3r Sep 17, 2024
4f8086b
Schedule the kevsync Lambda via an EventBridge expression, as opposed…
dav3r Sep 18, 2024
327d5ab
Add the .ssh directory to the .gitignore file
dav3r Sep 18, 2024
7c537c0
Use an EventBridge TF module instead of doing it myself
dav3r Sep 18, 2024
e8e7ea6
Remove examples directory inherited from skeleton
dav3r Sep 20, 2024
3397a6d
Improve comments
dav3r Sep 20, 2024
8fd3f75
Remove the input data to the kevsync Lambda
dav3r Sep 20, 2024
98a0f3d
Add a new optional variable for the retention duration of CloudWatch …
dav3r Sep 20, 2024
b27bb2d
Update README with latest output from terraform-docs
dav3r Sep 24, 2024
bc5db67
Correct a note in the README
dav3r Sep 24, 2024
3df9199
Add variable to control DocumentDB cluster size
dav3r Oct 8, 2024
190c7a9
Put variable names in backticks
dav3r Oct 16, 2024
e8a54bf
Put variable name in backticks
dav3r Oct 16, 2024
1141bf1
Invoke kevsync lamdba to initially load KEV data into the database
dav3r Oct 16, 2024
e5aebf2
Reword a variable description and add a period
dav3r Oct 16, 2024
833a4b0
Include the version of the lambda deployment package S3 object in the…
dav3r Oct 25, 2024
12c29bf
Remove a couple of TODO comments
dav3r Oct 25, 2024
0feaa66
Disable volume tags on the EC2 instance
dav3r Oct 25, 2024
c3e0424
Update README to include all Terraform resources
dav3r Oct 25, 2024
9cfefc6
Add some potentially-useful outputs
dav3r Oct 25, 2024
7539f4f
Add a blank line per our usual style
dav3r Nov 1, 2024
81605e1
Document Terraform outputs in README
dav3r Nov 4, 2024
2ef53d9
Remove unused "Examples" section from README
dav3r Nov 5, 2024
d788698
Replace default variables with more useful ones
dav3r Sep 13, 2024
4bcb6be
Add variables related to cyhy-cvesync-lambda
dav3r Nov 5, 2024
e927ed3
Add the cyhy-cvesync-lambda
dav3r Nov 5, 2024
524985f
Update README to reflect the addition of the CVE sync Lambda
dav3r Nov 5, 2024
ba73936
Mark all input variables as non-nullable
dav3r Nov 6, 2024
61b7c0f
Add an explanatory comment
dav3r Nov 6, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@
# Files already tracked by Git are not affected.
# See: https://git-scm.com/docs/gitignore

# Project Specific #
.ssh/

## Python ##
__pycache__
.mypy_cache
Expand Down
150 changes: 98 additions & 52 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,92 +2,138 @@

[![GitHub Build Status](https://github.com/cisagov/cyhy-tf-root/workflows/build/badge.svg)](https://github.com/cisagov/cyhy-tf-root/actions)

This is a generic skeleton project that can be used to quickly get a
new [cisagov](https://github.com/cisagov) [Terraform
module](https://www.terraform.io/docs/modules/index.html) GitHub
repository started. This skeleton project contains [licensing
information](LICENSE), as well as [pre-commit
hooks](https://pre-commit.com) and
[GitHub Actions](https://github.com/features/actions) configurations
appropriate for the major languages that we use.

See [here](https://www.terraform.io/docs/modules/index.html) for more
details on Terraform modules and the standard module structure.

## Usage ##

```hcl
module "example" {
source = "github.com/cisagov/cyhy-tf-root"

aws_region = "us-west-1"
aws_availability_zone = "b"
subnet_id = "subnet-0123456789abcdef0"
}
```

## Examples ##

- [Basic usage](https://github.com/cisagov/cyhy-tf-root/tree/develop/examples/basic_usage)
This is a [Terraform root module](https://www.terraform.io/docs/modules/index.html)
that can be used to create a Cyber Hygiene (CyHy) environment in AWS.

## Pre-requisites ##

- [Terraform](https://www.terraform.io/) installed on your system.
- An accessible AWS S3 bucket to store Terraform state
(specified in [`backend.tf`](backend.tf)).
- An accessible AWS DynamoDB database to store the Terraform state lock
(specified in [`backend.tf`](backend.tf)).
- To configure a CyHy account within a COOL environment, we strongly recommend
using
[`cisagov/cool-accounts-cyhy`](https://github.com/cisagov/cool-accounts-cyhy).
- A cyhy-cvesync Lambda deployment package stored in an S3 bucket (see the
`cvesync_lambda_s3_bucket` and `cvesync_lambda_s3_key` variables).
- A cyhy-kevsync Lambda deployment package stored in an S3 bucket (see the
`kevsync_lambda_s3_bucket` and `kevsync_lambda_s3_key` variables).
- Valid CyHy configurations stored in the Systems Manager (SSM) Parameter
Store of the Cyber Hygiene account for:
- The cyhy-cvesync Lambda (see the `cvesync_lambda_config_ssm_key` variable)
- The cyhy-kevsync Lambda (see the `kevsync_lambda_config_ssm_key`
variable)
- A Terraform [variables](variables.tf) file customized for your use case, for
example:

```hcl
cvesync_lambda_s3_bucket = "my-lambda-deployment-artifacts"
kevsync_lambda_s3_bucket = "my-lambda-deployment-artifacts"
ssh_public_key_path = "/home/.ssh"

tags = {
Team = "DevSecOps"
Application = "Cyber Hygiene"
}
```

<!-- BEGIN_TF_DOCS -->
## Requirements ##

| Name | Version |
|------|---------|
| terraform | >= 1.1 |
| aws | >= 4.9 |
| terraform | ~> 1.1 |
| aws | ~> 5.0 |

## Providers ##

| Name | Version |
|------|---------|
| aws | >= 4.9 |
| aws.provisionaccount | ~> 5.0 |

## Modules ##

No modules.
| Name | Source | Version |
|------|--------|---------|
| aws\_key\_pair | cloudposse/key-pair/aws | 0.18.3 |
| cvesync\_eventbridge | terraform-aws-modules/eventbridge/aws | 3.11.0 |
| cvesync\_lambda | terraform-aws-modules/lambda/aws | 7.9.0 |
| documentdb-cluster | cloudposse/documentdb-cluster/aws | 0.27.0 |
| ec2 | cloudposse/ec2-instance/aws | 1.6.0 |
| kevsync\_eventbridge | terraform-aws-modules/eventbridge/aws | 3.11.0 |
| kevsync\_lambda | terraform-aws-modules/lambda/aws | 7.9.0 |
| subnets | cloudposse/dynamic-subnets/aws | 2.4.2 |
| vpc | cloudposse/vpc/aws | 2.1.1 |

## Resources ##

| Name | Type |
|------|------|
| [aws_instance.example](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/instance) | resource |
| [aws_ami.example](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ami) | data source |
| [aws_default_tags.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/default_tags) | data source |
| [aws_lambda_invocation.cvesync](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_invocation) | resource |
| [aws_lambda_invocation.kevsync](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_invocation) | resource |
| [aws_security_group_rule.egress_from_ec2_to_documentdb](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource |
| [aws_security_group_rule.ingress_from_ec2_to_documentdb](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource |
| [aws_caller_identity.cyhy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |
| [aws_s3_object.cvesync_lambda](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/s3_object) | data source |
| [aws_s3_object.kevsync_lambda](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/s3_object) | data source |

## Inputs ##

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| ami\_owner\_account\_id | The ID of the AWS account that owns the Example AMI, or "self" if the AMI is owned by the same account as the provisioner. | `string` | `"self"` | no |
| aws\_availability\_zone | The AWS availability zone to deploy into (e.g. a, b, c, etc.). | `string` | `"a"` | no |
| aws\_region | The AWS region to deploy into (e.g. us-east-1). | `string` | `"us-east-1"` | no |
| subnet\_id | The ID of the AWS subnet to deploy into (e.g. subnet-0123456789abcdef0). | `string` | n/a | yes |
| aws\_availability\_zones | The list of AWS availability zones to deploy into (e.g. ["us-east-1a", "us-east-1b", "us-east-1c"]. | `list(string)` | ```[ "us-east-1a", "us-east-1b", "us-east-1c" ]``` | no |
| aws\_region | The AWS region to deploy into (e.g. "us-east-1"). | `string` | `"us-east-1"` | no |
| cvesync\_lambda\_cloudwatch\_logs\_retention\_in\_days | The number of days to retain CloudWatch logs for the Lambda function that syncs CVE data to the database in the Cyber Hygiene account. | `number` | `90` | no |
| cvesync\_lambda\_config\_ssm\_key | The SSM key that contains the configuration to use for the Lambda function that syncs CVE data to the database in the Cyber Hygiene account. | `string` | `"/cyhy-cvesync/config"` | no |
| cvesync\_lambda\_description | The description to associate with the Lambda function that syncs CVE data to the database in the Cyber Hygiene account. | `string` | `"Syncs CVE data to the database in the Cyber Hygiene account."` | no |
| cvesync\_lambda\_env\_variables | The environment variables to set for the Lambda function that syncs CVE data to the database in the Cyber Hygiene account. | `map(string)` | `{}` | no |
| cvesync\_lambda\_handler | The handler to use for the Lambda function that syncs CVE data to the database in the Cyber Hygiene account. | `string` | `"lambda_handler.handler"` | no |
| cvesync\_lambda\_memory | The amount of memory (in MB) to allocate to the Lambda function that syncs CVE data to the database in the Cyber Hygiene account. | `number` | `2048` | no |
| cvesync\_lambda\_name | The name to assign the Lambda function that syncs CVE data to the database in the Cyber Hygiene account. | `string` | `"cyhy-cvesync"` | no |
| cvesync\_lambda\_runtime | The runtime to use for the Lambda function that syncs CVE data to the database in the Cyber Hygiene account. | `string` | `"python3.12"` | no |
| cvesync\_lambda\_s3\_bucket | The name of the S3 bucket where the cyhy-cvesync Lambda deployment package is stored. | `string` | n/a | yes |
| cvesync\_lambda\_s3\_key | The key of the cyhy-cvesync Lambda deployment package in the S3 bucket. | `string` | `"cyhy-cvesync-lambda.zip"` | no |
| cvesync\_lambda\_schedule | The EventBridge expression that represents when to run the Lambda function that syncs CVE data to the database in the Cyber Hygiene account. The default value indicates that the Lambda will run every day at 5:00 AM UTC. See <https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-scheduled-rule-pattern.html> for details on EventBridge expression syntax. | `string` | `"cron(0 5 * * ? *)"` | no |
| cvesync\_lambda\_timeout | The timeout (in seconds) to use for the Lambda function that syncs CVE data to the database in the Cyber Hygiene account. | `number` | `900` | no |
| db\_cluster\_size | The number of instances to use for the DocumentDB cluster. | `number` | `3` | no |
| db\_instance\_class | The instance class to use for the DocumentDB cluster. | `string` | `"db.r5.large"` | no |
| db\_name | The name of the database to create. | `string` | `"cyhy"` | no |
| db\_password | The master password for the database user. | `string` | n/a | yes |
| db\_port | The port to use for the DocumentDB cluster. | `number` | `27017` | no |
| db\_username | The master username for the database user. | `string` | n/a | yes |
| ec2\_trusted\_ingress\_cidr\_blocks | The CIDR blocks to allow access to the EC2 instance. | `list(string)` | `[]` | no |
| kevsync\_lambda\_cloudwatch\_logs\_retention\_in\_days | The number of days to retain CloudWatch logs for the Lambda function that syncs KEV data to the database in the Cyber Hygiene account. | `number` | `90` | no |
| kevsync\_lambda\_config\_ssm\_key | The SSM key that contains the configuration to use for the Lambda function that syncs KEV data to the database in the Cyber Hygiene account. | `string` | `"/cyhy-kevsync/config"` | no |
| kevsync\_lambda\_description | The description to associate with the Lambda function that syncs KEV data to the database in the Cyber Hygiene account. | `string` | `"Syncs KEV data to the database in the Cyber Hygiene account."` | no |
| kevsync\_lambda\_env\_variables | The environment variables to set for the Lambda function that syncs KEV data to the database in the Cyber Hygiene account. | `map(string)` | `{}` | no |
| kevsync\_lambda\_handler | The handler to use for the Lambda function that syncs KEV data to the database in the Cyber Hygiene account. | `string` | `"lambda_handler.handler"` | no |
| kevsync\_lambda\_name | The name to assign the Lambda function that syncs KEV data to the database in the Cyber Hygiene account. | `string` | `"cyhy-kevsync"` | no |
| kevsync\_lambda\_runtime | The runtime to use for the Lambda function that syncs KEV data to the database in the Cyber Hygiene account. | `string` | `"python3.12"` | no |
| kevsync\_lambda\_s3\_bucket | The name of the S3 bucket where the cyhy-kevsync Lambda deployment package is stored. | `string` | n/a | yes |
| kevsync\_lambda\_s3\_key | The key of the cyhy-kevsync Lambda deployment package in the S3 bucket. | `string` | `"cyhy-kevsync-lambda.zip"` | no |
| kevsync\_lambda\_schedule | The EventBridge expression that represents when to run the Lambda function that syncs KEV data to the database in the Cyber Hygiene account. The default value indicates that the Lambda will run every day at 6:00 AM UTC. See <https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-scheduled-rule-pattern.html> for details on EventBridge expression syntax. | `string` | `"cron(0 6 * * ? *)"` | no |
| kevsync\_lambda\_timeout | The timeout (in seconds) to use for the Lambda function that syncs KEV data to the database in the Cyber Hygiene account. | `number` | `300` | no |
| ssh\_public\_key\_path | The local path to store the SSH public key used to access the EC2 instance. | `string` | n/a | yes |
| tags | Tags to apply to all AWS resources created. | `map(string)` | `{}` | no |
| vpc\_cidr\_block | The CIDR block to use for the VPC (e.g. "10.0.0.0/16"). | `string` | `"10.0.0.0/16"` | no |

## Outputs ##

| Name | Description |
|------|-------------|
| arn | The EC2 instance ARN. |
| availability\_zone | The AZ where the EC2 instance is deployed. |
| id | The EC2 instance ID. |
| private\_ip | The private IP of the EC2 instance. |
| subnet\_id | The ID of the subnet where the EC2 instance is deployed. |
| documentdb\_arn | The ARN of the DocumentDB cluster. |
| documentdb\_endpoint | The endpoint of the DocumentDB cluster. |
| documentdb\_sg\_arn | The ARN of the DocumentDB cluster security group. |
| ec2 | The EC2 instance that is allowed to access the DocumentDB cluster. |
| subnets | The subnets within the CyHy VPC. |
| vpc | The CyHy VPC. |
<!-- END_TF_DOCS -->

## Notes ##

Running `pre-commit` requires running `terraform init` in every directory that
contains Terraform code. In this repository, these are the main directory and
every directory under `examples/`.

## New Repositories from a Skeleton ##

Please see our [Project Setup guide](https://github.com/cisagov/development-guide/tree/develop/project_setup)
for step-by-step instructions on how to start a new repository from
a skeleton. This will save you time and effort when configuring a
new repository!
contains Terraform code. In this repository, this is only the main directory.

## Contributing ##

Expand Down
10 changes: 10 additions & 0 deletions backend.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
terraform {
backend "s3" {
bucket = "cisa-cool-terraform-state"
dynamodb_table = "terraform-state-lock"
encrypt = true
key = "cyhy-tf-root/terraform.tfstate"
profile = "cool-terraform-backend"
region = "us-east-1"
}
}
109 changes: 109 additions & 0 deletions cyhy-cvesync-lambda.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# Create a Lambda function that runs the cyhy-cvesync-lambda code on a schedule.
#
# Prerequisites:
# - A cyhy-cvesync Lambda deployment package stored in an S3 bucket (see the
# cvesync_lambda_s3_bucket and cvesync_lambda_s3_key variables)
# - A valid CyHy configuration stored in the Systems Manager (SSM) Parameter
# Store of the Cyber Hygiene account (see the cvesync_lambda_config_ssm_key
# variable)

# Fetch the Lambda deployment package from the S3 bucket where it is stored
# so that we can check its version ID and update the Lambda function when a new
# version is uploaded.
data "aws_s3_object" "cvesync_lambda" {
provider = aws.provisionaccount

bucket = var.cvesync_lambda_s3_bucket
key = var.cvesync_lambda_s3_key
}

module "cvesync_lambda" {
providers = {
aws = aws.provisionaccount
}

source = "terraform-aws-modules/lambda/aws"
version = "7.9.0"

allowed_triggers = {
cvesync = {
principal = "events.amazonaws.com"
source_arn = module.cvesync_eventbridge.eventbridge_rule_arns["${var.cvesync_lambda_name}"]
}
}
attach_network_policy = true
attach_policy_statements = true
cloudwatch_logs_retention_in_days = var.cvesync_lambda_cloudwatch_logs_retention_in_days

# This is necessary to avoid the following error:
# "InvalidParameterValueException: We currently do not support adding policies
# for $LATEST." For more, see
# https://github.com/terraform-aws-modules/terraform-aws-lambda/blob/v7.9.0/README.md#faq
create_current_version_allowed_triggers = false

create_package = false
description = var.cvesync_lambda_description
environment_variables = merge({ "CYHY_CONFIG_SSM_PATH" = var.cvesync_lambda_config_ssm_key }, var.cvesync_lambda_env_variables)
function_name = var.cvesync_lambda_name
handler = var.cvesync_lambda_handler
memory_size = var.cvesync_lambda_memory
policy_statements = {
ssm_read = {
effect = "Allow",
actions = ["ssm:GetParameter"],
resources = ["arn:aws:ssm:${var.aws_region}:${local.cyhy_account_id}:parameter${var.cvesync_lambda_config_ssm_key}"]
},
}
runtime = var.cvesync_lambda_runtime
s3_existing_package = {
bucket = var.cvesync_lambda_s3_bucket
key = var.cvesync_lambda_s3_key
version_id = data.aws_s3_object.cvesync_lambda.version_id
}
tags = var.tags
timeout = var.cvesync_lambda_timeout
vpc_security_group_ids = [module.ec2.security_group_id]
vpc_subnet_ids = module.subnets.private_subnet_ids
}

# Invoke the Lamdba function to initially load CVE data into the database
resource "aws_lambda_invocation" "cvesync" {
provider = aws.provisionaccount

function_name = module.cvesync_lambda.lambda_function_name
input = "{}"
}

# Schedule the Lambda function
module "cvesync_eventbridge" {
providers = {
aws = aws.provisionaccount
}

source = "terraform-aws-modules/eventbridge/aws"
version = "3.11.0"

# We are using the default bus, so no need to create it here.
create_bus = false
# The role allowing the Lambda to be triggered by this EventBridge rule is
# created by the Lambda module, so no need to create it here.
create_role = false

rules = {
"${var.cvesync_lambda_name}" = {
description = format("Executes %s Lambda on a schedule", var.cvesync_lambda_name)
schedule_expression = var.cvesync_lambda_schedule
}
}

tags = var.tags

targets = {
"${var.cvesync_lambda_name}" = [
{
arn = module.cvesync_lambda.lambda_function_arn
name = var.cvesync_lambda_name
}
]
}
}
Loading