Bitops conditional helm (#4)

* Added file structure for other charts * changed names * Test * Changed grafana version * values.yaml edit * destroy tf stack * Added env file * env file rename * Added bitOps file * Changed namespace * Removed aws-auth config folder * Added two charts * Test new chart * test with env file * Added bitops files * Added prometheus chart * Added env file * Test with new format of prometheus * Added grafana chart for test * Added env file * Added ENvironment in correct folder * Added operations recruting chart * try with aws auth * Test without charts * Test after destroying * Test after destroying * Added env ƒile * added aws-auth * check with values file * Test with values mapping * Test with values mapping * Test with values mapping * uncomment namespace * Added grafana wrapper * Added fluent chart * Added fluent chart * Added fluent chart * Added fluent chart * Added grafana chart * Destroy cluster * Destroy cluster * New isntall after config map * destroy chart * TEST * Added aws-auth * No replica * Added prometehus chart * Added ENV file * default ns * Added kube prom stack chart * Added env file * Added bitops config file * Added all charts * Added conditions * Added shell property * Wrapped prometheus and grafana * Original charts * Added charts * Added charts * Changing branch * Test nginx * test in default cluster * test nginx * enable nginx * remove nginx * Added loki chart * old insta * old insta * Added bitops in loki * Use bitops main branch * Changing main version * Added nginx chart * nginx name fix * nginx chart test * content in bitops * Testing single nat GW * disable nat * Fixing nat gateway * Fixing inputs * moved aws auth outside * moved aws auth * added aws-auth back * Changed namespace * Changed namespace * Deploy grafana * Adding aws-auth as option * Adding required labels and annotations * Adding missing inputs * Changing commons to main, adding details, fixing README * Update README.md * Cosmetic bump * Adding missing inputs * Small typos in README * Tweaks in readme * Adding warning * Added boolean logic for charts * Fixes in README * Update README.md * Update README.md --------- Co-authored-by: LeoDiazL <[email protected]>
bitovi · Feb 26, 2024 · 4d357bb · 4d357bb
1 parent b178b6a
commit 4d357bb
Show file tree

Hide file tree

Showing 397 changed files with 38,192 additions and 30 deletions.
diff --git a/README.md b/README.md
@@ -1,7 +1,7 @@
 # Deploy Amazon EKS Cluster
 
-GitHub action to deploy an EKS cluster, defining VPC's, Secruity Groups, EC2 Instance templates and everything needed, taking minimum imputs from the user.
-Will generate a cluster of EC2 Instances running Amazon EKS Image, with version 1.27 as default.
+GitHub action to deploy an EKS cluster, defining VPC's, Security Groups, EC2 Instance templates and everything needed, taking minimum imputs from the user.
+Will generate a cluster of EC2 Instances running Amazon EKS Image, with version 1.28 as default.
 
 ## Requirements
 
@@ -28,9 +28,10 @@ jobs:
     - name: Create EKS Cluster
       uses: bitovi/[email protected]
       with:
-        aws_access_key_id: ${{ secrets.AWS_ACCESS_KEY_ID_SANDBOX}}
-        aws_secret_access_key: ${{ secrets.AWS_SECRET_ACCESS_KEY_SANDBOX}}
-        aws_default_region: us-east-1
+        aws_access_key_id: ${{ secrets.AWS_ACCESS_KEY_ID }}
+        aws_secret_access_key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+        aws_eks_cluster_admin_role_arn: arn:aws:iam::123456789012:role/AWSReservedSSO_AdministratorAccess_1234567890123456
+        aws_additional_tags: {"key1": "value1", "key2": "value2"}
 ```
 
 ### Advanced example
@@ -48,16 +49,17 @@ jobs:
     - name: Create EKS Cluster
       uses: bitovi/[email protected]
       with:
-        aws_access_key_id: ${{ secrets.AWS_ACCESS_KEY_ID_SANDBOX}}
-        aws_secret_access_key: ${{ secrets.AWS_SECRET_ACCESS_KEY_SANDBOX}}
+        aws_access_key_id: ${{ secrets.AWS_ACCESS_KEY_ID }}
+        aws_secret_access_key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
         aws_default_region: us-east-1
+        aws_eks_cluster_admin_role_arn: arn:aws:iam::123456789012:role/AWSReservedSSO_AdministratorAccess_1234567890123456
 
         # tf_stack_destroy: true
         tf_state_bucket_destroy: true 
 
         aws_eks_environment: qa
         aws_eks_stackname: qa-stack
-        aws_eks_cluster_version: 1.25
+        aws_eks_cluster_version: 1.29
         aws_eks_instance_type: t2.small
 
         aws_eks_max_size: 5
@@ -76,12 +78,18 @@ jobs:
 1. [Action Defaults](#action-defaults-inputs)
 1. [AWS](#aws-inputs)
 1. [EKS](#eks-inputs)
+1. [Extras](#eks-extras) ⚠️
 1. [VPC](#vpc-inputs)
 
+> ⚠️ Using any kind of **extras** can lead to the creation of load balancers. If doing so, manual intervention to delete them after will be needed. (You'll need to delete a load balancer and the VPC manually, then run the job )
+
+
+### Outputs
+1. [Action Outputs](#action-outputs)
+
 The following inputs can be used as `step.with` keys
 <br/>
 <br/>
-
 #### **Action defaults Inputs**
 | Name             | Type    | Description                        |
 |------------------|---------|------------------------------------|
@@ -112,15 +120,18 @@ The following inputs can be used as `step.with` keys
 | Name             | Type    | Description                        |
 |------------------|---------|------------------------------------|
 | `aws_eks_create` | Boolean | Define if an EKS cluster should be created. Defaults to `true`. |
-f| `aws_eks_security_group_name_master` | String | Define the security group name master. Defaults to `SG for ${var.aws_resource_identifier} - EKS Master`. |
-| `aws_eks_security_group_name_worker` | String | Define the security group name worker. Defaults to `SG for ${var.aws_resource_identifier} - EKS Worker`. |
+| `aws_eks_security_group_name_cluster` | String | Define the security group name master. Defaults to `SG for ${var.aws_resource_identifier} - EKS Cluster`. |
+| `aws_eks_security_group_name_node` | String | Define the security group name worker. Defaults to `SG for ${var.aws_resource_identifier} - EKS Node`. |
 | `aws_eks_environment` | String | Specify the eks environment name. Defaults to `env` |
 | `aws_eks_management_cidr` | String | Comma separated list of remote public CIDRs blocks to add it to Worker nodes security groups. |
 | `aws_eks_allowed_ports` | String | Allow incoming traffic from this port. Accepts comma separated values, matching 1 to 1 with `aws_eks_allowed_ports_cidr`. |
 | `aws_eks_allowed_ports_cidr` | String | Allow incoming traffic from this CIDR block. Accepts comma separated values, matching 1 to 1 with `aws_eks_allowed_ports`. If none defined, will allow all incoming traffic. |
 | `aws_eks_cluster_name` | String | Specify the k8s cluster name. Defaults to `${var.aws_resource_identifier}-cluster` |
-| `aws_eks_cluster_log_types` | String | Comma separated list of cluster log type. See [this AWS doc](https://docs.aws.amazon.com/eks/latest/userguide/control-plane-logs.html). Defaults to `none`. |
-| `aws_eks_cluster_version` | String | Specify the k8s cluster version. Defaults to `1.27` |
+| `aws_eks_cluster_admin_role_arn` | String | Role ARN to grant cluster-admin permissions. | 
+| `aws_eks_cluster_log_types` | String | Comma separated list of cluster log type. See [this AWS doc](https://docs.aws.amazon.com/eks/latest/userguide/control-plane-logs.html). Defaults to `api,audit,authenticator`. |
+| `aws_eks_cluster_log_retention_days` | String | Days to store logs. Defaults to `7`. | 
+| `aws_eks_cluster_logs_skip_destroy` | Boolean | Skip deletion of cluster logs if set to true. Defaults to `false`. |
+| `aws_eks_cluster_version` | String | Specify the k8s cluster version. Defaults to `1.28` |
 | `aws_eks_instance_type` | String | Define the EC2 instance type. See [this list](https://aws.amazon.com/ec2/instance-types/) for reference. Defaults to `t3a.medium`. |
 | `aws_eks_instance_ami_id` | String | AWS AMI ID. Will default to the latest Amazon EKS Node image for the cluster version. |
 | `aws_eks_instance_user_data_file` | String | Relative path in the repo for a user provided script to be executed with the EC2 Instance creation. See note. |
@@ -130,6 +141,16 @@ f| `aws_eks_security_group_name_master` | String | Define the security group nam
 | `aws_eks_max_size` | String | Enter the max_size for the worker nodes. Defaults to `4`. |
 | `aws_eks_min_size` | String | Enter the min_size for the worker nodes. Defaults to `2`. |
 | `aws_eks_additional_tags` | JSON | Add additional tags to the terraform [default tags](https://www.hashicorp.com/blog/default-tags-in-the-terraform-aws-provider), any tags put here will be added to eks provisioned resources.|
+<hr/>
+<br/>
+
+#### **EKS Extras**
+| Name             | Type    | Description                        |
+|------------------|---------|------------------------------------|
+| `prometheus_enable` | Boolean | Set to `true`to enable deployment through chart. |
+| `grafana_enable` | Boolean | Set to `true`to enable deployment through chart. |
+| `loki_enable` | Boolean | Set to `true` to enable deployment through chart. |
+| `nginx_enable` | Boolean | Set to `true` to enable deployment through chart. |
 | `input_helm_charts` | String | Relative path to the folder from project containing Helm charts to be installed. Could be uncompressed or compressed (.tgz) files. |
 <hr/>
 <br/>
@@ -146,12 +167,27 @@ f| `aws_eks_security_group_name_master` | String | Define the security group nam
 | `aws_vpc_id` | String | **Existing** AWS VPC ID to use. Accepts `vpc-###` values. |
 | `aws_vpc_subnet_id` | String | **Existing** AWS VPC Subnet ID. If none provided, will pick one. (Ideal when there's only one). |
 | `aws_vpc_enable_nat_gateway` | String | Adds a NAT gateway for each public subnet. Defaults to `true`. |
-| `aws_vpc_single_nat_gateway` | String | Toggles only one NAT gateway for all of the public subnets. Defaults to `false`. |
+| `aws_vpc_single_nat_gateway` | String | Toggles only one NAT gateway for all of the public subnets. Defaults to `true`. |
 | `aws_vpc_external_nat_ip_ids` | String | **Existing** comma separated list of IP IDs if reusing. (ElasticIPs). |
 | `aws_vpc_additional_tags` | JSON | Add additional tags to the terraform [default tags](https://www.hashicorp.com/blog/default-tags-in-the-terraform-aws-provider), any tags put here will be added to vpc provisioned resources.|
 <hr/>
 <br/>
 
+#### **Action Outputs**
+| Name             | Description                        |
+|------------------|------------------------------------|
+| `aws_vpc_id` | The selected VPC ID used. |
+| `ecs_load_balancer_dns` | ECS ALB DNS Record. |
+| `ecs_dns_record` | ECS DNS URL. |
+| `ecs_sg_id` | ECS SG ID. |
+| `ecs_lb_sg_id` | ECS LB SG ID. |
+<hr/>
+<br/>
+
+
+## Helm charts
+We have **aws-auth**,**ingress**, **grafana**, **prometheus** and **loki** as helm charts which can be called by deployment repo to install in the aws eks cluster. User can pass inputs like `grafana_enable`, `loki_enable`, `nginx_enable` and/or `prometheus_enable` in the deployment repo along with aws access information, and these charts would be installed along with eks creation in aws.
+
 ## Note about resource identifiers
 
 Most resources will contain the tag `${GITHUB_ORG_NAME}-${GITHUB_REPO_NAME}-${GITHUB_BRANCH_NAME}`, some of them, even the resource name after. 
@@ -161,9 +197,15 @@ We use the kubernetes style for this. For example, kubernetes -> k(# of characte
 
 For some specific resources, we have a 32 characters limit. If the identifier length exceeds this number after compression, we remove the middle part and replace it for a hash made up from the string itself. 
 
+## Note about tagging
+
+There's the option to add any kind of defined tags to each grouping module. Will be added to the commons tagging.
+See first example for the correct formatting.
+
 ### S3 buckets naming
 
-Buckets names can be made of up to 63 characters. If the length allows us to add -tf-state, we will do so. If not, a simple -tf will be added.
+Buckets names can be up to 63 characters. If the length allows, -tf-state will be suffixed to the name. Otherwise, only -tf will be added.
+In all cases, the name hashing described above will be used to keep the lengths within limit.
 
 ## EC2 User data
 
@@ -175,8 +217,14 @@ As a default, if not setting any instance ami_id, we will take care of setting u
 [BitOps](https://bitops.sh) allows you to define Infrastructure-as-Code for multiple tools in a central place.  This action uses a BitOps [Operations Repository](https://bitops.sh/operations-repo-structure/) to set up the necessary Terraform and Ansible to create infrastructure and deploy to it.
 
 ## Contributing
-We would love for you to contribute to [bitovi/github-actions-deploy-docker-to-ec2](https://github.com/bitovi/github-actions-deploy-docker-to-ec2).
-Would you like to see additional features?  [Create an issue](https://github.com/bitovi/github-actions-deploy-docker-to-ec2/issues/new) or a [Pull Requests](https://github.com/bitovi/github-actions-deploy-docker-to-ec2/pulls). We love discussing solutions!
+We would love for you to contribute to [bitovi/github-actions-deploy-docker-to-ec2](https://github.com/bitovi/github-actions-deploy-eks).
+Would you like to see additional features?  [Create an issue](https://github.com/bitovi/github-actions-deploy-eks/issues/new) or a [Pull Requests](https://github.com/bitovi/github-actions-deploy-eks/pulls). We love discussing solutions!
 
 ## License
-The scripts and documentation in this project are released under the [MIT License](https://github.com/bitovi/github-actions-deploy-docker-to-ec2/blob/main/LICENSE).
+The scripts and documentation in this project are released under the [MIT License](https://github.com/bitovi/github-actions-deploy-eks/blob/main/LICENSE).
+
+# Provided by Bitovi
+[Bitovi](https://www.bitovi.com/) is a proud supporter of Open Source software.
+
+# We want to hear from you.
+Come chat with us about open source in our Bitovi community [Discord](https://discord.gg/zAHn4JBVcX)!
diff --git a/action.yaml b/action.yaml
@@ -1,4 +1,4 @@
-name: 'Deploy ESK to AWS'
+name: 'Deploy EKS to AWS'
 description: 'Deploy Kubernetes Service (EKS) to AWS'
 branding:
   icon: upload-cloud
@@ -57,10 +57,10 @@ inputs:
     description: 'Define if an EKS cluster should be created'
     required: false
     default: true
-  aws_eks_security_group_name_master:
+  aws_eks_security_group_name_cluster:
     description:  "SG for ${var.aws_resource_identifier} - ${var.aws_eks_environment} - EKS Master"
     required: false
-  aws_eks_security_group_name_worker:
+  aws_eks_security_group_name_node:
     description:  "SG for ${var.aws_resource_identifier} - ${var.aws_eks_environment} - EKS Worker"
     required: false
   aws_eks_environment:
@@ -78,8 +78,17 @@ inputs:
   aws_eks_cluster_name:
     description: "EKS Cluster name. Defaults to eks-cluster"
     required: false
+  aws_eks_cluster_admin_role_arn:
+    description: "Role ARN to grant cluster-admin permissions"
+    required: false
   aws_eks_cluster_log_types:
-    description: "EKS Log types, csv list"
+    description: "EKS Log types, comma separated list. Defaults to api,audit,authenticator"
+    required: false
+  aws_eks_cluster_log_retention_days:
+    description: "Days to store logs. Defaults to 7."
+    required: false
+  aws_eks_cluster_log_skip_destroy:
+    description: "Skip deletion of cluster logs if set to true"
     required: false
   aws_eks_cluster_version:
     description: 'Specify the k8s cluster version'
@@ -112,6 +121,23 @@ inputs:
     description: 'A JSON object of additional tags that will be included on created resources. Example: `{"key1": "value1", "key2": "value2"}`'
     required: false
 
+  # EKS Extras
+  prometheus_enable:
+    description: 'Specifies if this action should checkout the code for this deployment'
+    required: false
+  grafana_enable:
+    description: 'Specifies if this action should checkout the code for this deployment'
+    required: false
+  loki_enable:
+    description: 'Specifies if this action should checkout the code for this deployment'
+    required: false
+  nginx_enable: 
+    description: 'Specifies if this action should checkout the code for this deployment'
+    required: false
+  input_helm_charts: 
+    description: 'Relative path to the folder from project containing Helm charts to be installed. Could be uncompressed or compressed (.tgz) files.'
+    required: false
+
   # AWS VPC Inputs
   aws_vpc_create:
     description: 'Define if a VPC should be created'
@@ -149,25 +175,56 @@ inputs:
   aws_vpc_single_nat_gateway:
     description: 'Creates only one NAT gateway'
     required: false
+    default: true
   aws_vpc_external_nat_ip_ids:
     description: 'Comma separated list of IP IDS to reuse in the NAT gateways'
     required: false
   aws_vpc_additional_tags:
     description: 'A JSON object of additional tags that will be included on created resources. Example: `{"key1": "value1", "key2": "value2"}`'
     required: false
 
-  # Helm input
-  input_helm_charts: 
-    description: 'Relative path to the folder from project containing Helm charts to be installed. Could be uncompressed or compressed (.tgz) files.'
-    required: false
+outputs:
+  # VPC
+  aws_vpc_id:
+    description: "The selected VPC ID used."
+    value: ${{ steps.deploy.outputs.aws_vpc_id }}
+  eks_cluster_name:
+    description: "EKS Cluster name"  
+    value: ${{ steps.deploy.outputs.eks_cluster_name }}
+  eks_cluster_role_arn:
+    description: "EKS Role ARN"
+    value: ${{ steps.deploy.outputs.eks_cluster_role_arn }}
 
 runs:
   using: 'composite'
   steps:
+    - name: If grafana is enabled
+      if: ${{ inputs.grafana_enable == 'true' }}
+      shell: bash
+      run: |
+        mv $GITHUB_ACTION_PATH/helm-charts/grafana $GITHUB_ACTION_PATH/operations/deployment/helm
+
+    - name: If prometheus is enabled
+      if: ${{ inputs.prometheus_enable == 'true' }}
+      shell: bash
+      run: |
+        mv $GITHUB_ACTION_PATH/helm-charts/prometheus $GITHUB_ACTION_PATH/operations/deployment/helm
+
+    - name: If loki is enabled
+      if: ${{ inputs.loki_enable == 'true' }}
+      shell: bash
+      run: |
+        mv $GITHUB_ACTION_PATH/helm-charts/loki $GITHUB_ACTION_PATH/operations/deployment/helm
+
+    - name: If nginx is enabled
+      if: ${{ inputs.nginx_enable == 'true' }}
+      shell: bash
+      run: |
+        mv $GITHUB_ACTION_PATH/helm-charts/ingress-nginx $GITHUB_ACTION_PATH/operations/deployment/helm
 
     - name: Deploy with BitOps
       id: deploy
-      uses: bitovi/github-actions-commons@main
+      uses: bitovi/github-actions-commons@v0.0.13
       with:
         bitops_code_only: ${{ inputs.bitops_code_only }}
         bitops_code_store: ${{ inputs.bitops_code_store }}
@@ -192,14 +249,17 @@ runs:
 
         # EKS
         aws_eks_create: ${{ inputs.aws_eks_create }}
-        aws_eks_security_group_name_master: ${{ inputs.aws_eks_security_group_name_master }}
-        aws_eks_security_group_name_worker: ${{ inputs.aws_eks_security_group_name_worker }}
+        aws_eks_security_group_name_cluster: ${{ inputs.aws_eks_security_group_name_cluster }}
+        aws_eks_security_group_name_node: ${{ inputs.aws_eks_security_group_name_node }}
         aws_eks_environment: ${{ inputs.aws_eks_environment }}
         aws_eks_management_cidr: ${{ inputs.aws_eks_management_cidr }}
         aws_eks_allowed_ports: ${{ inputs.aws_eks_allowed_ports }}
         aws_eks_allowed_ports_cidr: ${{ inputs.aws_eks_allowed_ports_cidr }}
         aws_eks_cluster_name: ${{ inputs.aws_eks_cluster_name }}
+        aws_eks_cluster_admin_role_arn: ${{ inputs.aws_eks_cluster_admin_role_arn }}
         aws_eks_cluster_log_types: ${{ inputs.aws_eks_cluster_log_types}}
+        aws_eks_cluster_log_retention_days: ${{ inputs.aws_eks_cluster_log_retention_days }}
+        aws_eks_cluster_log_skip_destroy: ${{ inputs.aws_eks_cluster_log_skip_destroy }}
         aws_eks_cluster_version: ${{ inputs.aws_eks_cluster_version }}
         aws_eks_instance_type: ${{ inputs.aws_eks_instance_type }}
         aws_eks_instance_ami_id: ${{ inputs.aws_eks_instance_ami_id }}

diff --git a/...tions/deployment/helm/aws-auth/Chart.yaml → helm-charts/aws-auth/Chart.yaml b/...tions/deployment/helm/aws-auth/Chart.yaml → helm-charts/aws-auth/Chart.yaml
diff --git a/...ployment/helm/aws-auth/bitops.config.yaml → helm-charts/aws-auth/bitops.config.yaml b/...ployment/helm/aws-auth/bitops.config.yaml → helm-charts/aws-auth/bitops.config.yaml
@@ -4,7 +4,7 @@ helm:
     timeout: 200s
     debug: true
     atomic: false
-    force: false
+    force: true
     dry-run: false
   options:
     release-name: aws-auth

diff --git a/.../helm/aws-auth/example.bitops.config.yaml → ...harts/aws-auth/example.bitops.config.yaml b/.../helm/aws-auth/example.bitops.config.yaml → ...harts/aws-auth/example.bitops.config.yaml
diff --git a/...ent/helm/aws-auth/templates/aws-auth.yaml → helm-charts/aws-auth/templates/aws-auth.yaml b/...ent/helm/aws-auth/templates/aws-auth.yaml → helm-charts/aws-auth/templates/aws-auth.yaml
@@ -3,6 +3,11 @@ apiVersion: v1
 kind: ConfigMap
 metadata:
   name: {{ .Values.configmap.name }}
+  labels:
+    app.kubernetes.io/managed-by: Helm
+  annotations:
+    meta.helm.sh/release-name: aws-auth
+    meta.helm.sh/release-namespace: kube-system
 data:
 {{- if .Values.data.mapAccounts }}
   mapAccounts: |

diff --git a/...ions/deployment/helm/aws-auth/values.yaml → helm-charts/aws-auth/values.yaml b/...ions/deployment/helm/aws-auth/values.yaml → helm-charts/aws-auth/values.yaml
diff --git a/helm-charts/grafana/.helmignore b/helm-charts/grafana/.helmignore
@@ -0,0 +1,23 @@
+# Patterns to ignore when building packages.
+# This supports shell glob matching, relative path matching, and
+# negation (prefixed with !). Only one pattern per line.
+.DS_Store
+# Common VCS dirs
+.git/
+.gitignore
+.bzr/
+.bzrignore
+.hg/
+.hgignore
+.svn/
+# Common backup files
+*.swp
+*.bak
+*.tmp
+*~
+# Various IDEs
+.vscode
+.project
+.idea/
+*.tmproj
+OWNERS
diff --git a/helm-charts/grafana/Chart.yaml b/helm-charts/grafana/Chart.yaml
@@ -0,0 +1,33 @@
+apiVersion: v2
+name: grafana
+version: 7.0.17
+appVersion: 10.2.2
+kubeVersion: "^1.8.0-0"
+description: The leading tool for querying and visualizing time series and metrics.
+home: https://grafana.com
+icon: https://artifacthub.io/image/b4fed1a7-6c8f-4945-b99d-096efa3e4116
+sources:
+  - https://github.com/grafana/grafana
+  - https://github.com/grafana/helm-charts
+annotations:
+  "artifacthub.io/license": AGPL-3.0-only
+  "artifacthub.io/links": |
+    - name: Chart Source
+      url: https://github.com/grafana/helm-charts
+    - name: Upstream Project
+      url: https://github.com/grafana/grafana
+maintainers:
+  - name: zanhsieh
+    email: [email protected]
+  - name: rtluckie
+    email: [email protected]
+  - name: maorfr
+    email: [email protected]
+  - name: Xtigyro
+    email: [email protected]
+  - name: torstenwalter
+    email: [email protected]
+type: application
+keywords:
+  - monitoring
+  - metric
diff --git a/helm-charts/grafana/ENV_FILE b/helm-charts/grafana/ENV_FILE