Milestone 3

Zilean WeatherApp is hosted at http://149.165.153.6:30000/

Deliverables

• Incorporate plotting the MERRA-2 dataset
• Build CI-CD Pipelines for all the microservices.
• Incorporate message queue based communication between the services
• Solving the Thundering herd problem

Updated Architecture

New endpoints for plotting MERRA data can be found here here

Jetstream Deployment

We have configured one jetstream instance as a Master node for deploying our Kubernetes cluster.

After creating a master, we followed the below steps to deploy our Kubernetes cluster

1. Install Kubernetes on the machine - We followed this article to install Kubernetes on the machine.

2. Get our YAML file onto the machine

git clone https://github.com/airavata-courses/Zilean.git

3. Deploy cluster

cd Zilean && kubectl apply -f dp

After these steps, our Kubernetes cluster was deployed successfully. PFB screenshots for reference.

1) Incorporate plotting the MERRA-2 dataset

As you can see from the updated architecture, we have added MERRA-2 plotting functionality. For endpoints refer here

2) Build CI-CD Pipelines for all the microservices.

We have used CircleCI for creating CI-CD pipelines for each microservice.

How does it work??

• After modifying code for a particular microservice, push the code to the main branch for the microservice or submit an pull request to the main branch for the microservice.
• CircleCI will run the test that we have written for that particular microservice
• If the tests run successfully, CircleCI will build a docker image and push it to docker hub
• CirclCI will ssh into the Jetstream server where our Kubernetes cluster is deployed.
• CircleCI will delete and recreate the deployment for that particular service. While recreating the deployment, Kubernetes will pull the new image from the docker hub

3) Incorporate message queue based communication between the services

We have used Kafka for asynchronous communication between the services. We refereed this article to run and deploy Kafka inside our Kubernetes cluster

4) Solving the Thundering herd problem

We have used Rate Limiters in API's by taking the limits from the project 2 load testing.