Pattern 4: Helm Chart for Distributed API-M Deployment with Traffic Manager Separated from the Control Plane
Resources for building a Helm chart for deployment of a simple scalable deployment of WSO2 API Manager.
For advanced details on the deployment pattern, please refer to the official documentation.
- Prerequisites
- Quick Start Guide
- Configuration
- Runtime Artifact Persistence and Sharing
- Managing Java Keystores and Truststores
- Configuring SSL in Service Exposure
-
WSO2 product Docker images used for the Kubernetes deployment.
WSO2 product Docker images available at DockerHub package General Availability (GA) versions of WSO2 products with no WSO2 Updates.
For a production grade deployment of the desired WSO2 product-version, it is highly recommended to use the relevant Docker image which packages WSO2 Updates, available at WSO2 Private Docker Registry. In order to use these images, you need an active WSO2 Subscription.
-
Install Git, Helm and Kubernetes client in order to run the steps provided in the following quick start guide.
-
An already setup Kubernetes cluster.
-
Install NGINX Ingress Controller.
-
Add the WSO2 Helm chart repository.
helm repo add wso2 https://helm.wso2.com && helm repo update
You can install the relevant Helm chart either from WSO2 Helm Chart Repository or by source.
Note:
NAMESPACEshould be the Kubernetes Namespace in which the resources are deployed.
Install Chart From WSO2 Helm Chart Repository
Helm version 2
helm install --name <RELEASE_NAME> wso2/am-pattern-4 --version 4.1.0-1 --namespace <NAMESPACE>
Helm version 3
-
Deploy the Kubernetes resources using the Helm Chart
helm install <RELEASE_NAME> wso2/am-pattern-4 --version 4.1.0-1 --namespace <NAMESPACE> --create-namespace
The above steps will deploy the deployment pattern using WSO2 product Docker images available at DockerHub.
If you are using WSO2 product Docker images available from WSO2 Private Docker Registry,
please provide your WSO2 Subscription credentials via input values (using --set argument).
Please see the following example.
helm install --name <RELEASE_NAME> wso2/am-pattern-4 --version 4.1.0-1 --namespace <NAMESPACE> --set wso2.subscription.username=<SUBSCRIPTION_USERNAME> --set wso2.subscription.password=<SUBSCRIPTION_PASSWORD>
In the context of this document,
KUBERNETES_HOMEwill refer to a local copy of thewso2/kubernetes-apimGit repository.HELM_HOMEwill refer to<KUBERNETES_HOME>/advanced.
git clone https://github.com/wso2/kubernetes-apim.git
Helm version 2
helm install --dep-up --name <RELEASE_NAME> <HELM_HOME>/am-pattern-4 --namespace <NAMESPACE>
Helm version 3
-
Deploy the Kubernetes resources using the Helm Chart
helm install <RELEASE_NAME> <HELM_HOME>/am-pattern-4 --namespace <NAMESPACE> --dependency-update --create-namespace
The above steps will deploy the deployment pattern using WSO2 product Docker images available at DockerHub.
If you are using WSO2 product Docker images available from WSO2 Private Docker Registry,
please provide your WSO2 Subscription credentials via input values (using --set argument).
Please see the following example.
helm install --name <RELEASE_NAME> <HELM_HOME>/am-pattern-4 --namespace <NAMESPACE> --set wso2.subscription.username=<SUBSCRIPTION_USERNAME> --set wso2.subscription.password=<SUBSCRIPTION_PASSWORD>
Or else, you can configure the default configurations inside the am-pattern-4 helm chart values.yaml file. Refer this for to learn more details about the values.yaml file.
Note:
From the above Helm commands, base image of a Micro Integrator is deployed (without any integration solution). To deploy your integration solution with the Helm charts follow the below steps.
- Create an integration service using WSO2 Integration Studio and expose it as a Managed API. Then create a Docker image and push it to your private or public Docker registry.
-INTEGRATION_IMAGE_REGISTRYwill refer to the Docker registry that created Docker image has been pushed
-INTEGRATION_IMAGE_NAMEwill refer to the name of the created Docker image
-INTEGRATION_IMAGE_TAGwill refer to the tag of the created Docker image- If your Docker registry is a private registry, create an imagePullSecret.
-IMAGE_PULL_SECRETwill refer to the created image pull secret- Deploy the helm resource using following command.
helm install <RELEASE_NAME> wso2/am-pattern-4 --version 4.1.0-1 --namespace <NAMESPACE> --set wso2.deployment.mi.dockerRegistry=<INTEGRATION_IMAGE_REGISTRY> --set wso2.deployment.mi.imageName=<INTEGRATION_IMAGE_NAME> --set wso2.deployment.mi.imageTag=<INTEGRATION_IMAGE_TAG> --set wso2.deployment.mi.imagePullSecrets=<IMAGE_PULL_SECRET>
If you need to enable Choreo Analytics with WSO2 API Manager, please follow the documentation on Register for Analytics to obtain the on-prem key for Analytics.
The following example shows how to enable Analytics with the helm charts.
Helm v2
helm install --name <RELEASE_NAME> wso2/am-pattern-4 --version 4.1.0-1 --namespace <NAMESPACE> --set wso2.choreoAnalytics.enabled=true --set wso2.choreoAnalytics.endpoint=<CHOREO_ANALYTICS_ENDPOINT> --set wso2.choreoAnalytics.onpremKey=<ONPREM_KEY>
Helm v3
helm install <RELEASE_NAME> wso2/am-pattern-4 --version 4.1.0-1 --namespace <NAMESPACE> --set wso2.choreoAnalytics.enabled=true --set wso2.choreoAnalytics.endpoint=<CHOREO_ANALYTICS_ENDPOINT> --set wso2.choreoAnalytics.onpremKey=<ONPREM_KEY> --create-namespace
You will be able to see the Analytics data when you log into Choreo Analytics Portal.
Obtain the external IP (EXTERNAL-IP) of the API Manager Ingress resources, by listing down the Kubernetes Ingresses.
kubectl get ing -n <NAMESPACE>
The output under the relevant column stands for the following.
API Manager Control Plane
- NAME: Metadata name of the Kubernetes Ingress resource (defaults to
wso2am-pattern-4-am-cp-ingress) - HOSTS: Hostname of the WSO2 API Manager's Control Plane service (
<wso2.deployment.am.cp.ingress.hostname>) - ADDRESS: External IP (
EXTERNAL-IP) exposing the API Manager's Control Plane service to outside of the Kubernetes environment - PORTS: Externally exposed service ports of the API Manager's Control Plane service
API Manager Gateway
- NAME: Metadata name of the Kubernetes Ingress resource (defaults to
wso2am-pattern-4-am-gateway-ingress) - HOSTS: Hostname of the WSO2 API Manager's Gateway service (
<wso2.deployment.am.gateway.ingress.hostname>) - ADDRESS: External IP (
EXTERNAL-IP) exposing the API Manager's Gateway service to outside of the Kubernetes environment - PORTS: Externally exposed service ports of the API Manager's Gateway service
API Manager Websub
- NAME: Metadata name of the Kubernetes Ingress resource (defaults to wso2am-pattern-4-am-websub-ingress)
- HOSTS: Hostname of the WSO2 API Manager's Websub service (
<wso2.deployment.am.websub.ingress.hostname>) - ADDRESS: External IP (EXTERNAL-IP) exposing the API Manager's Websub service to outside of the Kubernetes environment
- PORTS: Externally exposed service ports of the API Manager's Websub service
Micro Integrator Management APIs
- NAME: Metadata name of the Kubernetes Ingress resource (defaults to wso2am-pattern-4-mi-1-management-ingress)
- HOSTS: Hostname of the WSO2 Micro Integrator service (
<wso2.deployment.mi.ingress.management.hostname>) - ADDRESS: External IP (EXTERNAL-IP) exposing the Micro Integrator service to outside of the Kubernetes environment PORTS: Externally exposed service ports of the Micro Integrator service
If the defined hostnames (in the previous step) are backed by a DNS service, add a DNS record mapping the hostnames and
the external IP (EXTERNAL-IP) in the relevant DNS service.
If the defined hostnames are not backed by a DNS service, for the purpose of evaluation you may add an entry mapping the
hostnames and the external IP in the /etc/hosts file at the client-side.
-
API Manager Publisher:
https://<wso2.deployment.am.cp.ingress.hostname>/publisher -
API Manager DevPortal:
https://<wso2.deployment.am.cp.ingress.hostname>/devportal
The following tables lists the configurable parameters of the chart and their default values.
| Parameter | Description | Default Value |
|---|---|---|
wso2.subscription.username |
Your WSO2 Subscription username | - |
wso2.subscription.password |
Your WSO2 Subscription password | - |
wso2.choreoAnalytics.enabled |
Chorero Analytics enabled or not | false |
wso2.choreoAnalytics.endpoint |
Choreo Analytics endpoint | https://analytics-event-auth.choreo.dev/auth/v1 |
wso2.choreoAnalytics.onpremKey |
On-prem key for Choreo Analytics | - |
If you do not have an active WSO2 subscription, do not change the parameters wso2.subscription.username and wso2.subscription.password.
| Parameter | Description | Default Value |
|---|---|---|
wso2.deployment.dependencies.mysql |
Enable the deployment and usage of WSO2 API Management MySQL based Helm Chart | true |
wso2.deployment.dependencies.nfsProvisioner |
Enable the deployment and usage of NFS Server Provisioner (https://github.com/helm/charts/tree/master/stable/nfs-server-provisioner) | true |
| Parameter | Description | Default Value |
|---|---|---|
wso2.deployment.persistentRuntimeArtifacts.storageClass |
Appropriate Kubernetes Storage Class | nfs |
wso2.deployment.persistentRuntimeArtifacts.sharedArtifacts.capacity.executionPlans |
Capacity for execution plans shared between the Traffic Manager profile instances | 20M |
wso2.deployment.persistentRuntimeArtifacts.sharedArtifacts.capacity.synapseConfigs |
Capacity for synapse artifacts of APIs shared between the Gateway profile instances | 50M |
wso2.deployment.persistentRuntimeArtifacts.apacheSolrIndexing.enabled |
Indicates if persistence of the runtime artifacts for Apache Solr-based indexing is enabled | false |
wso2.deployment.persistentRuntimeArtifacts.apacheSolrIndexing.capacity.carbonDatabase |
Capacity for persisting the H2 based local Carbon database file | 50M |
wso2.deployment.persistentRuntimeArtifacts.apacheSolrIndexing.capacity.solrIndexedData |
Capacity for persisting the Apache Solr indexed data | 50M |
| Parameter | Description | Default Value |
|---|---|---|
wso2.deployment.am.dockerRegistry |
Registry location of the Docker image to be used to create API Manager instances | - |
wso2.deployment.am.imageName |
Name of the Docker image to be used to create API Manager instances | wso2am |
wso2.deployment.am.imageTag |
Tag of the image used to create API Manager instances | 4.1.0 |
wso2.deployment.am.imagePullPolicy |
Refer to doc | Always |
wso2.deployment.am.resources.requests.memory |
The minimum amount of memory that should be allocated for running API Manager product profiles with profile optimization | 1Gi |
wso2.deployment.am.resources.requests.cpu |
The minimum amount of CPU that should be allocated for running API Manager product profiles with profile optimization | 1000m |
wso2.deployment.am.resources.limits.memory |
The maximum amount of memory that should be allocated for running API Manager product profiles with profile optimization | 2Gi |
wso2.deployment.am.resources.limits.cpu |
The maximum amount of CPU that should be allocated for running API Manager product profiles with profile optimization | 2000m |
wso2.deployment.am.livenessProbe.initialDelaySeconds |
Initial delay for the live-ness probe for API Manager optimized profile | 60 |
wso2.deployment.am.livenessProbe.periodSeconds |
Period of the live-ness probe for API Manager optimized profile | 10 |
wso2.deployment.am.readinessProbe.initialDelaySeconds |
Initial delay for the readiness probe for API Manager optimized profile | 60 |
wso2.deployment.am.readinessProbe.periodSeconds |
Period of the readiness probe for API Manager optimized profile | 10 |
wso2.deployment.am.websub.ingress.enabled |
If enabled, create ingress resource for WebSub service | true |
wso2.deployment.am.websub.ingress.hostname |
Hostname for API Manager WebSub service | websub.am.wso2.com |
wso2.deployment.am.websub.ingress.annotations |
Ingress resource annotations for API Manager WebSub | Community NGINX Ingress controller annotations |
wso2.deployment.am.gateway.ingress.enabled |
If enabled, create ingress resource for API Manager Gateway | true |
wso2.deployment.am.gateway.ingress.hostname |
Hostname for API Manager Gateway | gateway.am.wso2.com |
wso2.deployment.am.gateway.ingress.annotations |
Ingress resource annotations for API Manager Gateway | Community NGINX Ingress controller annotations |
wso2.deployment.am.gateway.replicas |
Number of replicas of API Manager Gateway to be started | 2 |
wso2.deployment.am.gateway.strategy.rollingUpdate.maxSurge |
Refer to doc | 2 |
wso2.deployment.am.gateway.strategy.rollingUpdate.maxUnavailable |
Refer to doc | 0 |
wso2.deployment.am.gateway.config |
Custom deployment configuration file for Gateway profile (<WSO2AM>/repository/conf/deployment.toml) |
- |
wso2.deployment.am.cp.livenessProbe.initialDelaySeconds |
Initial delay for the live-ness probe for API Manager Control Plane profile | 60 |
wso2.deployment.am.cp.livenessProbe.periodSeconds |
Period of the live-ness probe for API Manager Control Plane profile | 10 |
wso2.deployment.am.cp.readinessProbe.initialDelaySeconds |
Initial delay for the readiness probe for API Manager Control Plane profile | 60 |
wso2.deployment.am.cp.readinessProbe.periodSeconds |
Period of the readiness probe for API Manager Control Plane profile | 10 |
wso2.deployment.am.cp.ingress.enabled |
If enabled, create ingress resource for API Manager management consoles | true |
wso2.deployment.am.cp.ingress.hostname |
Hostname for API Manager Control Plane | am.wso2.com |
wso2.deployment.am.cp.ingress.annotations |
Ingress resource annotations for API Manager Control Plane | Community NGINX Ingress controller annotations |
wso2.deployment.am.cp.resources.requests.memory |
The minimum amount of memory that should be allocated for running API Manager API Manager Control Plane | 1Gi |
wso2.deployment.am.cp.resources.requests.cpu |
The minimum amount of CPU that should be allocated for running API ManagerAPI Manager Control Plane | 1000m |
wso2.deployment.am.cp.resources.limits.memory |
The maximum amount of memory that should be allocated for running API Manager API Manager Control Plane | 2Gi |
wso2.deployment.am.cp.resources.limits.cpu |
The maximum amount of CPU that should be allocated for running API Manager API Manager Control Plane | 2000m |
wso2.deployment.am.cp.config |
Custom deployment configuration file for Control Plane profile (<WSO2AM>/repository/conf/deployment.toml) |
- |
wso2.deployment.am.trafficmanager.livenessProbe.initialDelaySeconds |
Initial delay for the live-ness probe for API Manager Traffic Manager profile | 180 |
wso2.deployment.am.trafficmanager.livenessProbe.periodSeconds |
Period of the live-ness probe for API Manager Traffic manager profile | 10 |
wso2.deployment.am.trafficmanager.readinessProbe.initialDelaySeconds |
Initial delay for the readiness probe for API Manager Control Plane profile | 180 |
wso2.deployment.am.trafficmanager.readinessProbe.periodSeconds |
Period of the readiness probe for API Manager Control Plane profile | 10 |
wso2.deployment.am.trafficmanager.replicas |
Number of replicas of API Manager Traffic Manager to be started | 2 |
wso2.deployment.am.trafficmanager.resources.requests.memory |
The minimum amount of memory that should be allocated for running API Manager Traffic Manager | 2Gi |
wso2.deployment.am.trafficmanager.resources.requests.cpu |
The minimum amount of CPU that should be allocated for running API Manager Traffic Manager | 2000m |
wso2.deployment.am.trafficmanager.resources.limits.memory |
The maximum amount of memory that should be allocated for running API Manager Traffic Manager | 3Gi |
wso2.deployment.am.trafficmanager.resources.limits.cpu |
The maximum amount of CPU that should be allocated for running API Manager Traffic Manager | 3000m |
wso2.deployment.am.trafficmanager.resources.jvm.heap.memory.xms |
The minimum Resource allocation for the Java Heap size for running API Manager Traffic Manager | 1024m |
wso2.deployment.am.trafficmanager.resources.jvm.heap.memory.xmx |
The maximum Resource allocation for the Java Heap size for running API Manager Traffic Manager | 1024m |
| Parameter | Description | Default Value |
|---|---|---|
wso2.deployment.mi.dockerRegistry |
Registry location of the Docker image to be used to create Micro Integrator instances | - |
wso2.deployment.mi.imageName |
Name of the Docker image to be used to create API Manager instances | wso2mi |
wso2.deployment.mi.imageTag |
Tag of the image used to create API Manager instances | 4.1.0 |
wso2.deployment.mi.imagePullPolicy |
Refer to doc | Always |
wso2.deployment.mi.livenessProbe.initialDelaySeconds |
Initial delay for the live-ness probe for Micro Integrator node | 35 |
wso2.deployment.mi.livenessProbe.periodSeconds |
Period of the live-ness probe for Micro Integrator node | 10 |
wso2.deployment.mi.readinessProbe.initialDelaySeconds |
Initial delay for the readiness probe for Micro Integrator node | 35 |
wso2.deployment.mi.readinessProbe.periodSeconds |
Period of the readiness probe for Micro Integrator node | 10 |
wso2.deployment.mi.resources.requests.memory |
The minimum amount of memory that should be allocated for a Pod | 512Mi |
wso2.deployment.mi.resources.requests.cpu |
The minimum amount of CPU that should be allocated for a Pod | 500m |
wso2.deployment.mi.resources.limits.memory |
The maximum amount of memory that should be allocated for a Pod | 1Gi |
wso2.deployment.mi.resources.limits.cpu |
The maximum amount of CPU that should be allocated for a Pod | 1000m |
wso2.deployment.mi.config |
Custom deployment configuration file (<WSO2AM>/repository/conf/deployment.toml) |
- |
wso2.deployment.mi.ingress.management.hostname |
Hostname for Micro Integrator management apis | management.mi.wso2.com |
wso2.deployment.mi.ingress.management.annotations |
Ingress resource annotations for API Manager Gateway | Community NGINX Ingress controller annotations |
Note: The above mentioned default, minimum resource amounts for running WSO2 API Manager server profiles are based on its official documentation.
| Parameter | Description | Default Value |
|---|---|---|
kubernetes.serviceAccount |
Name of the Kubernetes Service Account to which the Pods are to be bound | `wso2am-pattern-4- |
| svc-account` |
-
It is mandatory to set an appropriate Kubernetes StorageClass in this deployment, for persistence and sharing.
-
By default, this deployment uses the
nfsKubernetes StorageClass created using the official, stable NFS Server Provisioner. -
Only persistent storage solutions supporting
ReadWriteManyaccess mode are applicable forwso2.deployment.persistentRuntimeArtifacts.storageClass. -
Please refer to the official WSO2 container guide for advanced details with regards to WSO2 recommended, storage options.
-
By default, this deployment uses the default keystores and truststores provided by the relevant WSO2 product.
-
For advanced details with regards to managing custom Java keystores and truststores in a container based WSO2 product deployment please refer to the official WSO2 container guide.
- For WSO2 recommended best practices in configuring SSL when exposing the internal product services to outside of the Kubernetes cluster, please refer to the official WSO2 container guide.
If you want to setup API Manager only without Micro Integrator, you have to install the charts from source after removing MI templates.
-
Clone the repository
git clone https://github.com/wso2/kubernetes-apim.git -
Remove the MI templates by removing the
mifolder in<KUBERNETES_HOME>/advanced/am-pattern-4/templates/. -
Deploy Helm charts
helm install <RELEASE_NAME> <HELM_HOME>/am-pattern-4 --version 4.1.0-1 --namespace <NAMESPACE> --dependency-update --create-namespace