diff --git a/clr-k8s-examples/10-cri-resource-manager/README.md b/clr-k8s-examples/10-cri-resource-manager/README.md new file mode 100644 index 00000000..6376dfe7 --- /dev/null +++ b/clr-k8s-examples/10-cri-resource-manager/README.md @@ -0,0 +1,58 @@ +CRI Resource Manager +==================== +CRI Resource Manager serves as a relay/proxy between kubelet and the container runtime, +relaying requests and responses back and forth between these two, potentially alrering requests +as they fly by. + +This document explains a very simple use case for the `cri-resource-manager`, for more details and tweaks +on CRI Resource Manager service, you can go to https://github.com/intel/cri-resource-manager. + +Install +------- +[`install.sh`](install.sh) script will download the binary and install it as an `systemd` service unit. Below you can see the available +variables you can use to customize the usage of your CRI Resource Manager service. + +| Variable | Description | Default Value | +|-----------------------------|-------------------------------------------|--------------------------------------------------| +| `RUNNER` | Default Container Runtime | `containerd` | +| `CRI_RESMGR_POLICY` | CRI Resource Manager Policy type | `null` | +| `CRI_RESMGR_POLICY_OPTIONS` | CRI Resource Manager extra policy options | `-dump='reset,full:.*' -dump-file=/tmp/cri.dump` | +| `CRI_RESMGR_DEBUG_OPTIONS` | CRI Resource Manager debugging options | | + +``` +RUNNER=containerd ./install.sh +``` + +- Install verification + - Verify that the cri-resource-manager service is actually running. +``` +systemctl status cri-resource-manager +``` + - Verify that the `/var/run/cri-resmgr/cri-resmgr.sock` is created, it will indicate that `cri-resource-manager` is ready to receive requests. + + +Setup as a container runtime in `kubelet` +---------------------------------------- +The [`setup.sh`](setup.sh) script will configure the `kubelet` service to use the `cri-resource-manager` relay as its remote container runtime. +``` +./setup.sh +``` + +- Setup verification + - Kubelet service should be restarted and now using `cri-resource-manager` as its container runtime + - `cri-resource-manager` service's logs will be located at `/tmp/cri.dump` +``` +tail /tmp/cri.dump +``` + +Cleanup +------- +The [`clean.sh`](clean.sh) will first clean the `kubelet` service as it was before the `cri-resource-manager` and restarts `kubelet` service. +Then. it will proceed to stop the `cri-resource-manager` service. +``` +./clean.sh +``` + +More kubernetes native approach (experimental) +---------------------------------------------- +In case that you're interested in a more Kubernetes native way of deploying the CRI Resource manager, take a look on: https://github.com/obedmr/cri-resource-manager/blob/cloud-native/cmd/cri-resmgr/deployment.yaml \ No newline at end of file diff --git a/clr-k8s-examples/10-cri-resource-manager/clean.sh b/clr-k8s-examples/10-cri-resource-manager/clean.sh new file mode 100755 index 00000000..06cf3fa9 --- /dev/null +++ b/clr-k8s-examples/10-cri-resource-manager/clean.sh @@ -0,0 +1,18 @@ +#!/bin/bash + +set -o errexit +set -o nounset + +# Kubelet +KUBEADM_FLAGS="/var/lib/kubelet/kubeadm-flags.env" +sudo rm -f /etc/systemd/system/kubelet.service.d/99-cri-resource-manager.conf +sudo systemctl daemon-reload +sudo systemctl restart kubelet + +if sudo test -f "$KUBEADM_FLAGS.bkp" ; then + sudo mv $KUBEADM_FLAGS.bkp $KUBEADM_FLAGS +fi + +# CRI Resource Manager +sudo systemctl stop cri-resource-manager +sudo systemctl disable cri-resource-manager diff --git a/clr-k8s-examples/10-cri-resource-manager/install.sh b/clr-k8s-examples/10-cri-resource-manager/install.sh new file mode 100755 index 00000000..0b32bb41 --- /dev/null +++ b/clr-k8s-examples/10-cri-resource-manager/install.sh @@ -0,0 +1,31 @@ +#!/bin/bash + +set -o errexit +set -o nounset + +RUNNER=${RUNNER:-"containerd"} +CRI_RESMGR_POLICY=${CRI_RESMGR_POLICY:-"null"} +CRI_RESMGR_POLICY_OPTIONS=${CRI_RESMGR_POLICY_OPTIONS:-"-dump='reset,full:.*' -dump-file=/tmp/cri.dump"} +CRI_RESMGR_DEBUG_OPTIONS=${CRI_RESMGR_DEBUG_OPTIONS:-""} + +curl https://raw.githubusercontent.com/obedmr/cri-resource-manager/master/godownloader.sh | bash +sudo cp ./bin/* /usr/bin/ + +runtime_socket=$(sudo find /run/ -iname $RUNNER.sock | head -1) +CRI_RESMGR_POLICY_OPTIONS+=" -runtime-socket=$runtime_socket -image-socket=$runtime_socket" + +sudo mkdir -p /etc/sysconfig/ +cat <<EOF | sudo tee /etc/sysconfig/cri-resource-manager +POLICY=$CRI_RESMGR_POLICY +POLICY_OPTIONS=$CRI_RESMGR_POLICY_OPTIONS +DEBUG_OPTIONS=$CRI_RESMGR_DEBUG_OPTIONS +EOF + +sudo mkdir -p /etc/systemd/system/ +curl https://raw.githubusercontent.com/obedmr/cri-resource-manager/master/cmd/cri-resmgr/cri-resource-manager.service | sudo tee /etc/systemd/system/cri-resource-manager.service + +sudo sed -i '/Requires=/d' /etc/systemd/system/cri-resource-manager.service +sudo systemctl daemon-reload +sudo systemctl restart cri-resource-manager.service +sudo systemctl enable cri-resource-manager.service + diff --git a/clr-k8s-examples/10-cri-resource-manager/setup.sh b/clr-k8s-examples/10-cri-resource-manager/setup.sh new file mode 100755 index 00000000..e50b7836 --- /dev/null +++ b/clr-k8s-examples/10-cri-resource-manager/setup.sh @@ -0,0 +1,24 @@ +#!/bin/bash + +set -o errexit +set -o nounset + +CRI_RESMGR_SOCKET="/var/run/cri-resmgr/cri-resmgr.sock" +KUBEADM_FLAGS="/var/lib/kubelet/kubeadm-flags.env" + +if sudo test -S "$CRI_RESMGR_SOCKET" ; then + sudo mkdir -p /etc/systemd/system/kubelet.service.d/ + cat <<EOF | sudo tee /etc/systemd/system/kubelet.service.d/99-cri-resource-manager.conf +[Service] +Environment=KUBELET_EXTRA_ARGS= +Environment=KUBELET_EXTRA_ARGS="--container-runtime remote --container-runtime-endpoint unix://${CRI_RESMGR_SOCKET}" +EOF + + if sudo test -f "$KUBEADM_FLAGS" ; then + sudo mv $KUBEADM_FLAGS $KUBEADM_FLAGS.bkp + fi + + sudo systemctl daemon-reload + sudo systemctl restart cri-resource-manager + sudo systemctl restart kubelet +fi diff --git a/clr-k8s-examples/README.md b/clr-k8s-examples/README.md index e63fc74f..2e78947c 100644 --- a/clr-k8s-examples/README.md +++ b/clr-k8s-examples/README.md @@ -146,6 +146,9 @@ kubectl -n monitoring port-forward svc/grafana 3000 Grafana is available at this URL http://localhost:3000 . Default credentials are `admin/admin`. Upon entering you will be asked to chose a new password. +### CRI Resource Manager +Go to [`10-cri-resource-manager`](./10-cri-resource-manager). + ## Cleaning up the cluster (Hard reset to a clean state) Run `reset_stack.sh` on all the nodes