Skip to content

Latest commit

 

History

History
236 lines (166 loc) · 11 KB

using-cinder-csi-plugin.md

File metadata and controls

236 lines (166 loc) · 11 KB
Raw

Table of Contents generated with DocToc

CSI Cinder driver

The Cinder CSI Driver is a CSI Specification compliant driver used by Container Orchestrators to manage the lifecycle of OpenStack Cinder Volumes.

CSI Compatibility

This plugin is compatible with CSI versions v1.2.0 , v1.1.0, and v1.0.0

Downloads

Stable released version images of the plugin can be found at Docker Hub

Kubernetes Compatibility

For each kubernetes official release, there is a corresponding release of Cinder CSI driver which is compatible with k8s release. It is recommended to use the corresponding version w.r.t kubernetes.

For sidecar version compatibility with kubernetes, please refer Compatibility Matrix of each sidecar.

Driver Config

Implementation of cinder-csi-plugin relies on following OpenStack services.

Service API Version(s) Deprecated Required
Identity (Keystone) v2 Yes No
Identity (Keystone) v3 No Yes
Compute (Nova) v2 No Yes
Block Storage (Cinder) v3 No Yes

For Driver configuration, parameters must be passed via configuration file specified in $CLOUD_CONFIG environment variable. The following sections are supported in configuration file.

Global

For Cinder CSI Plugin to authenticate with OpenStack Keystone, required parameters needs to be passed in [Global] section of the file. For all supported parameters, please refer Global section.

Block Storage

These configuration options pertain to block storage and should appear in the [BlockStorage] section of the $CLOUD_CONFIG file.

  • node-volume-attach-limit Optional. To configure maximum volumes that can be attached to the node. Its default value is 256.
  • rescan-on-resize Optional. Set to true, to rescan block device and verify its size before expanding the filesystem. Not all hypervizors have a /sys/class/block/XXX/device/rescan location, therefore if you enable this option and your hypervizor doesn't support this, you'll get a warning log on resize event. It is recommended to disable this option in this case. Defaults to false
  • ignore-volume-az Optional. When Topology feature enabled, by default, PV volume node affinity is populated with volume accessible topology, which is volume AZ. But, some of the openstack users do not have compute zones named exactly the same as volume zones. This might cause pods to go in pending state as no nodes available in volume AZ. Enabling ignore-volume-az=true, ignores volumeAZ and schedules on any of the available node AZ. Default false.

Metadata

These configuration options pertain to metadata and should appear in the [Metadata] section of the $CLOUD_CONFIG file.

  • search-order: This configuration key influences the way that the driver retrieves metadata relating to the instance (s) in which it runs. The default value of configDrive,metadataService results in the provider retrieving metadata relating to the instance from the config drive first if available and then the metadata service. Alternative values are:

    • configDrive - Only retrieve instance metadata from the configuration drive.
    • metadataService - Only retrieve instance metadata from the metadata service.
    • metadataService,configDrive - Retrieve instance metadata from the metadata service first if available, then the configuration drive.

    Influencing this behavior may be desirable as the metadata on the configuration drive may grow stale over time, whereas the metadata service always provides the most up to date view. Not all OpenStack clouds provide both configuration drive and metadata service though and only one or the other may be available which is why the default is to check both.

Driver Deployment

You can either use the manifests under manifests/cinder-csi-plugin or the Helm chart charts/cinder-csi-plugin.

Using the manifests

All the manifests required for the deployment of the plugin are found at manifests/cinder-csi-plugin

Configuration file specified in $CLOUD_CONFIG is passed to cinder CSI driver via kubernetes secret. If the secret cloud-config is already created in the cluster, you can remove the file, manifests/cinder-csi-plugin/csi-secret-cinderplugin.yaml and directly proceed to the step of creating controller and node plugins.

To create a secret:

  • Encode your $CLOUD_CONFIG file content using base64.

$ base64 -w 0 $CLOUD_CONFIG

  • Update cloud.conf configuration in manifests/cinder-csi-plugin/csi-secret-cinderplugin.yaml file by using the result of the above command.

  • Create the secret.

$ kubectl create -f manifests/cinder-csi-plugin/csi-secret-cinderplugin.yaml

This should create a secret name cloud-config in kube-system namespace.

Once the secret is created, Controller Plugin and Node Plugins can be deployed using respective manifests

$ kubectl -f manifests/cinder-csi-plugin/ apply

This creates a set of cluster roles, cluster role bindings, and statefulsets etc to communicate with openstack(cinder). For detailed list of created objects, explore the yaml files in the directory. You should make sure following similar pods are ready before proceed:

$ kubectl get pods -n kube-system
NAME                                READY   STATUS    RESTARTS   AGE
csi-cinder-controllerplugin         6/6     Running   0          29h
csi-cinder-nodeplugin               3/3     Running   0          46h

To get information about CSI Drivers running in a cluster -

$ kubectl get csidrivers.storage.k8s.io
NAME                       CREATED AT
cinder.csi.openstack.org   2019-07-29T09:02:40Z

NOTE: If using certs(ca-file), make sure to update the manifests (controller and node plugin) to mount the location of certs as volume onto container as well.

Using the Helm chart

NOTE: With default values, this chart assumes that the cloud-config is found on the host under /etc/kubernetes/ and that your OpenStack cloud has cert under /etc/cacert.

You can specify a K8S Secret for cloud-config :

secret:
  enabled: true
  name: yousecretname

You can also tell Helm to create the K8S Secret :

secret:
  enabled: true
  name: cinder-csi-cloud-config
  data:
    cloud-config: |-
      ...

To install the chart, use the following command:

helm install --namespace kube-system --name cinder-csi ./charts/cinder-csi-plugin

Supported Features

Sidecar Compatibility

Supported Parameters

Parameter Type Paramerter Name Default Description
StorageClass parameters availability nova String. Volume Availability Zone
StorageClass parameters type Empty String String. Name/ID of Volume type. Corresponding volume type should exist in cinder
VolumeSnapshotClass parameters force-create false Enable to support creating snapshot for a volume in in-use status
Inline Volume volumeAttributes capacity 1Gi volume size for creating inline volumes
Inline Volume VolumeAttributes type Empty String Name/ID of Volume type. Corresponding volume type should exist in cinder

Local Development

Build

To build the plugin, run

$ export ARCH=amd64 # Defaults to amd64
$ make build-cmd-cinder-csi-plugin

To build cinder-csi-plugin image

$ export ARCH=amd64 # Defaults to amd64
$ make image-cinder-csi-plugin

Testing

Unit Tests

To run all unit tests:

$ make test

Sanity Tests

Sanity tests ensures the CSI spec conformance of the driver. For more info, refer Sanity check

Run sanity tests for cinder CSI driver using:

$ make test-cinder-csi-sanity

Optionally, to test the driver csc tool could be used. please refer, usage guide for more info.

In-tree Cinder provisioner to cinder CSI Migration

Starting from Kubernetes 1.18, CSI migration is supported as beta feature. If you have persistence volumes that are created with in-tree kubernetes.io/cinder plugin, you could migrate to use cinder.csi.openstack.org Container Storage Interface (CSI) Driver.

  • The CSI Migration feature for Cinder, when enabled, shims all plugin operations from the existing in-tree plugin to the cinder.csi.openstack.org CSI Driver.
  • In order to use this feature, the OpenStack Cinder CSI Driver must be installed on the cluster.
  • To turn on the migration, set CSIMigration and CSIMigrationOpenstack feature gates to true for kube-controller-manager and kubelet.
  • For more info, please refer Migrate to CCM with CSI Migration guide