A Complete Guide to Kubernetes CSI

Early in the cloud-native era, storage vendors created drivers specific to container orchestrators like Kubernetes. For Kubernetes, they were also required to write a separate driver integrated into Kubernetes code, which meant users had to wait for Kubernetes releases to enable new features or bug fixes in Kubernetes storage. What’s more, many users were not ready to upgrade Kubernetes but wanted the storage-related fixes, which was another complication.

In 2019, the community introduced the Container Storage Interface (CSI). We’ll explain the CSI with its architecture, setup, monitoring, and real-world use cases.

What is Kubernetes CSI (Container Storage Interface)?

CSI is a standard that helps provide drivers as an extension to expose arbitrary block and file storage systems to containerized workloads on Kubernetes, and establish connectivity between storage systems and Kubernetes. Using CSI, third-party storage providers can write and deploy plugins and expose storage systems in Kubernetes, without touching the core Kubernetes code.

Why is Storage Important in Containerized Environments?

In containerized environments, applications need properly configured storage to reliably store, manage, and access data, and fully leverage the benefits of Kubernetes.
kubernet

• Persistent Data Storage: Containers were not originally designed to implement stateful applications. In containerized environments, persistent storage is needed to support services like Database-Platform-as-a-Service so applications can maintain state across sessions or migrations.
• Scalability and Flexibility: Storage in containerized environments helps ensure that the data required by an application is accessible regardless of where the container is running in a cluster. Developers have the flexibility to scale up or down more effectively and also efficiently manage backup built for Kubernetes.

Understanding CSI Architecture

CSI architecture must establish seamless communication across the container orchestration systems and storage:

Centralized Controller Plugin CSI Architecture

Components of CSI

Controller

The Controller manages the lifecycle of storage volumes. It responds requests for:

• Volume creation and deletion.
• Attach and detach volumes to the nodes.
• Create snapshots of the volumes.

Node

The node is responsible for mounting and unmounting volumes on the worker nodes. Once the volume is mounted, it is also responsible for actions performed on the volume.

Volume Plugins

A volume plugin connects storage systems to Kubernetes. It is deployed on a Kubernetes cluster for out-of-tree implementation. It doesn’t require access to the underlying operating system or the root file system dependencies.

How CSI Connects with Kubernetes

With CSI, Kubernetes can interact with multiple storage solutions without modifying its core code. Kubernetes connects with CSI through gRPCs, a cross-platform Remote Procedure Call framework.

Remote Procedure Calls (RPCs) execute a procedure (subroutine) on a remote system. The CSI’s RPCs are idempotent, ensuring that a call should not change the result beyond its initial application. Here are three sets of gRPCs provided by the CSI Plugin:

• Identity Service RPCs: Provides endpoints like `GetPluginCapabilities` that allow Kubernetes to query the capabilities of the CSI plugin, `Probe` for its health, and `GetPluginInfo` that returns the metadata associated with the plugin.
• Controller Service RPCs: Provides endpoints like `CreateVolume` and `DeleteVolume` that allow the creation or deletion of volumes, respectively. Endpoints like `CreateSnapshot` allow the creation of a volume. It also has endpoints that enable attaching or detaching a volume to a node, and querying the storage pool for its capacity.
• Node Service RPCs: Provides endpoints like `NodeGetInfo` that give the details of where the volume has to be mounted and `NodeExpandVolume` that can expand the volume mounted on a node.

Getting Started with CSI in Kubernetes

Understanding CSI is critical whether you’re deploying a new application or upgrading an existing one. In this section, we’ll look at the prerequisites, installation, and configuration of various CSI providers.

Pre-requisites

Before you install and configure CSI drivers in Kubernetes, you need:

• The latest version of Kubernetes.
• Cluster-admin privileges to install and configure CSI drivers.
• To verify network connectivity and configuration to ensure uninterrupted communication between CSI components, Kubernetes nodes, and the
  storage backend.

Installing CSI Drivers

Installing a CSI driver is a multi-step process that involves deploying the driver, configuring necessary Kubernetes resources like StorageClass, PersistentVolume, and PersistentVolumeClaim, and ensuring proper integration with your storage backend. This section covers both official and third-party CSI drivers.

Official CSI Drivers

Official CSI drivers are developed and maintained by the Kubernetes community or cloud providers. These are designed to integrate cloud storage solutions like AWS EBS, Google Persistent Disks, and Azure Disks with Kubernetes and other storage backends (see the complete list of official CSI drivers)/

Let’s install Azure CSI Driver.

1. On a Kubernetes cluster, install the Azure CSI driver using the following command:
   

   “`sh
   curl -skSL https://raw.githubusercontent.com/kubernetes-sigs/azuredisk-csi-driver/v1.30.3/deploy/install-driver.sh | bash -s v1.30.3 snapshot –
   “`

    

   This will install the latest version (at the time of writing), v1.30.3, on your cluster. Refer to the latest available Azure CSI driver.

2. Check the pod status.
   

   “`sh
   kubectl get pods -n kube-system -l app=csi-azuredisk

    

   Sample output:

   “`sh
   NAME READY STATUS RESTARTS AGE
   csi-azuredisk-node-4bz2t 3/3 Running 0 5m
   csi-azuredisk-node-8fjhc 3/3 Running 0 5m
   csi-azuredisk-node-k9s4g 3/3 Running 0 5m
   csi-azuredisk-controller-0 5/5 Running 0 5m
   csi-azuredisk-controller-1 5/5 Running 0 5m
   
   “`

    

3. Create a StorageClass that specifies the Azure Disk CSI driver as the provisioner.
   

   “`yaml
   apiVersion: storage.k8s.io/v1
   kind: StorageClass
   metadata:
   name: azure-disk-sc
   provisioner: disk.csi.azure.com
   parameters:
   skuname: Standard_LRS
   kind: Managed
   reclaimPolicy: Delete
   volumeBindingMode: Immediate“`

    

4. Create a PVC to request storage resources from the cluster based on the configuration defined in `StorageClass`.
   

   “`yaml
   apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
   name: azure-disk-pvc
   spec:
   accessModes:
   – ReadWriteOnce
   storageClassName: azure-disk-sc
   resources:
   requests:
   storage: 50Gi
   “`

    

5. Configure your application to use the above-configured PVC.
   

   “`yaml
   …
   spec:
   containers:
   – name: my-app-container
   image: nginx:latest
   ports:
   – containerPort: 80
   volumeMounts:
   – mountPath: /usr/share/nginx/html
   name: azure-disk-storage
   volumes:
   – name: azure-disk-storage
   persistentVolumeClaim:
   claimName: azure-disk-pvc
   …
   “`

    

With the above configuration, the Nginx web server will use the storage mounted at `/usr/share/nginx/html`, backed by the Azure-managed disk provisioned by the Azure Disk CSI driver. Any data written to this path will persist even if the pod is deleted or rescheduled, as it is stored on a persistent disk managed by Azure.

Managing Persistent Storage with CSI

Persistent storage is critical for running stateful applications on Kubernetes. While CSI provides a standard way to expose storage systems to Kubernetes, key components like PVs and PVCs help you effectively utilize your CSI driver.

Provisioning Persistent Volumes (PVs)

Persistent Volumes (PVs) represent physical storage on your storage backend, such as disks or network-attached storage (NAS) in the cluster that has been provisioned. PVs persist beyond the lifecycle of any pod that uses them.

Kubernetes provides two ways to provision PVs:

• Static Provisioning
• Dynamic Provisioning

Static Provisioning

Static provisioning refers to manually creating Persistent Volumes (PVs). These PVs are pre-bound to specific storage resources, which can then be claimed by Persistent Volume Claims (PVCs). Admins can create PVs and specify the underlying storage resource, and these PVs are then available to be claimed by PVCs created by applications.

Benefits:
It provides complete control over the storage resource and is ideal for scenarios where the exact storage requirements are known before application deployment.
Allows admins to pre-allocate storage based on expected usage patterns.

Dynamic Provisioning

Dynamic provisioning automatically creates a PV when the application requests a PVC. This approach provides a seamless way to allocate storage without needing pre-provisioned volumes.

Benefits:

• Simplifies storage management by automatically handling PV creation and reduces the need for manual intervention.
• It supports a wide range of storage backends through CSI drivers.

Persistent Volume Claims (PVCs)

Persistent Volume Claims (PVCs) are user requests for storage. They abstract the details of the underlying storage resource and allow the users to request specific storage capacity, access modes, and other things without knowing the underlying storage infrastructure.

A user creates a PVC specifying the required storage capacity, access modes (such as ReadWriteOnce), and StorageClass. Kubernetes then attempts to find a matching PV or create one dynamically.

Benefits:

• Simplifies the storage request process and provides a flexible and user-friendly interface to manage storage.
• Supports various access modes and storage backends using CSI drivers.

Binding PVs and PVCs

Binding PVs to PVCs ensures your application receives the storage it needs. When a PVC is created, Kubernetes matches it with an appropriate PV to fulfill the request. This process connects the abstract storage request (PVC) and the concrete storage request (PV).

When a PVC is created, Kubernetes finds a matching PV or creates one, which is marked as “bound” to the PVC. The storage becomes available for use by the application.

Benefits:

• Ensures the requested storage by the application is allocated and ready to use.
• Automates the process of matching storage requests with available storage.

Read our guide on Kubernetes storage solutions to learn more about various storage solutions on Kubernetes and how they work.

Container Storage Interface Features and Capabilities

CSI is integral to Kubernetes’ storage architecture. It provides a standardized way to integrate storage systems with containers. Let us look at CSI’s key features and capabilities that can be leveraged with Kubernetes.

Snapshots and Clones

Snapshots and clones enable you to create point-in-time copies of your persistent volumes (PVs). While a snapshot is a read-only copy, a clone is a writable copy derived from an existing snapshot or volume.

Snapshots are created through the Kubernetes API, where a VolumeSnapshot object is defined. Clones are created by specifying a `dataSource` in the PVC that refers to either an existing snapshot or another PVC.

Snapshots can be used to create periodic backups and configure disaster recovery, and clones can be used to replicate environments.

Volume Expansion

Volume expansion allows you to increase the size of an existing PV without disrupting the application. This is performed by editing the PVC and specifying a larger storage size. The underlying storage system must support expansion, and the StorageClass the PVC uses must have the allowVolumeExpansion field set to true.

It allows you to easily adjust storage resources as data grows and expand volumes without downtime or the need to re-provision.

Volume Topology

Volume topology configures volumes based on location-specific racks, availability zones, or regions and provisions storage in locations compatible with the nodes where your workloads are running.

When provisioning a volume, Kubernetes uses topology constraints using nodeSelectorTerms specified in the PVC or the storage class to determine where the volume should be created, ensuring it matches the node’s topology.

This ensures low-latency access to storage by aligning volume location with the workload and helps maintain application resilience by ensuring storage is distributed across failure domains.

Raw Block Volumes

Databases or other high-performing workloads require direct access to storage as a block device. Raw Block Volumes allow you to configure direct access to a storage device bypassing the filesystem layer.

A PVC can be configured to use `volumeMode: Block`, which provides storage as a raw block device. The application must be capable of handling raw block storage.

It is suitable for applications that need direct control over storage and provides high performance by eliminating filesystem overhead.

Advanced Usage Of Kubernetes CSI

CSI standardizes storage management across different platforms and provides advanced features that enable customization and optimization of storage operations. This section explores these advanced capabilities and the critical role of CSI sidecars in Kubernetes environments.

Custom CSI Drivers Development

By creating a custom CSI-compliant driver, you can extend your Kubernetes storage capabilities to support new or proprietary storage systems. CSI drivers can be written in any language and are designed to interact with Kubernetes through standardized APIs to perform storage operations like provisioning, attaching, mounting, and snapshotting.

One must adhere to the CSI specification, ensuring drivers implement mandatory functions such as CreateVolume, DeleteVolume, ControllerPublishVolume, and NodeStageVolume.

Custom drivers are ideal when integrating specialized storage hardware or software not supported by existing CSI drivers.

CSI Sidecars and Their Roles

While the main CSI driver integrates with external storage, sidecars handle specific tasks such as provisioning, attaching, resizing, and snapshotting volumes. Each sidecar focuses on a specific function, making the CSI architecture more modular and easier to maintain.

External Provisioner

An external provisioner is a CSI sidecar responsible for dynamically provisioning PVs based on PVCs. It watches for PVCs that require a PV and updates the CSI driver to create a new volume when needed. It also ensures PVs are provisioned based on the configuration provided in the StorageClass.

External Attacher

The external attacher is responsible for attaching and detaching volumes to nodes. It handles the communication between Kubernetes and the storage backend to ensure volumes are attached to/detached from the correct nodes. It watches for VolumeAttachment objects in the Kubernetes API and manages the attachment process, ensuring the volume is accessible from the node where the pod is scheduled.

External Resizer

An external resizer is a sidecar that enables PV resizing. It listens for PVC updates where the requested storage size has increased and coordinates with the CSI driver to expand the underlying volume. The external resizer updates the storage backend to reflect the new size and adjusts Kubernetes API objects to match. Depending on the capability of the storage and CSI driver, the resizing can happen whether the volume is online or offline.

Snapshotter

Snapshotter handles creating, deleting, and listing snapshots, interfacing with the CSI driver to perform these tasks.

It listens for VolumeSnapshot objects, instructs the CSI driver to create snapshots, and ensures that snapshots consistently capture the data at a point in time.

Monitoring and Troubleshooting CSI

CSI can encounter issues like any other component, requiring monitoring and troubleshooting to ensure application reliability. Let’s look at some common issues and then at monitoring.

Common Issues and Resolutions

Pod Stuck in Pending State

• PVC Binding Issues: Ensure that a PV can satisfy your PVC or your StorageClass must support dynamic provisioning.
• StorageClass Misconfigurations: Ensure the correct ‘provisioner’ is specified so you can connect with the correct backend storage.
• Topology Constraints: Verify that your volume’s topology matches the node’s availability zone where the pod is scheduled.

Volume Attach/Detach Failures

• Node Affinity Mismatch: Ensure that the node selector in PVC matches the node’s label to avoid mismatches when attaching/detaching volumes.
• CSI Version Mismatch: Incorrect CSI driver and Kubernetes versions can cause issues utilizing the underlying storage system.

Volume Expansion Issues

• Backend Storage Limitation: Ensure that the underlying storage backend supports volume expansion and the volume is not at its full capacity.
• Online vs Offline Resize: Validate if the storage supports online resizing, otherwise detach the volume and then resize.

Snapshot Creation Failure

• Backend Security Capability: Validate that the storage supports snapshotting and that adequate permissions are granted to create snapshots.
• Consistency: Ensure that snapshotting happens in a way that your application isn’t disrupted and the data doesn’t corrupt.

Check the top 5 reasons CSI drivers fall short in Enterprise-Scale Kubernetes to avoid making those mistakes.

Tools and Best Practices for Monitoring

Kubernetes Native Tools

• kubectl Commands: Use `kubectl describe` and `kubectl logs` to inspect CSI driver pods, PVCs, and PVs for any issues.
• Events Monitoring: Use `kubectl get events` to catch events like failed volume provisioning or attachment errors in real time.

Logging and Metrics

• Prometheus and Grafana: Integrate Prometheus with CSI drivers to expose metrics like error rates and other critical metrics for monitoring performance. Build Grafana dashboards to visualize these metrics.
• Centralized Logging: Implement centralized logging using tools like Fluentd or Elasticsearch to aggregate and analyze logs from CSI drivers, making it easier to trace issues across your cluster.

Best Practices

• Regular Health Checks: Implement automated health checks for PVs and PVCs to detect and resolve issues before they impact workloads and application availability.
• Resource Limits and Quotas: Set appropriate limits and quotas on CSI driver pods to prevent resource exhaustion and ensure consistent performance.
• Disaster Recovery Planning: Regularly test snapshot creation and restore processes to ensure they work as expected in disaster recovery scenarios.

Monitoring helps you understand why your CSI driver can’t keep up with the storage demands of your workload.

Security Considerations for Container Storage Interfaces

CSI includes several configurations that ensure data safety during various operations. Let’sllook at some key security considerations.

Authentication and Authorization

Kubernetes offers different ways to implement authentication and authorization for CSI driver, and ensures only authorized users and systems can interact with storage resources managed by CSI.

• Role-Based Access Control (RBAC): Role-based Access Control (RBAC) enforces policies governing which users and service accounts can create, modify, or delete PVCs and other storage-related resources. Regularly audit RBAC roles and bindings to ensure minimal and appropriate permissions.
• CSI Driver Authentication: Many CSI providers integrate with third-party authentication services, like IAMs, to authenticate requests to the underlying storage.
• Secrets Management: Use Kubernetes of external secrets to store tokens and manage credentials CSI drivers use for authentication. Rotate secrets and credentials periodically to reduce the risk of unauthorized access.

Data Encryption

Data encryption in this context refers to safeguarding sensitive data stored on volumes managed by CSI drivers. Kubernetes provides support for both at-rest and in-transit encryption.

In-transit Encryption

This refers to the encryption of data when it’s moving between the storage volumes and Kubernetes pods to prevent tampering or unauthorized access during transit. Here’s how you can enable in-transit encryption and associated best practices:

• TLS Encryption: Use Transport Layer Security (TLS) to encrypt data as it moves between the CSI driver, Kubernetes components, and the storage backend. Ensure that all communications between these components are encrypted.
• Mutual TLS (mTLS): For higher security, implement mutual TLS to authenticate Kubernetes nodes and the storage backend during data transfer. Regularly update TLS certificates using automated tools.

At-rest Encryption

Data stored on physical disks must be encrypted to prevent unauthorized access to sensitive information. Here’s a few at-rest encryption methods and best practices:

• Storage Backend Encryption: Most backed storage systems provide means to implement at-rest encryption. Leverage these capabilities, such as AWS KMS for EBS volumes or Google Cloud’s encryption services for persistent disks. Remember to enable at-rest encryption for all sensitive data irrespective of storage backend.
• Kubernetes Integration: Some CSI drivers support native integration with cloud provider-managed encryption services, enabling and managing encryption at rest easier. Monitor and enforce encryption policies to ensure compliance with regulatory requirements.

Securing CSI Components

Until now, we discussed ways to protect data and communication between Kubernetes resources, CSI components, and storage backends. However, securing the CSI components themselves is crucial to maintaining the overall security posture of the Kubernetes storage system.

• Pod Security Policies: Use Pod Security Standards (PSS) to restrict the privileges of CSI driver pods, minimizing their attack surface.
• Network Policies: Apply Kubernetes Network Policies to control the network traffic to and from CSI components, ensuring they can only communicate with trusted entities.
• Image Security: Ensure that the CSI driver images are sourced from trusted repositories, are regularly scanned for vulnerabilities, and are signed to verify their integrity.
• Isolation: Isolate CSI components in different namespaces and ensure the right access controls are provided.

Real-World Applications of Kubernetes CSI

Kubernetes CSI plays a crucial role in dealing with storage-intensive workloads. Let us look at some real-world applications.

Enterprise Applications

Enterprise applications often use persistent storage for critical data like customer records, financial transactions, and business intelligence. With Kubernetes CSI, these applications can use enterprise-grade storage solutions from vendors like NetApp, Dell EMC, and IBM and ensure data durability and high availability.

Data-Intensive Workloads

Applications like big data analytics, machine learning, and media processing handle large volumes of data, and benefit from the scalability and flexibility of Kubernetes CSI. By integrating with file systems such as Gluster and Ceph or object storage solutions like MinIO, such workloads can be efficiently managed, and the applications can process extensive datasets.

Cloud Native Applications

Cloud native stateful applications need scalable, persistent storage integrated with cloud services. Kubernetes CSI allows these applications to use cloud storage options like Amazon Elastic Block Store (EBS), Google Persistent Disk, and Azure Disk Storage, making it easier to take advantage of cloud storage’s scalability and cost-effectiveness.

Kubernetes Storage Solutions with Portworx

Kubernetes storage solutions help manage persistent data in containerized stateful applications. You can maintain the state across container restarts or migrations and ensure data consistency and availability. Portworx provides a standard storage interface for containerized environments. Portworx is the first containerized storage system to support the CSI on Kubernetes fully.

For cloud-native applications, Portworx offers a comprehensive Kubernetes storage and data management solution to automate, protect, and unify data for modern applications across any on-prem, cloud, or hybrid environment. It’s a purpose-built storage solution to keep up with the demands of Kubernetes orchestration and lifecycle management. To get started, head to central.portworx.com to sign up for a trial.