How do I deal with specific data placement and distribution issues in the flexible and loosely defined Kubernetes storage world? PersistentVolumeClaims with StorageClasses are a great way to specify and flexibly handle storage needs for deployments. As users begin deploying more complex and data-dependent applications, they will begin to face situational needs for data locality, redundancy distribution, and interaction with other data resources that cannot be addressed with the basic functionality. Portworx Enterprise from Portworx by Pure Storage provides advanced options for fine-grained control over where volumes land on storage throughout the cluster.
Redundancy And Distribution
User Scenario: I want to spread out replicas of one application’s volume across separate datacenter racks to protect from small-scale power events, and spread another application’s replicas across separate zones to protect from large-scale cloud disruption.
Solution: Use a Portworx Volume Placement Strategy (VPS) for each StorageClass with replicaAntiAffinity to racks and zones respectively.
Rack Example:
apiVersion: portworx.io/v1beta2 kind: VolumePlacementStrategy metadata: name: avoid-same-rack spec: replicaAntiAffinity: - topologyKey: failure-domain.beta.kubernetes.io/rack
Zone Example:
apiVersion: portworx.io/v1beta2 kind: VolumePlacementStrategy metadata: name: avoid-same-zone spec: replicaAntiAffinity: - topologyKey: failure-domain.beta.kubernetes.io/zone
This core functionality of VPS will avoid placing a volume on the same rack/zone as an already placed replica for this volume, using the failure-domain or similar labels on nodes. All VPS rules can be marked as “preferred” to allow placement to still go through if the rule cannot be met.
Data Locality
User Scenario: I want two volumes for the same StatefulSet pod to land on the same node so their frequent interactions can benefit from the lowest latency by not going across the datacenter network.
Solution: Use a volumeAffinity rule to force the volumes associated with the same StatefulSet pod to land on the same node, so they will always have local data transfer speed/latency.
apiVersion: portworx.io/v1beta2
kind: VolumePlacementStrategy
metadata:
name: anti-ssd-pool-placement
spec:
volumeAffinity:
- matchExpressions:
- key: px/statefulset-pod
operator: In
values:
- "${pvc.statefulset-pod}"
- key: app
operator: In
values:
- "app1"
This key/value template will automatically use the parent StatefulSet pod information to group all volumes for the same application pod using this VPS with each other.
Interaction With Other Storage Resources
User Scenario: One team on a cluster wants to ensure volumes for their database app are spread across different hosts to improve response to single node recovery and don’t care about placement relative to any other team’s deployment of the same common application.
Solution: Limit the rule to their current namespace so VPS rules don’t conflict with another team’s pods/volumes.
apiVersion: portworx.io/v1beta2
kind: VolumePlacementStrategy
metadata:
name: anti-ssd-pool-placement
spec:
volumeAntiAffinity:
- matchExpressions:
- key: "namespace"
operator: In
values:
- "${pvc.namespace}"
- key: app
operator: In
values:
- "redis"
This key/value template will automatically inherit the namespace from its PVC, and logically AND it with the other expression(s) to only apply to other volumes in the same parent namespace.
Summary
Portworx advanced Volume Placement Strategies allow users to solve specific cluster data management problems by combining rules to influence the placement of volumes and replicas. The CRD gives advanced options to users who are unsatisfied with the limited control of built-in storage options. These features and more are available with Portworx Enterprise 2.10 release. For more details: https://docs.portworx.com/portworx-install-with-kubernetes/storage-operations/create-pvcs/volume-placement-strategies/crd-reference/
You can also take a look at our demo video which shows examples of TopologyKey mapping, statefulset modifiers, and the new namespace modifier now available in Portworx Enterprise 2.10 here:
https://www.youtube.com/watch?v=ywpkj4jVs5M
Share
Subscribe for Updates
About Us
Portworx is the leader in cloud native storage for containers.
Thanks for subscribing!
