Hear how our largest telecom customer tackled open-source challenges with Portworx. Register Now

How do I deal with specific data placement and distribution issues in the flexible and loosely defined Kubernetes storage world? PersistentVolumeClaims with StorageClasses are a great way to specify and flexibly handle storage needs for deployments. As users begin deploying more complex and data-dependent applications, they will begin to face situational needs for data locality, redundancy distribution, and interaction with other data resources that cannot be addressed with the basic functionality. Portworx Enterprise from Portworx by Pure Storage provides advanced options for fine-grained control over where volumes land on storage throughout the cluster.

Redundancy And Distribution

User Scenario: I want to spread out replicas of one application’s volume across separate datacenter racks to protect from small-scale power events, and spread another application’s replicas across separate zones to protect from large-scale cloud disruption.

Solution: Use a Portworx Volume Placement Strategy (VPS) for each StorageClass with replicaAntiAffinity to racks and zones respectively.

Rack Example:

apiVersion: portworx.io/v1beta2

kind: VolumePlacementStrategy

metadata:

  name: avoid-same-rack

spec:

  replicaAntiAffinity:

    -  topologyKey: failure-domain.beta.kubernetes.io/rack

Zone Example:

apiVersion: portworx.io/v1beta2

kind: VolumePlacementStrategy

metadata:

  name: avoid-same-zone

spec:

  replicaAntiAffinity:

    -  topologyKey: failure-domain.beta.kubernetes.io/zone

This core functionality of VPS will avoid placing a volume on the same rack/zone as an already placed replica for this volume, using the failure-domain or similar labels on nodes. All VPS rules can be marked as “preferred” to allow placement to still go through if the rule cannot be met.

Data Locality

User Scenario: I want two volumes for the same StatefulSet pod to land on the same node so their frequent interactions can benefit from the lowest latency by not going across the datacenter network.

Solution: Use a volumeAffinity rule to force the volumes associated with the same StatefulSet pod to land on the same node, so they will always have local data transfer speed/latency.

apiVersion: portworx.io/v1beta2

kind: VolumePlacementStrategy

metadata:

  name: anti-ssd-pool-placement

spec:

  volumeAffinity:

    - matchExpressions:

      - key: px/statefulset-pod

        operator: In

        values:

        - "${pvc.statefulset-pod}"

      - key: app

        operator: In

        values:

        - "app1"

This key/value template will automatically use the parent StatefulSet pod information to group all volumes for the same application pod using this VPS with each other.

Interaction With Other Storage Resources

vds-blog-image2

User Scenario: One team on a cluster wants to ensure volumes for their database app are spread across different hosts to improve response to single node recovery and don’t care about placement relative to any other team’s deployment of the same common application.

Solution: Limit the rule to their current namespace so VPS rules don’t conflict with another team’s pods/volumes.

apiVersion: portworx.io/v1beta2

kind: VolumePlacementStrategy

metadata:

  name: anti-ssd-pool-placement

spec:

  volumeAntiAffinity:

    - matchExpressions:

      - key: "namespace"

        operator: In

        values:

        - "${pvc.namespace}"

      - key: app

        operator: In

        values:

        - "redis"

This key/value template will automatically inherit the namespace from its PVC, and logically AND it with the other expression(s) to only apply to other volumes in the same parent namespace.

Summary

Portworx advanced Volume Placement Strategies allow users to solve specific cluster data management problems by combining rules to influence the placement of volumes and replicas. The CRD gives advanced options to users who are unsatisfied with the limited control of built-in storage options. These features and more are available with Portworx Enterprise 2.10 release. For more details: https://docs.portworx.com/portworx-install-with-kubernetes/storage-operations/create-pvcs/volume-placement-strategies/crd-reference/

You can also take a look at our demo video which shows examples of TopologyKey mapping, statefulset modifiers, and the new namespace modifier now available in Portworx Enterprise 2.10 here:
https://www.youtube.com/watch?v=ywpkj4jVs5M

Share
Subscribe for Updates

About Us
Portworx is the leader in cloud native storage for containers.

Luke Pitstick

Luke Pitstick

Portworx Research and Development Technical Staff
link
VolumeReplication
January 7, 2020 Lightboard
Lightboard Session: Understanding Volume Replication
Ryan Wallner
Ryan Wallner
link
VolumeSnapshots
January 7, 2020 Lightboard
Lightboard Session: Volume Snapshot Types
Ryan Wallner
Ryan Wallner
link
tw
August 13, 2019 Technical Insights
Scaling Elasticsearch on Kubernetes with the Portworx Data Platform
Vick Kelkar
Vick Kelkar