Cloud Native Storage: A View From Production

There has been a lot of discussion recently about whether you can or should run stateful services like databases in containers in production. As the leader in data services for containers and cloud native storage, Portworx categorically says yes, you can AND should run databases in containers. As the community discusses these important issues, Portworx customers like GE, Dreamworks, Lufthansa Airlines, TGen and many more are living proof that the time is now for stateful containers. 

That said, an important caveat is that success with stateful containers requires a cloud native storage solution. As customers come to Portworx, we hear all the time about problems they’ve run into using storage systems designed for VMs. It is not because these solutions are bad. It is because they were not designed specifically with containers in mind.

So what exactly is cloud native storage? Below, in a nutshell is how we define it and how Portworx fits in.

What is cloud native storage?

Cloud native storage solutions have five attributes.

1. Container granular operations

All volumes are virtually provisioned at container granularity. Operations such as snapshots, encryption, compression and others are not a cluster, or storage wide property, but rather per container. This is a key aspect, because it turns the operational experience over to the application owner (DevOps teams) and not the IT admin (so you can avoid slow, static and out-of-band storage provisioning).

How Portworx is cloud native?

Portworx’s volumes are dynamically and programmatically allocated and operated on directly by the end user, without involving IT.

2. Application pod aware

State should be managed at the pod level. An application is not just a single container, but rather a stack of related containers.

How Portworx is cloud native?

Portworx allows for data placement and management at pod granularity. An example is anti-affinity, where Portworx will automatically place volumes associated with various containers in a pod on failure-domain separated servers.  This makes your app naturally resilient to host and network failures.

3. Scheduler converged

Storage provisioning and operations should be handled through the same scheduler that manages the stateless parts of the app, not a separate CLI

How Portworx is cloud native?

Portworx is integrated with Kubernetes, Mesosphere DC/OS and Marathon, and Docker Swarm to allow for hyperconverged deployment of containers. Containers run on nodes that have the data local to the node when possible. This radically improves performance.

4. Cloud native provisioning

The experience of running data services should be as smooth in any public or private cloud.

How Portworx is cloud native?

Portworx integrates with various cloud providers to fully take advantage of a cloud-native experience. Examples include integration with AWS auto scaling groups (ASG).  With this integration you can scale a Portworx cluster directly using ASG rules. Another example is automatic snap-and-backup of data volumes to S3 buckets or similar object store. These are just a few of many features that enable IT to take advantage of the resident features offered by their cloud provider.

5. Software-defined, built for high-performance databases

Cloud native storage must be 100% software defined and optimized for high-performance database workloads like Cassandra, Hadoop, ElasticSearch, Kafka, MongoDB, and other modern databases

How Portworx is cloud native?

Portworx differentiates itself from other SDS products like CEPH and GlusterFS by being the only software-defined storage solution built specifically to run high-performance workloads where data locality is important. CEPH and GlusterFS were not built for these type of workloads. For example, CEPH stripes data across many servers increasing the network overhead for databases like Cassandra.  Portworx customer Dreamworks describes this as an “anti-pattern” that they wanted to avoid, opting instead for using local disks on their commodity server based-cloud.

So to recap Cloud Native Storage must have the following five attributes:

• Container granular operations
• Application pod aware
• Scheduler converged
• Cloud native provisioning
• Software-defined, built for high-performance databases

Are your apps using Cloud Native storage?  Are you already seeing the benefits?  Let us know in the comments.

If you want to learn more about cloud native storage, checkout this CNCF webinar on Cloud Native storage with Eric Han, VP of Product at Portworx.