Missing: Application Aware Infrastructure

Working on Kubernetes from its early days and co-founding Container Engine at Google, these products solved customers’ critical needs. But, I also met numerous customers who needed something else. What underlying infrastructure unlocks the promise of containers, especially if I’m on-premise?

By infrastructure, I mean the compute, network, and storage layers that gets offered up to an container orchestrator and ultimately– the application. Merely bolting onto virtual machine infrastructure– misses the win.

The arc of what we can do with containers is long. We have not scratched the surface and are collectively missing the point. Being a goal oriented person, let me try to summarize, as a set of goals and proposed report card for 2016. This is a public school, so no grade inflation.

Why should a container infrastructure be any different than the past?

Containers give us the opportunity to replatform our infrastructure, at a lower cost and higher productivity optimum. If we make it too complex, costs may be off worse than before. If we think too narrowly, productivity won’t improve. (Credit to this point goes to Frederick. Bastardization of his point, is my own.)

What is too thinking narrow?

Infrastructure platforms are two-sided markets. There are two primary user groups: (a) developers who write applications and  (b) devOps who deploy and manage those applications. Infrastructure will natively think about developers and devOps, as a duality.

Grading for 2016 will be based on the following experiences:

• Managed Impedance — tools & infrastructure will build holistically for a tussle, rooted in finding a healthier tension between app-dev and ops. As developers write their applications, they can express app requirements, and as ops deploys, ops will add their policy that gets expressed at execution time.Example: developers express a desired IO priority within app pieces, and infrastructure lets ops teams arbitrate, among competing users’ apps.
• Reversible — as apps move forward from dev, to test, through QA, and into production, tools & infrastructure will support porting, in the reverse.Example: instead of debugging in production, tools will help devOps & app dev reproduce in non-production environments. With CR/IU and snapshots, we can do better.
• Deeper Inspection — as density increases, node-centric tools lose their meaning. Bringing the issues closer to the business, infrastructure & tools will be able to show the tradeoffs and customer impact.Example: a single pane of glass to visualize both resource usage (cpu, disk, mem) and customer experiences (http latency, queuing), so that  infrastructure knobs give real-time feedback. I recently saw a demo of Sysdig, and I think Sysdig shows the benefits a unified view that spans apps and (slices) of infrastructure. Otherwise, operations teams have to stitch together and interpret from across views of 2-3 tools.
• Packaging stuff — everyone discusses the below, so I’ll briefly summarize my view at the end*.

In 2017, we should be moving onto ISVs applications easily packaged as distributed stacks. Prior, we will first go through building reactive infrastructure as platforms. I think these platforms will be purpose built;  naysayers will disagree. NP.

Feedback welcome.

*End:

• Composeable — you say microservices, I say composition. Applications call other applications. Microservices is fine term, but composition is the fulcrum that gets folks there.
• Full-stack — you say immutable, I just mean that how it was developed & tested continues, e.g. when other applications run side-by-side on the same server.Yes, mileage may vary, with issues like CVEs. That’s why Twistlock’s service is exciting.
• Portable — not just from a no-lock-in point but more simply from moving from laptops, through QA environments, and into production.

Did you enjoy what you just read? Learn more about Docker storage, Kubernetes storage or DC/OS storage.