Unlocking the Power of Platform Engineering: Build Once, Port Anywhere, Run Everywhere

In the first blog of the Platform Engineering series, we discussed how Platform Engineering helps organizations realize the goal of continuous application development. In this blog, we will focus on how Platform Engineering and internal developer platforms allow developers to focus on building applications to solve business problems and increase revenue and customer retention, rather than focusing on learning how different cloud platforms operate.

What is Platform Engineering?

According to Humanitec, Platform Engineering is the discipline of designing and building internal developer platforms, toolchains, and workflows that enable self-service capabilities for software engineering organizations. In other words, Platform engineering is about moving the organization away from ticket ops – the practice where developers create tickets for tasks like infrastructure provisioning, and the operations team spends their time doing repetitive tasks – and towards complete self-service capabilities, where developers can use internal developer platforms to abstract away the complexities and just focus on their application code. The promise of Platform Engineering is to build golden paths for developers – a templated configuration of code and capabilities for accelerated development. Golden paths can range from a quick getting started tutorial for new developers on the team to CI/CD pipeline templates that developers can use to promote their application code from development to production.

For Platform Engineering teams to deliver a holistic Internal Developer Platform (IDP), they need to focus on not just providing a user interface along with a couple of sample scripts or Infrastructure as Code (IaC) manifest files but also delivering a complete platform that can be used by multiple applications teams or lines of businesses. Each organization is different, and we see organizations where there is a single platform team that builds one unified platform for all of its internal customers, while at the same time, we also see organizations that build different platforms for different parts of their business. Platform Engineering doesn’t have one final goal or end state but is a way for organizations to keep evolving and keep increasing the speed with which they bring applications to production.

Kubernetes and Platform Engineering

Source: https://tag-app-delivery.cncf.io/whitepapers/platforms/

The increased adoption of Kubernetes and the entire cloud native landscape has allowed organizations to use consistent tools and solutions to reduce the complexity of building their IDPs. This approach has the following benefits:

Consistency: Instead of platform teams reinventing the wheel for each section of the IDP stack shown in the above figure, they can leverage community-supported tools that work with large systems and different infrastructure stacks. By adopting containers to build applications, organizations can use the same tools for secrets management, for building CI/CD pipelines, and for Day 2 operations like monitoring and observability across different cloud environments.

Self-Service Access: Platform teams can use tools like Backstage, Crossplane, or Helm to deliver self-service capabilities to their application teams. Developers can follow golden paths defined around these tools to automate infrastructure provisioning rather than having to wait for the Operations team to manually build and deliver infrastructure resources to them.

Avoiding Cloud Lock-In: Utilizing Kubernetes and the cloud-native landscape, platform engineers can build automation capabilities that work with any cloud or on-premises infrastructure stack – delivering a cloud-agnostic solution, rather than using cloud-specific tools and creating vendor lock-in.

This almost seems too easy, right? The answer to that question is yes and no! Let’s start from the infrastructure provisioning layer – although platform teams can use tools like Crossplane, Terraform, or OpenTofu to provision Kubernetes clusters for applications teams to use, it doesn’t completely deliver on the promise of consistency and reducing cognitive load for developers. When using different cloud platforms, automation tools can help you provision Kubernetes clusters, but, the plugin-based architecture (CSI and CNI) that Kubernetes uses creates a lot of feature inconsistencies when it comes to storage and network resource provisioning and management. Let’s talk about some of the inconsistencies when it comes to storage resources that lead to the creation of custom scripts:

Lack of Consistent Storage Features: Each cloud provider has their own Container Storage Interface (CSI plugin) that allows users to leverage native storage services for containerized applications. This means that AWS gives platform teams access to an EBS CSI plugin by default that allows developers to dynamically provision ReadWriteOnce (RWO) or block volumes, but they will need to perform additional installation steps to get access to ReadWriteMany (RWX) volumes backed by Amazon EFS. If we compare this to Azure, RWO, and RWX storage classes are available by default in any Azure Kubernetes Service (AKS) cluster. This leads to platform teams having to build custom scripts to ensure that their developers get storage feature parity across Kubernetes clusters regardless of which cloud gets selected – which increases the operational overhead.

Replication and High Availability: Each cloud provider has their own way to deliver replication and high availability capabilities to the applications deployed. This might not seem like a big deal when using cloud environments for development and testing, but it matters when you want to run applications in production. Having inconsistent environments for dev/test and production might lead to unseen issues in production, and to work around this scenario, developers might have to start adding complexity to their application code and building high availability inside the application layer – which defeats the purpose of reducing the cognitive load for developers.

Day 2 Operations: Each cloud provider will have their own recommendations and best practices for how customers should perform operations like data protection, disaster recovery, cross-region resiliency, etc. This leads to additional customizations platform teams have to build into their operational processes when using multiple cloud services. Although a majority of the developers don’t want to worry about these Day 2 operations, the “You Build It, You Run It” philosophy has added more pressure on the developers and the operators to find a solution that works across clouds.

Portworx to the Rescue

Don’t throw away your Platform Engineering OKRs just yet! Similar to how Kubernetes provides a consistent orchestration layer for your containers and manages deployment and scaling, Portworx provides a uniform storage layer on top of Kubernetes to deliver a consistent set of features for your multi-cloud environments. Portworx can offer the following benefits not just for platform engineers, but also developers:

Consistent Storage Experience: Portworx is a cloud-native storage solution that can run on any Kubernetes cluster and can consume any block-based backing storage. This means that platform teams or application teams can run Portworx on Amazon EKS or on Azure AKS, and deliver a consistent set of features while at the same time abstracting away the discrepancies between the native storage offerings from different cloud providers. Portworx allows developers to trust that their infrastructure stack will also have the ability to dynamically provision RWO or RWX persistent volumes – and all they need to do is have their applications ask for the type of volume they need.

Resiliency and Reliability: Portworx provides in-cluster replication and high availability for cluster-level resiliency, and provides asynchronous and synchronous disaster recovery solutions for cross-regions and cross-cluster resiliency, respectively. Developers can let Portworx and Kubernetes handle the data and application high availability and focus on their application code to generate revenue for their business.

New Golden Paths: Using Portworx, platform engineers can build golden paths that allow developers to create a primary-secondary Kubernetes cluster setup, test their applications in a DR setup, and verify that it will work as expected when they push their applications to production. Portworx also allows developers to clone their production data and test their applications against real-world data and scale, before promoting changes to production. The best part, Portworx can be integrated with developers’ existing Continuous delivery frameworks like Flux or ArgoCD, so the entire process can be fully automated.

Database As A Service: Portworx also allows platform teams to leverage its database platform as a service offering (Portworx Data Services) and integrate PDS into their IDPs. This eliminates the need to identify, study, deploy, and validate multiple open-source operators to deploy each database, messaging queue, or other data service. Using Portworx Data Services, platform engineers can offer multiple databases as a service to their developers across different Kubernetes distributions and cloud platforms.

Application Migration: Although developers don’t have to worry about migrating their applications across different cloud regions or across multiple clouds if there are feature assumptions made while building the applications that can really lead to last minute duck tape fixes to the application to ensure that it works with the new cloud platform. Portworx provides a consistent experience that the developers can rely on, and it also allows platform teams to perform these cross-region or cross-cloud migrations seamlessly.

Summary

Platform Engineering services are critical in enhancing the developer experience, streamlining deployment processes that traditionally consume a significant chunk of time for Platform Engineering teams. By utilizing emerging technology, Platform Engineering solutions are built to provide amore efficient, self-service environment, enabling developers to manage their applications with greater productivity.

Platform Engineering is a journey for organizations to keep improving their process to do things better, but along this journey, platform teams need to watch out for time sinks that will slow things down. Portworx has enabled many Global 2000 companies with their journey, and can be your partner and help you realize the promises of Platform Engineering!

To learn more, check out the Portworx webinar focusing on building resilient data platforms for Kubernetes and continuous app development.