Getting started with Amazon EKS Anywhere and Portworx for your hybrid cloud deployments

Contact Sales

How To

on-Pream-Datacenter

AWS announced general availability for Amazon Elastic Kubernetes Service (Amazon EKS) Anywhere in September 2021. Amazon EKS Anywhere provides another deployment option for Amazon EKS that enables users to deploy Kubernetes clusters on-premises running on VMware vSphere, with support for bare metal nodes coming later this year. EKS Anywhere is built using EKS Distro, an open-source distribution for Kubernetes used by Amazon EKS, which means that users get a consistent experience across their cloud and on-premises environments.

In this blog, we look at how you can deploy EKS Anywhere on your VMware vSphere environment and then install Portworx as the Kubernetes storage layer so you get a consistent set of data management features across your cloud and on-premises environments. Installing and using Portworx as the Kubernetes data management layer on Amazon EKS and EKS Anywhere clusters allows you to leverage features like replication, high availability, security, role-based access control, snapshots, io-profiles, and unified block and file storage across your hybrid cloud deployment.

To get started with EKS Anywhere, we will create an admin machine to run the eksctl-anywhere utility, and then we will create a production workload cluster.

For your admin machine, you can either use Mac OS (10.15) or deploy a new Ubuntu Server (20.04.2 LTS) VM on your VMware vSphere cluster.
The admin machine should meet the following prerequisites:
- Docker 20.x.x: If you are using Ubuntu, use the Docker CE installation instructions and not Snap installation.
- 4 CPU cores / 16GB memory / 30GB free disk space
Once the admin machine is up and running, install eksctl and the eksctl-anywhere plugin.

### Install eksctl on the admin machine
curl "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" \
    --silent --location \
    | tar xz -C /tmp
sudo mv /tmp/eksctl /usr/local/bin/

### Install eksctl-anywhere plugin
export EKSA_RELEASE="0.7.0" OS="$(uname -s | tr A-Z a-z)" RELEASE_NUMBER=5
curl "https://anywhere-assets.eks.amazonaws.com/releases/eks-a/${RELEASE_NUMBER}/artifacts/eks-a/v${EKSA_RELEASE}/${OS}/amd64/eksctl-anywhere-v${EKSA_RELEASE}-${OS}-amd64.tar.gz" \
    --silent --location \
    | tar xz ./eksctl-anywhere
sudo mv ./eksctl-anywhere /usr/local/bin/

### Verify installed eksctl anywhere version
eksctl anywhere version

Next, let’s go ahead and confirm that your VMware environment meets the following prerequisites:
- A vSphere 7+ environment running vCenter with capacity to deploy 6-10 VMs
- DHCP service running in vSphere environment in the primary VM network for your workload cluster
- One network in vSphere to use for the cluster. This network must have inbound access into vCenter.
- An Ubuntu OVA imported into vSphere and converted into a template for the workload VMs. For this blog, we have used the Ubuntu template to run EKS Anywhere nodes.
- User credentials to create VMs and attach networks, etc.
- One IP address routable from the cluster but excluded from the DHCP offering
- Each VM will require 2 vCPUs / 8GB RAM / 25GB disk at a minimum.
- The administrative machine and the target workload environment will need network access to. These endpoints allow the installer to download the necessary binaries.
  - public.ecr.aws
  - anywhere-assets.eks.amazonaws.com (to download the EKS Anywhere binaries, manifests and OVAs)
  - distro.eks.amazonaws.com (to download EKS Distro binaries and manifests)
  - d2glxqk2uabbnd.cloudfront.net (for EKS Anywhere and EKS Distro ECR container images)
  - api.github.com (only if GitOps is enabled)
Next, let’s generate a management cluster config. This command generates a configuration file that needs to be customized to match your environment-specific details.

CLUSTER_NAME=mgmt
eksctl anywhere generate clusterconfig $CLUSTER_NAME \
   --provider vsphere > eksa-mgmt-cluster.yaml

Here is a sample configuration that was customized for our lab environment:

apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: Cluster
metadata:
  name: mgmt-eks-demo-1
spec:
  clusterNetwork:
    cni: cilium
    pods:
      cidrBlocks:
      - 192.168.0.0/16
    services:
      cidrBlocks:
      - 10.96.0.0/12
  controlPlaneConfiguration:
    count: 3
    endpoint:
      host: "10.21.143.55"
    machineGroupRef:
      kind: VSphereMachineConfig
      name: mgmt
  datacenterRef:
    kind: VSphereDatacenterConfig
    name: mgmt-dc
  externalEtcdConfiguration:
    count: 3
    machineGroupRef:
      kind: VSphereMachineConfig
      name: mgmt
  kubernetesVersion: "1.21"
  managementCluster:
    name: mgmt
  workerNodeGroupConfigurations:
  - count: 3
    machineGroupRef:
      kind: VSphereMachineConfig
      name: mgmt
    name: md-0
---
apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: VSphereDatacenterConfig
metadata:
  name: mgmt-dc
spec:
  datacenter: "eks-px"
  insecure: false
  network: "VM Network"
  server: "<<vcenter IP>>"
  thumbprint: "<<vcenter thumbprint>>"
---
apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: VSphereMachineConfig
metadata:
  name: mgmt
spec:
  datastore: "EKS-Infra-1"
  diskGiB: 25
  folder: "eksvm"
  memoryMiB: 8192
  numCPUs: 2
  osFamily: ubuntu
  resourcePool: "/eks-px/host/eks-cluster/Resources/eks-rp"
  users:
  - name: capv

Next, let’s set environment variables for your vCenter credentials on your EKS Anywhere admin machine. The deployment automation verifies these variables have been set and passes these credentials to the EKS Anywhere VMs that get deployed. Ensure that these credentials have the needed privileges.

export EKSA_VSPHERE_USERNAME='administrator@vsphere.local'
export EKSA_VSPHERE_PASSWORD='t0p$ecret'

Next, let’s create the management cluster using the following command. You can also add a -v 6 or -v 9 to get verbose output.

eksctl anywhere create cluster -f eksa-mgmt-cluster.yaml

This creates a bootstrap cluster on the admin machine, followed by deployment of ETCD VMs, control plane VMs and worker VMs on your VMware environment. Once all the VMs are up and running, the cluster management is moved from the bootstrap cluster to the new VM-based management cluster. Once the migration is successful, the bootstrap cluster is deleted, and you will see a cluster creation successful message.