Portworx Guided Hands On-Labs. Register Now

oke1

Getting Started with an Oracle Kubernetes Engine (OKE) Cluster

Let’s start by creating an Oracle Kubernetes Engine cluster on Oracle Cloud Infrastructure (OCI).

You can use the OCI Web UI to manually create an OKE cluster within your OCI compartment.

Or if you prefer, you may want to consider using my Terraform automation, which is available in GitHub.

Before you start, consult the Portworx documentation for the latest supported Kubernetes and Kernel versions.

For this blog, I will be using the Portworx 2.8 supported Kubernetes v1.20.11 and Oracle Linux 7.8.

My Terraform script created a new 3-node OKE cluster in the UK-London-1 region.

I have also created three 100GB Block Volumes (one per AD) and attached a block volume to each compute node again using Terraform.

Note: UK-London-region has 3 Availability Domains (ADs), but some regions are limited to a single AD; check OCI Region documentation.

To access the OKE cluster from your laptop, select Compartment, navigate to the Containers -> Clusters -> Cluster Details, click Access Cluster, and follow the instructions for Local Access.

% oci ce cluster create-kubeconfig --cluster-id ocid1.cluster.oc1.uk-london-1.aaaaaaaayss.... --file $HOME/.kube/config --region uk-london-1 --token-version 2.0.0

Note: Existing Kubeconfig file found at /Users/rekins/.kube/config and new config merged into it.

Using the kubectl get nodes, we can see the three workers running Kubernetes v1.20.11 and Oracle Linux 7.8:

% kubectl get nodes -o wide

NAME         STATUS   ROLES   AGE   VERSION    INTERNAL-IP   EXTERNAL-IP     OS-IMAGE                  KERNEL-VERSION                     CONTAINER-RUNTIME

10.0.1.157   Ready    node    48m   v1.20.11   10.0.1.157    132.145.33.86   Oracle Linux Server 7.8   4.14.35-1902.306.2.el7uek.x86_64   cri-o://1.20.2

10.0.1.163   Ready    node    48m   v1.20.11   10.0.1.163    144.21.51.173   Oracle Linux Server 7.8   4.14.35-1902.306.2.el7uek.x86_64   cri-o://1.20.2

10.0.1.205   Ready    node    48m   v1.20.11   10.0.1.205    132.145.69.16   Oracle Linux Server 7.8   4.14.35-1902.306.2.el7uek.x86_64   cri-o://1.20.2

Again, we can use kubectl get nodes, but with the –show-labels option this time, we can see further details.

% kubectl get nodes --show-labels 

NAME STATUS ROLES AGE VERSION LABELS 

10.0.1.157 Ready node 52m v1.20.11 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=VM.Standard2.1,beta.kubernetes.io/os=linux,displayName=oke-cgr3bekcbka-n5k6rizv7zq-sbgpx554ada-1,failure-domain.beta.kubernetes.io/region=uk-london-1,failure-domain.beta.kubernetes.io/zone=UK-LONDON-1-AD-3,hostname=oke-cgr3bekcbka-n5k6rizv7zq-sbgpx554ada-1,internal_addr=10.0.1.157,kubernetes.io/arch=amd64,kubernetes.io/hostname=10.0.1.157,kubernetes.io/os=linux,node-role.kubernetes.io/node=,node.info.ds_proxymux_client=true,node.info/compartment.id_prefix=ocid1.compartment.oc1,node.info/compartment.id_suffix=aaaaaaaauktztjuam57uhird7up5gxp32svboo6bmjfxvotkja75tfndc2sq,node.info/compartment.name=tf-compartment,node.info/kubeletVersion=v1.20,oci.oraclecloud.com/fault-domain=FAULT-DOMAIN-2,oke.oraclecloud.com/node.info.private_subnet=false,oke.oraclecloud.com/node.info.private_worker=true,oke.oraclecloud.com/tenant_agent.version=1.38.5-8f5c194266-712,px/enabled=true,px/metadata-node=true 10.0.1.163 Ready node 52m v1.20.11 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=VM.Standard2.1,beta.kubernetes.io/os=linux,displayName=oke-cgr3bekcbka-n5k6rizv7zq-sbgpx554ada-0,failure-domain.beta.kubernetes.io/region=uk-london-1,failure-domain.beta.kubernetes.io/zone=UK-LONDON-1-AD-2,hostname=oke-cgr3bekcbka-n5k6rizv7zq-sbgpx554ada-0,internal_addr=10.0.1.163,kubernetes.io/arch=amd64,kubernetes.io/hostname=10.0.1.163,kubernetes.io/os=linux,last-migration-failure=get_kubesvc_failure,node-role.kubernetes.io/node=,node.info.ds_proxymux_client=true,node.info/compartment.id_prefix=ocid1.compartment.oc1,node.info/compartment.id_suffix=aaaaaaaauktztjuam57uhird7up5gxp32svboo6bmjfxvotkja75tfndc2sq,node.info/compartment.name=tf-compartment,node.info/kubeletVersion=v1.20,oci.oraclecloud.com/fault-domain=FAULT-DOMAIN-1,oke.oraclecloud.com/node.info.private_subnet=false,oke.oraclecloud.com/node.info.private_worker=true,oke.oraclecloud.com/tenant_agent.version=1.38.5-8f5c194266-712,px/enabled=true,px/metadata-node=true 10.0.1.205 Ready node 52m v1.20.11 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=VM.Standard2.1,beta.kubernetes.io/os=linux,displayName=oke-cgr3bekcbka-n5k6rizv7zq-sbgpx554ada-2,failure-domain.beta.kubernetes.io/region=uk-london-1,failure-domain.beta.kubernetes.io/zone=UK-LONDON-1-AD-1,hostname=oke-cgr3bekcbka-n5k6rizv7zq-sbgpx554ada-2,internal_addr=10.0.1.205,kubernetes.io/arch=amd64,kubernetes.io/hostname=10.0.1.205,kubernetes.io/os=linux,node-role.kubernetes.io/node=,node.info.ds_proxymux_client=true,node.info/compartment.id_prefix=ocid1.compartment.oc1,node.info/compartment.id_suffix=aaaaaaaauktztjuam57uhird7up5gxp32svboo6bmjfxvotkja75tfndc2sq,node.info/compartment.name=tf-compartment,node.info/kubeletVersion=v1.20,oci.oraclecloud.com/fault-domain=FAULT-DOMAIN-3,oke.oraclecloud.com/node.info.private_subnet=false,oke.oraclecloud.com/node.info.private_worker=true,oke.oraclecloud.com/tenant_agent.version=1.38.5-8f5c194266-712,px/enabled=true,px/metadata-node=true

Creating a Specification Using Portworx Enterprise

Now that we have our Kubernetes Cluster up and running, we can head over to https://central.portworx.com/ to create a Portworx specification using the wizard.

For this post, I will be using the Portworx Enterprise edition for my three OKE nodes.

And I’ll be using a Kubernetes Operator as my deployment method.

Basic Tab

Storage Tab

On the Storage Tab, we need to select On-Premises, as the Spec Generator currently does not include a wizard for Oracle Cloud Infrastructure (OCI).

Network Tab

Customize Tab

From the Operator wizard, we can decide to Download or Save Spec.

Operator Tab

Installing the Portworx Operator

From our shell, we’ll install the Portworx Operator using kubectl apply:

% kubectl apply -f 'https://install.portworx.com/2.8?comp=pxoperator'

serviceaccount/portworx-operator created

podsecuritypolicy.policy/px-operator created

clusterrole.rbac.authorization.k8s.io/portworx-operator created

clusterrolebinding.rbac.authorization.k8s.io/portworx-operator created

deployment.apps/portworx-operator created

Applying the Specification

Apply the specification created from PX-Central:

% kubectl apply -f 'https://install.portworx.com/2.8?operator=true&mc=false&kbver=&b=true&c=px-cluster-21a6ec62-0ed7-48b8-b6ac-b6833005b533&stork=true&csi=true&mon=true&st=k8s&promop=true'

storagecluster.core.libopenstorage.org/px-cluster-21a6ec62-0ed7-48b8-b6ac-b6833005b533 created

Monitoring Progress

Enter the kubectl get pods command below and wait until all Portworx nodes show a status of running:

% kubectl get pods -n kube-system -l name=portworx --watch

NAME                                                    READY   STATUS              RESTARTS   AGE

px-cluster-21a6ec62-0ed7-48b8-b6ac-b6833005b533-c6tq6   0/3     ContainerCreating   0          82s

px-cluster-21a6ec62-0ed7-48b8-b6ac-b6833005b533-jj5gc   0/3     ContainerCreating   0          82s

px-cluster-21a6ec62-0ed7-48b8-b6ac-b6833005b533-xqfw9   0/3     ContainerCreating   0          82s

px-cluster-21a6ec62-0ed7-48b8-b6ac-b6833005b533-xqfw9   1/3     Running             0          2m7s

px-cluster-21a6ec62-0ed7-48b8-b6ac-b6833005b533-c6tq6   1/3     Running             0          2m13s

px-cluster-21a6ec62-0ed7-48b8-b6ac-b6833005b533-jj5gc   1/3     Running             0          2m14s

px-cluster-21a6ec62-0ed7-48b8-b6ac-b6833005b533-jj5gc   2/3     Running             0          2m30s

px-cluster-21a6ec62-0ed7-48b8-b6ac-b6833005b533-c6tq6   2/3     Running             0          2m35s

px-cluster-21a6ec62-0ed7-48b8-b6ac-b6833005b533-xqfw9   2/3     Running             0          2m38s

px-cluster-21a6ec62-0ed7-48b8-b6ac-b6833005b533-jj5gc   3/3     Running             0          3m42s

px-cluster-21a6ec62-0ed7-48b8-b6ac-b6833005b533-jj5gc   3/3     Running             0          3m43s

px-cluster-21a6ec62-0ed7-48b8-b6ac-b6833005b533-c6tq6   2/3     Running             0          3m48s

px-cluster-21a6ec62-0ed7-48b8-b6ac-b6833005b533-xqfw9   2/3     Running             0          3m55s

px-cluster-21a6ec62-0ed7-48b8-b6ac-b6833005b533-c6tq6   2/3     Running             0          3m55s

px-cluster-21a6ec62-0ed7-48b8-b6ac-b6833005b533-c6tq6   2/3     Running             0          3m55s

px-cluster-21a6ec62-0ed7-48b8-b6ac-b6833005b533-xqfw9   2/3     Running             0          4m1s

px-cluster-21a6ec62-0ed7-48b8-b6ac-b6833005b533-xqfw9   2/3     Running             0          4m1s

px-cluster-21a6ec62-0ed7-48b8-b6ac-b6833005b533-c6tq6   3/3     Running             0          4m4s

px-cluster-21a6ec62-0ed7-48b8-b6ac-b6833005b533-xqfw9   3/3     Running             0          4m9s

px-cluster-21a6ec62-0ed7-48b8-b6ac-b6833005b533-jj5gc   3/3     Running             0          4m12s

% PX_POD=$(kubectl get pods -l name=portworx -n kube-system -o jsonpath='{.items[0].metadata.name}')

% echo $PX_POD

px-cluster-21a6ec62-0ed7-48b8-b6ac-b6833005b533-c6tq6

Now we need to confirm that we are using Portworx 2.8:

% kubectl exec -it $PX_POD -n kube-system -- /opt/pwx/bin/pxctl -v          

Defaulted container "portworx" out of: portworx, csi-node-driver-registrar, telemetry

pxctl version 2.8.0.0-1ef62f8

And then we’ll check the status of the Portworx cluster:

% kubectl exec -it $PX_POD -n kube-system -- /opt/pwx/bin/pxctl status

Defaulted container "portworx" out of: portworx, csi-node-driver-registrar, telemetry

Status: PX is operational

Telemetry: Healthy

License: Trial (expires in 31 days)

Node ID: b2a4cd36-f0d0-4c06-9de2-378a68dafe5a

IP: 10.0.1.205 

  Local Storage Pool: 1 pool

POOL IO_PRIORITY RAID_LEVEL USABLE USED STATUS ZONE REGION

0 MEDIUM raid0 100 GiB 7.1 GiB Online UK-LONDON-1-AD-1 uk-london-1

Local Storage Devices: 1 device

Device Path Media Type Size Last-Scan

0:1 /dev/sdb STORAGE_MEDIUM_MAGNETIC 100 GiB 09 Oct 21 13:05 UTC

* Internal kvdb on this node is sharing this storage device /dev/sdb  to store its data.

total - 100 GiB

Cache Devices:

 * No cache devices

Cluster Summary

Cluster ID: px-cluster-21a6ec62-0ed7-48b8-b6ac-b6833005b533

Cluster UUID: faf0a08a-d456-436e-81c8-7d6dbab1525e

Scheduler: kubernetes

Nodes: 3 node(s) with storage (3 online)

IP ID SchedulerNodeName Auth StorageNode Used Capacity Status StorageStatus Version Kernel OS

10.0.1.163 d5e96ff9-94d0-4004-9753-6a2a4568a79c 10.0.1.163 Disabled Yes 7.1 GiB 100 GiB Online Up2.8.0.0-1ef62f8 4.14.35-1902.306.2.el7uek.x86_64 Oracle Linux Server 7.8

10.0.1.205 b2a4cd36-f0d0-4c06-9de2-378a68dafe5a 10.0.1.205 Disabled Yes 7.1 GiB 100 GiB Online Up (This node) 2.8.0.0-1ef62f8 4.14.35-1902.306.2.el7uek.x86_64 Oracle Linux Server 7.8

10.0.1.157 3dea3719-f7b3-4cfa-8d4c-b934f615e357 10.0.1.157 Disabled Yes 7.1 GiB 100 GiB Online Up2.8.0.0-1ef62f8 4.14.35-1902.306.2.el7uek.x86_64 Oracle Linux Server 7.8

Warnings: 

 WARNING: Insufficient CPU resources. Detected: 2 cores, Minimum required: 4 cores

 WARNING: Persistent journald logging is not enabled on this node.

 WARNING: Internal Kvdb is not using a dedicated drive on nodes [10.0.1.157 10.0.1.205]. This configuration is not recommended for production clusters.

Global Storage Pool

Total Used     :  21 GiB

Total Capacity :  300 GiB

Next, we’ll check the status of the Portworx storage nodes:

% kubectl -n kube-system get storagenodes -l name=portworx

NAME         ID                                     STATUS   VERSION           AGE

10.0.1.157   3dea3719-f7b3-4cfa-8d4c-b934f615e357   Online   2.8.0.0-1ef62f8   53m

10.0.1.163   d5e96ff9-94d0-4004-9753-6a2a4568a79c   Online   2.8.0.0-1ef62f8   53m

10.0.1.205   b2a4cd36-f0d0-4c06-9de2-378a68dafe5a   Online   2.8.0.0-1ef62f8   53m

Finally, using kubectl get sc, we can see the OCI & Portworx storage classes:

% kubectl get sc

NAME                             PROVISIONER                       RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE

oci (default)                    oracle.com/oci                    Delete          Immediate              false                  165m

oci-bv                           blockvolume.csi.oraclecloud.com   Delete          WaitForFirstConsumer   false                  165m

px-db                            kubernetes.io/portworx-volume     Delete          Immediate              true                   59m

px-db-cloud-snapshot             kubernetes.io/portworx-volume     Delete          Immediate              true                   59m

px-db-cloud-snapshot-encrypted   kubernetes.io/portworx-volume     Delete          Immediate              true                   59m

px-db-encrypted                  kubernetes.io/portworx-volume     Delete          Immediate              true                   59m

px-db-local-snapshot             kubernetes.io/portworx-volume     Delete          Immediate              true                   59m

px-db-local-snapshot-encrypted   kubernetes.io/portworx-volume     Delete          Immediate              true                   59m

px-replicated                    kubernetes.io/portworx-volume     Delete          Immediate              true                   59m

px-replicated-encrypted          kubernetes.io/portworx-volume     Delete          Immediate              true                   59m

stork-snapshot-sc                stork-snapshot                    Delete          Immediate              true                   59m

Summary

From the example above, we can see that the Portworx Enterprise deployment is using the three OCI compute instances as Portworx storage nodes and has discovered the 100 GB volumes for a total storage capacity of 300GB.

Share
Subscribe for Updates

About Us
Portworx is the leader in cloud native storage for containers.

ron_ekins_

Ron Ekins

Principal Field Solutions Architect | Office of the CTO, Pure Storage
link
OracleFI
September 24, 2021 How To
Getting started with the Oracle MySQL Kubernetes Operator with Portworx on OCI
Ron Ekins
Ron Ekins
link
TanzuPXCentral
August 26, 2021 How To
Deploying Portworx on VMware Tanzu
Bhavin Shah
Bhavin Shah
link
one
September 2, 2020 Technical Insights
OpenShift Virtualization with Portworx
Ryan Wallner
Ryan Wallner