Many of the leading Docker images such as MongoDB, MySQL and WordPress require data…
March 14, 2017
Automating Container Storage for AWS and Packet using Terraform and Portworx (aka “Go Get Terraporx”!)
The Importance of Automation
Portworx provides container-granular storage that is scheduler integrated. Working with customers running on-prem and in public clouds, the importance of automation is clear. And the one framework that customers consistently request integration with is Terraform from HashiCorp. Terraform is probably the best automation framework that allows for DevOps Programmable Infrastructure — and most importantly, in a cloud-agnostic manner that works equally well for (___ fill in your own) cloud provider platform.
The Importance of Community
And like so many of the advances of which we regularly take advantage — it really takes a community. Good ideas thrive, given good light and proper feeding.
Earlier this month, we announced LCFS, for speeding up Docker builds and commits. Today we are making public our “Terraporx” repository for automating Portworx through Terraform. The name “Terraporx” actually harkens back to to the early days of GoLang-based Portworx development:
Deploying Portworx using Terraporx
There are various ways to automate the deployment and startup Portworx using Terraporx. The Terraform examples in this repository include three different methods:
- “docker run”. The “centos_type01” script provides an example of this vanilla “docker run” approach.
- “systemd”. The “coreos” script shows how to deploy Portworx through “systemd”. This also shows how to use “cloud-init” to create the corresponding systemd service file.
- Mesosphere Universe. For Mesosphere/DCOS environments Portworx can be deployed through the Mesosphere Universe catalog.
So what’s the prefered way? That’s the point: whatever works best/easiest for you should be considered first.
Understanding Terraporx Details and Deployment Options
Okay, this is a technical blog, so let’s get into the weeds a bit and look at the syntax of Terraporx.
1. “Here Doc”. One format for providing “user_data” is inline as a string. For example:
user_data = "#cloud-config\n\nmanage_etc_hosts: \"localhost\"\nssh_authorized_keys:\n - \"${file("${var.dcos_ssh_public_key_path}")}\"\n"
It’s kind of hard to read and even harder to parse. Which is why I found “Here Docs” to be quite useful. Especially for longer format “user_data”, as demonstrated next below with “cloud-init”
2. Cloud-Init. Cloud-init has emerged as the default way to provide bootstrap information for a new host. Cloud-init is the defacto multi-distribution package that handles early initialization of a cloud instance.
Cloud-init configuration data is among the most typical use cases for “user_data”. Here is the sample code from the coreos Terraform script, using a “Here Doc” format:
user_data = <<EOF
#cloud-config
coreos:
update:
reboot-strategy: off
etcd2:
discovery: "${var.etcd_discovery_url}"
advertise-client-urls: http://$private_ipv4:2379
initial-advertise-peer-urls: http://$private_ipv4:2380
listen-client-urls: http://0.0.0.0:2379,http://0.0.0.0:4001
listen-peer-urls: http://$private_ipv4:2380
units:
- name: etcd2.service
command: start
- name: portworx.service
command: start
content: |
[Unit]
Description=Portworx Container
Wants=docker.service
After=docker.service
[Service]
Restart=always
TimeoutSec=0
ExecStartPre=-/usr/bin/docker stop %n
ExecStartPre=-/usr/bin/docker rm -f %n
ExecStartPre=/usr/bin/bash -c "/usr/bin/systemctl set-environment DMPATH=`multipath -ll|grep dm-| awk '{print $2}'`"
ExecStart=/usr/bin/docker run --net=host --privileged=true \
--cgroup-parent=/system.slice/portworx.service \
-v /run/docker/plugins:/run/docker/plugins \
-v /var/lib/osd:/var/lib/osd:shared \
-v /dev:/dev \
-v /etc/pwx:/etc/pwx \
-v /opt/pwx/bin:/export_bin:shared \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /var/cores:/var/cores \
-v /lib/modules:/lib/modules \
--ipc=host \
--name=%n \
portworx/px-dev -s /dev/$${DMPATH} -d bond0 -m bond0 -k etcd://127.0.0.1:2379 -c px-cluster-coreos
KillMode=control-group
ExecStop=/usr/bin/docker stop -t 10 %n
[Install]
WantedBy=multi-user.target
EOF
It’s interesting to note here are the multiple types of variable interpolation:
discovery: “${var.etcd_discovery_url}”
Interpolates from the Terraform context
advertise-client-urls: http://$private_ipv4:2379
Interpolates from the CoreOS Cloud-init context.
ExecStartPre=/usr/bin/bash -c “/usr/bin/systemctl set-environment DMPATH=`multipath -ll|grep dm-| awk ‘{print $2}’`”
[…]
portworx/px-dev -s /dev/$${DMPATH} -d bond0 -m bond0 -k etcd://127.0.0.1:2379 -c px-cluster-coreos
Sets and then uses the DMPATH variable in the ‘systemd’ context.
3. Local disks vs EBS and attached volumes. This realm required subdividing the logic. Portworx works equally well with both deployment models. But the EBS/external volume model does require some extra coding for the volume create-and-attach. Having slightly “fatter” servers that include local disk will make the Terraform scripts a bit easier — but if the cost is an obstacle, then the EBS/external methods can be used. As always — whatever works best for you.
“Phil and the Blanks”
We intentionally left a number of targets blank in this effort. For example, we have methods for deploying Mesosphere/DCOS, but we don’t (yet) have methods for deploying Kubernetes. And we have working examples for Packet.net and AWS, but we don’t yet have methods for deploying in GCP, Digital Ocean or Azure. If you have experience on these fronts, or if you’ve already developed your own TF scripts, then you are welcome and encouraged to submit them here. And should this repo be ordered first by cloud provider and then by scheduler — or the other way around? What do you think? Please let us know.
This is where things get exciting. This is where we get to launch something new and watch it grow. Let’s give our little friend some good light and proper feeding 😉
