Resize a MySQL container volume without taking your app offline

With today’s growing storage needs, DevOps teams regularly discover that they have outgrown the previously provisioned storage for their various services. Provisioning additional storage in today’s solutions would require DevOps to open a ticket for IT or storage admins to perform the task, which would end up taking hours, or even days. At the admin level, provisioning additional storage would require either migrating data to new larger volumes or performing multiple steps which could require additional time from the underlying storage arrays. That is downtime for some services that enterprises cannot afford.

Consider a business-critical MySQL server that was provisioned with a 50GB volume because the DevOps user didn’t think they would need more space any time soon. But then there was a burst of orders that came in, causing the database to run out of space and hence causing the company to not be able to fulfil any new orders.

At Portworx, our solution is to provide a programmatic way for DevOps to instantly increase the size of already-provisioned volumes without having to take either the application or the the underlying volumes offline. It involves running just one command which provisions more storage on the existing PX nodes, increases the size of the block device and then resizes the filesystem, all in a matter of seconds.

Here’s how you can check if your volume is full and then increase its size.

1. Run the volume inspect command.This should tell you the capacity of the volume and how much is used.

[email protected]:~$ sudo  /opt/pwx/bin/pxctl v i mysql_volume

            Volume                       :  658175664581050143
            Name                         :  mysql_volume
            Size                         :  14 GiB
            Format                       :  ext4
            HA                           :  3
            IO Priority                  :  LOW
            Creation time                :  Feb 25 22:52:17 UTC 2017
            Shared                       :  no
            Status                       :  up
            State                        :  Attached: 643ca9a6-972e-41d3-8a84-a2b27b21a1cc
            Device Path                  :  /dev/pxd/pxd658175664581050143
            Reads                        :  32
            Reads MS                     :  44
            Bytes Read                   :  352256
            Writes                       :  61
            Writes MS                    :  104
            Bytes Written                :  58744832
            IOs in progress              :  0
            Bytes used                   :  14 GiB
            Replica sets on nodes:
                        Set  0
                                    Node  :  192.168.56.101
                                    Node  :  192.168.56.106
                                    Node  :  192.168.56.105

You can see that the volume above is full, since all the space is used up.

2. Run the resize command to increase the size of the volume per your new requirement.

[email protected]:~$ sudo  /opt/pwx/bin/pxctl volume resize mysql_volume –size 100
Resize Volume: Volume mysql_volume resized to 100GB

3. Run volume inspect again and you’ll see that the size of the volume has been increased.

[email protected]:~$ sudo  /opt/pwx/bin/pxctl v i mysql_volume
            Volume                       :  658175664581050143
            Name                         :  mysql_volume
            Size                         :  100 GiB
            Format                       :  ext4
            HA                           :  3
            IO Priority                  :  LOW
            Creation time                :  Feb 25 22:52:17 UTC 2017
            Shared                       :  no
            Status                       :  up
            State                        :  Attached: 643ca9a6-972e-41d3-8a84-a2b27b21a1cc
            Device Path                  :  /dev/pxd/pxd658175664581050143
            Reads                        :  32
            Reads MS                     :  44
            Bytes Read                   :  352256
            Writes                       :  122
            Writes MS                    :  232
            Bytes Written                :  117497856
            IOs in progress              :  0
            Bytes used                   :  14GiB
            Replica sets on nodes:
                        Set  0
                                    Node    :  192.168.56.101
                                    Node    :  192.168.56.106
                                    Node    :  192.168.56.105

Check it out in action:

 

What’s the bottom line? With Portworx, DevOps can can reallocate storage for a running database with no support ticket or downtime. How cool is that?

Learn more about Portworx features and integration with MySQL in this three-minute video.

Dinesh Israni
Senior Software Engineer
Portworx