TL;DR: With the Volume Manager UI you can take full control of your Kubernetes cluster’s PersistentVolumeClaims. You can list which ones exist in each namespace, delete them, and create new ones either empty or from existing snapshots. Most importantly, you can browse, edit, and upload or download files with only a few clicks. All of the above from inside an intuitive, user friendly UI that takes care of all the underneath technical details and allows you to seamlessly perform your data operations on top of PersistentVolumeClaims.
Introduction
Most software solutions and projects running on top of a Kubernetes cluster will eventually need to either store or load data at some point. Kubernetes provides the PersistentVolume — PersistentVolumeClaim mechanism to allow applications to consume the cluster’s provided storage infrastructure to accommodate such needs.
But despite having a solid mechanism for leveraging the cluster’s storage infrastructure, the UX of creating PVCs, initializing them with data and inspecting them at any point in time leaves a lot to be desired. The users will need to have a configured kubectl that can access that cluster, be familiar with a terminal and the commands that are required to create a PVC. On top of that, they should also be aware of how to deploy and expose a container on top of that PVC, in order to browse and modify its file contents. This adds a significant complexity for using PVCs for handling application data.
This leads to two possible solutions for handling application data. On the one hand the applications could consume some third-party storage service that would handle the data. However, this would be adding some complexity on the application’s code since it would require additional code for communicating and interacting with that service. On the other hand the use of PVCs is more transparent since the applications would just store their data on a specific path where the PVC would be mounted. But still, maintainers should have strong K8s knowledge to be able to handle and manage the data inside the PVCs themselves.
We believe that the solution with the best UX and at the same time transparent to the applications would be to encourage the use of PVCs, but also to make it easier to manage and work with them. The most important missing part for achieving this workflow is the easy management of PVCs. This led us to the creation of the Volume Manager UI, a web app that aims to ease the use of PVCs. It achieves that by taking care of all the low level technical details required to list, create, and delete PVCs as well as allow users to browse, edit, download or upload files to them via an intuitive UI.
In this blog post we are going to elaborate on Volume Management UI’s major features and the workflows it allows, as well as how existing projects’ UX could be improved by including this component.
Simplifying PVC management
The aim of this web app is to simplify the user interaction with PVCs and how they manage their lifecycle via a UI.
In order to achieve this, the web app needs to expose to the user the state of the PVCs in the different Kubernetes namespaces in a central dashboard page.
From there the user can see the status of each PVC as well as other information such as its capacity, storage class and access mode. The user can also delete PVCs that they don’t need anymore. The app will restrict the user from deleting a PVC if it is being used from a Pod in order to prevent data loss.
The web app contains a minimalist form for allowing the users to create either new empty PVCs or initialized with data from snapshots. The form allows the user to customize their PVC’s attributes such as its capacity, storage class, and access mode.
All of the above would be a tedious process that would require the creation of yaml files as well as the use of kubectl and a terminal to create the new resources. Volume Manager UI makes the life of end users much easier by hiding all the K8s PVC internals. Now one can create, delete, and list PVCs via an elegant dashboard, without having to run kubectl commands or create and edit yaml files.
Browsing and editing PVC contents
But Volume Manager UI is not just a GUI abstraction on top of kubectl for managing PVCs. It also provides a new unique feature, which is the ability to browse the file contents of any PVC directly from the browser and only with a few clicks!
The user clicks on the folder icon, a spinner appears for some seconds while the backend makes preparations for viewing the PVC, and after a little while the spinner will be replaced by a folder with a search icon. The user will then be able to start browsing the contents of their PVC on a new browser window by pressing this button.
From this window the user can:
- View and navigate the file system tree of that PVC
- Download files and folders of the PVC directly to their local machine
- Open and inspect files
- Upload files and folders from their local machine to the PVC by drag and dropping them inside of the folder they want, in the Viewer window
- Upload files from different cloud sources
- Have a terminal emulator from the Viewer window for more advanced file manipulation
This can make the process of managing the data that lives inside PVCs much easier. One can just browse an existing PVC, view and edit its contents fast from the UI, without having to go to a terminal, run kubectl commands to discover the PVC, spin up a new container, attach the Volume, and connect to that container to find out what’s happening.
This is achieved by creating a Viewer Pod that mounts the requested PVC and gets exposed to the end user via a Service. This Viewer Pod is always able to mount the PVC even if other Pods are using it, and independently of the PVC’s access mode. This way the users can immediately view the contents of a PVC without disrupting their existing workflow or having to stop Pods that use that PVC.
This workflow that the web app allows also makes the process of instantiating Volumes and ingesting data much easier. There are a lot of cases when one wants to prepare a PVC with the right data for later use. The Volume Manager UI allows you to either:
- clone an existing snapshot into a new PVC, which in that case will already contain all the data you need, or
- create an empty PVC and then use the Viewer to upload data to it from different sources.
All the above with just a few clicks!
Give it a try
This web app could either be deployed as a standalone component in a Kubernetes platform to allow users to manage their PVCs or it could be a part of existing solutions deployed on top of Kubernetes.
To showcase the power of this new app we’ve integrated it with Kubeflow to make it easier for Data Scientists to manage the data they use inside their Jupyter Notebooks, all from within Kubeflow’s UI and without the need to run a single kubectl command. You can give it a try by installing MiniKF via the Google Cloud Marketplace or on your laptop.
We would love to hear what you think of it and what other features you would like to see!
We have also created some bite-sized videos, so that you can see this new web app in action. In these videos, you will learn how to
- Create a new, empty K8s Volume
- Create a K8s Volume from an existing Rok snapshot
- Upload files to a K8s Volume by drag and drop
- Download files from a K8s Volume to your laptop
- Upload files and folders to your Notebooks
- Create a time machine for your K8s Volumes