TL;DR
Bitnami’s Helm charts (which are available in the VMware Application Catalog as well) can now be moved across OCI repositories with two simple commands - no configuration or extra steps are needed; air gap-friendly. Keep reading to learn more.
Introducing the Distribution Tooling for the Helm OSS project
The distribution tooling for Helm, a.k.a. dt, is a new open source Helm Plugin that distributes a set of utilities for making offline work with Helm charts easier. It is meant to be used for creating reproducible and relocatable packages for Helm charts that can be easily moved across registries without hassles. This is particularly useful for distributing Helm charts into air-gapped environments, which is typically where businesses operating in highly regulated verticals prefer having their software run.
In its simplest form, dt can be used to create tarballs from any compatible Open Container Initiative(OCI)-compliant Helm chart, like Bitnami’s, with a single command:
# Wrap
$ helm dt wrap oci://docker.io/bitnamicharts/kibana
...
🎉 Helm chart wrapped into
"/Users/martinpe/workspace/kibana/kibana-10.4.8.wrap.tgz"
And then distribute that tarball into any OCI registry.
# Unwrap
$ helm dt unwrap kibana-10.4.8.wrap.tgz demo.goharbor.io/helm-plugin/ --yes
...
🎉 Helm chart unwrapped successfully: You can use it now by running "helm install oci://demo.goharbor.io/helm-plugin/kibana --generate-name"
These tarballs or wraps contain not only the Helm Chart and its metadata but also all the container images that you possibly need to run the Helm Chart independently on the configuration values.
But how does it work?
Automation and standardization come to the rescue. One of the advantages of Bitnami’s Helm charts is that they follow very rigorous standards and practices. All Helm charts are created the same way and share the same characteristics. This makes automation easier. But until now, we have not taken a step in the direction of annotating our Helm Charts with images. Something that a few software providers were already doing and was proposed and approved as a Helm Improvement Process (HIP).
annotations:
category: MachineLearning
licenses: Apache-2.0
images: |
- name: configurable-http-proxy
image: docker.io/bitnami/configurable-http-proxy:4.5.6-debian-11-r8
- name: jupyter-base-notebook
image: docker.io/bitnami/jupyter-base-notebook:4.0.2-debian-11-r8
- name: jupyterhub
image: docker.io/bitnami/jupyterhub:4.0.2-debian-11-r9
- name: os-shell
image: docker.io/bitnami/os-shell:11-debian-11-r43
We have fully embraced this proposal and from now on, all Bitnami Helm charts will contain annotations that specify the container images needed by a Helm chart along with the tags (e.g. MongoDB or JupyterHub). But we have gone a step further. In this other Helm Improvement Proposal we have proposed a mechanism to create a lock file that uses the Helm chart image annotations to create a point-in-time snapshot of the Helm chart. This is similar to what many programming languages offer for tracking dependencies. The proposed mechanism brings the ability to capture all the hashes of the container images that a Helm chart might need at a given point in time.
apiversion: v0
kind: ImagesLock
metadata:
generatedAt: "2023-08-04T13:36:09.398772Z"
generatedBy: Distribution Tooling for Helm
chart:
name: mariadb
version: 13.0.0
images:
- name: mariadb
image: docker.io/bitnami/mariadb:11.0.2-debian-11-r2
chart: mariadb
digests:
- digest: sha256:d3006a4d980d82a28f433ae7af316c698738ba29a5a598d527751cb9139ab7ff
arch: linux/amd64
- digest: sha256:3ec78b7c97020ca2340189b75eba4a92ccb0d858ee62dd89c6a9826fb20048c9
arch: linux/arm64
- name: mysqld-exporter
image: docker.io/bitnami/mysqld-exporter:0.15.0-debian-11-r5
chart: mariadb
digests:
- digest: sha256:6f257cc719f5bbde118c15ad610dc27d773f80216adabf10e315fbcaff078615
arch: linux/amd64
- digest: sha256:e0c141706fd1ce9ec5276627ae53994343ec2719aba606c1dc228f9290698fc1
arch: linux/arm64
- name: os-shell
image: docker.io/bitnami/os-shell:11-debian-11-r22
chart: mariadb
digests:
- digest: sha256:7082ebf5644cf4968ac635986ded132dd308c0b9c13138f093834f343cd47d7b
arch: linux/amd64
- digest: sha256:232ca2da59e508978543c8b113675c239a581938c88cbfa1ff17e9b6e504dc1a
arch: linux/arm64
This file can be used in very fancy and creative ways especially if we think about how it can be integrated into Helm itself but in dt, we have focused on a feature that is among the most demanded by our customers - distribution and relocation. Let’s for example look at an air gap scenario and how dt could be used.
The above figure shows a typical air gap scenario but with this new tool. An operator would point the dt tool to any Bitnami Helm chart and use it to create a wrap tarball that would then be picked up by an approved operator and taken into the air-gapped zone. From there dt can be pointed to the USB stick and to an OCI registry and will take care of moving the Helm chart along with all the container images, making sure that all references are properly updated so there is never an attempt to obtain images from the outside network.
What else can I do with this plugin?
Wrapping and unwrapping are the most powerful coarse-grained commands. But this plugin ships with many fine-grained commands as well, which can be used to better customize your distribution flows. Some of the most interesting things that the plugin provides at the time of writing this piece are:
- Ability to create single-architecture wraps that are lighter to store and distribute.
- Lock a Helm chart by generating the images lock. This has many possibilities to explore like Helm chart reproducibility without needing to use hashes which have many benefits but are obscure.
- Verify a Helm chart against the images lock.
- Relocate any Helm chart.
- Pull and push images from and to registries.
- Get information from any Helm chart at the digest level.
It’s also very important to note that the wrap generation does not only work with simple Helm charts but also with more complex charts that have subchart dependencies. dt takes care of automatically navigating the Helm chart structure and creating the appropriate images lock. Operators are also supported. As long as the Chart ships with annotated images, lock generation as well as distribution and relocation should work independently of the type of application.
I have many ideas for this! Can I contribute?
Of course, we do welcome contributions! We are very excited about the possibilities that this tool opens and with the prospect of the Helm community starting to test it and reporting any issues, ideas, improvements or directly sending pull requests for enhancements. Start by reading our README, getting familiar with this plugin and trying it yourself. Then feel free to reach out to us through GitHub issues.