New Package Repository API and UI in version 2.5.0
Authored by Pepe Baena, R&D Manager at VMware
The Kubeapps team is continuously looking for ways to improve the user experience when browsing and deploying applications on Kubernetes clusters. The new Kubeapps release ﹣ version 2.5.0 ﹣ extends the repository management feature by including Carvel packages. Now, users will be able to configure package repositories for both Helm and Carvel packages in just a few clicks.
Keep reading to learn more about the key points that make it possible: a new Package Repository API and a revamped form that allows you to easily configure package repositories directly from the UI.
Context of this Enhancement
Before releasing version 2.4.3, Kubeapps was mainly focused on Helm chart management, and as a consequence, was tied closely to Helm repositories and OCI registries for Helm chart storage. When we added support for Carvel packages and Helm charts via Flux, it became necessary to standardize the management of different package repository types in Kubeapps. This brought the following challenges for the team:
- We needed to design and implement a new API, similar to the Kubeapps core API for managing packages, but in this case for managing package repositories as well as ensuring that the new API satisfies the requirements of the existing Helm functionality.
- We needed to implement a Package Repository API for the different plugins (Helm, Flux, and Carvel).
- We needed to revamp the current UI form to standardize the experience when configuring existing repositories (Helm and OCI) as well as Carvel and Flux sources.
Let’s see in detail what implementation strategy and steps were performed to successfully add this new feature to Kubeapps.
Design and Implementation of a New Package Repository API
The first step we followed was to design a new Package Repository API that allowed us to manage new package repositories on Kubeapps. The main component of the domain model is the PackageRepository (a repository of Kubernetes application packages that are available for installation). A repository may be served by any one of the backends supported by the configured plugins (Helm, Flux, and Carvel plugins), including, but not limited to:
- a Helm chart repository (e.g. https://charts.bitnami.com/bitnami ) – via Helm or via Flux plugins
- an OCI helm repository ﹣ via Helm plugin or via Flux plugin
- a Git repository containing artifacts ﹣ gzip compressed TAR archives, via Flux plugin
- object storage buckets containing artifacts coming from S3 compatible storage such as Minio, Amazon S3, Google Cloud Storage, Alibaba Cloud OSS, and others ﹣ via Flux plugin
- a Git repository or HTTP tarball of installable packages ﹣ for Carvel kapp﹣controller plugin
- a Docker image reference for an Image or ImagePackageBundle ﹣ used by Carvel kapp﹣controller to define a repository of installable packages
PackageRepositories may be either global (cluster-wide) or namespace-scoped, in which case only users with access to that namespace will have access to the repositories﹣Flux does not have the concept of global repositories. Furthermore, PackageRepository may be public (i.e. require no authentication to access its contents) or private (i.e. require some form of authentication to access its packages). Taking all those requirements into consideration, the Repositories API definition was defined and made publicly available in order to provide a complete reference to the new API. In addition, the API is also available for each Kubeapps instance and includes a documentation portal accessible using user credentials: https://<kubeapps url>/#/docs.
Once the initial version of the Package Repository API was defined, the next step was to implement the API for each available plugin in Kubeapps ﹣Helm, Flux, and Carvel plugins. We decided to begin the work by implementing the API for the Flux plugin ﹣as any other could have been selected ﹣ while continuing in parallel the development of the API for Helm and Carvel. Fortunately, we could speed up the development thanks to the pluggable architecture of Kubeapps implemented in prior versions. During the implementation for each plugin, we verified that the new API satisfied the needs of the UI and found some gaps in the existing form that needed to be addressed.
Revamping of the Kubeapps UI
When the new Package Repository API had been totally implemented, it was time to adapt the UI to comply with the new API, and at the same time, to simplify the user experience in managing package repositories in Kubeapps.
The main challenge we found was how to group and display all the fields in a way that the Kubeapps UI will be able to guide users through the configuring process. The revamping process of the form was tackled in three stages:
- Using the new repository API in the existing UI to manage new API calls, report issues back, and improve the API when needed. The main goal of all these actions was to ensure feature parity between the new API and what Kubeapps supports with a consistent implementation across plugins.
- Completing support for secret management in private repos. To do so, we had to unblock all the issues related to authentication and the existing inconsistencies across plugins that were identified in previous stages.
- Working on code and UI enhancements. In this last stage, the team focused on cleaning the code including the removal of pending calls to deprecate kubeops, adding some improvements in the form such as the scope field to select for global or namespaced repos, and fixing some issues identified when using the new UI in different scenarios.
As a result, the new UI for Package Repository management includes the following sections:
Basic information
In this section, users will include information about the repository such as litname, URL, packaging format, and storage type.
To improve the user experience, we have added a new field in this form: the repository scope. This was previously set by default depending on the namespace. Adding an extra field to set the scope of the repository allows Kubeapps users to perform this configuration explicitly.
Note that the storage types field for Helm differs from the one set for Carvel packages as mentioned above. Refer to the next image to see how it looks when you select Carvel packages as the packaging format:
Authentication
Once the basic information has been entered in the form, depending on whether the repository where the packages live is public or private, Kubeapps will require authentication data. We originally planned to support scenarios where the user may choose to provide a reference to an existing secret or string cert_authority, which Kubeapps would convert into a new secret on the backend but we found that approach to be complex, particularly in those cases that require an update in the Package Repository. To simplify the process, the form now includes only two options for secrets:
a. The user will manage secrets
b. Secrets will be managed by Kubeapps
These options are set on a per-repository basis; thus, different users can use whatever option they prefer in the same Kubeapps installation. When the Auth and/or TlsConfig options are configured, the user has the choice to select an existing secret (option A) or to enter the credentials (option B). Any updates thereafter must be according to the original option selected.
- If the secret was created by choosing option A, the plugin will return the secret name and any update must be done via secrets;
- If the secret was created by choosing option B, the plugin will return the credentials and any update must be done by providing the credentials again.
The plugin will automatically detect and manage whether the package repository is using option A or B. In the case of option B, it is recommended to leverage the Kubernetes ownerReference mechanism along with a managed-by annotation.
From a UI perspective, the revamped form includes a section for Authentication with all these different options:
Advanced
Finally, some advanced fields have been included to configure different aspects of the repository such as the synchronization interval ﹣ available for Flux and Carvel plugins ﹣ CA certificate, skip TLS verification, or passing credentials to 3rd party's URL as shown in the image below:
Conclusion and Next steps
Thanks to the implementation of the Package Repository API and the pluggable architecture in Kubeapps, the team was able to move forward and boost development when the API was defined. The new form lays down the foundation for making Kubeapps simple by guiding users through the configuration process.
Next, we plan to tackle challenges such as the support for OCI registries in the Flux plugin, adding sync intervals for the Helm plugin as well as exploring repository configuration for multi-cluster and air-gapped environments.
I would like to highlight the great work done by the Kubeapps team in implementing the new Package Repository API and revamping the user interface.
Support and Resources
Since Kubeapps is an open-source software project, support will be provided on a best-effort basis. To get a resolution on issues such as deployment support, operational support, and bug fixes, please open an issue in the Kubeapps GitHub repository. A Markdown template is provided by default to open new issues, with certain information requested to help us prioritize and respond as soon as possible. Also, if you want to contribute to the project, feel free to send us a pull request, and the team will guide you in the process of a successful merge.
In addition, you can reach out to Kubeapps developers at #kubeapps on Kubernetes Slack (click here to sign up).
For more information about the topics discussed in this blog post, refer to the following links: