Kubeapps Extends Package Repository Management

New Package Repository API and UI in version 2.5.0

Authored by Pepe Baena, R&D Manager at VMware

The Kubeapps team is continuously looking for ways to improve the user experience when browsing and deploying applications on Kubernetes clusters. The new Kubeapps release ﹣ version 2.5.0 ﹣ extends the repository management feature by including Carvel packages. Now, users will be able to configure package repositories for both Helm and Carvel packages in just a few clicks. 

Keep reading to learn more about the key points that make it possible: a new Package Repository API and a revamped form that allows you to easily configure package repositories directly from the UI. 

Context of this Enhancement

Before releasing version 2.4.3, Kubeapps was mainly focused on Helm chart management, and as a  consequence, was tied closely to Helm repositories and OCI registries for Helm chart storage. When we added support for Carvel packages and Helm charts via Flux, it became necessary to standardize the management of different package repository types in Kubeapps. This brought the following challenges for the team:

• We needed to design and implement a new API, similar to the Kubeapps core API for managing packages, but in this case for managing package repositories as well as ensuring that the new API satisfies the requirements of the existing Helm functionality.
• We needed to implement a Package Repository API for the different plugins (Helm, Flux, and Carvel).
• We needed to revamp the current UI form to standardize the experience when configuring existing repositories (Helm and OCI) as well as Carvel and Flux sources.

Let’s see in detail what implementation strategy and steps were performed to successfully add this new feature to Kubeapps.

Design and Implementation of a New Package Repository API

The first step we followed was to design a new Package Repository API that allowed us to manage new package repositories on Kubeapps. The main component of the domain model is the PackageRepository (a repository of Kubernetes application packages that are available for installation). A repository may be served by any one of the backends supported by the configured plugins (Helm, Flux, and Carvel plugins), including, but not limited to:

• a Helm chart repository (e.g. https://charts.bitnami.com/bitnami ) – via Helm or via Flux plugins
• an OCI helm repository ﹣ via Helm plugin or via Flux plugin
• a Git repository containing artifacts ﹣ gzip compressed TAR archives, via Flux plugin
• object storage buckets containing artifacts coming from S3 compatible storage such as Minio, Amazon S3, Google Cloud Storage, Alibaba Cloud OSS, and others ﹣ via Flux plugin
• a Git repository or HTTP tarball of installable packages ﹣ for Carvel kapp﹣controller plugin
• a Docker image reference for an Image or ImagePackageBundle ﹣ used by Carvel kapp﹣controller to define a repository of installable packages

PackageRepositories may be either global (cluster-wide) or namespace-scoped, in which case only users with access to that namespace will have access to the repositories﹣Flux does not have the concept of global repositories. Furthermore, PackageRepository may be public (i.e. require no authentication to access its contents) or private (i.e. require some form of authentication to access its packages). Taking all those requirements into consideration, the Repositories API definition was defined and made publicly available in order to provide a complete reference to the new API. In addition, the API is also available for each Kubeapps instance and includes a documentation portal accessible using user credentials: https://<kubeapps url>/#/docs.

Once the initial version of the Package Repository API was defined, the next step was to implement the API for each available plugin in Kubeapps ﹣Helm, Flux, and Carvel plugins. We decided to begin the work by implementing the API for the Flux plugin ﹣as any other could have been selected ﹣ while continuing in parallel the development of the API for Helm and Carvel. Fortunately, we could speed up the development thanks to the pluggable architecture of Kubeapps implemented in prior versions. During the implementation for each plugin, we verified that the new API satisfied the needs of the UI and found some gaps in the existing form that needed to be addressed.

Revamping of the Kubeapps UI

When the new Package Repository API had been totally implemented, it was time to adapt the UI to comply with the new API, and at the same time, to simplify the user experience in managing package repositories in Kubeapps. 

The main challenge we found was how to group and display all the fields in a way that the Kubeapps UI will be able to guide users through the configuring process. The revamping process of the form was tackled in three stages:

1. Using the new repository API in the existing UI to manage new API calls, report issues back, and improve the API when needed. The main goal of all these actions was to ensure feature parity between the new API and what Kubeapps supports with a consistent implementation across plugins.
2. Completing support for secret management in private repos. To do so, we had to unblock all the issues related to authentication and the existing inconsistencies across plugins that were identified in previous stages.
3. Working on code and UI enhancements. In this last stage, the team focused on cleaning the code including the removal of pending calls to deprecate kubeops, adding some improvements in the form such as the scope field to select for global or namespaced repos, and fixing some issues identified when using the new UI in different scenarios.

As a result, the new UI for Package Repository management includes the following sections:

Basic information

In this section, users will include information about the repository such as litname, URL,  packaging format, and storage type. 

To improve the user experience, we have added a new field in this form: the repository scope. This was previously set by default depending on the namespace. Adding an extra field to set the scope of the repository allows Kubeapps users to perform this configuration explicitly.

Note that the storage types field for Helm differs from the one set for Carvel packages as mentioned above. Refer to the next image to see how it looks when you  select Carvel packages as the packaging format: 

 

Authentication

Once the basic information has been entered in the form, depending on whether the repository where the packages live is public or private, Kubeapps will require authentication data. We originally planned to support scenarios where the user may choose to provide a reference to an existing secret or string cert_authority, which Kubeapps would convert into a new secret on the backend but we found that approach to be complex, particularly in those cases that require an update in the Package Repository. To simplify the process, the form now includes only two options for secrets: 

a. The user will manage secrets

b. Secrets will be managed by Kubeapps

These options are set on a per-repository basis; thus, different users can use whatever option they prefer in the same Kubeapps installation.  When the Auth and/or TlsConfig options are configured, the user has the choice to select an existing secret (option A) or to enter the credentials (option B). Any updates thereafter must be according to the original option selected.

• If the secret was created by choosing option A, the plugin will return the secret name and any update must be done via secrets; 
• If the secret was created by choosing option B, the plugin will return the credentials and any update must be done by providing the credentials again.

The plugin will automatically detect and manage whether the package repository is using option A or B. In the case of option B, it is recommended to leverage the Kubernetes ownerReference mechanism along with a managed-by annotation.

From a UI perspective, the revamped form includes a section for Authentication with all these different options:

Advanced

Finally,  some advanced fields have been included to configure different aspects of the repository such as the synchronization interval ﹣ available for Flux and Carvel plugins ﹣ CA certificate, skip TLS verification, or passing credentials to 3rd party's URL as shown in the image below:

Conclusion and Next steps

Thanks to the implementation of the Package Repository API and the pluggable architecture in Kubeapps, the team was able to move forward and boost development when the API was defined. The new form lays down the foundation for making Kubeapps simple by guiding users through the configuration process.

Next, we plan to tackle challenges such as the support for OCI registries in the Flux plugin, adding sync intervals for the Helm plugin as well as exploring repository configuration for multi-cluster and air-gapped environments. 

I would like to highlight the great work done by the Kubeapps team in implementing the new Package Repository API and revamping the user interface.  

Support and Resources

Since Kubeapps is an open-source software project, support will be provided on a best-effort basis. To get a resolution on issues such as deployment support, operational support, and bug fixes, please open an issue in the Kubeapps GitHub repository. A Markdown template is provided by default to open new issues, with certain information requested to help us prioritize and respond as soon as possible. Also, if you want to contribute to the project, feel free to send us a pull request, and the team will guide you in the process of a successful merge.

In addition, you can reach out to Kubeapps developers at #kubeapps on Kubernetes Slack (click here to sign up).

For more information about the topics discussed in this blog post, refer to the following links:

• Get Started with Kubeapps
• Managing Carvel Packages with Kubeapps
• Managing Flux V2 Packages with Kubeapps

Fix Available: Load Problems in Let’s Encrypt Servers due to Bitnami Cloud Images

Our colleagues from the Let’s Encrypt team have informed us that they have identified an issue with the certificate renewal process that is causing load problems in Let’s Encrypt servers.  

This issue affects Bitnami users that are using the Bitnami HTTPS Configuration tool (Bncert tool) to configure HTTPS on their Bitnami cloud deployments. To solve this issue, you must update your Lego installation as explained below. 

How to update your Lego installation 

A fix has been included in the Lego tool version 4.8.0 which adds a random 0-8 minute delay to avoid such spikes in the specific 0:00 minute. However, Bitnami cannot propagate this change to users unless they execute the tool. 

In order to avoid problems in case the renewal fails for several days in a row, and to avoid load problems, users should follow the steps below:  

• Execute Bncert again to renew the certificates. The tool will request to be updated - press “Yes”. This will also randomize the times in the crontab and add the user-agent to the crontab. 

 $ curl -L https://github.com/go-acme/lego/releases/download/v4.8.0/lego_v4.8.0_linux_amd64.tar.gz | tar xz -C /opt/bitnami/letsencrypt l 

ego  

• Manually update the lego version by running the following command:  

$ curl -L https://github.com/go-acme/lego/releases/download/v4.8.0/lego_v4.8.0_linux_amd64.tar.gz | tar xz -C /opt/bitnami/letsencrypt l 

ego  

• Randomize the renewal time. E.g. from 0:00: 

0 0 * * * sudo /opt/bitnami/letsencrypt/lego ... 

         To a random time, such as 21:40: 

40 21 * * * sudo /opt/bitnami/letsencrypt/lego … 

Once executed, the command {{sudo crontab -u bitnami -l}} should show something like this: 

 40 21 * * * sudo /opt/bitnami/letsencrypt/lego ... 

Where 21:40 is the new randomized time - you will probably see a different value - at which point the renewal will happen every day. 

Support and Resources   

Looking to learn more or have any questions? Check out the new Bitnami GitHub repository for virtual machines. If you need to get a resolution on issues or want to send us your feedback, please open an issue. A markdown template is provided by default to open new issues, with certain information requested to help us prioritize and respond as soon as possible.  

To learn more about how to generate and install a Let’s Encrypt certificate for a Bitnami application, refer to this tutorial.

 

Bitnami Sealed Secrets Team Collaborates with Students from the Aix-Marseille University

Authored by Alfredo García, R&D Manager at VMware

Bitnami’s Sealed Secrets has been a popular GitOps Secret Management solution ever since its launch back in 2017. With 5.4K starts and more than a million downloads per month, this project has a lot of traction and is widely adopted amongst the open-source community.

The Bitnami by VMware team encourages and fosters collaboration with university institutions. Examples of such collaborations can be found in the more than 20 training sessions delivered by our experts during this year's VMware Multi-cloud Academy.

While we regularly collaborate with institutions, opportunities to collaborate directly with computer science students are few and far between. For this reason, when Aix-Marseille University approached us with an offer to collaborate with some of their Reliability and IT Security Master’s Degree students, we quickly jumped on the idea. This proposal was translated into a two-month collaboration period in which several important features have been implemented in the Sealed Secrets project. 

Collaboration Scope

The collaboration started early in 2022 with some meetings with the faculty responsible for the Master’s Degree in Reliability and IT Security of the Aix-Marseille University, in order to define the scope and the approach of our cooperation. We agreed that five students will incorporate their work on the Sealed Secrets project as a part of their final dissertation for the Master’s degree they were undertaking. 

Those students had neither a previous background in collaborating with open source projects nor any proven experience in developing with Golang. To help them to be more efficient, the Bitnami by VMware Sealed Secrets team provided a minimum onboarding plan. This plan included the set-up of a GitHub account, a brief introduction to the project contributing guidelines, and a list of recommended readings that could help them better understand Sealed Secrets design and purpose.

The collaboration lasted from March to April 2022, and during that time, the students took ownership of several tasks in our project backlog focusing mainly on solving security and software supply chain issues. All these tasks were closely related to the content of their Master’s degree curriculum, so they dealt with them efficiently. 

These activities were grouped into three major blocks:

• Secure software supply chain
• Static code analysis and vulnerability scanning
• Sealed Secrets cryptographic review 

Secure Software Supply Chain

Needless to say, the software supply chain is a big concern for any organization. Given the importance of the Sealed Secrets project within the Kubernetes Security area, it is essential to control our dependencies and to provide a solid provenance for our deliverables.

Because of that, we asked the students to incorporate cosign verifications over the Sealed Secrets distroless and base images. They also included a cosign signature for the Controller images, Kubeseal CLI, and for the project’s official Helm chart. These improvements will make it easier for our users to verify the provenance of Sealed Secrets once included in their clusters.

Static Code Analysis and Vulnerability Scanning

Static code analysis is a great way to detect inefficiencies or security concerns on a codebase. Additionally, vulnerability scanning is a critical step in any continuous integration (CI) pipeline. In our case, we decided to include two complementary tools within the project CI process: gosec and trivy. For this last integration, in particular, Sealed Secrets leverages VMware Image Builder verification capabilities so that vulnerabilities are detected as part of the project release process. 

This task was related not only to integrating the tools but to analyzing the different reports and deciding which among various incidents were false positives and which could be added to the project as recommended code practices. The students included ten different Pull Requests (PRs) and some important improvements in Sealed Secrets security stance.

Sealed Secrets Cryptographic Review

The Sealed Secrets project had a few documents about cryptography, with little internal cohesion between them. It was difficult for new developers to understand the security stance of the project with these guides, so we decided to review and consolidate our Cryptography-related documentation.

A hot topic in cybersecurity is how to protect encrypted information against brute-force attacks executed with the help of quantum computers. These kinds of attacks are not yet possible with the current quantum processing power, but many security providers are designing algorithms that will be quantum resistant. To anticipate future developments, the students included in the Cryptographic documentation some recommendations and good practices regarding Post-quantum cryptography.  

Conclusion

The collaboration between the Aix-Marseille Cybersecurity Master students and the Sealed Secrets team has resulted in the merging of 18 Pull Requests into the project. These PRs include several important features that have improved the security posture of Sealed Secrets. Furthermore, the students have demonstrated great skills and determination by identifying key improvements and implementing them in the project. We recommend checking out their GitHub profiles and following them to discover this and other contributions to the open-source community. 

• Adrien MOLLET - @volker-carstein
• Luc BOBO - @luhko
• M'hamed BELHACHEMI M'HAMED - @Belhach
• Nicolas THOMAS - @JeNeComprendPas
• Nicolas BOURRAS - @vivescere

 We’d like to acknowledge their efforts and contributions and wish them the best in their next ventures!