Portable, Universal Single Sign-On for Your Kubernetes Clusters

Hi! I am Miguel, a software engineer at Bitnami and core contributor to open source Kubernetes-related projects like Helm, Monocular or Kubeapps.

Kubeapps is an application dashboard for Kubernetes. A web user interface that allows users to deploy and manage Kubernetes applications or interact with the Kubernetes Service Catalog.

Until now, our users needed to manually log in by providing the token associated with a Kubernetes Service Account previously created by a cluster operator. These manual steps had an impact in user experience and adoption.

We asked the community how we could improve this process and the most requested feature was Single Sign-On (SSO) support.

Kubeapps is not the only application in this scenario, any other application that talks to the Kubernetes API directly, such as kubectl or the Kubernetes Dashboard, belong to this category too. That’s why we decided to publish our research so other projects in the Kubernetes community can leverage our findings.

Why Single Sign-On?

To kick off our project, we looked at the main authentication methods in Kubernetes and analyzed four main attributes:

• Can a user self-serve credentials or it requires the intervention of a cluster operator?
• Can those credentials be rotated?
• Can they be revoked?
• What’s the user experience during both the provisioning and the usage of those credentials?

The results based on our experience are:

In our experience, using X509 Client Certs required a cluster operator team to verify the user identity, craft a certificate and share it securely. This AuthN mechanism is not great in terms of rotation/revocation either since all the certs crafted from the common CA needs to be revoked in order to revoke a single user.

Static Token/Service account and basic auth share similar properties. Self-serve is not generally possible but per-user rotation/revocation is possible. Although, in some cases, it requires restarting the Kubernetes API server.

Lastly, we looked at Single Sign-On powered by OpenID Connect. Once this AuthN mechanism is in place, users can self-serve. It has built-in credential rotation via refresh tokens. Revocation has the small caveat of the token being valid during its lifespan which depends on the Identity Provider.

The UX section is colored in green not only because of the usability enhancement from the User point of view, but also from the Cluster Operator’s who do not need to craft, rotate or transfer credentials anymore and just delegate all that to a third party Identity Provider (IdP).

SSO Support - Simple Scenario

After validating that, Single Sign-on seemed a good solution for our needs. We decided to go ahead with the implementation, which seemed straightforward from our research.

We will put an OpenID-Connect proxy in front of our application, configured to trust the same OIDC IdP than the Kubernetes API server.

The solution could look something like this:


In the example above, we are putting a proxy in front of our application that will take care of 1) intercepting and enforce authentication by performing the required Oauth2 dance and 2) inject a new set of headers including the user info in the form of a JSON Web Token (JWT).

For this solution to work, both the Kubernetes API server and the OIDC-proxy are configured to trust the same OIDC identity provider, and this is only possible if the k8s API server is customizable, which is not always the case.

SSO Support - Complex Scenario

K8s API server customization is not available in all providers, which made it our biggest challenge. Some managed services like Google’s GKE or AWS’ EKS do not support customizing the OIDC settings, while others like Azure’s offering just expose their own identity providers. 

We had to ask ourselves “How can we offer a solution that works in all platforms? We brainstormed a couple of solutions, from translating OIDC tokens to service accounts via a custom controller to using K8s impersonation.

Impersonation is a feature in Kubernetes that allows a user to act on behalf of another user or group as long as it has permissions to do so.

Impersonation can be performed via custom headers or even directly via kubectl. In the following example we are telling Kubernetes that we want to impersonate the “FooUser” from the “kubeapps-user” group. Then, Kubernetes will make sure my user can impersonate the FooUser and then FooUser can get pods in the current namespace.

$ kubectl get pods --as FooUser --as-group kubeapps-user

By leveraging impersonation, we could put an additional proxy that will take a valid (but not understood by the k8s API) server Id_token (JWT) and transform it into impersonation headers that Kubernetes will understand. 

By placing this proxy that leverages a k8s native feature like Impersonation between our application and the API server, we are able to offer a Single Sign-On experience that will work in all platforms.

If you want to learn more about these workarounds and see them in action, I’ll be speaking at Kubecon EU on May 21st, come by to say hi! See you in Barcelona!

Introducing Kubeapps

Today we are proud and excited to announce Kubeapps, a package agnostic application dashboard for Kubernetes. We see it as a compass that helps Kubernetes users find their way in the different ways to package a Kubernetes applications as well as a boostrapping mechanism so that all the necessary add-ons can be installed in your cluster easily.

For those of you who have been long-time Helm users, the website kubeapps.com may be familiar- until now it was a Chart discovery site. We decided to build upon the success of Kubeapps and extend the experience to bring applications of all types to Kubernetes users.

Indeed, now that it is clear that Kubernetes has won the container orchestration battle, it is time to focus on applications. At Bitnami we have been packaging applications for the cloud for almost a decade now. In building Kubeapps, we keep focusing on our core strength while bringing a useful service to the community and helping our customers that are embracing Kubernetes and migrating to container-based applications.

That’s why we decided to re-launch Kubeapps, making it more generic and able to handle all sorts of Kubernetes applications instead of just Helm charts. In particular, with the growing interest in serverless-based applications, our Kubeless project makes a perfect companion to Kubeapps. So Kubeapps is now made of three components:

• A command line interface that helps your bootstrap all the necessary Kubernetes cluster add-ons to deploy and manage your applications.
• A community site, hub.kubeapps.com, aimed at becoming the Docker Hub of Kubernetes applications. It now has social features like starring and commenting.
• An in-cluster application dashboard, built on Monocular, that also features the Kubeless user interface to bring in serverless capabilities to kubeapps.

From a technical standpoint, The Hub is an instance of Monocular with the added social features enabled thanks to the ability to log in to Kubeapps. The code and various microservices that it is now composed of are all available on GitHub in the Kubeapps organization. We believe that everyone in the community will enjoy starring and leaving comments on their favorite charts. In the near future, we plan to allow users to submit new applications and new application registries to Kubeapps so it can act as a true aggregator of shareable Kubernetes applications.

The command line interface is Golang binary that allows users to easily install the following add-ons into any cluster (caveat: only Minikube and GKE are tested right now, AKS is coming soon):

• Helm/tiller (in a secure, opinionated deployment configuration)
• Kubeless (our Kubernetes-native serverless solution)
• Sealed Secrets (a tool to give #gitops the ability to manage secrets securely)

As new packages and operational workflows to manage Kubernetes applications emerge we envision adding the appropriate add-ons to Kubeapps so that anyone can configure their cluster to meet their specific needs. Sealed Secrets might seem a little odd in this mix, but we believe that a significant number of users will start moving to a declarative mindset and will adopt a #gitops workflow like what is described by our friends from [Weave works](https://engineering.bitnami.com/articles/secure-gitops.html ).

Each add-on is defined in a jsonnet manifest available on GitHub. These manifests are turned into full YAML manifests and embedded in the code of `kubeapps`. This gives the ability to deploy the add-ons in a declarative manner, set labels, and configure garbage collection properly. A lot of the code that is in `kubeapps` was first introduced in our work on `kubecfg` as part of the ksonnet project. Once you download the binary you have two key commands to learn:

• `kubeapps up` (will create all the required resources to make the add-ons work)
• `kubeapps down` (if you wish to remove everything)

Here is a snapshot of the output of `kubeapps up`:



The third command in the Kubeapps CLI is:

`kubeapps dashboard`




This will connect you to a dashboard running in your cluster. This dashboard is based on Monocular as well but has the added functionality to deploy Helm Charts directly from the web interface. It also now integrates the Kubeless UI, making the Kubeapps dashboard a one stop shop for both Helm Charts and Serverless functions. Below is a snapshot of the Kubeless UI.




This is just the beginning for Kubeapps. As our friends from CoreOS mention in their 2018 predictions, 2 out of 5 critical areas are going to be Kubernetes applications and Serverless. With Kubeapps we are very excited at Bitnami to be addressing these needs already, offering a package agnostic launchpad for Kubernetes applications. One that brings serverless functions and any type of app formats together under one roof.

Look for an improved UI, more social features to build a great application hub for Kubernetes plus the ability to support declarative management of applications. This is going to be an exciting 2018 and Kubeapps is just the beginning.

See you at Kubecon 2017!

Kubecon is around the corner and we’re pumped up for this year’s event. We’ll be exhibiting, conducting training, speaking, announcing an exciting new project, and of course meeting with loads of customers, partners and friends. For those of you who are headed to Kubecon in Austin this year (and even if you are not), we wanted to provide a summary of the main activities that we’ve got going on.

New Bitnami Kubernetes Project

As you know, Bitnami is leading the charge in defining how containers and functions will be packaged and delivered for Kubernetes. Building on Bitnami’s contributions to leading open source projects (Helm, Monocular, Kubeless and more) and leveraging our expertise in application packaging, we’ll be launching a new tool that provides a complete application delivery environment that empowers users to launch, review and share applications all from within a Kubernetes cluster.

Visit bitnami.com or follow us on social media for more details coming Dec 4th. 

Speaking Session: Building Serverless Application Pipelines
Thursday, December 7 • 11:55am - 12:30pm

Session Description: The serverless paradigm is bringing a new type of applications to the forefront of application architecture. Distributed, containerized, scalable, event-driven and ephemeral with fine grained billing. In this talk we will go through several application use-cases that are driving the serverless movement (e.g data processing, IoT, mobile-backends, machine learning) and demonstrate how these applications can be developed and deployed on top of Kubernetes using an open source serverless solution called kubeless. Through live demos and examples, we will show that Kubernetes with its rich and stable core API is the perfect platform to build FaaS solutions.

Add this talk to your KubeCon schedule now!

Live Training: Kubernetes Core Concepts
Tuesday, December 5 • 8:30am - 5:00pm

This one-day course serves as a crash course to learn the basics of Kubernetes right before KubeCon NA. You will discover the Kubernetes architecture and how to install it. You will then learn how to use its basic primitives (i.e pods, deployments and services) to build your own distributed application. The course will be a mix of lectures, demos and hands-on exercises.

There are only a few spots left, so be sure to add this training to your ticket before it is too late!

Booth Activities
Visit us at Booth #S24

Join us at the Bitnami booth for a demo of any of our Kubernetes projects, to see simple ways to launch applications in your cluster, or just to chat about how you are using Kubernetes and how we can help.  We’ll have a few featured slots with our partners at Weaveworks and an exclusive book signing of the soon to be released Kubernetes Cookbook from O’Reilly Media, authored by our very own Sebastien Goasguen.

Live Booth Demos:  GitOps using Weave Cloud Deploy And Bitnami's Sealed Secrets

Come to the Weavework booth or Bitnami booth at the times below to learn how to secure GitOps using Weave Cloud Deploy And Bitnami's Sealed Secrets, and ask the experts any questions that you might have.

• Demo at Weaveworks Booth at 10:45 am on Wednesday, December 6th
• Demo at Bitnami Booth at 10:45 am on Thursday, December 7th 

Live Booth Demos: Securing Kubeless function endpoints using cert-manager, by Jetstack

Come to the Bitnami booth to meet James Munnelly, from Jetstack, demonstrate how you can use cert-manager to automatically secure your Kubeless function endpoints, and discuss the opportunities that Kubernetes-native TLS certificates opens for your own stack too!

• Demo at Bitnami Booth at 3:30 pm on Wednesday, December 6th

Book Signing

• O’Reilly Kubernetes Cookbook book signing at 7:00 pm -8:00 pm on Wednesday December 6th

Looking forward to seeing many of you there!