Kubeapps is an application dashboard for Kubernetes. A web user interface that allows users to deploy and manage Kubernetes applications or interact with the Kubernetes Service Catalog.
Until now, our users needed to manually log in by providing the token associated with a Kubernetes Service Account previously created by a cluster operator. These manual steps had an impact in user experience and adoption.
We asked the community how we could improve this process and the most requested feature was Single Sign-On (SSO) support.
Kubeapps is not the only application in this scenario, any other application that talks to the Kubernetes API directly, such as kubectl or the Kubernetes Dashboard, belong to this category too. That’s why we decided to publish our research so other projects in the Kubernetes community can leverage our findings.
Why Single Sign-On?
To kick off our project, we looked at the main authentication methods in Kubernetes and analyzed four main attributes:
- Can a user self-serve credentials or it requires the intervention of a cluster operator?
- Can those credentials be rotated?
- Can they be revoked?
- What’s the user experience during both the provisioning and the usage of those credentials?
In our experience, using X509 Client Certs required a cluster operator team to verify the user identity, craft a certificate and share it securely. This AuthN mechanism is not great in terms of rotation/revocation either since all the certs crafted from the common CA needs to be revoked in order to revoke a single user.
Static Token/Service account and basic auth share similar properties. Self-serve is not generally possible but per-user rotation/revocation is possible. Although, in some cases, it requires restarting the Kubernetes API server.
Lastly, we looked at Single Sign-On powered by OpenID Connect. Once this AuthN mechanism is in place, users can self-serve. It has built-in credential rotation via refresh tokens. Revocation has the small caveat of the token being valid during its lifespan which depends on the Identity Provider.
The UX section is colored in green not only because of the usability enhancement from the User point of view, but also from the Cluster Operator’s who do not need to craft, rotate or transfer credentials anymore and just delegate all that to a third party Identity Provider (IdP).
SSO Support - Simple Scenario
After validating that, Single Sign-on seemed a good solution for our needs. We decided to go ahead with the implementation, which seemed straightforward from our research.
We will put an OpenID-Connect proxy in front of our application, configured to trust the same OIDC IdP than the Kubernetes API server.
The solution could look something like this:
In the example above, we are putting a proxy in front of our application that will take care of 1) intercepting and enforce authentication by performing the required Oauth2 dance and 2) inject a new set of headers including the user info in the form of a JSON Web Token (JWT).
For this solution to work, both the Kubernetes API server and the OIDC-proxy are configured to trust the same OIDC identity provider, and this is only possible if the k8s API server is customizable, which is not always the case.
SSO Support - Complex Scenario
K8s API server customization is not available in all providers, which made it our biggest challenge. Some managed services like Google’s GKE or AWS’ EKS do not support customizing the OIDC settings, while others like Azure’s offering just expose their own identity providers.
We had to ask ourselves “How can we offer a solution that works in all platforms? We brainstormed a couple of solutions, from translating OIDC tokens to service accounts via a custom controller to using K8s impersonation.
Impersonation is a feature in Kubernetes that allows a user to act on behalf of another user or group as long as it has permissions to do so.
Impersonation can be performed via custom headers or even directly via kubectl. In the following example we are telling Kubernetes that we want to impersonate the “FooUser” from the “kubeapps-user” group. Then, Kubernetes will make sure my user can impersonate the FooUser and then FooUser can get pods in the current namespace.
$ kubectl get pods --as FooUser --as-group kubeapps-user
By leveraging impersonation, we could put an additional proxy that will take a valid (but not understood by the k8s API) server Id_token (JWT) and transform it into impersonation headers that Kubernetes will understand.
By placing this proxy that leverages a k8s native feature like Impersonation between our application and the API server, we are able to offer a Single Sign-On experience that will work in all platforms.
If you want to learn more about these workarounds and see them in action, I’ll be speaking at Kubecon EU on May 21st, come by to say hi! See you in Barcelona!