Securing Access to Kubernetes with Rancher
l

1 Nov, 2021

LinkedInTwitter

This blog describes approaches to securing your Kubernetes deployments with SUSE Rancher, an open-source suite of tools from SUSE: https://github.com/rancher .

If you would like to know more about how to implement modern data and cloud technologies into your business, we at Digitalis do it all: from the cloud and Kubernetes migrations to fully managed services, we can help you modernize your operations, data, and applications – on-premises, in the cloud and hybrid.

We provide consulting and managed services on a wide variety of technologies. Contact us today for more information or to learn more about each of our services.

Introduction

At Digitalis we take security seriously, we are ISO27001 accredited and work on critical systems for our customers. All of our DevOps engineers complete the CNCF Certified Kubernetes Administrator (CKA) and Certified Kubernetes Security Specialist (CKS) certifications and we are more aware than most of Kubernetes security best practices (or lack thereof). The CKS material is very interesting and covers great tools such as Falco and Trivy to help you make smarter decisions and ensure the bad guys cannot compromise your Kubernetes cluster.

But where the CKS fails to deliver is, in my opinion, the culture around DevOps and Developers.

What do you mean?

Securing Kubernetes from intruders outside on the Internet is a big task, so much so that often we forget the bad guys may be closer than we think, they could be sat next to you at the office.
Not Production Ready

We spend a lot of time securing Kubernetes, the physical servers or virtual machines, the network with firewalls, two-factor authentication, etc. But then everyone has the same kubeconfig with admin rights!

A kubeconfig is a yaml file with the information required to access the Kubernetes clusters. It contains server addresses and SSL certificates. Keeping this configuration file secure is paramount.

you have been hacked

Really?

Yes! Very few companies spend time creating custom kubeconfig for the different teams in the company. A developer that only needs to check the logs on some pods is unlikely to require permissions to create or delete. You may not want to give access to critical namespaces to a junior DevOps/SRE.

If you are using a managed Kubernetes service, you may want to restrict access to it and disallow people from downloading or generating a kubeconfig. Surprisingly I have seen that security teams restrict access to Kubernetes with custom permissions but forget this and unhappy users download their own admin kubeconfig from the managed Kubernetes provider portal.

What should I do?

The most obvious answer is that you should limit access to Kubernetes with RBAC (Role-Based Access Control). Better still, don’t give them access at all.

A senior SRE may need full access to investigate and resolve problems, but a Developer probably does not need access at all if you have the right tools in place.

  • Invest in automation, be it GitOps like Rancher Fleet, Jenkins, Rundeck, you name it.
  • Centralise the logs into an analytics solution such as Elasticsearch or Grafana Loki, removing the need to access Kubernetes at all for many users.
  • Make sure unprivileged users can only access the sections of the cluster they need through your administration tools.
  • Make good use of the Kubernetes namespaces to separate not just workloads but users and teams as well.

Another alternative is to create a custom kubeconfig per team or even individual granting the permissions that person or group requires to do their daily job.

Lastly, my preferred option is to use Rancher RBAC.

Rancher

Rancher provides a powerful UI for the administration of multiple Kubernetes clusters from a single point. It also offers a simple yet powerful RBAC to create users within roles and give them just the right amount of permissions.

Another great feature is it supports multiple Auth Providers allowing you to connect the Rancher authentication to Google, Active Directory, LDAP, etc. and therefore use your company’s existing centralised user database.

Rancher Auth Providers
With this in mind, you can now create roles granting permissions to the group of users either locally configured or centrally managed.

You can create custom Roles or use the built-in ones to grant permissions to a user or group:

Rancher RBAC

The built-in ones are good for most use cases. They are pretty granular covering access to Rancher itself down to individual projects or namespaces. From here you can create your own with stricter permissions if you need.

The roles are then applied to Projects which is something native to Rancher. A Project is, in essence, a group of namespaces.
In terms of hierarchy:

  • Clusters contain projects
  • Projects contain namespaces

You can use projects to perform actions such as:

  • Assign users to a group of namespaces (i.e., project membership).
  • Assign users specific roles in a project. A role can be owner, member, read-only, or custom.
  • Assign resources to the project.
  • Assign Pod Security Policies.

As you can see, you can get pretty granular when configuring access.

I don’t like using a Web UI

I hear you saying? No trouble. You can also use the command line with Rancher as well, and there is a shell within Rancher you can use for quick jobs:
Kubectl Shell

You can also download a kubeconfig from Rancher which will be configured with your user permissions only.

Download KubeConfig

Another alternative is the Rancher CLI command. This can be especially handy if you manage multiple clusters and want a way to switch between them.

Once you have created your API key, you can log in:

$ rancher login https://<SERVER_URL> --token <BEARER_TOKEN>
If you have multiple clusters, switch between them with:
$ rancher context switch
NUMBER    CLUSTER NAME   PROJECT ID              PROJECT NAME   
1         cluster-2      c-7q96s:p-h4tmb         project-2      
2         cluster-2      c-7q96s:project-j6z6d   Default        
3         cluster-1      c-lchzv:p-xbpdt         project-1      
4         cluster-1      c-lchzv:project-s2mch   Default       
Select a Project:
Now you should be able to run any kubectl command using:
$ rancher kubectl cluster-info

Conclusion

Hacking is not all about teens wearing hoodies in dark rooms full of leftover pizza trying to break into your company. A good portion of the hacking attempts (up to 60% according to some reports) comes from inside organizations.

Remember to apply the least privilege access principle, secure access to the managed Kubernetes portal, use two-factor authentication when possible and apply RBAC, either with Rancher or by creating custom kubeconfig files per user or group of users.

Kubernetes Management For Dummies

Categories

Archives

Related Articles