Using secure Kubernetes settings¶
When you install Kubernetes, there are numerous configuration settings that can affect security. In this section you'll learn about checking the settings in your cluster against the best practices advised by the Center for Internet Security.
The CIS Kubernetes Benchmark¶
The CIS have many different benchmarks recommending how to configure software components with security best practices in mind. One such benchmark is for Kubernetes - in fact there are several editions for managed version of Kubernetes, like EKS or GKE, as well as for upstream Kubernetes installations.
There are hundreds of recommendations in these benchmarks, so running them manually on every node would be a time-consuming process.
Running benchmark checks with kube-bench¶
The open source tool kube-bench makes it easy to run the tests defined in the CIS Kubernetes benchmark. In this tutorial, you will use kube-bench to identify some insecure Kubernetes settings, and you'll remediate one of the settings to turn a failing test into a pass.
You could run
kube-bench in a cluster of your choice but for this tutorial we are showing it running in a kind (Kubernetes in Docker) single-node cluster that runs on your laptop as a Docker container.
Run kube-bench in kind¶
What we are about to do is TERRIBLE practice but it makes it easier to write a platform-independent set of instructions for this tutorial. Never run YAML directly from the internet like this in your production cluster - check what's in it first!
Create kube-bench job¶
You can watch the job until it has completed:
Hit Ctrl-C once the job has finished.
Get job output from logs¶
The job applies the label
app: kube-bench to the pod, so you can easily retrieve the logs like this:
Scroll back through the logs to see how it is divided into sections, each with its own set of results, remediation recommendations, and a summary.
Most of the tests pass but there are a few results marked with
[FAIL]means that the test failed
[WARN]indicates that you need to do something manually to verify whether the test should pass or not.
For more detail on the output check the kube-bench documentation.
Remediate a test¶
This tutorial was written using Kubernetes 1.18.2 and testing against the CIS Kubernetes Benchmark v1.5.1. If you are using a later version of Kubernetes, it's possible that the default configuration settings have changed and the results you get might not match what is described here.
Scroll back through the results to find the result and (further down the results) the remediation information for the test 4.2.6.
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Kind uses a kubelet configuration file that lives at
/var/lib/kubelet/config.yaml, so only the first line of the remediation text applies - you don't have to worry about editing the kubelet service file or restarting the service.
When using kind, there is a Docker container running your control plane. This image for this container is based on Ubuntu so we can exec into the running container and then treat it much as if it were a virtual machine running a Kubernetes node.
Edit kubelet configuration¶
First, open a shell into the kind container.
Assuming that it doesn't already have an editor installed, you can add one.
Edit the Kubelet config file
Add the line
protectKernelDefaults: true so that the file looks something like this:
1 2 3 4 5 6 7 8 9
Save the file. The kubelet will spot that the configuration has changed and update itself, but meanwhile you can
exit to leave the container so that you are back at your terminal where you can run
kubectl commands on the kind cluster.
First delete the previous job:
Run the kube-bench job, as before:
Once the job has completed, get the test results from the logs
This time you should see that test 4.2.6 passes. Congratulations, you have remediated a security setting on a Kubernetes node!
This only remediates the running node, of course! If you are managing your own Kubernetes nodes, it would be better to update the configuration settings you use in deployment scripts, so that the nodes are configured to run from the outset with the settings you want.
Using kube-bench via Starboard¶
You can also use Starboard to run kube-bench and store the results in a Kubernetes CRD.
These results can be easily viewed using Octant and the Octant Starboard plugin.
Using Starboard has the advantage that it will automatically run a
kube-bench job on all the nodes in the cluster.
If you download the
job.yaml file used above, you can modify it to try some optional exercises.
Run a specific test¶
Sometimes you might want to run an individual test rather than the whole benchmark. For example, try to modify the command run in the job so that it only runs the test 4.2.6 that you remediated earlier. You can do this by specifying
--check=4.2.6 as a parameter to the
Specify worker node tests only¶
There are different CIS Kubernetes Benchmark tests for different node types in the cluster (control plane nodes, worker nodes, etcd nodes). On a managed Kubernetes system you might only have access to worker nodes, so you only need to run the tests that apply to those nodes.
kube-bench tries to auto-detect which tests to run on any given node, but to keep things simple you may wish to specify worker node tests only. You might like to try out the
job-node.yaml configuration which does just that.