How The NSA Says You Should Secure Kubernetes

Security has become a primary consideration in any technological solution with the increased number of cyber threats across the board. As the leading orchestration engine, Kubernetes is getting targeted by more and more attacks. Thus it is paramount that users secure Kubernetes properly. In this post, let’s look at some recommendations by the National Security Agency (NSA) to secure Kubernetes.

Authentication and Authorization

Implement authentication and authorization mechanisms

Administrators implement authentication methods to facilitate authentication and authorization mechanisms as there are no default automatic authentication methods in Kubernetes for user accounts. It can be done either via client certificates, bearer tokens, authentication plugins, or any other supported authentication protocol. However, weak methods such as static password files should be avoided at all costs.

Configure role-based access control

RABC can be used to restrict access to all the resources within the cluster. Roles set permissions for a particular namespace, while ClusterRoles set permissions at a cluster level. Meanwhile, RoleBindings and ClusterRoleBindings tie these permissions to a specific user, group, or service account. All these bindings should follow the principle of least privilege granting with only the absolutely necessary permissions.

Control Plane Security

Control the access to Kubernetes control plane

As the brains of the cluster, the control plane must be secured by enforcing TLS encryption and using a robust authentication method. Additionally, access to the control plane should be restricted with RBAC policies and network controls like a firewall that blocks unauthorized access to the Kubernetes API. The API should not be exposed to the internet or any untrusted network.

Harden Etcd backend database

Etcd database stores the state information and cluster secrets to protect the Etcd. It should only be accessible via the API server with the cluster authentication method and RBAC restricting the users. Administrators should also enforce HTTPS communication between the API server and Etcd using TLS certificates. Moreover, they should configure Etcd to only trust certificates assigned to API servers.

Protect Kubeconfig Files

The kubeconfig files in the $HOME/.kube directory of worker and control nodes contain sensitive information about the cluster, including user, namespace, authentication details, etc. These files should be protected from unintended modifications, and their access should be blocked to any unauthenticated non-root users to prevent them from getting compromised.

Kubernetes Pod Security

Run application as non-privileged users

Run applications as non-root users of non-root groups in a container or by utilizing a rootless container engine to run in an unprivileged context. It helps minimize the impact if the container is compromised.

Create immutable container file-system

Kubernetes can be used to lock down the container file system, which mitigates the risk of attackers interacting with the file system. K8s administrators should mount secondary file systems for specific directories in applications that require write access.

Build secure container images

When building a container either by scratch or on top of an existing image, users must use trusted repositories and utilize image scanning to detect outdated libraries, known vulnerabilities, misconfigurations, etc. This scanning should also be a regular occurrence on already built images to keep them up to date and free of vulnerabilities.

Apply Pod Security Policies

Pod Security Policies allow for enforcing cluster-wide security for the Pods to be executed in the cluster. It establishes a minimum security threshold for all the Pods even if security mechanisms are not specified for the Pod or deployment. These policies can range from denying privileged containers and container features to applying security services like SELinux and AppArmor.

Disable automatic Pod service account token mounting

K8s automatically creates a services account and mount the secret token to the Pod at runtime. However, when an application does not need access to this service account directly, users should prevent these tokens from getting mounted to the Pod by disabling secret token mounting in the Pod specification.

Network and Resource Security

Create separate Namespaces

Users can create separate spaces that are isolated from each other by creating separate Namespaces for specific applications or workloads. Then these Namespaces can be managed separately by enforcing namespace-specific RBAC, network policies, resource limits and quotas.

Implement Network Policies

Network policies enable administrators to control the traffic flow between Pods, namespaces, and external IP addresses. As K8s allow unrestricted ingress and egress by default, controlling traffic is key to eliminating unnecessary exposure of container traffic.

Implement Resource Policies

Resource policies allow limiting the resources that can be utilized on namespace and node levels. LimtRanges constrains individual resources per Pod or container, and ResourceQuota limits aggregate resource usage for an entire namespace.

Encrypt Entire Network

All the traffic between the components, nodes, and control plane should be encrypted using either TLS 1.2 or 1.3 standard. Moreover, the appropriate certificates should be distributed and kept up to date across all the nodes within the cluster.

Encrypt Kubernetes Secrets at REST

Kubernetes secrets are stored in an unencrypted manner by default. These secrets can be encrypted either by configuring encryption on the API server or by using an external Key Management Service. Additionally, access to secrets should be controlled by applying RBAC policies.

Implement Security at the Infrastructure Level

Underlying resources used to run Kubernetes should be secured in addition to the cluster itself. Virtual instances that act as workers in a cloud environment will have access to sensitive cloud metadata services. Users can restrict access to these data via network policies or cloud configuration policies to prevent misusing them.

Kubernetes Monitoring

Implement Logging for all Resources

Logging should be configured at the application, host, and cloud levels to gain a complete overview of the environment. Everything from API requests, deployments and resource usage to network traffic should be logged. These logs can then be aggregated via external tools for further analysis and anomaly detection. If required, specialized logging agents can be run across nodes as DaemonSets ensure continuous delivery of logs.

Setup Alerts for the Cluster

There is no default alerting mechanism in K8s. However, there are many monitoring tools with monitoring capabilities. These alerts should be configured to monitor all aspects of the cluster, including abnormal resource utilization, services going offline, anomalous requests, unusual user behavior, etc. Alerting can act as a preventive measure to quickly detect changes in the K8s cluster.

Properly configuring all the items mentioned above on your Kubernetes environment can increase the security posture of your environment and create a more resilient environment for cyber attacks. A Platform like CloudPlex allows to bring different cloud platforms and technologies together and simplifies the development and deployment of containerized applications. Moreover, it provides comprehensive monitoring capabilities to monitor both applications and the environment.

Asad Faizi

Founder CEO

Cloudplex.io, Inc

asad@cloudplex.io