Label standard and best practices for Kubernetes security

This article talks about label standard and best practices for Kubernetes security, a common area where I see organizations struggle to define the set of labels required to meet their security requirements. My advice is to always start with a hierarchical security design that can achieve your enterprise security and compliance requirements, then define your label standard in alignment with your design.

This is not meant to be a comprehensive guide for all your label requirements, but rather a framework that guides you through developing your own label standard to meet your specific security requirements.

Kubernetes labels for network policies

Labels are key/value pairs that are attached to Kubernetes objects to identify attributes that are intuitive for users and that are required for specific purposes, such as inventory reporting or the enforcement of an intent.

Label classification

Kubernetes network policies represent the intent of enforcing security controls to pods using labels to match intended endpoints. Label prefixes can be used to identify label classification. The following short-list is a high-level classification of endpoints required for developing a Kubernetes network policies design:

• Multi-tenancy
• Application micro-segmentation
• External endpoints
• Host endpoints

Label scope

Labels that are required for defining network policies can have the following scope:

• Namespace: Can be used to define multi-tenancy controls. Role-based access control (RBAC) can be used to restrict creation or modification of namespaces to ensure compliance with high-level multi-tenancy controls.
• Pod: Can be used to define application micro-segmentation controls within the context of one or multiple namespaces.
• Networkset: Calico namespaced resource used for defining endpoints external to the cluster that are accessed by or require access to pods within a given namespace.
• GlobalNetworkset: Calico global resource for defining endpoints external to the cluster that are accessed by or require access to pods in any namespace.

Reserved label keys

You will typically want to restrict the use of certain label keys that can be essential for the integrity of your cluster. Restrictions can be implemented at runtime or in the CI/CD pipeline through mutating or validating webhooks. The following are examples of labels you need to reserve:

• *kubernetes*: reserved for Kubernetes nodes
• *tigera* and *calico*: reserved for Calico components
• Any other identifiers for platform applications or hosts for your Kubernetes distribution or management Kubernetes environment

Multi-tenancy labels

Multi-tenancy labels can be used to identify endpoints belonging to a given tenant, or potentially multiple tenants in the case of shared services. Those endpoints can then be selected in network policies for defining East-West or North-South tenant controls. It is recommended to attach multi-tenancy labels to both pods and namespaces. Depending on your enterprise architecture, your definition of tenant might include any of the following:

• Business unit
• Development team
• Application
• Client
• Shared services
• Environment
• Compliance
• Asset classification

Typical standard of multi-tenancy labels:

• sec.<organization_domain>/tenant-id: <tenant-uid>
• sec.<organization_domain>/environment: <dev|test|prod>
• sec.<organization_domain>/compliance: <compliance-requirement-uid>

Notes:

• sec: prefix that stands for security and identifies labels required for multi-tenancy
• tenant-uid: a unique identifier of a tenant in your environment (e.g., sec.tigera.io/tenant-id:tigera-customer-success-dev)
• compliance-requirement-uid: a unique identifier for a given compliance environment (e.g., sec.tigera.io/compliance: pci-dss-marketplace)

Application micro-segmentation labels

Application micro-segmentation labels can be used to identify endpoints backing microservices of a given application. Those endpoints can then be selected in network policies for defining East-West and North-South controls for microservice communication.

Typically, a deployment model following one application per namespace fits most customer requirements, except for shared services, which typically need to be deployed in their own namespaces to be accessed by multiple tenants. It is recommended that you attach application micro-segmentation labels to pods.

Depending on your application architecture, application micro-segmentation labels might include any of the following:

• Application
• Service
• Application tier
• Version

Typical standard of application micro-segmentation labels:

• app.<organization_domain>/name: <app-uid>
• app.<organization_domain>/instance: <app-instance-uid>
• app.<organization_domain>/version: <app-version>
• app.<organization_domain>/service: <svc-uid>
• app.<organization_domain>/tier: <app-tier>

External endpoint labels

External endpoint labels can be used to identify endpoints external to the cluster. Those endpoints can then be selected in network policies for defining North-South controls for egress and ingress communication from and to your cluster endpoints.

Ingress

• From trusted load-balancers for your tenant for exposed services
• From trusted subnets in your legacy infrastructure
• From trusted partners

Egress

• To services in your legacy infrastructure
• To trusted external API services
• To trusted repository
• To trusted fabric service

Typical standard of external endpoint labels:

• eep.<organization_domain>/ingress-lb: <tenant-id-load-balancer>
• eep.<organization_domain>/egress-api: <tenant-id-api>
• eep.<organization_domain>/egress-db: <tenant-id-db>
• eep.<organization_domain>/egress-repo: <tenant-id-repo>

Note:

• eep: prefix that identifies external endpoints

Host endpoint labels

Host endpoint labels can be used to identify cluster hosts or external hosts. Those endpoints can then be selected in network policies for defining the following controls:

• Cluster hosts: North-South and East-West controls for hosts-to-host, pod-to-host and pods with host networking communication. This is required for cluster hardening and for implementing control-plane and management-plane controls.
• External hosts: North-South and East-West controls for host and legacy applications hosted on bare metal or virtual machines. This is effective for extending cloud-native controls to legacy hosts in integration or migration scenarios.

Depending on your Kubernetes distribution and the role of the node in your cluster, host endpoint labels might include any of the following:

• Kubernetes node role
• Host function
• Architecture
• Operating system
• Region
• Datacenter
• Rack
• Affinity criteria

External host endpoint labels might include multi-tenancy, application micro-segmentation, and any of the above host endpoint labels as appropriate to secure the hosted legacy services.

Typical examples of host endpoint labels, some of which can be native to your Kubernetes distribution:

• kubernetes.io/arch: amd64
• kubernetes.io/os: linux
• node-role.kubernetes.io/worker: true
• node-role.kubernetes.io/egress-gateway: true
• topology.kubernetes.io/region: ca-central-1
• topology.kubernetes.io/zone: ca-central-1a

Best practices

• Develop and communicate a label standard in your organization.
• Define labels for security controls according to a hierarchical design that achieves your organization’s security and compliance objectives.
• Use an intuitive language in your label definition that enables a quick and simple identification of labeled Kubernetes objects.
• Use label key prefixes and suffixes to identify attributes required for asset classification.
• Ensure the right labels are applied to Kubernetes objects by implementing label governance checks in your CI/CD pipeline or at runtime.
• Develop a comprehensive set of labels that meets the deployment, reporting, and security requirements of different stakeholders in your organization.
• Do not abuse the use of labels. Labels should be defined to achieve a specific and explicit purpose.