Security Context Constraints
This section covers how to add SCCs to your operator for running privileged app or agent workloads on OpenShift.
A Security Context Constraint is an OpenShift extension to the Kubernetes security model, that restricts which Security Contexts can be applied to a pod. This both prevents unprivileged (non cluster-admin) users from being able to apply Security Contexts that would allow a pod definition to forcefully override which UID (possibly as root / UID 0), GID, or fsGroup that a pod can run with, as well as which host volumes and node networking interfaces can be accessed by a pod.
To summarize, SCCs prevent unprivileged users from being able to run privileged containers on an OpenShift cluster, by restricting which user and/or group that a pod is able to run as on the worker nodes in the cluster. SCC's also prevent pods from gaining access to local resources on the nodes. SCCs restrict all of these things by default, unless explicitly overridden by a cluster-admin user or service account.
By default, OpenShift runs all pods with the
restricted SCC. This causes pods to run with a randomized UID in a very high numerical range (100000+) and disregards the USER or UID specified in the container image Dockerfile (unless explicitly set to root, in which the pod will be prevented from running at all). The next section will describe alternative SCCs and when to use them.Security Context Constraints are covered in depth in the official OpenShift documentation at https://docs.openshift.com/container-platform/latest/authentication/managing-security-context-constraints.html.
Given the nature of the restricted SCC, it's quite commonplace that a pod might need to run with a certain UID (as with many databases), or sometimes even as root, when the use case justifies doing so (as with host monitoring agents). This sort of classifies container workloads into three cases:
- Containers that are completely agnostic as to what UID or GID they run with (most operators)
- Containers that simply need to run as a certain UID (even root)
- Containers that require privileged, host-level access in order to run
The first case is handled easily by the default,
restricted SCC. The second case requires the anyuid SCC, which will allow the container to run with the USER specified in the Dockerfile (even root). The third, and most special case is one that absolutely requires not only running as root, but access to node resources at the host level, and is handled by the privileged SCC. As such, the privileged SCC should be used with care and where it's justified, as mentioned previously. OpenShift does provide several other SCCs which allow more granular access, but SCC's aren't stackable, therefore you can't make a concoction of, say anyuid, hostaccess, and hostnetwork SCCs to grant each set of permissions to your pod. You can assign multiple SCC's to a service account, but the SCC with the least amount of privileges will take precedence over all the others. This makes the other SCC's less useful in common practice.It's not common for an operator to require a non-default SCC, since they only make API calls back to the K8s environment where they're being run. However, it's quite common for an operand (pod or pods deployed and managed by an operator) to require a non-default SCC.
There are ways to get around issues with container filesystem permissions with the restricted SCC, but doing so would require making changes to the container's Dockerfile, and therefore rebuilding the container image to allow read and/or write access to certain files/directories by the root group (GID 0).
If required, the last step is to apply either the
anyuid or the privileged SCC to your operator or operand. This is done by modifying the set of RBAC rules being shipped with the operator. This will soon be managed directly by shipping a ClusterRole manifest in a containerized operator bundle, but today this requires modifying the ClusterServiceVersion from your metadata bundle to add a clusterPermissions field with the necessary RBAC rules.For example, to add the
anyuid SCC to a service account named my-operand , the following block would get added to the ClusterServiceVersion of your operator bundle:spec:
install:
spec:
clusterPermissions:
- rules:
- apiGroups:
- security.openshift.io
resources:
- securitycontextconstraints
resourceNames:
- anyuid
verbs:
- use
serviceAccountName: my-operand
You can see where we define the
anyuid SCC under resourceNames, and we set the serviceAccountName that this ClusterRole will apply to as my-operand. To set the privileged SCC instead, set the field under resourceNames to privileged. Note that hyphens are required in the specified places, as these data types are enforced by the K8s API as arrays, which are denoted by a hyphen in the YAML syntax.A full, albeit commented out example of including a clusterPermissions field into a ClusterServiceVersion definition can be found on our GitHub project: https://github.com/RHC4TP/operators/blob/master/examples/operator_sdk/helm/mariadb-operator/bundle/mariadb.v0.0.4.clusterserviceversion.yaml#L269-#L280