Replace PSP with Kubewarden policy

October 28, 2022 | By: Karthikeyan Rajendran

Kubewarden – ClusterAdmissionPolicy

Kubewarden is a policy engine for Kubernetes. Its mission is to simplify the adoption of policy-as-code . Since PodSecurityPolicy (PSP) is being deprecated in Kubernetes 1.21, you can use Kubewarden as a replacement to PSP policies .

The Kubewarden team has written a script that leverages the migration tool written by AppVia, to migrate PSP automatically. The tool is capable of reading PSPs YAML and can generate the equivalent policies in many different policy engines. Our simple script migrates your PSPs to their equivalent Kubewarden policies.

In below section we will learn how to perform following tasks ,

* . Install Kubewarden stack
* . Enforce Admission control policy

Now add helm chart and install kubewarden on an existing kubernetes cluster . I have used SUSE Rancher’s RKE2 kubernetes cluster in my setup .

The Kubewarden stack is made of the following components:

An arbitrary number of ClusterAdmissionPolicy resources: this is how policies are defined inside Kubernetes
An arbitrary number of PolicyServer resources: this component represents a Deployment of a Kubewarden PolicyServer. The policies defined by the administrators are loaded and evaluated by the Kubewarden PolicyServer
A Deployment of kubewarden-controller: this is the controller that monitors the ClusterAdmissionPolicy resources and interacts with the Kubewarden PolicyServer components

In order to create Policies we will have to install kubewarden-crds , kubewarden-controller and kubewarden-defaults

Step 1 ) Installation of Cert-manager

Kubewarden chart depends on cert-manager . Since it is a dependency we will have to first install cert-manager .

To Install latest version of cert-manager, on Rancher server UI click on left most corner near Rancher logo ->Home -> rke2-cluster1 -> Kubectl icon !

Run below commands in Kubectl shell :

$ kubectl apply -f https://github.com/jetstack/cert-manager/releases/latest/download/cert-manager.yaml

You should see an output similar to below screen-shot ,

$ kubectl wait –for=condition=Available deployment –timeout=2m -n cert-manager –all

You should see an output similar to below screen-shot ,

Now we have successfully deployed Certmanager in our cluster . The next step would be to install kubewarden stack .

Step 2 ) Deploy Kubewarden stack

The following charts should be installed inside the kubewarden namespace in your Kubernetes cluster:

kubewarden-crds, which will register the ClusterAdmissionPolicy and PolicyServer Custom Resource Definitions
kubewarden-controller, which will install the Kubewarden controller
kubewarden-defaults, which will create a PolicyServer resource named default. It can also installs a set of recommended policies to secure your cluster by enforcing some well known best practices.

Open Kubectl shell . Add kubewarden helm chart using below command ,

$ helm repo add kubewarden https://charts.kubewarden.io

Kubewarden stack can be deployed from above helm chart . Copy paste below commands in kubectl shell ,

$ helm install –wait -n kubewarden –create-namespace kubewarden-crds kubewarden/kubewarden-crds

$ helm install –wait -n kubewarden kubewarden-controller kubewarden/kubewarden-controller

$ helm install –wait -n kubewarden kubewarden-defaults kubewarden/kubewarden-defaults

Wait until you see an output similar to below screen-shot ,

Now we have deployed Kubewarden stack . Next step is to deploy actual policies .

Step 3 ) Enforce Admission Control Policy to avoid ARP spoofing attack

Once you have the Kubewarden instance running, it is time to deploy some policies to replace the PodSecurityPolicy object . The ClusterAdmissionPolicy resource is the core of the Kubewarden stack. This resource defines how policies evaluate requests.

Enforcing policies is the most common operation which a Kubernetes administrator will perform. You can declare as many policies as you want, and each policy will target one or more specific Kubernetes resources (i.e., pods, Custom Resource). You will also specify the type of operation(s) that will be applied for the targeted resource(s). The operations available are CREATE, UPDATE, DELETE and CONNECT.

Kubernetes by default connects all the containers running in the same node (even if they belong to different namespaces) down to Layer 2 (ethernet). This allows a malicious containers to perform an ARP spoofing attack to the containers on the same node and capture their traffic.

In order to avoid such ARP spoofing attack it is important , not to allow NET_RAW capability . The Kubewarden Policy psp-capabilities controls Container Capabilities . In below example you can see NET_RAW capability under required_drop_capabilities section . These are capabilities which must be dropped from containers and are removed from the default set .

Create a yaml file clusteradmissionpolicy.yaml with psp-capabilities kubewarden policy (that replaces the earlier PSP) below content and save it .

apiVersion: policies.kubewarden.io/v1alpha2
kind: AdmissionPolicy
metadata:
     name: drop-cap-net-raw
     namespace: default
spec:
    policyServer: default
    module: registry://ghcr.io/kubewarden/policies/psp-capabilities:v0.1.7
    rules:
    – apiGroups: [“”]
    apiVersions: [“v1”]
      resources:
      – pods
      – deployments
      operations:
      – CREATE
      – UPDATE
   mutating: true
   settings:
       required_drop_capabilities:
       – NET_RAW

Once deployed you should see an output similar to below screen-shot ,

Now let us Create a manifest file named bcisle15default.yaml with below content and save and execute it ,

apiVersion: apps/v1
kind: Deployment
metadata:
      name: bci-sle15
      labels:
          app: sle15
spec:
     replicas: 1
     strategy:
        type: RollingUpdate
     selector:
        matchLabels:
             app: sle15
     template:
        metadata:
              labels:
                  app: sle15
        spec:
            containers:
            – name: sle15
            image: registry.suse.com/suse/sle15:latest
            imagePullPolicy: IfNotPresent
            command: [‘sh’, ‘-c’, ‘echo Container 1 is Running ; sleep 3600’]

This pod should have NET_RAW capability enabled by default as it inherits the same . But since we have enabled the drop-cap-net-raw policy , this capability must be dropped . You can check this by logging into this pod bci-sle15 and run below commands ,

$ zypper install -y libcap-progs
$ capsh –decode=$( cat /proc/$$/status | grep CapEff | cut -d : -f 2 | xargs )

You can see an output similar to below screenshot . You can see the NET_RAW capabilities is gone/dropped in the pod, because of the enforcement by the admission policy in Kubewarden)