FreeBSD Storage for OpenShift with Democratic CSI | 𝚟𝚎𝚛𝚖𝚊𝚍𝚎𝚗

Today quite different topic, using ZFS/NFS on FreeBSD as backend storage for containers/pods running on OpenShift platform.

Serving storage for OpenShift is not as simple as OpenShift will not just consume NFS or iSCSI volumes "just like that" … OpenShift requires a special CSI driver in between to make that even work.

My first association with CSI is of course CSI series like CSI: Miami … but the real name is Container Storage Interface of course :

While the Democratic CSI driver is suited mostly for Linux based systems like TrueNAS SCALE for example – its zfs-generic-nfs driver also works with FreeBSD – and this is what we will use today. The FreeBSD setup is very simple – its just NFSv4 only server and a dedicated NFS share for OpenShift along with dedicated ZFS dataset. One can also use any regular user with added ZFS permissions done via zfs allow command or a regular user with sudo(8) to lift up the permissions.

The FreeBSD/ZFS/NFS part was done my be – as I am not expert in the OpenShift domain – the OpenShift commands were done by luckyonesl and shared with his approval – thank You for help.

FreeBSD Server ZFS/NFS Setup

Latest FreeBSD 15.1-RELEASE was used for installation with Auto (ZFS) option. One can also use ready to download one of the FreeBSD VM-IMAGES that project also creates – in that case something ZFS based is needed. Just set the root password to something You will later use in the config. Using SSH keys instead of password is also possible. FreeBSD will use 10.0.0.9 IP address.

freebsd # echo "something+VERY-insecure-123" | pw usermod -n root -h 0

First create the ZFS dataset – as this is only for demonstration purposes I will just create zroot/openshift with /zroot/openshift as its mountpoint.

freebsd # zfs create -o /zroot/openshift zroot/openshift

As /etc/rc.conf is very basic – I will only focus on the NFS part here.

nfsv4_server_enable=YES<br>nfsv4_server_only=YES<br>mountd_enable=YES<br>nfs_server_enable=YES<br>nfs_server_flags="-t -n 64"<br>nfs_server_maxio=131072

The /etc/exports file for the NFS share. The hosts 10.0.0.10-13 are the nodes of OpenShift cluster.

V4: / -sec=sys<br>/zroot/openshift -sec=sys -maproot=root 10.0.0.10 10.0.0.11 10.0.0.12 10.0.0.13

Now start the NFS server.

freebsd # service nfsd start

OpenShift Setup

Next are the needed OpenShift commands which were done by luckyonesl and shared with his approval as I am not that proficient in OpenShift.

One can use privateKey: instead of password: for more security – this is only for demonstration purposes.

Now the installation of Democratic CSI driver with helm(8) command.

openshift # cat /root/democratic-csi-install.yaml<br>image:<br>repository: ghcr.io/democratic-csi/democratic-csi<br>tag: v1.9.3<br>csiDriver:<br>name: "org.democratic-csi.controller-zfs-generic"

controller:<br>hostNetwork: true<br>dnsPolicy: ClusterFirstWithHostNet

driver:<br>config:<br>logLevel: debug<br>driver: zfs-generic-nfs<br>sshConnection:<br>host: 10.0.0.9<br>port: 22<br>username: root<br>password: something+VERY-insecure-123<br>zfs:<br>cli:<br>paths:<br>zfs: /sbin/zfs<br>zpool: /sbin/zpool<br>sudo: /usr/local/bin/sudo<br>datasetParentName: "zroot/openshift"<br>detachedSnapshots:<br>enabled: false<br>datasetPermissionsMode: "0777"<br>nfs:<br>shareStrategy: "setDatasetProperties"<br>shareStrategySetDatasetProperties:<br>properties:<br>sharenfs: "on"<br>shareHost: 10.0.0.9

storageClasses:<br>- name: democratic-nfs<br>defaultClass: false<br>reclaimPolicy: Delete<br>volumeBindingMode: Immediate<br>allowVolumeExpansion: true

volumeSnapshotClasses:<br>- name: democratic-nfs-snapshots<br>parameters:<br>detachedSnapshots:<br>enabled: false

openshift # helm repo add democratic-csi https://democratic-csi.github.io/charts/

openshift # helm repo update

openshift # helm upgrade \<br>--install democratic-csi democratic-csi/democratic-csi \<br>-n democratic-csi \<br>--create-namespace \<br>-f /root/democratic-csi-install.yaml

Next check how the installation went.

openshift # oc get deployment,ds -n democratic-csi -o yaml | grep -iE 'hostNetwork'

openshift # oc get deployment,ds -n democratic-csi -o yaml | grep -iE 'hostNetwork|mountPropergation|privileged'

Sometimes additional polices/privileges are needed – so here are the commands for them.

openshift # oc adm policy add-scc-to-user privileged system:serviceaccount:democratic-csi:democratic-csi-controller-sa

openshift # oc adm policy add-scc-to-user privileged system:serviceaccount:democratic-csi:democratic-csi-node-ss

Tests on OpenShift

Now the Democratic CSI driver seems to be installed – lets use it as storage for OpenShift containers and create some snapshot(s). First the YAML files that will be used.

openshift # cat /root/csi-snapshot-test.yaml<br>apiVersion: v1<br>kind: Namespace<br>metadata:<br>name: csi-test<br>apiVersion: v1<br>kind: PersistentVolumeClaim<br>metadata:<br>name: source-pvc<br>namespace: csi-test<br>spec:<br>storageClassName: democratic-nfs<br>accessModes:<br>- ReadWriteMany<br>resources:<br>requests:<br>storage: 1Gi<br>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>name:...