Using Secrets Store CSI Driver with HashiCorp Vault
This walkthrough deploys PingDirectory and PingFederate using the ping-devops Helm chart with all secrets sourced from HashiCorp Vault via the Secrets Store CSI Driver.
|
In order for this example to work, you must be on the |
|
This walkthrough uses Vault in development mode — in-memory storage, auto-unsealed, single node. Do not use this configuration in production. Production deployments should use Vault HA with TLS and a dedicated namespace. In addition, the credentials shown here are samples and should not be used in production. |
The Getting Started page has instructions on configuring your environment for using Ping ping-devops Helm charts.
For more examples, see Helm Chart Example Configurations.
Overview
The Secrets Store CSI Driver fetches secrets from Vault and makes them available inside pods as mounted files. Ping product containers already know how to consume secrets this way: the source_secret_envs hook function sources every *.env file found in SECRETS_DIR and exports its KEY=VALUE pairs as environment variables.
This walkthrough uses that native mechanism as the primary delivery path:
-
Each product’s secrets are stored in Vault as a single
contentkey containingKEY=VALUEpairs. -
The CSI driver mounts that content as a
product.envfile at a configurable path. -
Setting
SECRETS_DIRto the mount path tells the Ping startup hooks where to find the file. -
No Kubernetes
Secretobjects are created — secret values exist only as files inside the pod.
|
The Secrets Store CSI Driver fetches one Vault key per However, if you attempt to fetch the entire secret in one shot without specifying a The fix is a storage convention: put all of a product’s credentials into a single Vault key named |
The ping-devops chart integrates with the CSI driver through the secretProviderClass values block: set enabled: true and the chart auto-wires a CSI volume and mount into every pod spec, and optionally creates the SecretProviderClass resource itself.
|
Ping container images default |
Prerequisites
-
kubectlconfigured to a running cluster (Docker Desktop or kind works for this walkthrough) -
helmv3 -
A
PING_IDENTITY_DEVOPS_USERandPING_IDENTITY_DEVOPS_KEY
Infrastructure Setup (Steps 1–4)
|
Steps 1–4 install and configure the Secrets Store CSI Driver, HashiCorp Vault, and the Kubernetes auth plumbing those components require. In a real environment these would already be managed by your platform team. If you have them in place, skip to Step 5 where the |
Step 1: Install the Secrets Store CSI Driver
The CSI driver runs as a DaemonSet on every node.
helm repo add secrets-store-csi-driver \
https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts
helm repo update
helm install csi-secrets-store \
secrets-store-csi-driver/secrets-store-csi-driver \
--namespace kube-system \
--set syncSecret.enabled=true \
--set enableSecretRotation=true
Verify the DaemonSet is running:
kubectl get daemonset -n kube-system \
-l app.kubernetes.io/instance=csi-secrets-store
DESIRED and READY should match (one pod per schedulable node).
Step 2: Install Vault with the CSI Provider
The HashiCorp Helm chart installs both the Vault server and the Vault CSI provider DaemonSet in a single release. The injector (sidecar-based secret delivery) is disabled because we are using the CSI path instead.
helm repo add hashicorp https://helm.releases.hashicorp.com
helm repo update
helm install vault hashicorp/vault \
--namespace vault \
--create-namespace \
--set "server.dev.enabled=true" \
--set "server.dev.devRootToken=root" \
--set "csi.enabled=true" \
--set "injector.enabled=false"
Wait for Vault to be ready:
kubectl wait pod vault-0 \
-n vault \
--for=condition=Ready \
--timeout=120s
Step 3: Seed Vault with Secrets
Open a shell into the Vault pod:
kubectl exec -it vault-0 -n vault -- /bin/sh
The following commands are run inside that shell.
3.1 Authenticate and enable the KV engine
export VAULT_TOKEN=root
export VAULT_ADDR=http://127.0.0.1:8200
vault secrets enable -path=ping kv-v2
3.2 PingDirectory DevOps credentials
DevOps credentials and EULA acceptance are stored as a content key sourced by the Ping startup hooks. Replace the placeholder values with your credentials.
vault kv put ping/pd-env content="PING_IDENTITY_DEVOPS_USER=<username>
PING_IDENTITY_DEVOPS_KEY=<devops_key>
PING_IDENTITY_ACCEPT_EULA=YES"
3.3 PingDirectory password and license secrets
PingDirectory resolves credentials from files at startup via *_FILE environment variables. Store the passwords — and optionally the license file — as individual keys in a separate Vault secret so each can be mounted as its own file.
vault kv put ping/pdpwd - <<EOF
{
"admin-user-password": "SecretAdm1nPa55word",
"encryption-password": "SuperLongEncryptionPassword",
"root-user-password": "3FederateMuchMore",
"pd-license": "<base64-encoded-PingDirectory.lic-content>"
}
EOF
|
This walkthrough uses |
3.4 PingFederate secrets (not for production use!)
vault kv put ping/pf-env content="PING_IDENTITY_DEVOPS_USER=<username>
PING_IDENTITY_DEVOPS_KEY=<devops_key>
PING_IDENTITY_ACCEPT_EULA=YES
PING_IDENTITY_PASSWORD=<admin-password>
PF_RUN_PF_CLUSTER_AUTH_PWD=<cluster-auth-password>"
|
|
Exit the shell:
exit
Step 4: Configure Kubernetes Auth in Vault
The Vault Kubernetes auth method lets pods authenticate to Vault using the ServiceAccount token Kubernetes automatically mounts into every pod. When a pod requests a secret via the CSI driver, the Vault provider presents that token; Vault validates it against the cluster and, if the token’s ServiceAccount matches a configured role, issues a short-lived Vault token scoped to the policy you define.
4.1 Create the namespace and ServiceAccount
kubectl create namespace ping
kubectl create serviceaccount ping-vault-auth -n ping \
--dry-run=client -o json \
| jq '.automountServiceAccountToken = false' \
| kubectl apply -f -
Setting automountServiceAccountToken: false prevents Kubernetes from mounting the SA token at the default /var/run/secrets/kubernetes.io/serviceaccount path. The Vault CSI provider does not use that auto-mounted token — it mounts its own projected token — so disabling automounting removes an unnecessary credential from every pod.
4.2 Enable Kubernetes auth
kubectl exec -it vault-0 -n vault -- vault auth enable kubernetes
Configure the auth method using the Vault pod’s own cluster credentials:
kubectl exec -it vault-0 -n vault -- /bin/sh -c '
vault write auth/kubernetes/config \
kubernetes_host="https://$KUBERNETES_PORT_443_TCP_ADDR:443" \
token_reviewer_jwt="$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \
kubernetes_ca_cert=@/var/run/secrets/kubernetes.io/serviceaccount/ca.crt
'
4.3 Write a policy granting read access to the ping secrets
kubectl exec -it vault-0 -n vault -- vault policy write ping-policy - <<'EOF'
path "ping/data/pd-env" {
capabilities = ["read"]
}
path "ping/data/pdpwd" {
capabilities = ["read"]
}
path "ping/data/pf-env" {
capabilities = ["read"]
}
EOF
4.4 Bind the policy to the ServiceAccount
kubectl exec -it vault-0 -n vault -- vault write auth/kubernetes/role/ping-role \
bound_service_account_names=ping-vault-auth \
bound_service_account_namespaces=ping \
policies=ping-policy \
ttl=1h
Any pod in the ping namespace using the ping-vault-auth ServiceAccount can now request secrets.
Step 5: PingDirectory Values
|
The next two sections provide snippets from a values file for exploration and discussion. Refer to Step 7 for the location of this file. |
The objects block requests three files from Vault: the content key of ping/pd-env (mounted as pingdirectory.env and sourced by the Ping startup hooks), the root-user-password key from ping/pdpwd (mounted as a standalone file), and optionally PingDirectory.lic for static-license deployments (see the note in Step 3.3). Setting SECRETS_DIR injects the .env file’s key-value pairs as environment variables; ROOT_USER_PASSWORD_FILE tells PingDirectory to read its root password from the mounted file.
|
|
pingdirectory:
enabled: true
envs:
SERVER_PROFILE_URL: https://github.com/pingidentity/pingidentity-server-profiles.git
SERVER_PROFILE_PATH: getting-started/pingdirectory
SECRETS_DIR: /run/vault-secrets
ROOT_USER_PASSWORD_FILE: "/run/vault-secrets/root-user-password"
secretProviderClass:
enabled: true
create: true
provider: vault
mountPath: /run/vault-secrets
parameters:
vaultAddress: http://vault.vault:8200
roleName: ping-role
objects: |
- objectName: "pingdirectory.env"
secretPath: "ping/data/pd-env"
secretKey: "content"
- objectName: "root-user-password"
secretPath: "ping/data/pdpwd"
secretKey: "root-user-password"
- objectName: "PingDirectory.lic"
secretPath: "ping/data/pdpwd"
secretKey: "pd-license"
Step 6: PingFederate Values
PingFederate follows the same pattern — one *.env file per pod type.
|
The CSI driver propagates updated Vault secrets to the mounted file automatically, but PingFederate does not re-read environment variables from that file while running. A pod restart is required to pick up rotated secrets. Depending on your operating pattern (Deployment vs. StatefulSet, single node vs. clustered), a rolling restart of the affected pods is typically sufficient. If you rotate |
pingfederate-admin:
enabled: true
envs:
SERVER_PROFILE_URL: https://github.com/pingidentity/pingidentity-server-profiles.git
SERVER_PROFILE_PATH: getting-started/pingfederate
SECRETS_DIR: /run/vault-secrets
secretProviderClass:
enabled: true
create: true
provider: vault
mountPath: /run/vault-secrets
parameters:
vaultAddress: http://vault.vault:8200
roleName: ping-role
objects: |
- objectName: "pingfederate.env"
secretPath: "ping/data/pf-env"
secretKey: "content"
pingfederate-engine:
enabled: true
envs:
SERVER_PROFILE_URL: https://github.com/pingidentity/pingidentity-server-profiles.git
SERVER_PROFILE_PATH: getting-started/pingfederate
SECRETS_DIR: /run/vault-secrets
secretProviderClass:
enabled: true
create: true
provider: vault
mountPath: /run/vault-secrets
parameters:
vaultAddress: http://vault.vault:8200
roleName: ping-role
objects: |
- objectName: "pingfederate.env"
secretPath: "ping/data/pf-env"
secretKey: "content"
Step 7: Deploy
The complete values file combining Steps 5 and 6 is in the getting-started repository at 30-helm/vault-spc/ping-values.yaml. Clone it if you haven’t already, then deploy.
The global.rbac block tells the chart to assign the ping-vault-auth ServiceAccount — which is bound to the Vault role — to every workload pod. Without this, pods authenticate with the default ServiceAccount, which is not authorized in Vault.
git clone https://github.com/pingidentity/pingidentity-devops-getting-started.git
cd pingidentity-devops-getting-started
helm install ping pingidentity/ping-devops \
-n ping \
--set global.rbac.serviceAccountName=ping-vault-auth \
--set global.rbac.applyServiceAccountToWorkload=true \
-f 30-helm/vault-spc/ping-values.yaml
Watch the pods reach Running:
kubectl get pods -n ping -w
Verify
Verify the .env file is mounted
Confirm the CSI driver wrote the secret file into the pod:
kubectl exec -n ping \
$(kubectl get pod -n ping -l app.kubernetes.io/name=pingdirectory -o name | head -1) \
-- ls /run/vault-secrets
Expected:
PingDirectory.lic pingdirectory.env root-user-password
Verify environment variables were sourced
Confirm ROOT_USER_PASSWORD_FILE is set and the .env file was consumed at startup:
kubectl exec -n ping \
$(kubectl get pod -n ping -l app.kubernetes.io/name=pingdirectory -o name | head -1) \
-- env | grep ROOT_USER_PASSWORD_FILE
Expected:
ROOT_USER_PASSWORD_FILE=/run/vault-secrets/root-user-password
|
|
Confirm the password file itself is present:
kubectl exec -n ping \
$(kubectl get pod -n ping -l app.kubernetes.io/name=pingdirectory -o name | head -1) \
-- cat /run/vault-secrets/root-user-password
Alternative: Kubernetes Secret Sync
If you cannot set SECRETS_DIR on the container — for example when a platform team manages the deployment spec — you can use the Kubernetes Secret sync mechanism instead. The secretObjects block instructs the CSI driver to create a Kubernetes Secret from the mounted files; container.envFrom then injects that Secret’s keys as environment variables.
This approach requires one objectName entry per secret key (rather than one per product) and creates Kubernetes Secret objects in the cluster.
|
The Kubernetes |
For this path, seed Vault with individual per-key secrets instead of the content key format used in Step 3. You will also need to update the Vault policy in Step 4.3 to cover these paths:
vault kv put ping/devops \
PING_IDENTITY_DEVOPS_USER="<username>" \
PING_IDENTITY_DEVOPS_KEY="<devops_key>" \
PING_IDENTITY_ACCEPT_EULA="YES"
vault kv put ping/pingdirectory \
ROOT_USER_PASSWORD="3FederateMuchMore" \
ROOT_USER_DN="cn=administrator"
vault kv put ping/pingfederate \
PING_IDENTITY_PASSWORD="<admin-password>"
The values configuration for PingDirectory then becomes:
pingdirectory:
enabled: true
envs:
SERVER_PROFILE_URL: https://github.com/pingidentity/pingidentity-server-profiles.git
SERVER_PROFILE_PATH: getting-started/pingdirectory
secretProviderClass:
enabled: true
create: true
provider: vault
mountPath: /run/vault-secrets
parameters:
vaultAddress: http://vault.vault:8200
roleName: ping-role
objects: |
- objectName: "PING_IDENTITY_DEVOPS_USER"
secretPath: "ping/data/devops"
secretKey: "PING_IDENTITY_DEVOPS_USER"
- objectName: "PING_IDENTITY_DEVOPS_KEY"
secretPath: "ping/data/devops"
secretKey: "PING_IDENTITY_DEVOPS_KEY"
- objectName: "PING_IDENTITY_ACCEPT_EULA"
secretPath: "ping/data/devops"
secretKey: "PING_IDENTITY_ACCEPT_EULA"
- objectName: "ROOT_USER_PASSWORD"
secretPath: "ping/data/pingdirectory"
secretKey: "ROOT_USER_PASSWORD"
- objectName: "ROOT_USER_DN"
secretPath: "ping/data/pingdirectory"
secretKey: "ROOT_USER_DN"
secretObjects:
- secretName: ping-pd-env
type: Opaque
data:
- objectName: PING_IDENTITY_DEVOPS_USER
key: PING_IDENTITY_DEVOPS_USER
- objectName: PING_IDENTITY_DEVOPS_KEY
key: PING_IDENTITY_DEVOPS_KEY
- objectName: PING_IDENTITY_ACCEPT_EULA
key: PING_IDENTITY_ACCEPT_EULA
- objectName: ROOT_USER_PASSWORD
key: ROOT_USER_PASSWORD
- objectName: ROOT_USER_DN
key: ROOT_USER_DN
container:
envFrom:
- secretRef:
name: ping-pd-env
Optional: Using a Pre-Existing SecretProviderClass
If your platform team manages SecretProviderClass resources centrally, set create: false and provide the name. The chart will wire up the CSI volume pointing at that resource without attempting to create or own it. For example:
pingdirectory:
secretProviderClass:
enabled: true
create: false
name: platform-team-pd-spc
Secret Rotation
This section walks through rotating the PingDirectory root user password end-to-end: updating it in Vault, confirming the CSI driver propagates it to the pod, and restarting the pod so the product picks it up.
Before you rotate: add PD_REBUILD_ON_RESTART
PingData products (PingDirectory, PingDirectoryProxy, PingDataSync) write credentials into an internal configuration database at first startup. On subsequent restarts, manage-profile replace-profile checks whether the server profile has changed. Secret files delivered via *_FILE environment variables are not tracked in the profile manifest, so replace-profile detects "no changes" and exits without re-applying credentials — even though the mounted file now contains the rotated value. The pod starts successfully using the old internally-stored password; the rotation silently fails.
Setting PD_REBUILD_ON_RESTART: "true" forces replace-profile to run in full replace mode, re-applying all credentials from the mounted files on every restart.
Add this to the pingdirectory envs block before rotating any credential:
pingdirectory:
envs:
PD_REBUILD_ON_RESTART: "true"
Then apply it:
helm upgrade ping pingidentity/ping-devops \
-n ping \
--reuse-values \
--set-string pingdirectory.envs.PD_REBUILD_ON_RESTART=true
|
Use |
Step 1: Update the secret in Vault
kubectl exec -it vault-0 -n vault -- /bin/sh -c '
export VAULT_TOKEN=root
export VAULT_ADDR=http://127.0.0.1:8200
vault kv patch ping/pdpwd root-user-password="R0tatedNewPa55word!"
'
Step 2: Confirm the CSI driver has propagated the new value
The CSI driver polls Vault on a rotation interval (default: two minutes). Wait for the mounted file to update before restarting the pod:
kubectl exec -n ping \
$(kubectl get pod -n ping -l app.kubernetes.io/name=pingdirectory -o name | head -1) \
-- cat /run/vault-secrets/root-user-password
Repeat until the output shows the new password.
Step 3: Restart the pod
Delete the pod so the StatefulSet recreates it, picking up the rotated secret and running replace-profile with --replaceFullProfile:
kubectl delete pod \
$(kubectl get pod -n ping -l app.kubernetes.io/name=pingdirectory -o name | head -1 | sed 's|pod/||') \
-n ping
|
In this single-pod walkthrough, deleting the pod is safe — the StatefulSet recreates it immediately. In a production deployment with multiple replicas, rotate one pod at a time: delete it, wait for |
Watch it return to Running:
kubectl get pods -n ping -w
Step 4: Verify the new password is active
kubectl exec -n ping \
$(kubectl get pod -n ping -l app.kubernetes.io/name=pingdirectory -o name | head -1) \
-- ldapsearch --useSSL --trustAll -h localhost -p 1636 \
-D "cn=administrator" -w "R0tatedNewPa55word!" \
-b "" -s base "(objectClass=*)" namingContexts
A Result Code: 0 (success) response confirms the rotated password is active.
|
A rotated |
|
Environment variables injected via |
Troubleshooting
Pod stuck in ContainerCreating
kubectl describe pod <pod-name> -n ping
Common events and their causes:
-
failed to get secretproviderclass ping/…-spc— theSecretProviderClassresource does not exist or is in the wrong namespace. Checkkubectl get secretproviderclass -n ping. -
failed to mount secrets store objects— the Vault provider could not authenticate. The most common cause is a mismatch between the pod’s ServiceAccount name or namespace and the Vault role’sbound_service_account_names/bound_service_account_namespaces. Verify the pod is using the correct ServiceAccount withkubectl get pod <pod-name> -n ping -o jsonpath='{.spec.serviceAccountName}'and confirm the Vault role withkubectl exec -it vault-0 -n vault — vault read auth/kubernetes/role/ping-role. -
secrets-store.csi.x-k8s.io/v1 is not available in Capabilities.APIVersions— the CSI driver CRDs are not installed. Re-run Step 1. -
make mountpoint … read-only file systemon the SA token path — the CSI mountPath conflicts with the Kubernetes ServiceAccount token path. SetmountPath: /run/vault-secretsin thesecretProviderClassblock.
Vault authentication errors
kubectl logs -n vault -l app.kubernetes.io/name=vault-csi-provider
Verify the role binding matches the pod’s ServiceAccount:
kubectl exec -it vault-0 -n vault -- vault read auth/kubernetes/role/ping-role
Environment variables not set after pod starts
If the pod reaches Running but expected env vars are missing, confirm the startup hooks found the .env file:
kubectl logs <pod-name> -n ping | grep -E "SECRETS_DIR|\.env"
The log should show SECRETS_DIR : /run/vault-secrets. If it shows the default /run/secrets instead, the envs.SECRETS_DIR override was not applied — check the values file.