How to configure deployment file for Azure Keyvault + SecretProviderClass + imagePullSecrets+ Private docker repository - azure

Question

How to configure deployment file for the combination of Azure Keyvault + SecretProviderClass + imagePullSecrets+ Private docker repository.
We have private docker repository to maintain images, now we have a requirement maintaining the credentials of that Docker repository in Azure key vault, import it into AKS using SecretProviderClass, use that secret under 'imagePullSecrets'
# This is a SecretProviderClass example using system-assigned identity to access your key vault
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
name: azure-kvname-system-harbor
spec:
provider: azure
secretObjects:
- secretName: harborcredentialvault
data:
- key: harborcredentialvaultkey
objectName: harborcredentialvault
type: kubernetes.io/dockerconfigjson
parameters:
usePodIdentity: "false"
useVMManagedIdentity: "true" # Set to true for using managed identity
userAssignedIdentityID: "" # If empty, then defaults to use the system assigned identity on the VM
keyvaultName: "<Keyvault name>"
cloudName: "" # [OPTIONAL for Azure] if not provided, the Azure environment defaults to AzurePublicCloud
objects: |
array:
- |
objectName: harborcredentialvault
objectType: secret # object types: secret, key, or cert
objectVersion: "" # [OPTIONAL] object versions, default to latest if empty
tenantId: "<tenant ID>" # The tenant ID of the key vault
- name: harborcredentialvault
valueFrom:
secretKeyRef:
name: keyvault-secret
key: harborcredentialvaultkey
imagePullSecrets:
- name: ${harborcredentialvault}
volumeMounts:
- mountPath: "/mnt/secrets-store"
name: secrets-store01-inline
readOnly: true
- name: secrets-store01-inline
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: "azure-kvname-system-harbor"

Answer 1

As you do not provided a real question or an error im will be a bit general:
For the AKS/KeyVault integration it is important to understand that you are accessing the Key Vault with the Kubelet Identity of the Nodepool and not with the Managed Identity of the AKS as described here. So if you are using Managed Identity userAssignedIdentityID should not be empty.
So we need to give the Kubelet Identity access to the Key Vault, for example like this:
export KUBE_ID=$(az aks show -g <resource group> -n <aks cluster name> --query identityProfile.kubeletidentity.objectId -o tsv)
export AKV_ID=$(az keyvault show -g <resource group> -n <akv name> --query id -o tsv)
az role assignment create --assignee $KUBE_ID --role "Key Vault Secrets Officer" --scope $AKV_ID
The result of $KUBE_ID needs to be also added the the SecretProviderClass:
userAssignedIdentityID: "RESULT"
From this official example here your SecretProviderClass looks good for this use case.
This would be the pod config:
spec:
containers:
- name: demo
image: demo
volumeMounts:
- name: secrets-store-inline
mountPath: "/mnt/secrets-store"
readOnly: true
imagePullSecrets:
- name: harborcredentialvault
volumes:
- name: secrets-store-inline
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: "azure-kvname-system-harbor"
This should sync the Key Vault secret to a Kubernetes secret. Here is also the documentation.
One thing you should consider is = The secrets will only sync once you start a pod mounting the secrets. Solely relying on the syncing with Kubernetes secrets feature thus does not work.
That being said you maybe would need another pod with a public image to sync your private pull secrets for your cluster bcs your pod would not start as it can not pull the image from your private registry.

Answer 2

#Philip Welz answer helped me to find the below solution
SecretProviderClass sample yaml
# This is a SecretProviderClass example using system-assigned identity to access your key vault
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
name: azure-kvname-system-harbor
spec:
provider: azure
secretObjects:
- secretName: dockerconfig
type: kubernetes.io/dockerconfigjson
data:
- objectName: harborcredentialvault
key: .dockerconfigjson
parameters:
usePodIdentity: "false"
useVMManagedIdentity: "true" # Set to true for using managed identity
userAssignedIdentityID: "" # If empty, then defaults to use the system assigned identity on the VM
keyvaultName: "<Keyvault name>"
cloudName: "" # [OPTIONAL for Azure] if not provided, the Azure environment defaults to AzurePublicCloud
objects: |
array:
- |
objectName: harborcredentialvault
objectType: secret # object types: secret, key, or cert
objectVersion: "" # [OPTIONAL] object versions, default to latest if empty
tenantId: "<tenant ID>" # The tenant ID of the key vault
Deployment sample yaml file
spec:
containers:
- name: demo
image: demo
volumeMounts:
- name: secrets-store-inline
mountPath: "/mnt/secrets-store"
readOnly: true
imagePullSecrets:
- name: dockerconfig
volumes:
- name: secrets-store-inline
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: "azure-kvname-system-harbor"
Create Secret in Keyvault, make sure value should be in below JSON format
Key: harborcredentialvault
Value: {
"auths": {
"dockerwebsite.com": {
"username": "username",
"password": "password"
}
}
}