I used AzureFileShare to create a dynamic PVC for a pod where I deployed a NodeJS application.
Below is the yaml of the storageclass I used to create the pvc,
apiVersion: storage.k8s.io/v1
metadata:
name: my-azurefile
provisioner: kubernetes.io/azure-file
mountOptions:
- dir_mode=0777
- file_mode=0777
- uid=0
- gid=0
- mfsymlinks
- cache=strict
- actimeo=30
parameters:
skuName: Standard_LRS
The yaml file I used to create the pvc,
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: dynamic-pvc
namespace: test
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 10Gi
storageClassName: my-azurefile
I took the backup of the namespace where the pod is deployed using velero. When I restored the backup in a different cluster, I see no data present in the pod. But when I use dynamic azuredisk pvc, I am able to restore the pod with the data.
NOTE: Before restoring the velero backup, I have created the my-azurefile storageclass in the new cluster where I performed the restoration.
Can anyone please explain why the restoration is not happening properly with the data when I use dynamic azurefile pvc? Thanks in Advance!
kubectl patch storageclass/<YOUR_AZURE_FILE_STORAGE_CLASS_NAME>
--type json
--patch '[{"op":"add","path":"/mountOptions/-","value":"nouser_xattr"}]'
As per the above patch method, I am able to restore the data from azurefile pv.
Related
The way to mount azure file as a PV is shown in tutorials as below:
apiVersion: v1
kind: PersistentVolume
metadata:
name: azurefile
spec:
capacity:
storage: 5Gi
accessModes:
- ReadWriteMany
azureFile:
secretName: azure-secret
shareName: aksshare
readOnly: false
mountOptions:
- dir_mode=0777
- file_mode=0777
- uid=1000
- gid=1000
- mfsymlinks
- nobrl
Currently I need to create a secret with the storage account key to get the pv working.
Is there a way to mount the file share without creating a secret, preferably using environment variables? For example, putting the account key in environment variables or using service principal credentials.
I also attempted to use kustomise secretGenerator to create the secret from env variables, but the secret name is different each time and I can't use it in the pv yaml file.
Kubernetes is secret or configmap is one way of injecting secret only to deployments.
If you are storing something in secret that means you can access it as an environment variable.
You can create a storage class and use it something like:
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: azurefile
provisioner: kubernetes.io/azure-file
mountOptions:
- dir_mode=0777
- file_mode=0777
- uid=0
- gid=0
- mfsymlinks
- cache=strict
- actimeo=30
parameters:
skuName: Standard_LRS
With this only you can add the secret so you don't have to add it multiple time.
I am using azure csi disk driver method for implementing K8 persistent volume. I have installed azure-csi-drivers in my K8 cluster and using below mentioned files as end-to-end testing purpose but my deployment is getting failed due to following error :
Warning FailedAttachVolume 23s (x7 over 55s) attachdetach-controller
AttachVolume.Attach failed for volume "pv-azuredisk-csi" : rpc error:
code = NotFound desc = Volume not found, failed with error: could not
get disk name from
/subscriptions/464f9a13-7g6o-730g-hqi4-6ld2802re6z1/resourcegroups/560d_RTT_HOT_ENV_RG/providers/Microsoft.Compute/disks/560d-RTT-PVDisk,
correct format:
./subscriptions/(?:.)/resourceGroups/(?:.*)/providers/Microsoft.Compute/disks/(.+)
Note: I have checked multiple times, my URL is correct but I am not sure if underscore in resource group name is creating any problem, RG = "560d_RTT_HOT_ENV_RG". Please suggest if anyone have any idea what is going wrong?
K8 version : 14.9
CSI drivers : v0.3.0
My YAML files are :
csi-pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv-azuredisk-csi
namespace: azure-static-diskpv-csi-fss
spec:
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
csi:
driver: disk.csi.azure.com
readOnly: false
volumeHandle: /subscriptions/464f9a13-7g6o-730g-hqi4-6ld2802re6z1/resourcegroups/560d_RTT_HOT_ENV_RG/providers/Microsoft.Compute/disks/560d-RTT-PVDisk
volumeAttributes:
cachingMode: ReadOnly
fsType: ext4
-------------------------------------------------------------------------------------------------
csi-pvc.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: pvc-azuredisk-csi
namespace: azure-static-diskpv-csi-fss
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
volumeName: pv-azuredisk-csi
storageClassName: ""
nginx-csi-pod.yaml
kind: Pod
apiVersion: v1
metadata:
name: nginx-azuredisk-csi
namespace: azure-static-diskpv-csi-fss
spec:
nodeSelector:
beta.kubernetes.io/os: linux
containers:
image: nginx
name: nginx-azuredisk-csi
command:
"/bin/sh"
"-c"
while true; do echo $(date) >> /mnt/azuredisk/outfile; sleep 1; done
volumeMounts:
name: azuredisk01
mountPath: "/mnt/azuredisk"
volumes:
name: azuredisk01
persistentVolumeClaim:
claimName: pvc-azuredisk-csi
It seems you create the disk in another resource group, not the AKS nodes group. So you must grant the Azure Kubernetes Service (AKS) service principal for your cluster the Contributor role to the disk's resource group firstly. For more details, see Create an Azure disk.
Update:
Finally, I found out the reason why it cannot find the volume. I think it's a stupid definition. It's case sensitive about the resource Id of the disk which you used for the persist volume. So you need to change your csi-pv.yaml file like below:
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv-azuredisk-csi
namespace: azure-static-diskpv-csi-fss
spec:
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
csi:
driver: disk.csi.azure.com
readOnly: false
volumeHandle: /subscriptions/464f9a13-7g6o-730g-hqi4-6ld2802re6z1/resourcegroups/560d_rtt_hot_env_rg/providers/Microsoft.Compute/disks/560d-RTT-PVDisk
volumeAttributes:
cachingMode: ReadOnly
fsType: ext4
In addition, the first paragraph of the answer is also important.
Update:
Here are the screenshots of the result that the static disk for the CSI driver works on my side:
I have been trying to deploy Cassandra using following documentation
https://kubernetes.io/docs/tutorials/stateful-application/cassandra/
deployment of Cassandra works fine but when i try to create statefull set it gives following error :
Cassandra 0
pod has unbound immediate PersistentVolumeClaims (repeated 2 times)
can any one help me where am i doing wrong ?
A stateful set requires a persistent volume where store the state, the docs you provide there is a section that shows it:
volumeClaimTemplates:
- metadata:
name: cassandra-data
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: fast
resources:
requests:
storage: 1Gi
---
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: fast
provisioner: k8s.io/minikube-hostpath
parameters:
type: pd-ssd
these are the docs to create a PV and/or Storage class in Azure as you need
https://learn.microsoft.com/en-us/azure/aks/azure-files-dynamic-pv
then you can associate the object with your StatefulSet
Did you create correct storage class and named it fast?
Try with this one (should work on azure):
...
volumeClaimTemplates:
- metadata:
name: cassandra-data
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: fast
resources:
requests:
storage: 1Gi
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: fast
parameters:
fsType: xfs
kind: Managed
storageaccounttype: Premium_LRS
provisioner: kubernetes.io/azure-disk
reclaimPolicy: Delete
volumeBindingMode: Immediate
In Kubernetes (Azure AKS), how do I create a PersistentVolume resource that is bound to my own managed disk Azure resource that has a specific diskName and diskURI (resource id)?
Here is one example but for a Pod:
kind: Pod
apiVersion: v1
metadata:
name: mypodrestored
spec:
containers:
- name: myfrontendrestored
image: nginx
volumeMounts:
- mountPath: "/mnt/azure"
name: volume
volumes:
- name: volume
azureDisk:
kind: Managed
diskName: pvcRestored
diskURI: /subscriptions/<guid>/resourceGroups/MC_myResourceGroupAKS_myAKSCluster_eastus/providers/Microsoft.Compute/disks/pvcRestored
You can create your own managed disk in the AKS cluster and then attach it to the pod which you need. For more details about the steps, see Volumes with Azure disks.
The result will like this:
I'm trying to run the Jenkins Helm chart. As part of this setup, I'd like to pass in a persistent volume that I provisioned ahead of time (or perhaps exported from another cluster during a migration).
I'm trying to get my persistent volume (PV) and persistent volume claim (PVC) setup in a such a way that when Jenkins starts, it uses my predefined PV and PVC.
I think the problem originates from the persistent storage definition for the Azure disk points to a VHD in my storage account. Is there any way to point it to an existing managed disk -and not a blob?
This is how I setup my persistent storage using Azure Managed Disk
apiVersion: v1
kind: PersistentVolume
metadata:
name: jenkins-home
spec:
capacity:
storage: 10Gi
storageClassName: default
azureDisk:
diskName: jenkins-home
diskURI: https://<storageaccount>.blob.core.windows.net/jenkins-data/jenkins-home.vhd
fsType: ext4
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
claimRef:
name: jenkins-home-pvc
namespace: default
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: jenkins-home-pvc
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
storageClassName: default
I then start helm like this...
helm install --name jenkins stable/jenkins --values=values.yaml
Where my values.yaml file looks like
Persistence:
ExistingClaim: jenkins-home-pvc
Here is the error I receive when the Jenkins' pod starts.
AttachVolume.Attach failed for volume "jenkins-home" : Attach volume "jenkins-home" to instance "aks-agentpool-40897452-0" failed with compute.VirtualMachinesClient#CreateOrUpdate: Failure responding to request: StatusCode=409 -- Original Error: autorest/azure: Service returned an error. Status=409 Code="OperationNotAllowed" Message="Addition of a blob based disk to VM with managed disks is not supported."
I posed this question to the Azure team here.
Through their help I arrived at the following solution...
I had tried to use the managed disk resource ID before but it yelled at me saying it expected a .vhd file. But after adding 'kind: Managed', it was perfectly happy to take the managed disk resource id.
Creating an empty and formatted managed disk is of course a pre-requisite for this to work. Copying the managed disk into the same resource group as the AKS cluster was also required.
So now my PV and PVC look like this and it's working...
apiVersion: v1
kind: PersistentVolume
metadata:
name: jenkins-home
spec:
capacity:
storage: 10Gi
storageClassName: default
azureDisk:
kind: Managed
diskName: jenkins-home
diskURI: /subscriptions/{subscription-id}/resourceGroups/{aks-controlled-resource-group-name}/providers/Microsoft.Compute/disks/jenkins-home
fsType: ext4
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
claimRef:
name: jenkins-home-pvc
namespace: default
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: jenkins-home-pvc
annotations:
volume.beta.kubernetes.io/storage-class: default
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
storageClassName: default