Been working on a disaster recovery plan for my Kubernetes cluster and I am able to make snap shots of my managed disks but im not sure how to bind a recovered manager disk to an existing volumn cliam so I can re hydrate my data after a loss of data.
You can mount any disk manually as a volume in a POD to recover data. Better approach would be to use Velero to take k8s configuration backup. It will save the disk and PVC information and should restore the volume claims smoothly.
Additionally, have you looked at AzureCSI drivers? That's the hot stuff in AKS right now. It does support Volume snapshotting and recovery from within the cluster. Best practice still would be to use Velero for configuration with CSI to backup whenever possible.
Related
Not able to set up persistent volume using Azure disk
We are trying to deploy an application on AKS and the application is to use persistent volume. If we use Azure disk, we have noticed if the node having the pod running the application container is stopped / not working , another pod from another node is spinned up but it is no longer accessing the persistent volume.
As per documentation ,azure disk is mapped to a particular node and file share is shared across nodes. What is the way to ensure that a application running on AKS using persistent volume is not lost if a pod/node does not work ?
We are looking for a solution with regard to persistent storage so that an application with 3 pods as a replica set can use an Azure disk persistent volume in AKS.
The Azure disk to work as the persistent storage volume in AKS, it should associates to the actual node, so it cannot share the files between multiple pods. So if you want to share files and persist files between pods whenever the pods in any node, the Azure File Share is a good way for you.
Finally, all of all, if you have multiple nodes and the deployment has 3 replicas. Then the best way to share and persist data between pods is using the Azure File Share or the NFS.
I just installed a DC/OS Cluster on Azure using Terraform. Now I was wondering if it's possible to automatically mount Data Disks of agent nodes under /dcos/volume<N>. As far as I understood the docs, this is a manual task. Wouldn't it be possible to automate this step with Terraform? I was looking through the DC/OS docs and Terraform docs but I couldn't find anything related to auto mounting.
It seems you just can mount the Data disks to the node of AKS manual as a volume. It's a Kubernetes task, not Azure's. Azure only can manage the data disk for you.
What you can do through the Terraform is attach the data disk to the node itself of AKS as a disk, not a volume of the AKS. And the volume, you only can create it through Kubernetes, not Azure. So Terraform also cannot help you achieve it automated.
In Azure, is it possible to have master VM that writes to a disk which has read-only slave replicas on other VMs?
Our app needs to download ~100GB of files when scaling to a new VM. This is loaded slowly from an external provider but we want to make it available quickly when we scale out more VMs.
I don't think you can do streaming replication (which I think is what you're asking for), or read only slave through the Azure service without implementing this yourself over network or through a relational database management system.
As of this writing, one disk cannot be connected to multiple Azure VMs (See FAQ for Managed Disks. One option would be to create a snapshot of the disk, and create a new disk from the snapshot. You could automate this via the Azure Managed Disk Service API (eg: an Azure Powershell script), and it would have to happen on a VM that isn't running.
If your data is same and doesn't change per new VM created then you can have it stored on the Azure File Storage Standard/ Premium. Then have Azure File storage attached to every new VM whenever it is created. snapshot disk will make it pretty complex. Azure Files Storage is good choice in this scenario.
I have worked with EXSi Servers lot of times.They provide snapshot option which can be used to revert back the same server to any point of snapshot that we taken.
I was unable to find the same in AWS and Azure.These cloud enterprises provide the option to backup the server.
AWS backups the whole volume.
Azure provide vault800 backup wizard which is incremental.
We can create a new Server with that backup, but we cannot revert back the same server.The EXSi Server take snapshot 10% of 100% volume of server and revert back as per our requirement.
For Azure, take a look at blob snapshots.
Azure Storage provides the capability to take snapshots of blobs. Snapshots capture the blob state at that point in time.
Pretty much the same story with AWS:
You can back up the data on your Amazon EBS volumes to Amazon S3 by taking point-in-time snapshots. Snapshots are incremental backups, which means that only the blocks on the device that have changed after your most recent snapshot are saved
how about using a 3rd party backup solution like Veeam or cloudberry to take image based backup copies and replicate them onto preferred cloud storage.
Veeam also supports instant VM recovery, you can immediately restore a VM into your production environment by running it directly from the backup file. Instant VM recovery helps improve recovery time objectives (RTO), minimise disruption and downtime of production VMs. It is like having a "temporary spare" for a VM: users remain productive while you can troubleshoot an issue with the failed VM.
I have replicated cassandra database and would like to know the best way to maintain its data.
Currently im using kubernetes emptyDir for cassandra container volume.
Can i use google's Persistent disks for replicated cassandra db pods?
If i have 3 cassandra nodes and one of them fails / destroyed what happens to the google's Persistent disks data?
If all 3 nodes fail, will i still be able to populate db data from google's persistant disks to new pods that spins up?
How to backup db's data which is in google's persistent disks?
I will answer your questions in the same order:
1: You can use Google's persistent disks for the master Cassandra node and then all the other cassandra replicas will just use their local emptyDir.
2: When deploying to the cloud, the expectation is that instances are ephemeral and might die at any time. Cassandra is built to replicate data across the cluster to facilitate data redundancy, so that in the case that an instance dies, the data stored on the instance does not, and the cluster can react by re-replicating the data to other running nodes. You can use DaemonSet to place a single pod on each node in the Kubernetes cluster which will give u data redundancy.
Is it possible to provide more information here? how the new pods will spin up?
Taking a snapshot of the disk, or use epmtyDir with a sidecar container in order to periodically snapshot the directory and upload it to Google Cloud Storage.