In Azure Batch when creating a pool in the portal you can create a DataDisk and set it's size in GB as well as choose between Standard LRS and Premium LRS.
When using Powershell and/or the .NET libraries you can also set up a MountConfiguration to a FileShare (as well as Blobs, etc).
I'm confused as to what the difference is between the two. Specifically between a DataDisk and a Mounted FileShare.
For my scenario I want to use the lowest powered Linux VM possible but need at least 500GB of storage isolated to each node (no need for sharing across nodes).
I added a DataDisk to my pool since it seemed simpler than mounting a FileShare but my nodes do not have access to the additional file storage. Are there additional configurations that need to be made to the job or task? Does it need to be mounted to a drive letter like a FileShare does?
If I add a 500GB DataDisk to my pool is that shared across all the nodes that are running or does each new node get their own 500GB partition?
There does not seem to be much documentation on DataDisks for Azure Batch. In fact searching for the term within the Batch documentation has 0 results!
• When you add a data disk of a particular size to a batch pool, it is added to all the nodes existing or created in that batch pool, i.e., if you are adding a data disk of 500 GB to a batch pool and you created 4 nodes in that pool, then all the 4 nodes will be attached with a data disk of 500 GB individually. If these nodes are Linux VMs, then they will be attached with the data disk individually and you need to initialize these data disks from within the VM. To mount the disks and partition them, please follow the below documentation: -
https://learn.microsoft.com/en-us/azure/virtual-machines/linux/attach-disk-portal#connect-to-the-linux-vm-to-mount-the-new-disk
By following the above documentation, you will be able to mount these data disks individually to all the nodes from within the VM.
• When you add a data disk in a VM, you won’t be able to see them until you initialize them or format them from within the VM, thus you will need to login to every node and then partition it or initialize the disk for it to be visible and used.
Data disks are dedicated storage spaces or attached disks to a system/VM which can be shared with another resource likewise unless enabled but File shares are network mounted and partitioned storage volumes that are available over the network to all provisioned resources/VMs/systems. File shares like data disks have a fixed disk space/size but it is shared equally amongst the shared resources unless quota is allocated to each resource accessing the file share.
The above is same for nodes in an Azure batch pool also.
Please find the below links for your reference: -
https://learn.microsoft.com/en-us/azure/batch/virtual-file-mount?tabs=linux
Related
I use AKS to deploy the application services. I have used the VM size of Standard_D4a_v4 with 4vCPU and 16Gi Memory configuration for the worker nodes.
Max data disks specified for the above mentioned configuration is 8. I need to clarify that, if only 8 pvcs of azure disks provisioner can be mounted to the worker node or is that possible to mount more pvcs (>8) with azure file provisioner?
• Since you are using a VM size of Standard_D4a_v4 with the said configuration of 4 vCPUs and 16GB of memory, the maximum number of data disks that can be created for this size are ‘8’ as for each VM vCPU, two data disks can be attached up to an absolute maximum of 64 disks per virtual machine. Also, I tested the above said in my environment also as below: -
Also, after attaching the maximum number of data disks as above, I also tried to connect an Azure file share as a volume in a VM as below and I was able to successfully mount it and access it. Thus, though you cannot add more data disks to a VM according to the said limitations, but you can surely connect file shares successfully and access them as network volumes through azure file provisioner.
Please find the below links for more information: -
https://learn.microsoft.com/en-us/azure/virtual-machine-scale-sets/tutorial-use-disks-cli#azure-data-disks
https://learn.microsoft.com/en-us/azure/virtual-machines/dav4-dasv4-series#dav4-series
I own a virtual machine (classic) on Azure that uses 4 data disks, each making 50 GB. These disks are grouped together in a storage pool.
Is it possible to increase the size (up to 100 GB for example) of disks despite the storage pool ?
I have already made a large increase on a single disk successfully (with powershell) but never on a storage pool.
I want to be sure that there is no danger to the data currently on the disks.
Thanks for your help.
Based on my knowledge, it is not possible to increase physical disk to expand storage pool.
As a workaround, maybe you can resize your Azure VM to a high size, then extend the pool by adding more physical disks.
You can use PowerShell to add physical disk Add-PhysicalDisk.
$toadd = Get-PhysicalDisk -FriendlyName "Msft Virtual Disk"
Add-PhysicalDisk -StoragePoolFriendlyName poolname -PhysicalDisks $toadd
I have an Azure Windows Server VM running for several months.
Because of some historical reasons, I have two almost empty volumes on my Windows Server, I want to delete them to get rid of expensive bills.
I'm looking into ways to Delete volumes from Azure Windows Sever, and I believe you can achieve this by delete resources from Azure Portal.
However, I'm struggling finding the Mapping between DataDisk Resource and Windows Volume. (from my research, I think there likely to be an one-to-one mapping)
And for certain reason, I'm saving starting a new VM and migrate everything as the last resort.
Thanks!
UPDATE with details:
As I'm really struggling with this AZURE structure, I'm updating it with a lot of screenshots:
this is I have from all resources:
You can see I have 2 1TB disks, one is premium and another is standard
this is I have from clicking my virtual machine then click disks:
BUT IN HERE, I END UP HAVE TWO PREMIUM DISKS
And the following is powershell output on my Windows Server (disk4 does not have a LUN in the output):
I MUST BE MISSING SOMETHING, THINGS JUST DONT ADD UP!
I'm struggling finding the Mapping between DataDisk Resource and
Windows Volume.
You are right, Azure data disk and windows Volume to be an one-to-one mapping.
We can use get-disk to find the windows volume information, in this way, we can find LUN of this volume.
Here is my test, Windows server 2016.
Run get-disk command in Azure VM:
Via Azure portal we can find the LUN:
In this way, we can find which Azure VM disk map to windows volume.
Update:
Module: storage space, this means that disk create from storage pools(storage space).
Like this:
In this scenario, we can use this command Get-PhysicalDisk to get the disk information:
Also we can find the storage pool in Azure VM:
Update2:
Q1: Yes, we map Physical disk to storage pool, the create one or more disks in from this storage pool. For example, Physical disk about 1TB, we map this disk to storage pool, so the storage pool is 1TB, then we can create a disk from this storage pool, this disk should be smaller than 1TB, if we create 3 disks, disk1 + disk2 + disk3 = 1TB.
Q2: because he only create a disk from this storage pool, and the disk = 1TB, that means the disk = the storage space = TB, so the free space is 0.00B
Q3: yes, Unattached means this disk not in use at that time.
I got into an issue with the size of the Log folder on the first node of a Service Fabric cluster.
It seems that there isn't any upper limit on the disk space it can use and it will eventually eat up the entire local disk.
The cluster was created via an ARM template and I set up two storage accounts associated to the cluster. The variable names for the storage accounts are: supportLogStorageAccountName and applicationDiagnosticsStorageAccountName
However, the etl files are written only to local cluster nodes disks and not to the storage (where I could find dtr files).
Is there any way to set the destination to the etl files to an external storage or at least to limit the size of the Log folder? I wonder if the parameter overallQuotaInMB in the ARM template could be related to that.
You can override the MaxDiskQuotaInMb setting from the default of 10240 to reduce the disk usage by etl's. (We use the local logs in cases where Azure storage isn't available for some reason)
https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-cluster-fabric-settings
When creating the VM I'm asked about Storage configuration. When I select IOPS=0 (the minimum is otherwise 5000), Throughput=0 and Storage size=0, the info text is
0 data disks will be added to the virtual machine. This value was computed based on the value of IOPS, throughput, and storage size.
When the VM is created and I go to the Storage account, select Blobs and Container named vhds I see two disks, one 127GB and one 1TB disk.
Since the 1TB premium disks costs >100€/month I don't want that.
I tried removing the disk from a created machine but when I tried to add a new I got the error that "LUN :0 is already in use".
Preferably I would like to create machine correctly from the start. How can I do that?
This is correct. The current SQL Server IaaS experience on Azure Portal would creates one disk of 1TB even if specify 0 IOPS. We will add a fix to ensure the user cannot specify IOPS below 1 TB disk. If you need SQLVM without disks or any other configurations, you may use Azure PowerShell to create the VM.