I have two VMs using the same vnet and I would like to be able to copy a directory that has thousands of files and is about 400mb.
I can use a UNC path to copy the files, but that takes 2 minutes. I’ve also tried using a storage account and created a file share, but that is also slow.
Are there any other Azure resources that might make getting files from one VM to another faster?
As the comment point it out, if you have two VMs in the same VNet, you should use its private address. Traffic between the two VMs is kept on the Azure backbone network. You could directly copy/paste the files from one VM to another VM when you RDP to that VM.
Also, different VM size has different performance. For the best performance, It's recommended that you migrate any VM disk that requires high IOPS to Premium Storage. VM disks that use Premium Storage store data on solid-state drives (SSDs).
High-performance Premium Storage and managed disks for VMs
Related
we are using Azure VM Scale sets to compute a larger job. In some stages we want the machines to share data with each other. We have tried Blob Storage for this but it's way to slow.
We are looking at either make the computers talking to each other or a simpler solution having them share a network drive (a fast one, being close to the actual hardware). Is this available in Azure? How we have understood it Azure Files is as slow as Blob storage because it's on top of blob storage.
Is it possible to create a disk that is shared between VM's in an Azure Scale Set?
No, this is not possible. you might use network shares.
well, you can implement Software Defined Storage cluster, but thats probably overkill
Is it's possible to attach the same premium storage to multiple VMs so the files stored in the storage can be access in all of them.
The idea is to have a VM optimized for CPU that will calculate something and write results to the storage and have a low cost VM that will read the results and do other operations.
So if by saying "same" you mean same storage account - yes, you can do that, if by "same" you mean same VHD, no, you cant simultaneously attach same VHD to different VM's.
But you can have Azure Storage Files take on that role, it works like an SMB share, were you can store the results and other nodes will read them. Or you could just create a share on some VM that is supposed to read the results and store the results there.
Either way, its perfectly doable.
I am using two Microsoft Azure Virtual Machines (marked as classic), both running on Linux. One is used for test purposes and internal demos, the other is production and running few of clients' instances.
What I would like to do is change the size of Virtual Machine. I understand this is quite common process and can easily be done from the Azure Management Portal and that this is not affecting data. However, when I have changed the size of our testing machine, exactly this has happened and we have lost all data.
Azure Support answer received was:
"We recommend you delete the VM by keeping the attached disks and create a new VM with the required size." Not sure why this would be better?
Any data stored on the ephemeral (internal-to-chassis) scratch disk is at risk, as it's a non-durable disk (and will in all likelihood be destroyed/recreated upon resizing a VM).
The only way to have durable data is to use Azure Storage (blobs, vhd as attached disk, Azure File storage) or external database. Azure Storage is durable (minimum 3 copies), and is not stored with your VM.
One more thing: The VM's OS Disk is a VHD in Azure Storage (so the OS disk is durable, just like attached vhd's).
You have more than one way to do that and keep in mind what David said, data on OS disks, attached disks and blobs is the only durable one.
To prevent losing data and since you're using Classic VMs, you can do the following:
1- Go to your VM on portal and capture an image out of it.
2- Go to your new image and create a new VM out of it, while specifying the new specs that you need.
3- When done, connect to your new VM while keeping the old one without termination.
4- Check if all your data is there, if yes, then you can remove the old one. (In case you need the old IP, you can still assign it to the new one).
Cheers.
Is it possible to create a Azure VM (medium or large) running an instance of SQL Server. This machine would contain various databases (one of each client). These databases would then need to be accessible from different Azure VM's which would be hosting either a single website per vm or multiple website per vm each requiring to access different databases.
If it is possible:
How do the other Azure VMs connect to the relevant VM that's running SQL server and connect to a specific database?
What cost implication are there? We store a lot of data including documents as blobs.
Our software supports both storing the documents as blobs or as actual files. Should we consider storing the files directly onto the hard drive instead of storing them in the database? Would this improve performances when running on Azure VM and/or reduce cost?
What's the recommended solution for backup? I assume that Azure VM can go down, though less likely, just like any other machines. What the recommended solution to restore a back?
Same for SQL Server. What is the recommended solution to backup database. Since we'd be running a full version of SQL on a Azure VM, is the best solution to have a separate storage drive and backup databases on this?
Thanks.
1-)You can create a Virtual Network and after that, create your VM's and set them to use this virtual network. Doing that, your vm's would call the vm with your sql server.
2-) You'll be billed for network, storage and vm compute (including vm size). The best you can do is create a vm and test for a little (with this settings). So you'll know how much your are spending.
3-) When you create a vm, the vhd is stored in a storage account. There's a backup for that, but you can download the vhd and make a version of that on premises.
4-) My suggestion, attach a extra disk to your VM and use it to SQL Server Backup.
I have just moved my web site to an Azure Virtual Machine and have been up and running since last weekend. So far I'm very happy with the results and looking forward to taking advantage of Azure further in due course.
I do have what would seem to be a pretty common scenario - and, to my surprise, I can't find an obvious solution. I have a couple of VMs - one my primary server and the other which will be suspended and ready to kick in (manually is fine) if the first one has an issue. I backup my web site to Azure Storage (my backup utility supports saving to an Azure blob). That's the good news.
I had assumed that I could somehow mount the storage blob as a drive, therefore effectively having shared storage across the two VMs. However, to my surprise, I haven't found an obvious way to do that. I have found a third party utility (Gladinet Cloud Desktop) but it seems painfully slow. As I say, I admit I just assumed this would be an easy thing to do.
So, stepping back, what is the most straightforward way to access a storage blob from multiple VMs? I really don't want to set up a private network and then set up network file sharing - that seems so old school :) and places a specific dependency on one specific VM.
Any suggestions?
Thanks.
This is now not just possible, but very easy, and it looks just like a filesystem. Check out the new Azure File Service (in preview as of this writing).
http://blogs.msdn.com/b/windowsazurestorage/archive/2014/05/12/introducing-microsoft-azure-file-service.aspx
Quoting from the announcement:
"The Azure File service exposes file shares using the standard SMB 2.1 protocol. Applications running in Azure can now easily share files between VMs using standard and familiar file system APIs like ReadFile and WriteFile."
It is better than just an SMB drive, as the announcement goes on to mention:
"In addition, the files can also be accessed at the same time via a REST interface, which opens a variety of hybrid scenarios. Finally, Azure Files is built on the same technology as the Blob, Table, and Queue Services, which means Azure Files is able to leverage the existing availability, durability, scalability, and geo redundancy that is built into our platform."
In Azure Resource Manager "Storage Account" you can create a Network File Share that can be Mounted as a Drive to multiple VM's or to computers and devices not on Azure for both Unix, Linux and Windows.
In General, go to your Storage Account ➡ Files ➡ Create FileShare ➡ Name the Share and the Disk Space Quota ➡ Click Connect to obtain the command or windows or linux to mount the share to the respective devices. Note this ONLY WORKS for Local Redundant Storage, not Zone, not Geo Redundant.
https://www.youtube.com/watch?v=SGPJZMaSlis
The video tutorial above shows you step by step how to do this. The only restriction is needed is OS support of the SMB 3.0 protocol which Windows 8 or above does and Windows 2012 or above does. Requires Firewall Port 445 to be opened.
You can access blobs from multiple VMs. This is a very common pattern. What you can't do is mount a drive (stored in a blob) on multiple VMs simultaneously. That is, if you decide to create a VHD disk and attach it to a VM (whether Linux or Windows - doesn't matter), then the blob-backed disk is locked to a VM and that VM can then work with the vhd like it would a local file system.
If, on the other hand, you deal with blobs discretely as single objects, you can easily work with these blobs across any number of VMs.
If you're looking to do something like network sharing (e.g. SMB), you'd either need to use the Azure File Service or stage your own SMB server VM.
In the case where you absolutely must have a mounted file system, yet want to use the file system in a primary/backup fashion, you could always do something via the API to unmount from one VM and remount to another VM. This can be executed via PowerShell (Windows only) or via the cross-platform command-line interface on Linux/Mac/Windows. You'd do this if your primary VM failed for some reason.
this are good articles, I am also looking for that, hope find the right solution.
I hope you share your experience here with your choice.
Deciding when to use Azure Blobs, Azure Files, or Azure Disks
https://learn.microsoft.com/en-us/azure/storage/common/storage-decide-blobs-files-disks
there are premium disks
https://azure.microsoft.com/en-us/pricing/details/managed-disks/
Manually create and use a volume with Azure disks in Azure Kubernetes Service (AKS)
https://learn.microsoft.com/en-us/azure/aks/azure-disk-volume
Note : An Azure disk can only be mounted to a single pod at a time. If you need to share a persistent volume across multiple pods, use Azure Files.
Performance guidelines for SQL Server in Azure Virtual Machines
https://learn.microsoft.com/en-us/azure/virtual-machines/windows/sql/virtual-machines-windows-sql-performance
Deploy a SQL Server container in Kubernetes with Azure Kubernetes Services (AKS)
https://learn.microsoft.com/en-us/sql/linux/tutorial-sql-server-containers-kubernetes?view=sql-server-2017