I am currently working on achieving 99.95 % Availability of my application residing on Azure Virtual Machine.
What I want to achieve is that whenever the machine is restarted due to windows update or fault restart, another machine in a different "update domain"/ "fault domain" should take over. I ran across the following issues:
Storage High Availability: My application resides on wwwroot folder and has dynamic folders that contain Document database (CSVs, PDFs, Txt ...) Created by the application users. I cannot move to "Azure Web Application" because the files are accessed via virtual path: "~/Users/CSVs/4.csv". Moving to BLOB storage is a solution that requires a lot of changes on my infrastructure and web application code as multiple other components are involved (Windows Services & Scheduled Tasks Executables are also accessing the same files via absolute path: "C:/inetput/wwwroot/ApplicationFolder/Users/CSVs/4.csv"). I moved the whole application to Data Disk (VHD) different than the OS Disk and attached it as a new simple volume hoping to re-attach the same VHD to another instance. This is not allowed in MS Azure. Any recommendations away from using BLOB and away from SMB Protocol?? :)
SQL High Availability: I use SQL Dependency to query notifications from SQL Server to the application level and auto-update the UI using SignalR. This requires Service Broker to be enabled on the database. Service broker is not accessible in SQL Azure. Any other recommended way to query changes on a specific query without using SQL Service Broker and without overburdening the server by querying new notifications via Ajax timeout?
I am open to any hybrid high availability architecture suggestion for my IaaS solution rather than going full PaaS.
Any Suggestions Architects?
Thanks
I just tried it myself and it worked; specifically, I:
Spun up two Ubuntu VMs
Used the XPlat CLI to attach an empty disk to VM1 ('azure vm disk attach ...')
Mounted the disk and put a file on it called 'hi.txt'
detached the disk ('azure vm disk detach ...')
attached the same disk to VM2 ('azure vm disk attach ...')
Mounted the disk on VM2 and found the file 'hi.txt'
This was all in Linux, following this blog post for linux-specific commands: https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-linux-how-to-attach-disk/
Here is a link that will hopefully help for Windows: https://azure.microsoft.com/en-us/documentation/articles/storage-windows-attach-disk/
One thing to note is that I did everything in ARM mode; the tutorials above assume Classic/ASM mode, but this is not necessary.
Hope this helps! :)
Related
We have 2 Ubuntu VMs inside Virtual Machine Flexible Orchestration that are behind Application Gateway and are running Apache Tomcat web servers. When a client connects to one of the VMs and uploads the files that files also need to exist on another Virtual Machine.
I only found 2 options to do that:
Azure File Share - $80/month for 1 TB of Hot SKU, but the speed is only 1 MBs when mounted as SMB share on Ubuntu.
Azure NetApp Files - $600/month for 4 TB minimum.
Both of the options are not good, the first one is to slow and the second one is too expensive. What can we use in the development environment and production environment to achieve file sharing between Highly Available VMs?
1MBs is awfully low, I am not sure where this is coming from. I am fairly sure I get about 30MBs for Standard SSD/HDD deployments when mounting them into Linux docker containers, which should not perform worse.
An alternative to the mounted file shares would be to use shared disks. You can basically attach a disk to multiple VMs at the same time.
There are some limitations, for your case mainly mainly:
Shared disks can be attached to individual VMSS instances but can't be defined in the VMSS models or automatically deployed.
You can still expect to pay 50-200$ for the disk, but you should be able to get much better speeds than what you are currently getting.
Use a Blob and grant access via Managed Identity to your Virtual Machines:
https://learn.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/tutorial-vm-windows-access-storage
Blob Pricing and IOPS:
https://azure.microsoft.com/es-es/pricing/details/storage/page-blobs/
We've got two Windows Server 2019 virtual machines within the same Azure subscription and subnet. Recently, we have created a Premium SSD azure data disk with 'sharing' enabled and mounted it to those two VMs without any problems. It's perfectly fine to use the disk from both of the machines, but unfortunately files/folders added from one of them are not visible on the other.
Is it possible to somehow truly share the data between the machines using such azure disk attached to both of them? Maybe some super secret PowerShell option/flag when mounting the drives?
The machines are within same domain so obviously we can simply share a folder (which's what we do right now), but here the problem is that whenever our application is writing something to that share, it takes ages due to latencies/long upload times (effectively, it freezes the application for couple of minutes). Yes, they are in the same region (machines and disk). There's this Proximity Placement Groups thing available, but it does not seem to be applicable to disks, unfortunately.
We've also tried Azure Files but we've got exactly same problem as with 'shared folder' within the domain (long upload times whenever our application is writing something to the persistent storage).
I've went through Shared drive between Azure Virtual Machines but there's nothing about seeing the same contents from all machines which have the disk attached and mounted.
Thank you! Would appreciate any ideas.
Right so eventually I've found the answer. Basically the machines have to be joined into a failover cluster. Assuming the shared SSD:
is formatted to NTFS on both machines
has the same label on both machines (in my case F:)
if the conditions are met, that particular disk can be added to Cluster Shared Volumes. Program has no problems with writing the data smoothly from both machines.
I am trying to deploy a nodejs app on azure vm but I keep reading that the vm machine can crash or restart.
So lets say I opened the vm and installed nodejs and all the required tools I need then I get my code from github and started it, now every thing is running okay.
I am wondering what would happen if the vm restarted/crashed? Will the tools I downloaded and my code be lost? how can I make the vm when it start to redownload the tools, setup the environment, download my code then run it?
Azure Virtual Machines use Azure Blobs to back the OS disk VHD, as well as any attached data disk VHD. So, for those disks, everything is durable, regardless whether the VM is running or not (or crashes).
VMs also provide temporary storage on ephemeral disks. Assume anything placed on these temporary disks can, and will, disappear upon crash/restart.
So if you install any type of code libraries, apps, etc. on OS or attached disks, things remain in place unless you decommission the VM (and related storage), or delete it yourself.
Lots of documentation around Azure Virtual Machines and Storage, which can provide additional details.
I'm thinking about setting up 2 web VMs with a load balancer and availability set, and another VM for SQL server (not sure if I can set an availability set for a SQL Server as well - SQL Server Express / Standard?)
My main problem is how to keep both web servers in sync (prefer not to use the DFS) or having the files in more than one location...
Another issue - is user uploaded content that I want to be available in both web servers (I wonder if I can also direct cache objects to be saved on a specific storage disk)
So, I was thinking to setup a storage account and attach it to both web VMs for user uploaded content and images while each server still serve it's own separate web application with same shared access to content files...
Is that a good idea? I understand that Azure storage is a virtual disk that is supposed to be highly available and fast - is it true??
Do I get a major performance hit if using the same storage disk from 3 different VMs (is that even possible?)
UPDATE:
I found out that because I'm using the BizSpark program I can't really connect more than one server - and share resources between them (unless I pay extra for it). so this became irrelevant for now
Also, I'm talking about ASP.NET but this shouldn't matter
Azure Files enables you to run multiple IIS instances against a single file share and thus not have to worry about replicating files across the multiple shares - so this is definitely an option. See Getting Started with File Storage for more information.
This is scenario than a specific technical question.
I have two azure vm's who run a web application in load balanced mode.
as per this article http://asheej.blogspot.in/2014/03/load-balancing-using-windows-azure.html
both virtual machines are attached an additional disk which stores images which are referred from web application hosted in vm's IIS.
Now What would be the best way to keep contents on two vm hard drives in sync.
For example, If i delete, add a data from vhd of first vm then that should also be affected on second vm.
Is there anything possible, probably using a common vhd for both machines which will take sync out of question.
Before going into solution , let me briefly touch base on the VM and disk relationship.
Typically a VM contains 3 Disks attached to them 1. OS Disk 2. Temporary Disk and 3.Data Disks. The VM will have lease on all these disks ,the only way to write into data disks is via the VM.
The C: Disk is persistent, meaning when the VM get rebooted the data in the disk is retained. But the D:\ is non persistent , when you reboot the disk will be fully wiped clean. So at any point in time the D:\ shouldn't be used to store any user data.
So writing a process to sync between two VM's just to keep pictures in sync is not very ideal. You might know this already , but wanted to set context for the choice of options provided below.
Your potential options are as follows
You can setup File Share using the new Azure File Service (In Preview) http://blogs.technet.com/b/uspartner_ts2team/archive/2014/06/09/setting-up-a-file-share-for-the-new-azure-file-service.aspx. This will be single source for all your images and you don't need to worry about syncing of files.
2.Store the images in the Azure Blob and access them from the application that's running in the VM http://blogs.msdn.com/b/yaohuang1/archive/2012/07/02/asp-net-web-api-and-azure-blob-storage.aspx and http://www.nickharris.net/2012/11/how-to-upload-an-image-to-windows-azure-storage-using-mobile-services/
3.Host another VM as a Webserver and host your images from there. Then the two VM's can refer the image. The cost here will be to hosting the VM.
The key point with all the 3 potential options there is no need sync the files in two different places , everything is in single place.
Edited based on new information:-
In your scenario hosting your files into VM is not the right approach. You should take the following into consideration even for the short term solution , if you are using Azure LB.
Azure Load Balancer uses a 5 tuple (source IP, source port, destination IP, destination port, protocol type) to calculate the hash that and map traffic to available servers and also the distribution is fairly random. So if you load balance the VM, you cannot control which VM the images are accessed.
Manual updates is not possible in this scenario.
You either need to setup an virtual network to allow you to create and share a windows file share OR you should investigate the use of Azure File Service for creating a share that both VMs connect to (see: http://blogs.technet.com/b/uspartner_ts2team/archive/2014/06/09/setting-up-a-file-share-for-the-new-azure-file-service.aspx).