I am sorry if my question is very general - I just started with azure...
We are working with Azure and we currently have only one VM.
I would like to add a second VM and to place them both under the same availability set. The thing is that using our APP our users are posting files to the server and then a third party (twilio) should read the files. I need the two servers to be sync immediately to make sure the files are exists in both server.
Is it possible?
I be happy to get the outlines of how this should be done.
You can use azure files and attach it to both machines and store your files over there
Availability sets have nothing to do with VM syncing; it's about ensuring your VMs aren't upgraded at the same time or placed all in the same physical location.
You'll need to store your uploaded outside of your VMs in a common area (whether in blobs, or an Azure File share, or even in a database). Otherwise, you'll need to create your own way of syncing (copying) content between your VMs.
There's no right way to store your data, and the options I listed each have their pros and cons. You'll need to choose what's right for your app.
Related
I heard a lot that Azure Web Applications uploaded content such as images or any files should be stored in Azure Storage service, not in the app File System.
But I would like to keep the solution simple and store those files into local file system.
Does storing images or any files on the application's local file system hampers somehow the application deployment with more than one instance?
After researching a lot, I understand that, unlike Amazon Beanstalk, all instances of a Web App share the same file storage. despite of ecah apps runs into a different VM, they file system is the same.
The best way to think about it is that all the instances of your app map to the same network drive share that have your files.
when you spin up 2 or more instances the files in the file are using that same storage based file system even if you only have one instance.
You can see that easily by dropping a file (e.g. via FTP), and seeing it reflected instantly in all instances.
Sources: Microsoft, This question and this question
By storing files on the Web application, you're limiting your ability to scale.
The web server/app should do one thing: process your request and output the HTML. Everything else should be handled elsewhere if you truly want to take advantage of cloud computing. So for this instance you should store any files off the web app.
Now if you want to keep this as simple as possible and you're not overly concerned with achieving scale, there's really nothing wrong with your approach. It's my understanding that Web Apps, don't actually spin up new virtual machines for you, or if they do, they replicate exactly what is on the VM. For instance think about this -- if you had all your files stored on one VM and you spun up two new ones, you'd have to copy all those files over to the next two, and now you'd have to create a way to sync all the uploaded files among all your VMs.
I don't actually think you'd run into this problem with Azure Web Apps, but it i a problem that can arise if you're handling the VMs yourself or through an auto-scaling policy. You'll definitely run into the issue if you decide to spin up new web apps in different regions (Say the Ireland region to your EU customers get better performance - you'd now have two different locations where files could be uploaded to and you'd need to sync them as opposed to uploading them to Azure storage all along and keepin them centrally located)
I have a very special requirement which is:
Two web roles accessing a local shared file location.
I am aware of the "Local Storage" role settings, but those are only accessible within each role scope.
Does anyone know another option to accomplish this?
------- EDIT --------
As suggested I will explain more clearly what I'm trying to achieve here.
I'm implementing Only Office which is a web editor for office files. Their product requires to have a file saved on the file system to be opened on the editor.
I don't want to mix their ASP.NET MVC open source project with my code, so that's why I want to deploy their website as a separate webrole.
-------- END EDIT ------------
Thanks
In your question, you state that (my emphasis):
I'm implementing Only Office which is a web editor for office files. Their product requires to have a file saved on the file system to be opened on the editor.
If Only Office's requirement is to have temporary file storage that is used while the document is being edited, you may be able to get away with this in a Cloud Service Web Role. This is assuming that your users wouldn't be too angry if the temp. working document was 'lost' during a role re-start.
Web (and Worker) Roles are non-durable and the Azure Service Fabric might bounce them if they need to patch the underlying host or they might just crash due to a fault (which is usually why you deploy them in pairs - fault-tolerance etc.) If you save something to the file system on a Web Role, you are not guaranteed that it will be there if the role is bounced.
If however you need durability, you will need to implement something based around Azure Blob Storage and possibly something based on Blob Leases. However I imagine that Only Office doesn't have an implementation for Azure....
Failing that, you could try running on Azure Web App Service, however I imagine you would have the same issue re. backing storage and would need to implement something on Blob Storage.
So, finally, if you want complete control and something akin to running on-premise, take a look at using an IaaS Virtual Machine where you have all of the file system to play-with as you please.
==UPDATE==
Taking a look at the Only Office website, there is a SaaS offering Only Office SaaS Hosting which is probably cheaper to run for a year than the time taken for me to write this answer!
Failing that, if you look at the requirements for Only Office Document Server there is no way you're going to run that on a Web Role. Go Azure IaaS VM's.
You basically have 2 options here, both mentioned in the comments. You can use BLOB storage, or you can use an SMB share using Azure Files, which I believe is in preview still. We have used Azure files to mount an SMB share on several linux boxes. One thing we have noticed is that it is not particularly fast. It is also built on top of blob storage. Here is a link to Azure Files https://azure.microsoft.com/en-us/documentation/articles/storage-dotnet-how-to-use-files/.
If you choose to use blob storage and you will need to consider concurrency.
https://azure.microsoft.com/en-us/blog/managing-concurrency-in-microsoft-azure-storage-2/
I would suggest to use Azure File Services, you could have a share like URI to be used.
take a look at this:
https://azure.microsoft.com/en-us/documentation/articles/storage-dotnet-how-to-use-files/
I'm thinking about setting up 2 web VMs with a load balancer and availability set, and another VM for SQL server (not sure if I can set an availability set for a SQL Server as well - SQL Server Express / Standard?)
My main problem is how to keep both web servers in sync (prefer not to use the DFS) or having the files in more than one location...
Another issue - is user uploaded content that I want to be available in both web servers (I wonder if I can also direct cache objects to be saved on a specific storage disk)
So, I was thinking to setup a storage account and attach it to both web VMs for user uploaded content and images while each server still serve it's own separate web application with same shared access to content files...
Is that a good idea? I understand that Azure storage is a virtual disk that is supposed to be highly available and fast - is it true??
Do I get a major performance hit if using the same storage disk from 3 different VMs (is that even possible?)
UPDATE:
I found out that because I'm using the BizSpark program I can't really connect more than one server - and share resources between them (unless I pay extra for it). so this became irrelevant for now
Also, I'm talking about ASP.NET but this shouldn't matter
Azure Files enables you to run multiple IIS instances against a single file share and thus not have to worry about replicating files across the multiple shares - so this is definitely an option. See Getting Started with File Storage for more information.
This is scenario than a specific technical question.
I have two azure vm's who run a web application in load balanced mode.
as per this article http://asheej.blogspot.in/2014/03/load-balancing-using-windows-azure.html
both virtual machines are attached an additional disk which stores images which are referred from web application hosted in vm's IIS.
Now What would be the best way to keep contents on two vm hard drives in sync.
For example, If i delete, add a data from vhd of first vm then that should also be affected on second vm.
Is there anything possible, probably using a common vhd for both machines which will take sync out of question.
Before going into solution , let me briefly touch base on the VM and disk relationship.
Typically a VM contains 3 Disks attached to them 1. OS Disk 2. Temporary Disk and 3.Data Disks. The VM will have lease on all these disks ,the only way to write into data disks is via the VM.
The C: Disk is persistent, meaning when the VM get rebooted the data in the disk is retained. But the D:\ is non persistent , when you reboot the disk will be fully wiped clean. So at any point in time the D:\ shouldn't be used to store any user data.
So writing a process to sync between two VM's just to keep pictures in sync is not very ideal. You might know this already , but wanted to set context for the choice of options provided below.
Your potential options are as follows
You can setup File Share using the new Azure File Service (In Preview) http://blogs.technet.com/b/uspartner_ts2team/archive/2014/06/09/setting-up-a-file-share-for-the-new-azure-file-service.aspx. This will be single source for all your images and you don't need to worry about syncing of files.
2.Store the images in the Azure Blob and access them from the application that's running in the VM http://blogs.msdn.com/b/yaohuang1/archive/2012/07/02/asp-net-web-api-and-azure-blob-storage.aspx and http://www.nickharris.net/2012/11/how-to-upload-an-image-to-windows-azure-storage-using-mobile-services/
3.Host another VM as a Webserver and host your images from there. Then the two VM's can refer the image. The cost here will be to hosting the VM.
The key point with all the 3 potential options there is no need sync the files in two different places , everything is in single place.
Edited based on new information:-
In your scenario hosting your files into VM is not the right approach. You should take the following into consideration even for the short term solution , if you are using Azure LB.
Azure Load Balancer uses a 5 tuple (source IP, source port, destination IP, destination port, protocol type) to calculate the hash that and map traffic to available servers and also the distribution is fairly random. So if you load balance the VM, you cannot control which VM the images are accessed.
Manual updates is not possible in this scenario.
You either need to setup an virtual network to allow you to create and share a windows file share OR you should investigate the use of Azure File Service for creating a share that both VMs connect to (see: http://blogs.technet.com/b/uspartner_ts2team/archive/2014/06/09/setting-up-a-file-share-for-the-new-azure-file-service.aspx).
I setup a VM running Windows Server 2012 R2 (VM1) and configured it to run IIS. I then created another VM in the same region off an image I made from the first (VM2). These two are in their own availability set located in the West US data center.
Assuming I want to share my IIS configuration and website files between the two servers, I need to setup a location to store and access these files regardless of what server is making changes to them or is online at the time.
Attached storage almost provides a solution, but this can only be tied to a single machine at a time. Files can be stored in a separate Blob storage, but I don't know how to get IIS to read and write to it.
Once I do figure out how to share these files in a manner IIS will accept, I have a second problem where I would need a third VM (VM3) in Europe that would need access to this data as well.
Thank you for any assistance anyone can provide!
You would use DFS (Distributed File System) to accomplish this.
http://technet.microsoft.com/en-us/library/cc753479(v=ws.10).aspx
http://blogs.iis.net/rickbarber/archive/2012/12/05/keeping-multiple-iis-7-servers-in-sync-with-shared-configuration.aspx
Thanks.
-matt
Have you thought about using something like Dropbox to keep the folders synchronized?