I am using Windows Azure for a custom blog implementation. The blog uses CKEditor and the CKFinder file management plugin. Typically the file management plugin connects to a file system directory to store the files. I need to store these as if it was a local directory and serve them through HTTP requests. In Azure you cannot rely on the file system to maintain through recycles.
I assume you are to use Azure Storage, but am at a loss as to how to do this. Is there a way to "mount" these storage systems to the file system? Am I correct in my assumptions to use storage? If not any guidance as to what I am missing?
Thanks
Or, you could use AzureBlobDrive to mount blob storage as a drive in Azure directly (no VHD, no limitation on only one instance being able to write).
https://github.com/richorama/AzureBlobDrive
You can actually mount a page blob as an NTFS drive, which is then a "durable drive" (just like any other blob), and you access it via a drive letter, just like a locally-attached (but volatile) drive.
The issue is that, using mounted drives, you may only have one writer, so this might cause challenges when scaling to multiple instances.
Take a look at this MSDN post to see an example of mounting a drive. Notice that, while the example doesn't set up any cache, you can specify a cache size. The cache is stored on a local disk resource.
EDIT: For a tutorial, download the Windows Azure Training Kit. Go to hands-on labs, and open Exploring Windows Azure Storage. Check out Exercise 4: Working with Drives.
Related
As the title says, I'm looking for a way to access an azure files share (in preview) directly from an azure website. I cannot use any REST API or anything like this and I was looking for the possibility of mounting a SMB share directly into the website (through the new portal or any other way).
I found the following links, from which I understand that this is still under review (http://feedback.azure.com/forums/169385-web-apps-formerly-websites/suggestions/6084609-allow-map-azure-file-share-microsoft-azure-file-s) and also a SO question (Can the new Azure File Service be used from Azure WebSites?) that doesn't answer my question.
To be honest and for the sake of giving more details, my scenario is pretty simple - I have some websites and also some virtual machines that should access the files from the azure files service. Regarding the VM, the approach is pretty straight forward and easy but regarding the WebSites, I don't find any way at this moment.
On the other hand, regardless of the answer to the above question, does it make sense to (or do I have the possibility to) enable CDN over an Azure Files Share?
Thank you very much.
As of today, no single technology will serve your purpose. You can't use File Service as you don't have the capability to mount a share in an Azure Website as well as it is not suited for streaming purposes (all access to files there need to be authorized and there's no concept of Shared Access Signature in File Service today).
I guess, you would have to pick one of the two technologies (Blob Service and File Service) and make some compromises to make it work in both Websites and Virtual Machines.
Assuming you go with File Service, then you can mount them in the Virtual Machine and do the processing on the files there. On the website front, you would need to use Storage Client library to download the relevant files in some folder in your website and stream those files from there.
Assuming you go with Blob Service, then you can simply stream them in your website directly from blob storage (no need to have those files in your website). In the Virtual Machine, when you need to process those files (blobs), you would simply download them to your VM for processing and then re-upload them in blob storage.
Does it make sense to (or do I have the possibility to) enable CDN
over an Azure Files Share?
Currently it is not possible to serve Azure File Service files via CDN.
I have just moved my web site to an Azure Virtual Machine and have been up and running since last weekend. So far I'm very happy with the results and looking forward to taking advantage of Azure further in due course.
I do have what would seem to be a pretty common scenario - and, to my surprise, I can't find an obvious solution. I have a couple of VMs - one my primary server and the other which will be suspended and ready to kick in (manually is fine) if the first one has an issue. I backup my web site to Azure Storage (my backup utility supports saving to an Azure blob). That's the good news.
I had assumed that I could somehow mount the storage blob as a drive, therefore effectively having shared storage across the two VMs. However, to my surprise, I haven't found an obvious way to do that. I have found a third party utility (Gladinet Cloud Desktop) but it seems painfully slow. As I say, I admit I just assumed this would be an easy thing to do.
So, stepping back, what is the most straightforward way to access a storage blob from multiple VMs? I really don't want to set up a private network and then set up network file sharing - that seems so old school :) and places a specific dependency on one specific VM.
Any suggestions?
Thanks.
This is now not just possible, but very easy, and it looks just like a filesystem. Check out the new Azure File Service (in preview as of this writing).
http://blogs.msdn.com/b/windowsazurestorage/archive/2014/05/12/introducing-microsoft-azure-file-service.aspx
Quoting from the announcement:
"The Azure File service exposes file shares using the standard SMB 2.1 protocol. Applications running in Azure can now easily share files between VMs using standard and familiar file system APIs like ReadFile and WriteFile."
It is better than just an SMB drive, as the announcement goes on to mention:
"In addition, the files can also be accessed at the same time via a REST interface, which opens a variety of hybrid scenarios. Finally, Azure Files is built on the same technology as the Blob, Table, and Queue Services, which means Azure Files is able to leverage the existing availability, durability, scalability, and geo redundancy that is built into our platform."
In Azure Resource Manager "Storage Account" you can create a Network File Share that can be Mounted as a Drive to multiple VM's or to computers and devices not on Azure for both Unix, Linux and Windows.
In General, go to your Storage Account ➡ Files ➡ Create FileShare ➡ Name the Share and the Disk Space Quota ➡ Click Connect to obtain the command or windows or linux to mount the share to the respective devices. Note this ONLY WORKS for Local Redundant Storage, not Zone, not Geo Redundant.
https://www.youtube.com/watch?v=SGPJZMaSlis
The video tutorial above shows you step by step how to do this. The only restriction is needed is OS support of the SMB 3.0 protocol which Windows 8 or above does and Windows 2012 or above does. Requires Firewall Port 445 to be opened.
You can access blobs from multiple VMs. This is a very common pattern. What you can't do is mount a drive (stored in a blob) on multiple VMs simultaneously. That is, if you decide to create a VHD disk and attach it to a VM (whether Linux or Windows - doesn't matter), then the blob-backed disk is locked to a VM and that VM can then work with the vhd like it would a local file system.
If, on the other hand, you deal with blobs discretely as single objects, you can easily work with these blobs across any number of VMs.
If you're looking to do something like network sharing (e.g. SMB), you'd either need to use the Azure File Service or stage your own SMB server VM.
In the case where you absolutely must have a mounted file system, yet want to use the file system in a primary/backup fashion, you could always do something via the API to unmount from one VM and remount to another VM. This can be executed via PowerShell (Windows only) or via the cross-platform command-line interface on Linux/Mac/Windows. You'd do this if your primary VM failed for some reason.
this are good articles, I am also looking for that, hope find the right solution.
I hope you share your experience here with your choice.
Deciding when to use Azure Blobs, Azure Files, or Azure Disks
https://learn.microsoft.com/en-us/azure/storage/common/storage-decide-blobs-files-disks
there are premium disks
https://azure.microsoft.com/en-us/pricing/details/managed-disks/
Manually create and use a volume with Azure disks in Azure Kubernetes Service (AKS)
https://learn.microsoft.com/en-us/azure/aks/azure-disk-volume
Note : An Azure disk can only be mounted to a single pod at a time. If you need to share a persistent volume across multiple pods, use Azure Files.
Performance guidelines for SQL Server in Azure Virtual Machines
https://learn.microsoft.com/en-us/azure/virtual-machines/windows/sql/virtual-machines-windows-sql-performance
Deploy a SQL Server container in Kubernetes with Azure Kubernetes Services (AKS)
https://learn.microsoft.com/en-us/sql/linux/tutorial-sql-server-containers-kubernetes?view=sql-server-2017
I have just implemented Umbraco in an Azure Cloud Instance. I was able to migrate my existing SQL Database to run on SQL Azure and everything runs fine, except for the images and documents inside the media folder.
By default the media folder resides in [siteroot]/Media.
Is there a way to map this folder to azure storage? If not I don't think I'm going to be able to scale up my cloud instances, since the images depend on the virtual server's local storage.
Edit: Bounty Started
What I have so far is this:
Define a stand alone web role which would hold the media directory
and all the files.
Map this folder to the Azure Blobg Storage service with Cloud Drive, in order to minimize the risk of losing data and relying on a
single point of storage.
Somehow (and this is the part I don't know how to accomplish) keep all the folder of [siteRoot]/media synced with this shared drive on
all running instances.
I've seen a similar approach taken with the Azure Accelerator project from Umbraco here: http://azureaccelerators.codeplex.com/releases
But they haven't updated the release since 2011, and I'm not sure it would work with the current version of Azure.
Edit 2:
Umbraco has their own accelerator, but they've deprecated it in favor of using Websites instead of Web Roles:
https://github.com/Microsoft-DPE/wa-accelerator-umbraco
This release works with the 1.6 SDK. Current version is 1.8 I believe...
I'm not sure about a way of mapping the path to storage, but depending on the version of Umbraco you are using, I think from 4.9 (possibly 4.10) they introduced FileSystemProviders configuration which may help solve your problem.
My understanding of it is that it allows you to replace the default Umbraco FileSystemProvider, Umbraco.Core.IO.PhysicalFileSystem with your own custom implementation. I'm pretty sure you could implement an Azure-based provider that wrote and read from the blob storage. In the source, it looks fairly straightforward, a matter of implementing their IFileSystem.
Ended up using Matt Brailsford's Universal Media Picker solution:
http://our.umbraco.org/projects/backoffice-extensions/universal-media-picker
The final solution actually circumvents the Umbraco Media Folder and reads directly from Blob Storage, so I had to rewrite all the macros and templates that rendered images before and point them directly to the Blob Storage account
Unfortunately theres no way to map a NTFS directory to BlobStorage directly.
Have a look at the CloudDrive class of the Windows Azure SDK. This feature allows you to upload a Virtual Hard Disk file (.vhd file) into your blob storage and mount it as a local drive inside Windows Azure Instances.
You sould know that (if you're using multiple instances) only one cloud instance can mount the VHD in read/write mode. The rest of them has only read access to the drive. If the "Media" folder stores static content that you update manually only a few times, this is okay. But if user content is placed there, too, you might want only one instance to mount the VHD and grant other instances access to it via Network Share.
This package provided by Ali Sheikh Taheri solves the problem of the media folder
http://our.umbraco.org/projects/backoffice-extensions/ast-azure-media-sync
How would I write to a tmp/temp directory in windows azure website? I can write to a blob, but i'm using an NPM that requires me to give it file names so that it can directly write to those filenames.
Are you using Cloud Services (PaaS) or Virtual Machines (IaaS).
If PaaS, look at Windows Azure Local Storage. This option gives you up to 250gb of disk space per core. Its a great location for temporary storage of information in a way that traditional apps will be familiar with. However, its not persistent so if you put anything there you need to make sure will be available if the VM instance gets repaved, then copy it to Blob storage. Also, this storage is specific to a given role instance. So if you have two instances of the same role, they each have their own local storage buckets.
Alternatively, you can use Azure Drive, which allows you to keep the information persisted, but still doesn't allow multiple parallel writes.
If IaaS, then you can just mount a data disk to the VM and write to it directly. Data disks are already persisted to blob storage so there's little risk of data loss.
Just from my understanding and please correct me if anything wrong.
In Windows Azure Web Site, the content of your website will be stored in blob storage and mounted as a drive, which will be used for all instances your web site is using. And since it's in blob storage it's persistent. So if you need the local file system I think you can use the folders under your web site root path. But I don't think you can use the system tmp or temp folder.
I am looking at moving to Windows Azure rather than typical hosting however I'm unsure how best to store images. After searching I've found that there are 2 possible solutions - Blob storage or Azure drive.
I have looked into Blob storage and although I have begun to get used to the idea it will require quite a lot of modification to our CMS. In my searching I have just stumbled across Azure Drive which if I understand correctly creates a virtual hard drive which allows your application to run as it would on a normal server.
Are there any disadvantages to Azure Drive over blob storage? It sounds like migrating current applications to Azure will be much easier with Azure Drive rather than Blob storage but I just wanted to check that there weren't any major flaws in this.
Thanks
Pat
Yes, there are quite a few differences. First, the Windows Azure drive is actually a VHD uploaded as a page blob and mounted by a driver to provide a NTFS partition. So, to get any data on it, you must mount it (or a snapshot). Data is not directly accessible without mounting.
Next, Drives can only be mounted for RW by one instance. If you want anything else to even read that drive, you must snapshot and mount, which introduces a 'staleness' problem to read only instances that are mounting snapshots. You can work around this with an SMB share, but that is slightly complicated.
You would lose the ability to automatically get CDN capabilities if you used a drive as well.
Drives are great for their intended purpose - getting applications that must use NTFS to work in Windows Azure.
If you were to use Blobs natively, you would a.) get the storage subsystem to scale and remove the load from your instances for serving the data and b.) be able to use the CDN to get geoscale on the
images as well.
While it is some work, I would strongly recommend putting images in blob storage. It is ideal for it.