I have a problem that I have been wracking my brain about and figured I would need some perspective and insight from people who are a lot more knowledgeable about this.
What I have currently: Web based application hosted in azure uses azure blob store to store files that are generated as part of data import processes. We have a seperate application that extends the original web application that allows users to upload files and these files are currently also stored in azure blob store.
Where I am trying to go: I have a requirement that wants the ability to map network file shares on a users laptop and be able to access these files that currently reside in the blob.
Since Azure blob does not support SMB I have no way of actually doing this with a blob store.
I could use Azure files in conjunction with a File Server running the sync agent. However, this requires a lot of work both in terms of refactoring, setup and some custom service that add remove permissions on the file server.
I'm wondering if there is a service or a piece of software that exists in the market currently that allows me to continue using blob and perhaps sync the blob files into a file server that can then allow users to access and open files using windows file explorer? I found one that looks like an open source project but only does a one way sync from the blob to the file share. Ideally I'd like to find a solution that does a two way sync like azure file sync does.
Any thoughts and ideas will be appreciated.
Since the max number of blob containers, file shares is unlimited. Per my understanding, you could leverage the following approaches:
Migrate the data from blob storage to azure file share instead of blob storage, then the subsequent file store is azure file storage.
Note: Currently you must specify storage account key when mounting file shares, details you could follow this feedback. I recommend that you'd better do not map network file shares on a users laptop.
You could still use the blob storage, and you could create each blob container for each user and generate each blob container SAS token for your users, then the users could leverage Azure Storage Explorer to manage their blob files or use AzCopy and other command tools to download the blob files into their laptop file system.
Note: For security consideration, you could combine a stored access policy with a SAS, in order to revoke the permissions, you just need to invalidate the related access policy instead of regenerating the account key. Details you could follow Controlling a SAS with a stored access policy and Shared Access Signatures, Part 2: Create and use a SAS with Blob storage.
Related
I am designing a web application and it needs to be in the Azure web app. The app is focused on managing files, so it needs to upload files and store them.
As is a cloud app, I suppose that I am not able to create a directory in the web app service. My question is if I have to use the benefits of Azure and create a Storage Account and if this is the solution, What will be the best storage solution, Blob or File?
Thank you in advance.
Best wishes
Container is a Blob Storage, which is a great option for programmable storage, where our program can read and write to the storage account.
If we don't want to allow websites and the public to access the files, we can choose the below options.
Blob Storage Containers can contain any binary files/ binary large objects, there is no ordering and hierarchy, we can have a virtual folder structure.
Containers are usually programmed to share files to access using Shared Access Signature and Access Policy
I suppose that I am not able to create a directory in the web app service
Azure Files is more useful for mounting a file share to a server and multiple servers can mount the same file share. It can have a quota.
File share has a Directory Structure, we can create Directories and Subdirectories in a Hierarchical manner compared to Containers.
The Connect option in the File Share gives you details on how to mount drive onto a Windows/Linux machine.
Use file storage if you need the shared drive protocol, if not we can design the applications and use blob storage.
As per your requirement, if you want to create Directories you can choose AzureFile Share.
Reference link Azure Blob and Fileshare storage mentioned by #deherman-MSFT
I am a total newbie to Azure WebApps and storage, I need some clarification/confirmation. The main thing to take note of, my application (described below) requires a folder hierarchy. Blob is out of the question and file share doesn't allow anonymous access unless I use Shared Access Signature (SAS).
Am I understanding Azure storage correctly, it's either you fit into the Azure storage model or you don't?
Can anyone advise how I can achieve what's required by the CMS application as described below by using Blobs?
The only option I see is to find a way to change the CMS application so that it always has the SAS in the URL to every file it requests from storage in order to serve content on my Web App? If so, is it a problem if I set my SAS to expire sometime in the distant future?
https://<appname>.file.core.windows.net/instance1/site1/file1.jpg?<SAS>
Problem with using Blob
So far my understanding is that Blob storage doesn't allow "sub folders" as it's a container that holds unstructured data, therefore I'm unable to use this based on my application (described below) as it requires folder structure.
The problem with using File Share
File share seemed perfect as it allows for folder hierarchy, naturally that's what I've used.
However, no anonymous access is allowed for files stored in file storage, the access needs to be authorised. One way of authorising the access is to create a SAS on a file/share level with Read permission and then using that SAS URL to access the file.
Cannot access Windows azure file storage document
My application
I've created a Linux Web App running open source CMS application. This application allows creation of multiple websites, for each website's content such as images, docs, multimedia to be stored on a file server. These files are then served to the website via a defined URL.
The CMS application allows for a settings of the location where it should save its files, this would be a folder on the file server. It then creates a new sub folder for every site it hosts in that location.
Example folder hierarchy
/instance1
/site1
/file1
/file2
/site2
/file1
/file2
Am I understanding Azure storage correctly, it's either you fit into
the Azure storage model or you don't?
You can use Azure Storage Model for your CMS Application. You can use either Blob Storage or File Share
Can anyone advise how I can achieve what's required by the CMS
application as described below by using Blobs?
You can use Data Lake Gen 2 storage account if you want to use Azure Blob Storage.
Data Lake Gen 2 storage enables hierarchical namespace so that you can use subfolders in the Blob Storage as per your requirements
Problem with using Blob
Blob Storage allows subfolders if we use Data Lake Gen 2 storage account. You can enable Blob Public Anonymous access
The problem with using File Share
Azure File Share supports but does not allow public anonymous access. You can use Azure Managed Identity (System-Assigned) for your web app to access the Azure File Share.
Then your application would be able to access the Azure File Share without SAS token
The issue of not having real folders in a blob storage shouldn't be any issue for your use case. Just because it doesn't have your traditional folders doesn't mean it can't serve content on e.g. instance1/site1/file1. That's still possible but the instance1/site1/ will just be part of the name of the blob.
Tools like the Azure Portal or Storage Explorer will actually show folders by using the delimiter / and querying data that appears to be inside a folder by using the path as prefix.
I have an azure web app (mvc core 2). It uses an occasionally updated data file. It is stored in a folder off wwwroot. I wish to allow users in a specific role to be able to upload a new copy of the data file maybe once per day or two days. The datafile is less than 1MB.The datafile should not be generally viewable
Can I set permissions on the folder to let me do this and if so how?
Martin
I believe this is can be done with Azure blob storage. Blob storage is designed for serving images or documents directly to a browser, storing files for distributed access, streaming video and audio and much more. Users or client applications can access objects in Blob storage via HTTP/HTTPS.
As for the permissions on the folder, I believe the SAS token could implement it. With a SAS, you can grant clients access to resources in your storage account, without sharing your account keys. This is the key point of using shared access signatures in your applications--a SAS is a secure way to share your storage resources without compromising your account keys. About the deteails you could refer to this doc: Using shared access signatures (SAS).
I want to upload some static images that I will later access via some mobile apps. I have an Azure Account that I rarely use so I thought that was the best place and therefore I uploaded them to a "File Share" within Azure Storage.
I naievely thought I could them just access those files via a simple web request url
https://myplace.file.core.windows.net/app/images/bnb/shop/bugle_200_2.jpg
All this gets me is a BadRequest error. I realize that I could create a Shared Access Signature (SAS) for every file but that seems total overkill.
Is there a better Azure feature to use? I do not want to have to use the Azure APIs to get at these files
Adding a few more points to #CtrlDot's excellent answer.
I completely agree that you should use Blob Storage for storing static content.
On the container permissions, I would actually recommend setting the permission (ACL) to Blob so that user can only view the blob they have the URL for and not enumerate all blobs in a container (setting container ACL to Container will enable the users to list blobs in a container which may not be a desired behaviour for you).
Other than these, there are two distinct advantage of using Blob Storage:
Custom domain: You can map blob storage to a custom domain (e.g. static content.mywebsite.com) and use that to serve the content instead of using Azure Blob Storage standard endpoint (your account.blob.core.windows.net).
CDN: You can also CDN enable your blob storage endpoint. The content will then be replicated across many CDN nodes spread throughout the globe and will be served from a node near to your user thus improving the user experience.
I think the service you should be looking to use is blob storage, not file storage. File storage, as per the documentation, is meant more for SMB shares.
When you setup Azure blob storage, you have a couple of different options. If there is nothing sensitive/secure about these static images, you could consider making a public container and simply accessing the files like that.
If you require authentication, then you need to either use azure storage access keys, or azure storage access tokens. Of the two, the storage access tokens are by far the most secure.
You wouldn't need to create a SAS token for each file, rather, grant it read permission to the container. Once again, you will have to tailor this to the security/sensitivity needs of your application.
I've developed an application which users can upload their files and share them with each other. The files are some private files for each user and public files like profile pictures. I'm storing the files in Azure File Storage.
Assume that I have a method to retrieve a file with its id: I've implemented the permissions in the file access methods in WebApi controllers.
Is Azure File Storage is proper storage type for this scenario?
What is the best way to retrieve the files from Azure Storage? Should I read the files server-side (using Azure .NET SDK) and stream them to the clients? Is there any way to avoid streaming the file in WebApi then clients can access the file directly from Azure File Storage (considering the permission)?
Thanks
Azure File Storage exists mainly to allow lift and shift of legacy applications to the cloud.
I would recommend using Blob Storage combined with SAS tokens for your problem. Using the SAS tokens you can control access permissions on a blob level. And this avoids the need to get your files on the web server first, before relaying them to the end users.
SAS and REST access are supported in both Azure Files and Azure Blob Storage. Azure Files support other key scenarios besides life & shift. Although a bit old, this article (https://blogs.msdn.microsoft.com/windowsazurestorage/2014/05/12/introducing-microsoft-azure-file-service/) explains the difference between Azure Blob vs Azure Files vs. Azure Disks. You should also factors such as size limit for your share, folder structure, ability to natively mount to a VM, maximum file/object size, throughput requirements, pricing, SMB/REST support and etc. If you still have questions, please send an email to azurefiles AT microsoft.com and we will be happy to review your scenario and recommend the option suitable to your usage scenario.
Aung
As I know, we can use Azure blob storage to store your files. Azure blob Storage containers provide three access level: Full public read access, Public read access for blobs only, no public read access. Refer to this article for more details. For your scenario, please save private files in “no public read access”, and save public file in “Public read access for blobs only”. So that the user cannot access your private files but can read your public files. If you want to share private files to others, please try SAS as LoekD mentioned. If you want to expired the SAS token in server side. Please try to use SAS policy to do it. Read this article for more details.