We already allow our users to upload files through the app to the Azure Blob Storage, and then view them inside the app.
What we need now is to allow the upload of an entire folder containing web files (html, js, css, images...) maintaining the folder structure that it has and then be able to run these files in the browser. The link references between the files must be maintained also so it can work.
What will be the correct way to do this?
Is it possible through Blob Storage or do we need to upload the folder and its contents directly to the file system?
Thanks!
Note the Azure Blob storage doesn't have a concept of a "folder". The closest you would get would be to name a file "foldername/filename.ext". How you populate blob storage in this fashion would depend on how you allow a user to upload all their files. Perhaps as a zip file or through some form of ajax-based file upload UI on a web page... not sure. Ultimately you can't build folders but should be able to replicate the behaviours of one.
Related
I am a total newbie to Azure WebApps and storage, I need some clarification/confirmation. The main thing to take note of, my application (described below) requires a folder hierarchy. Blob is out of the question and file share doesn't allow anonymous access unless I use Shared Access Signature (SAS).
Am I understanding Azure storage correctly, it's either you fit into the Azure storage model or you don't?
Can anyone advise how I can achieve what's required by the CMS application as described below by using Blobs?
The only option I see is to find a way to change the CMS application so that it always has the SAS in the URL to every file it requests from storage in order to serve content on my Web App? If so, is it a problem if I set my SAS to expire sometime in the distant future?
https://<appname>.file.core.windows.net/instance1/site1/file1.jpg?<SAS>
Problem with using Blob
So far my understanding is that Blob storage doesn't allow "sub folders" as it's a container that holds unstructured data, therefore I'm unable to use this based on my application (described below) as it requires folder structure.
The problem with using File Share
File share seemed perfect as it allows for folder hierarchy, naturally that's what I've used.
However, no anonymous access is allowed for files stored in file storage, the access needs to be authorised. One way of authorising the access is to create a SAS on a file/share level with Read permission and then using that SAS URL to access the file.
Cannot access Windows azure file storage document
My application
I've created a Linux Web App running open source CMS application. This application allows creation of multiple websites, for each website's content such as images, docs, multimedia to be stored on a file server. These files are then served to the website via a defined URL.
The CMS application allows for a settings of the location where it should save its files, this would be a folder on the file server. It then creates a new sub folder for every site it hosts in that location.
Example folder hierarchy
/instance1
/site1
/file1
/file2
/site2
/file1
/file2
Am I understanding Azure storage correctly, it's either you fit into
the Azure storage model or you don't?
You can use Azure Storage Model for your CMS Application. You can use either Blob Storage or File Share
Can anyone advise how I can achieve what's required by the CMS
application as described below by using Blobs?
You can use Data Lake Gen 2 storage account if you want to use Azure Blob Storage.
Data Lake Gen 2 storage enables hierarchical namespace so that you can use subfolders in the Blob Storage as per your requirements
Problem with using Blob
Blob Storage allows subfolders if we use Data Lake Gen 2 storage account. You can enable Blob Public Anonymous access
The problem with using File Share
Azure File Share supports but does not allow public anonymous access. You can use Azure Managed Identity (System-Assigned) for your web app to access the Azure File Share.
Then your application would be able to access the Azure File Share without SAS token
The issue of not having real folders in a blob storage shouldn't be any issue for your use case. Just because it doesn't have your traditional folders doesn't mean it can't serve content on e.g. instance1/site1/file1. That's still possible but the instance1/site1/ will just be part of the name of the blob.
Tools like the Azure Portal or Storage Explorer will actually show folders by using the delimiter / and querying data that appears to be inside a folder by using the path as prefix.
On my Kentico project I have integrated Azure blob storage instead of saving files locally. Followed this article. https://docs.kentico.com/k12/custom-development/working-with-physical-files-using-the-api/configuring-file-system-providers/configuring-azure-storage
Things are working alright except for a one problem. Now all the files are accessible publically. There are some PDF files in the media library that I won't only the logged in users to view but now any one can view these files. Is there any workaround for this issue?
Files in Media Library are always accessible via the direct link and you can't restrict them to logged-in users only. Regardless it's Azure storage or local disk.
But there are two ways of achieving this:
Presentation-only restrictions. When you present those PDF links to the website user - display them only to logged-in users. The files will still be accessible via direct links but only logged-in users will see them.
Hard restrictions. As far as I know, these restrictions can be set up only for files stored in CMS tree. This approach will check permissions when accessing files via direct link.
If you are storing files in blob there is no way. You can restrict the access to the whole container with SAS token (or individual blob), but not a to a specific folder. Folder is purely virtual structure, it exists only in a file path.
I have few csv file present on my local system. I want to upload them into Azure blobs in a particular directory structure. I need to create the directory structure as well on azure.
Please suggest the possible options to achieve that.
1 - Create Your storage in Azure
2 - get Azure storage explorer
https://azure.microsoft.com/en-us/features/storage-explorer/
3- start the app, login and Navigate to your Blob
4- Drag and drop folder and files :)
basically this MS provided Software allows you to use your storage account in a classic folders and files structure
I have an Azure web app that stores documents in Azure blob "container X".
Now, we want to run a "job" to generate specialized reports for these documents.
This includes running an EXE file that takes a document path as argument, letting it generate a report on the file system, and uploading this to Azure blob "container Y".
Like: generate-report.exe document.doc generates report.txt.
How can this be done? Do we need to download the blob to the web app, or is it possible to somehow refer to a blob as we refer to a physical disk file?
You cannot refer to a blob as a local file object, since blob storage does not implement any type of file I/O abstraction layer. Yes, you can use File Storage service, which implements SMB on top of blob storage, but this is different than working with individual blobs.
If your app is built to deal just with file objects, you'd need to copy it from blob storage to local disk first, then upload the results from local disk to blob storage.
If you're just building your app, you can directly access blob content via the REST API (or one of the various language-specific SDK's that wrap the API).
Reading file from the blob can be done in form of stream that can later be used to create the text file inside the web app also.
You can also create web jobs under web app to accomplish this task in backend.
https://azure.microsoft.com/en-in/documentation/articles/storage-dotnet-how-to-use-blobs/
Currently i have some ftp where on it i have some deep structure of folders and files within it. It could be even 10 levels down from root folder. As i migrated already with success my local database to azure database, i wonder also whether is there any azure ftp i could use to migrate this as well. I know we have something like Azure storage and i could create Container for it of type File or Blobs - are one of those could be used like particural ftp - could i create folder structure there somehow using container and either File or Blob for that purpose, how it works there? Does either container blob or file for such purposes?
Let me add to what NDJ has written. So both Azure Blobs and Files would serve your purpose.
As mentioned by NDJ, Azure Blob Storage is a 2-level hierarchy system. At the top you have a blob container and the each blob container contains 0 or more files. So it does not support a folder structure per se but as NDJ mentioned, you can create an illusion of a sub folder by using appropriate blob delimiters (usually /). If you were to compare it with local file system, a directory at the root level (C:) is a container in blob storage and then the files would go in there. So imagine you have a folder called images in C:\ of your computer, that would be a container in blob storage. Now imagine that you have 2 sub folders beneath this folder (let's call them hires and lores) and both of them contains some files (say image1.png). When you move them to Azure Blob Storage, the container name would be images but the blob names would be hires/image1.png and lores/image1.png. Some of the storage explorers would take this delimiter (/) and show you that your container contains 2 folders and inside each folder you have an image called image1.png but in reality there are only 2 blobs in that blob container.
Azure File Service is a close match to your local file system. At the top level, you've got a Share and each share will container directories and files. Each directory can again contain many directories and files.
As NDJ mentioned, there's no FTP access to Azure Storage but there are many tools that will allow you to upload files from local computer to Azure Storage and many of them will preserve the file hierarchy. You can always write code to upload the files yourself. If you decide to use Azure Files, you can simply mount a File Storage Share as a network drive on your local computer and then transfer the files from your local computer to Azure Files as if you're transferring files from one drive to another.
UPDATE
Regarding difference between Azure Blob Storage and File Storage, both are used to store files. There are a few differences that I could think of:
A Share in Azure File Storage can be mount as a network drive on your local computer/Azure VM whereas a Blob Container in Azure Blob Storage can't. So if you have an application which writes files to local file system, you can take the application as is and make use of Azure File Storage and write the file to that network drive without making many changes to your code (typical example of Lift-And-Shift kind of application.
You can set ACL on a Blob Container whereas you can't do the same on a Share. This makes Azure Blob Storage ideal for storing static content (images, css, js) for your websites. For exposing files in File Storage, you would need to resort to Shared Access Signature.
You can set the size of a Share (default is 5GB) whereas no such thing exist for a Blob Container. A blob container can go up to the size of a storage account.
To understand Azure Files, I would recommend reading this: https://azure.microsoft.com/en-in/documentation/articles/storage-dotnet-how-to-use-files/.
Azure blob supports 10 levels down (up to 254. Basically the files are stored non hierarchically, but each / separator gives the appearance of directories.
It's relatively trivial to write something to move files to azure, as far as I know there is no ftp functionality yet - but it has been requested. It looks like some people have already created some code for this
You can now use Storage Explorer across all platforms to easily work within any folder structure.