I am trying to upload a Vhd (sizing atleast 30GB) to a page blob in azure storage in an mvc web application. Due to size of the file i can not upload this large file as a whole as browsers don't allow this large request to be sent. So, the only option is to upload file in chunks (i.e. 4mb). on client size i am able to do chunking and i am sending chunks to my server side controller through an ajax request (in a loop). But using .net sdk for azure i am not finding a way to upload chunks to a page blob.
P.S There is a way to upload file in chunks in block blob using putblock() and putblocklist() methods and i am able to achieve the uploading in that way but i need to create a VM image out of the uploaded vhd and for that purpose it needs to be a page blob.
So, i would welcome any guidance to show me the way to upload vhd in chunks in a Page Blob using azure .net sdk.
You can try AzCopy tool without writing any code.
AzCopy /Source:C:\myfolder
/Dest:https://myaccount.blob.core.windows.net/mycontainer
/DestKey:mykey /Pattern:abc.vhd /BlobType:Page
You could use CloudPageBlob.WritePages method to upload the chunks of data. Please see this blog post from Azure Storage Team for an example of using this method: http://blogs.msdn.com/b/windowsazurestorage/archive/2010/04/11/using-windows-azure-page-blobs-and-how-to-efficiently-upload-and-download-page-blobs.aspx.
Related
Recently I have been working on adding documents to Azure storage using blob and file share. But then I realized that in file share using rest API I can upload in two steps
Creating a file
Adding content
I am able to doing that but my requirement here is to upload the .pdf, .docx document at once
and then there should be a way to download them as well.
Could some one please help.
Thanks
Unfortunately, there's no batch download capability available in Azure Blob Storage. You will need to download each blob individually. What you could do is download blobs in parallel to speed things up.
There is an alternative way you can approach using C# or PowerShell.
Would recommend you to please go through this MS document :
https://learn.microsoft.com/en-us/azure/storage/blobs/storage-blob-scalable-app-download-files?tabs=dotnet
And this one also
https://azurelessons.com/upload-and-download-file-in-azure-blob-storage/
Reference: How to download multiple files in a single request from Azure Blob Storage using c#?
When uploading to Azure Storage, does FineUploader send the file directly to Azure Storage or to the server first?
I noticed on the website that with S3, one can upload directly but the fact that S3 was singled out got me curious.
I'm looking for a really robust solution to upload files - even large files up to 10 GB - to Azure Storage. Wanted to see if FineUploader could be the answer for me.
When uploading to Azure Storage, does FineUploader send the file directly to Azure Storage or to the server first?
Fine Uploader Azure sends the files directly to Azure Cloud Storage. You do need a server to generate Shared Access Signatures for each request. Fine Uploader Azure will contact your SAS server before each upload request (or before any request to Azure) to obtain a SAS. More information on the Azure feature page at http://docs.fineuploader.com/features/azure.html.
Fine Uploader S3 functions using a similar workflow, but there is also an option to upload files directly to S3 without maintaining your own signature server. That particular option is not available with Fine Uploader Azure.
I have seen few examples where a file is transferred to server side and then uploaded to Azure Blob Storage.
But I have files with size in few GBs.
Is there a way I can upload a file directly to Azure Blob Storage using Client Side scripts instead of doing it from Server Side to save time.
Updating my answer, now that CORS is supported in Windows Azure Storage and the OP has not accepted any answer :).
Yes, it is possible to upload large files directly from your browser to Windows Azure Storage. You may find these steps useful:
First create a Shared Access Signature URL (SAS) with at least Write permission on the blob container in which you wish to upload the files. Since you're uploading large files, I would recommend keeping SAS expiry time to be long enough.
Next enable CORS on your storage account. If you wish to do it programmatically, you may find this post useful: http://gauravmantri.com/2013/12/01/windows-azure-storage-and-cors-lets-have-some-fun/. If you want to use a tool, my company has released a Free tool to do just that. You can read more about this tool and download from here: http://blog.cynapta.com/2013/12/cynapta-azure-cors-helper-free-tool-to-manage-cors-rules-for-windows-azure-blob-storage/.
I wrote a blog post some time back on uploading very large files into blob storage which you can read here: http://gauravmantri.com/2013/02/16/uploading-large-files-in-windows-azure-blob-storage-using-shared-access-signature-html-and-javascript/. Once CORS is enabled on your storage account, code mentioned in the blog should work just fine.
Actually there's a way though there are some preconditions/caveats.
Because CORS is not supported in Blob Storage just yet, your HTML and JS file need to be present in same blob storage account. They should be in a public blob container.
Since you're uploading large files, they would need to be split into chunks less than 4 MB in size. HTML 5 has a File API which can split the file into chunks but not all browsers support this feature.
I wrote a blog post some time ago about uploading large files using pure JavaScript and Shared Access Signature. You can read that post here: http://gauravmantri.com/2013/02/16/uploading-large-files-in-windows-azure-blob-storage-using-shared-access-signature-html-and-javascript.
I have an MVC application which allows users to sell items and upload images associated with each item. I am using Azure Blob storage to store the images. Once the image is stored in Azure, the MVC backend might retrieve it and resize it (usually shrink it) before sending it to the client, depending on requirements. Is there anyway I can request a file from Azure and have Azure resize it for me? This would save on the expense of sending a large image file to my MVC controller, which I then resize and stream to the client. Of course, I know I could save multiple copies of each image in Azure, each with the size I might need.
No, Azure Blob storage is not designed process the content of your files in any way.
Simple binary upload and download.
Can anyone please advise me if it is possible to directly upload a file to a blob container without routing it through my web server? I'm thinking some sort of client-side JS/jQuery script or a 3rd party upload module that streams the file directly to the blob container.
With Amazon S3 I used a component called Flajaxian Direct Uploader to achieve this.
I have the need to upload zip files to an Azure blob container that are 50 mb - 200 mb in size and routing via the web server is slower and consumes additional bandwidth.
Yes, it is possible. This can be achieved by having the client contact your web server and ask for a Shared Access Signature with (w)rite only access and limited expiry. Your client can then use the simple REST API to upload the blob. The trick here is that if your blob is bigger than 64mb, you must use use the PUT block and PUT block list option. The latter is not as straightforward for a Javascript client. If your client can use curl, it works well.
Reference:
PUT Blob
PUT Block
PUT Block list