Is it possible to create an html form to allow web users to upload files directly to azure blob store without using another server as a intermediary? S3 and GAW blobstore both allow this but I cant find any support for azure blob storage.
EDIT November 2019
You can now refer to the official documentation:
Azure Storage JavaScript Client Library Sample for Blob Operations
Azure Storage client library for JavaScript
Initial answer
There is a New Azure Storage JavaScript client library for browsers (Preview).
(Everything from this post comes from the original article above)
The JavaScript Client Library for Azure Storage enables many web development scenarios using storage services like Blob, Table, Queue, and File, and is compatible with modern browsers
The new JavaScript Client Library for Browsers supports all the storage features available in the latest REST API version 2016-05-31 since it is built with Browserify using the Azure Storage Client Library for Node.js
We highly recommend use of SAS tokens to authenticate with Azure Storage since the JavaScript Client Library will expose the authentication token to the user in the browser. A SAS token with limited scope and time is highly recommended. In an ideal web application it is expected that the backend application will authenticate users when they log on, and will then provide a SAS token to the client for authorizing access to the Storage account. This removes the need to authenticate using an account key. Check out the Azure Function sample in our Github repository that generates a SAS token upon an HTTP POST request.
Code sample:
Insert the following script tags in your HTML code. Make sure the JavaScript files located in the same folder.
<script src="azure-storage.common.js"></script/>
<script src="azure-storage.blob.js"></script/>
Let’s now add a few items to the page to initiate the transfer. Add the following tags inside the BODY tag. Notice that the button calls uploadBlobFromText method when clicked. We will define this method in the next step.
<input type="text" id="text" name="text" value="Hello World!" />
<button id="upload-button" onclick="uploadBlobFromText()">Upload</button>
So far, we have included the client library and added the HTML code to show the user a text input and a button to initiate the transfer. When the user clicks on the upload button, uploadBlobFromText will be called. Let’s define that now:
<script>
function uploadBlobFromText() {
// your account and SAS information
var sasKey ="....";
var blobUri = "http://<accountname>.blob.core.windows.net";
var blobService = AzureStorage.createBlobServiceWithSas(blobUri, sasKey).withFilter(new AzureStorage.ExponentialRetryPolicyFilter());
var text = document.getElementById('text');
var btn = document.getElementById("upload-button");
blobService.createBlockBlobFromText('mycontainer', 'myblob', text.value, function(error, result, response){
if (error) {
alert('Upload filed, open browser console for more detailed info.');
console.log(error);
} else {
alert('Upload successfully!');
}
});
}
</script>
Do take a look at these blog posts for uploading files directly from browser to blob storage:
http://coderead.wordpress.com/2012/11/21/uploading-files-directly-to-blob-storage-from-the-browser/
http://gauravmantri.com/2013/02/16/uploading-large-files-in-windows-azure-blob-storage-using-shared-access-signature-html-and-javascript
The 2nd post (written by me) makes use of HTML 5 File API and thus would not work in all browsers.
The basic idea is to create a Shared Access Signature (SAS) for a blob container. The SAS should have Write permission. Since Windows Azure Blob Storage does not support CORS yet (which is supported by both Amazon S3 and Google), you would need to host the HTML page in the blob storage where you want your users to upload the file. Then you can use jQuery's Ajax functionality.
Now that Windows Azure storage services support CORS, you can do this. You can see the announcement here: Windows Azure Storage Release - Introducing CORS, JSON, Minute Metrics, and More.
I have a simple example that illustrates this scenario here: http://www.contentmaster.com/azure/windows-azure-storage-cors/
The example shows how to upload and download directly from a private blob using jQuery.ajax. This example still requires a server component to generate the shared access signature: this avoids the need to expose the storage account key in the client code.
You can use HTML5 File API, AJAX and MVC 3 to build a robust file upload control to upload huge files securely and reliably to Windows Azure blob storage with a provision of monitoring operation progress and operation cancellation. The solution works as below:
Client-side JavaScript that accepts and processes a file uploaded by user.
Server-side code that processes file chunks sent by JavaScript.
Client-side UI that invokes JavaScript.
Get the sample code here: Reliable Uploads to Windows Azure Blob Storage via an HTML5 Control
I have written a blog post with an example on how to do this, the code is at GitHub
It is based on Gaurav Mantris post and works by hosting the JavaScript on the Blob Storage itself.
Configure a proper CORS rule on your storage account.
Generate a Shared Access Signature from your target container.
Install the blob storage SDK: npm install #azure/storage-blob.
Assuming your file is Blob/Buffer/BufferArray, you can do something like this in your code:
import { ContainerClient } from "#azure/storage-blob";
const account = "your storage account name";
const container = "your container name";
const sas = "your shared access signature";
const containerClient = new ContainerClient(
`https://${account}.blob.core.windows.net/${container}${sas}`
);
async function upload(fileName, file) {
const blockBlobClient = containerClient.getBlockBlobClient(fileName);
const result = await blockBlobClient.uploadData(file);
console.log("uploaded", result);
}
Related
I am using these codes to save image's url:
await destA.save("base64Image", {
metadata: {
contentType: "image/jpeg",
},
});
const metaData = await destA.getMetadata();
const url = metaData[0].mediaLink;
I activated Uniform access control. But it is error when I want to display it om Flutter. The error I think is: "Anonymous caller does not have storage.objects.get access to the Google Cloud Storage object." Is this the correct way to save metadata file url? Or maybe i was wrong at the beginning.
This is the saved url
Cloud Storage for Firebase lets you upload and share user generated content, such as images and video, which allows you to build rich media content into your apps.
The basic steps involve the following in broad
Create a default Cloud Storage bucket
Set up access
Create a Cloud Storage reference on Flutter
Upload files with Cloud Storage on Flutter
Get a download URL
Add File Metadata
As mentioned above ,it is recommended to use the To upload a file to Cloud Storage, you first create a reference to the full path of the file, including the file name. You cannot upload data with a reference to the root of your Cloud Storage bucket. Your reference must point to a child URL.
You can find more information about Error Handling issues here.
And a similar example here.
My web app allows users to upload files.
I want to use cloud azure blob storage for this.
Since downloading will be very frequent (more than uploading)
I would like to save server computing time and bandwith and serve files directly from the azure blob server.
I believe it is made possible on Google cloud with Firebase (Firestorage).
Where you can upload and download directly from the client. (I know authentication and authorization are also managed by firebase so it makes things easier)
Does any similar mechanisms/service exist on Azure?
For example
When a user clicks an azure storage download link a trigger would check the JWT for authorization and data would be sent directly to the client from azure storage
Similar option is available with Azure Blob storage as well. You can use the Storage SDK to access the containers and list/download the blob
With a javascript backend You can either use SAS Token or Azure Storage JavaScript Client Library also supports creating BlobService based on Storage Account Key for authentication besides SAS Token. However, for security concerns, use of a limited time SAS Token, generated by a backend web server using a Stored Access Policy.
Example here
EDIT:
I have not answered the question completely above, However if you want to access the blob storage or download any files from the blob storage you can make use of normal http get request with SAS token generated with any JavaScript application.
With Angualr:
uploadToBLob(files) {
let formData: FormData = new FormData();
formData.append("asset", files[0], files[0].name);
this.http.post(this.baseUrl + 'insertfile', formData)
.subscribe(result => console.log(result));
}
downloadFile(fileName: string) {
return this.http.get(this.baseUrl + 'DownloadBlob/' + fileName, { responseType: "blob" })
.subscribe((result: any) => {
if (result) {
var blob = new Blob([result]);
let saveAs = require('file-saver');
let file = fileName;
saveAs(blob, file);
this.fileDownloadInitiated = false;
}
}, err => this.errorMessage = err
);
}
However the best practice (considering the security) is to have a backend API/Azure function to handle the file upload.
I have an azure resource group which contains Web App Service and Storage with BLOB container. My web app (.NET Core) tries to retrieve and show an image from container. The container has no public access to content (access level is private). I created system assigned identity for my app and gave it Reader role in storage access control (IAM).
This is how I get access to blobs in app's code:
const string blobName = "https://storagename.blob.core.windows.net/img/Coast.jpg";
string storageAccessToken = await GetStorageAccessTokenAsync();
var tokenCredential = new TokenCredential(storageAccessToken);
var storageCredentials = new StorageCredentials(tokenCredential);
var blob = new CloudBlockBlob(new Uri(blobName), storageCredentials);
ImageBlob = blob.Uri;
GetStorageAccessTokenAsync() does this:
var tokenProvider = new AzureServiceTokenProvider();
return await tokenProvider.GetAccessTokenAsync("https://storage.azure.com/");
Then the image is displayed by
<img src="#Model.ImageBlob" />
I don't get any exceptions in my code, but image from the BLOB container isn't shown with 404 error (specified resource doesn't exist) in browser console.
When I change container's access level to "blob" (public access), app works fine and the image is displayed.
Apparently, it is something wrong with getting credentials part, but I couldn't find any working example nor detailed explanations how it actually should work.
Any help is very appreciated.
UDPATE:
Thank you all who responded. So, it seems I've got two problems here.
1) I don't get credentials properly.
I can see that "AzureServiceTokenProvider" object (Microsoft.Azure.Services.AppAuthentication) that I create, has empty property PrincipalUsed at the runtime.
My application deployed to Azure App Service, which has system managed identity and that identity (service principal) is given permissions in Azure storage (I changed permission from account Reader to Storage Blob Data Reader as was suggested).
Shouldn't it get all data needed from the current context? If not, what I can do here?
2) I use wrong method to show image, but since the app has no access to storage anyway I can't fix it yet.
But still - what is the common way to do that in my case? I mean there is no public access to storage and I use "CloudBlockBlob" to reach images.
Reader gives access to read the control plane, but not the data plane. The role you need is Storage Blob Data Reader, which gives access to read blob contents.
For more details about this, check out: https://learn.microsoft.com/en-us/azure/role-based-access-control/role-definitions#data-operations-example
When you use <img src="#Model.ImageBlob" />, no authorization header is sent in the request by the browser. In your code, you are fetching the token, but the token is not being sent in the authorization header when the image is being fetched. So, storage API thinks this is an anonymous request. This is the reason you are getting a 404.
You need to send auth code when fetching the image. This code works for me
public async Task<ActionResult> Image()
{
const string blobName = "https://storage.blob.core.windows.net/images/image.png";
string storageAccessToken = await GetStorageAccessTokenAsync().ConfigureAwait(false);
var tokenCredential = new TokenCredential(storageAccessToken);
var storageCredentials = new StorageCredentials(tokenCredential);
var blob = new CloudBlockBlob(new Uri(blobName), storageCredentials);
Stream blobStream = blob.OpenRead();
return File(blobStream, blob.Properties.ContentType, "image.png");
}
In the view, I use
<img src="/Home/Image" />
Finally, I got it to work. First of all, the part of code regarding getting token and image from Azure storage was OK. The second problem with displaying image in RazorPages application I resolved, using this code in view:
<form asp-page-handler="GetImage" method="get">
<img src="/MyPageName?handler=GetImage" />
</form>
and corresponding code in model:
public async Task<ActionResult> OnGetGetImageAsync()
{
//getting image code and returning FileContentResult
}
But I'm still thinking: whether is more simple way to do that? Something like to add image collection to the model, fill it using "OnGet..." handler and then display its content using in view. I didn't find a way to use model properties in <img> tag. Does anyone have some suggestions?
With the announcement of Azure Storage support for Azure Active Directory based access control, is it possible to serve a blob (a specific file) over a web browser just by it's URI?
The use case I want to simplify is giving a few people access to files on the blob without the need of having to append a SAS token to the URI. Instead it would be brilliant to have the typical OAuth flow started when trying to open the plain URI in his/her web browser.
In my case we want to give access to files that have been uploaded to the blob storage by users through our support bot, build on Microsoft Bot framework. Links in our support system should be accessible by a support agent in their web browser of choice.
It this use case supported by this announcement or does this only work for coded OAuth flows, meaning we still have to implement some code?
If so, is there a good sample on how to start the OAuth flow from a Azure Function app and use the resulting token to download the file (over Azure Storage REST endpoint)?
While this answer is technically correct, it wasn't a direct response to my initial question.
I was looking for a way to provide the direct uri of any blob to business users, so they can simply open it in any web browser and see the file.
In my case we wanted to give access to files that have been uploaded to the blob storage by users through our support bot, build on Microsoft Bot framework. E.g. serving the attachment as a link in our support system to be accessed by a support agent.
After digging into this, I can answer the question my self:
With the announcement of Azure Storage support for Azure Active Directory based access control, is it possible to serve a blob (a specific file) over a web browser just by it's URI?
No, this is not possible. More specifically, simply opening the direct uri to a blob in the browser doesn't trigger the OAuth flow. Instead it will always give you ResourceNotFound response unless you provide a SAS query token or set the blob to public. Both solutions are bad from security perspective (when normal users involved) and obviously bad UX.
Solution
Looking for a way to achieve exactly what I want, I came up with the idea of a azure function serving the attachment to any business user by passing the fileName as url parameter and constructing the path using a route template.
Thinking of security and the need for an access token anyway, you could protect the function app through platform authentication (a.k.a. easyAuth).
However, this is not enough and configuring all parts of the solution is not straight forward. That is why I'm sharing it.
TL;DR high-level steps:
Create a new Function App (v2 recommended)
Enable the function App for authentication (easyAuth)
Create a service principal (a.k.a. app registration) for the function app (implicit by step 2)
Add additional allowed token audience https://storage.microsoft.com on the app registration
Edit the manifest of the app registration to include Azure Storage API permission (see special remarks below)
Modify authSettings in Azure Resource explorer to include additionalLoginParams for token response and resourceId
Give at least the Storage Blob Data Reader permission on the blob to all users accessing the files
Deploy your function app, call it, access the user token, call the blob storage and present the result to the user (see code samples below)
Remarks on Azure Storage API permission and access token (Step 5 & 6)
As stated in the latest documentation for AAD authentication support on azure storage, the app must grand user_impersonation permission scope for resourceId https://storage.azure.com/. Unfortunately the documentation did not state on how to set this API permission as it is not visible in the portal (at least I did not find it).
So the only way is to set it through its global GUID (can be found on the internet) by editing the app registration manifest directly in the azure portal.
Update:
As it turned out, not finding the right permission in the portal is a bug. See my answer here. Modifying the manifest manually results in the same, but directly doing it in the portal is much more convenient.
"requiredResourceAccess": [
{
"resourceAppId": "e406a681-f3d4-42a8-90b6-c2b029497af1",
"resourceAccess": [
{
"id": "03e0da56-190b-40ad-a80c-ea378c433f7f",
"type": "Scope"
}
]
},
{
"resourceAppId": "00000002-0000-0000-c000-000000000000",
"resourceAccess": [
{
"id": "311a71cc-e848-46a1-bdf8-97ff7156d8e6",
"type": "Scope"
}
]
}
]
The first one is the user_impersonation scope on Azure Storage and the second is the graph permission for User.Read, which in most cases is helpful or needed.
After you uploaded your modified manifest, you can verify it on the API Permissions tab on your app registration.
As easyAuth is using the v1 endpoint of AAD, your app needs to request those permission statically by passing resource=https://storage.azure.com/ when triggering the OAuth flow.
Additionally Azure Storage requires the bearer schema for authentication header and therefore a JWT token is needed. To get a JWT token from the endpoint, we need to pass response_type=code id_token as an additional login parameter.
Both can only be done through Azure Resource explorer or powershell.
Using Azure Resource explorer you have to navigate all your way down to the authSettings on your function app and set the additionalLoginParams accordingly.
"additionalLoginParams": [
"response_type=code id_token",
"resource=https://storage.azure.com/"
]
Code Sample
Here is a complete code sample for an easy azure function using all aboves mechanisms.
using System;
using System.IO;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Logging;
using Newtonsoft.Json;
using System.Linq;
using System.Net.Http;
using System.Net.Http.Headers;
namespace Controller.Api.v1.Org
{
public static class GetAttachment
{
private const string defaultContentType = "application/octet-stream";
[FunctionName("GetAttachment")]
public static async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = "v1/attachments")] HttpRequest req,
ILogger log)
{
if (!req.Query.ContainsKey("fileName"))
return new BadRequestResult();
// Set the file name from query parameter
string fileName = req.Query["fileName"];
string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
dynamic data = JsonConvert.DeserializeObject(requestBody);
fileName = fileName ?? data?.name;
// Construct the final uri. In this sample we have a applicaiton setting BLOB_URL
// set on the function app to store the target blob
var blobUri = Environment.GetEnvironmentVariable("BLOB_URL") + $"/{fileName}";
// The access token is provided as this special header by easyAuth.
var accessToken = req.Headers.FirstOrDefault(p => p.Key.Equals("x-ms-token-aad-access-token", StringComparison.OrdinalIgnoreCase));
// Construct the call against azure storage and pass the user token we got from easyAuth as bearer
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", accessToken.Value.FirstOrDefault());
client.DefaultRequestHeaders.Add("Accept-Encoding", "gzip, deflate");
client.DefaultRequestHeaders.Add("Accept", "*/*");
client.DefaultRequestHeaders.Add("x-ms-version", "2017-11-09");
// Serve the response directly in users browser. This code works against any browser, e.g. chrome, edge or even internet explorer
var response = await client.GetAsync(blobUri);
var contentType = response.Content.Headers.FirstOrDefault(p => p.Key.Equals("Content-Type", StringComparison.OrdinalIgnoreCase));
var byteArray = await response.Content.ReadAsByteArrayAsync();
var result = new FileContentResult(byteArray, contentType.Value.Any() ? contentType.Value.First() : defaultContentType);
return result;
}
}
}
}
If you want to use Azure Active Directory based access control for the storage, what you need to get is the access token. Here are the steps for your reference.
Register an application
2.Assign a build-in RBAC role to this application It depends on you which role to be assigned to the application.
3.Get the access token.
4.With the access token, now you can call the storage rest api.
I am attempting to perform a Google reverse image search using Google Cloud Vision on an Azure app service web app.
I have generated a googleCred.json, which the Google client libraries use in order to construct API requests. Google expects it to be available from an environment variable named GOOGLE_APPLICATION_CREDENTIALS.
The Azure app service that runs the web app has settings that mimic environment variables for the Google client libraries. The documentation is here, and I have successfully set the variable here:
Furthermore, the googleCred.json file has been uploaded to the app service. Here is the documentation I followed to use FTP and FileZilla to upload the file:
Also, the file permissions are as open as they can be:
However, when I access the web app in the cloud, I get the following error message:
Error reading credential file from location D:\site\wwwroot\Statics\googleCred.json: Could not find a part of the path 'D:\site\wwwroot\Statics\googleCred.json'. Please check the value of the Environment Variable GOOGLE_APPLICATION_CREDENTIALS
What am I doing wrong? How can I successfully use the Google Cloud Vision API on an Azure web app?
This error message is usually thrown when the application is not being authenticated correctly due to several reasons such as missing files, invalid credential paths, incorrect environment variables assignations, among other causes.
Based on this, I recommend you to validate that the credential file and file path are being correctly assigned, as well as follow the Obtaining and providing service account credentials manually guide in order to explicitly specify your service account file directly into your code; In this way, you will be able to set it permanently and verify if you are passing the service credentials correctly.
Passing the path to the service account key in code example:
// Imports the Google Cloud client library.
const Storage = require('#google-cloud/storage');
// Instantiates a client. Explicitly use service account credentials by
// specifying the private key file. All clients in google-cloud-node have this
// helper, see https://github.com/GoogleCloudPlatform/google-cloud-node/blob/master/docs/authentication.md
const storage = new Storage({
keyFilename: '/path/to/keyfile.json'
});
// Makes an authenticated API request.
storage
.getBuckets()
.then((results) => {
const buckets = results[0];
console.log('Buckets:');
buckets.forEach((bucket) => {
console.log(bucket.name);
});
})
.catch((err) => {
console.error('ERROR:', err);
});
I'm writing here since i can't comment, but at a quick glance, is the "D:" in the path necessary? I assume you uploaded the file to the app service so try with this value for the path "\site\wwwroot\Statics\googleCred.json"