We use Azure Mobile Services with Javascript (Node.js) back-end. The front-end is HTML/Javascript and runs as Azure Web App. We want to use Azure blob storage to store files (uploaded from front-end). I searched for a working example implementing this scenario, but I can't find it. There are examples with .NET back-end or Android/Windows Phone front-end. As a workaround it's possible to post the file to mobile service and do the storage from there, but the mobile service api body has a 1MB limit. I know I have to use a Shared Access Signature (SAS), but I don't know how to implement that. Generating the url from mobile service works. But it's not accepted when I use it in the client.
This guide is not working anymore: http://gauravmantri.com/2013/02/16/uploading-large-files-in-windows-azure-blob-storage-using-shared-access-signature-html-and-javascript/
Thanks in advance for your help!
As usually, custom APIs on mobile services are used in handling logic workflows or event triggers. So Azure mobile service limit the body size of custom APIs requests for better performance. To implement upload files from clients to Azure Storage, we recommend to leverage SAS URIs.
And lots samples use the backend project to generate the SAS URI and return back to front-end.We can leverage Azure Node.js SDK in Mobile service custom APIs scripts to generate SAS URI.
Here is the code snippet:
exports.get = function(request, response) {
var azure = require('azure');
var qs = require('querystring');
var accountName = { accountName };
var accountKey = { accountKey };
var host = accountName + '.blob.core.windows.net';
var blobService = azure.createBlobService(accountName, accountKey, host);
var startDate = new Date();
var expiryDate = new Date(startDate);
expiryDate.setMinutes(startDate.getMinutes() + 30);
startDate.setMinutes(startDate.getMinutes() - 30);
var sharedAccessPolicy = {
AccessPolicy: {
Permissions: azure.Constants.BlobConstants.SharedAccessPermissions.WRITE,
Start: startDate,
Expiry: expiryDate
},
};
// you can custom send container name and blob name via http get request
/** e.g. var containerName = request.query.container,
blobName = request.query.blob
client side use invokeApi method, e.g.
client.invokeApi('getSAS',{
method:'GET',
parameters:{container:'mycontainer',blob:'myblob'}
})
**/
var blobSAS = blobService.generateSharedAccessSignature('mycontainer', 'myblob', sharedAccessPolicy);
var sasQueryString = qs.stringify(blobSAS.queryString);
var sasUri = blobSAS.baseUrl + blobSAS.path;
response.send(sasUri+"?"+sasQueryString);
};
You can refer to Upload images to Azure Storage from an Android device and Work with a JavaScript backend mobile service for reference.
Furthermore, for a deep understanding of generating SAS URI, you can refer to Constructing a Service SAS and Shared Access Signatures, Part 1: Understanding the SAS Model
Additionally, here is a similar example built in this architecture, Upload files to Microsoft Azure Storage from JavaScript
Related
My web app allows users to upload files.
I want to use cloud azure blob storage for this.
Since downloading will be very frequent (more than uploading)
I would like to save server computing time and bandwith and serve files directly from the azure blob server.
I believe it is made possible on Google cloud with Firebase (Firestorage).
Where you can upload and download directly from the client. (I know authentication and authorization are also managed by firebase so it makes things easier)
Does any similar mechanisms/service exist on Azure?
For example
When a user clicks an azure storage download link a trigger would check the JWT for authorization and data would be sent directly to the client from azure storage
Similar option is available with Azure Blob storage as well. You can use the Storage SDK to access the containers and list/download the blob
With a javascript backend You can either use SAS Token or Azure Storage JavaScript Client Library also supports creating BlobService based on Storage Account Key for authentication besides SAS Token. However, for security concerns, use of a limited time SAS Token, generated by a backend web server using a Stored Access Policy.
Example here
EDIT:
I have not answered the question completely above, However if you want to access the blob storage or download any files from the blob storage you can make use of normal http get request with SAS token generated with any JavaScript application.
With Angualr:
uploadToBLob(files) {
let formData: FormData = new FormData();
formData.append("asset", files[0], files[0].name);
this.http.post(this.baseUrl + 'insertfile', formData)
.subscribe(result => console.log(result));
}
downloadFile(fileName: string) {
return this.http.get(this.baseUrl + 'DownloadBlob/' + fileName, { responseType: "blob" })
.subscribe((result: any) => {
if (result) {
var blob = new Blob([result]);
let saveAs = require('file-saver');
let file = fileName;
saveAs(blob, file);
this.fileDownloadInitiated = false;
}
}, err => this.errorMessage = err
);
}
However the best practice (considering the security) is to have a backend API/Azure function to handle the file upload.
I am building an Angular 6 application that will be able to make CRUD operation on Azure Blob Storage. I'm however using postman to test requests before implementing them inside the app and copy-pasting the token that I get from Angular for that resource.
When trying to read a file that I have inside the storage for test purposes, I'm getting: <Code>AuthorizationPermissionMismatch</Code>
<Message>This request is not authorized to perform this operation using this permission.
All in production environment (although developing)
Token acquired specifically for storage resource via Oauth
Postman has the token strategy as "bearer "
Application has "Azure Storage" delegated permissions granted.
Both the app and the account I'm acquiring the token are added as "owners" in azure access control IAM
My IP is added to CORS settings on the blob storage.
StorageV2 (general purpose v2) - Standard - Hot
x-ms-version header used is: 2018-03-28 because that's the latest I could find and I just created the storage account.
I found it's not enough for the app and account to be added as owners. I would go into your storage account > IAM > Add role assignment, and add the special permissions for this type of request:
Storage Blob Data Contributor
Storage Queue Data Contributor
Make sure to use Storage Blob Data Contributor and NOT Storage Account Contributor where the latter is only for managing the actual Storage Account and not the data in it.
I've just solved this by changing the resource requested in the GetAccessTokenAsync method from "https://storage.azure.com" to the url of my storage blob as in this snippet:
public async Task<StorageCredentials> CreateStorageCredentialsAsync()
{
var provider = new AzureServiceTokenProvider();
var token = await provider.GetAccessTokenAsync(AzureStorageContainerUrl);
var tokenCredential = new TokenCredential(token);
var storageCredentials = new StorageCredentials(tokenCredential);
return storageCredentials;
}
where AzureStorageContainerUrl is set to https://xxxxxxxxx.blob.core.windows.net/
Be aware that if you want to apply "STORAGE BLOB DATA XXXX" role at the subscription level it will not work if your subscription has Azure DataBricks namespaces:
If your subscription includes an Azure DataBricks namespace, roles assigned at the subscription scope will be blocked from granting access to blob and queue data.
Source: https://learn.microsoft.com/en-us/azure/storage/common/storage-auth-aad-rbac-portal#determine-resource-scope
Make sure you add the /Y at the end of the command.
Used the following to connect using Azure AD to blob storage:
This is code uses SDK V11 since V12 still has issues with multi AD accounts
See this issue
https://github.com/Azure/azure-sdk-for-net/issues/8658
For further reading on V12 and V11 SDK
https://learn.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-dotnet-legacy
https://learn.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-dotnet
using Microsoft.Azure.Services.AppAuthentication;
using Microsoft.Azure.Storage.Auth;
using Microsoft.Azure.Storage.Blob;
using Microsoft.Azure.Storage.Queue;
[Fact]
public async Task TestStreamToContainer()
{
try
{
var accountName = "YourStorageAccountName";
var containerName = "YourContainerName";
var blobName = "File1";
var provider = new AzureServiceTokenProvider();
var token = await provider.GetAccessTokenAsync($"https://{accountName}.blob.core.windows.net");
var tokenCredential = new TokenCredential(token);
var storageCredentials = new StorageCredentials(tokenCredential);
string containerEndpoint = $"https://{accountName}.blob.core.windows.net";
var blobClient = new CloudBlobClient(new Uri(containerEndpoint), storageCredentials);
var containerClient = blobClient.GetContainerReference(containerName);
var cloudBlob = containerClient.GetBlockBlobReference(blobName);
string blobContents = "This is a block blob contents.";
byte[] byteArray = Encoding.ASCII.GetBytes(blobContents);
using (MemoryStream stream = new MemoryStream(byteArray))
{
await cloudBlob.UploadFromStreamAsync(stream);
}
}
catch (Exception e)
{
Console.WriteLine(e.Message);
Console.ReadLine();
throw;
}
}
Is it possible to use the Azure Fluent API to add a WebJob to a WebApp? I'm not finding any documentation describing this.
I believe the answer is no, and that you are required to use the Azure Kudu WebJob API
More information about that can be found:
https://github.com/projectkudu/kudu/wiki/WebJobs-API
https://blogs.msdn.microsoft.com/benjaminperkins/2016/02/01/using-the-azure-webjob-api/
As Lachie said, it seems that there is no Fluent API support to add WebJobs, you could use WebJobs KUDU API to achieve it.
I do a demo for that. It works correctly in my site. The following is my detail steps:
Preparation:
1.The WebJob API require Basic Authentication using the publishing credentials, you could go to your webapp in azure portal to click Get publish profile and download it to get username and userpassword.
2.Zip the WebJob to be published file.
Steps:
1.Create a C# console project.
2.Add the following code in the Program.cs file.
string userName = "$name";
string userPassword = "pass";
string webAppName = "webappname";
string webJobName = "webjobname";
var base64Auth = Convert.ToBase64String(Encoding.Default.GetBytes($"{userName}:{userPassword}"));
var file = File.ReadAllBytes(#"webjob zip file path");
MemoryStream stream = new MemoryStream(file);
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Add("Authorization", "Basic " + base64Auth);
client.DefaultRequestHeaders.Add("ContentType", "application/zip");
var baseUrl = new Uri($"https://{webAppName}.scm.azurewebsites.net/");
var requestURl = baseUrl + "api/continuouswebjobs/"+webJobName;
var httpContent = new StreamContent(stream);
httpContent.Headers.Add("Content-Disposition", "attachement;filename="+ webjob.exe);
var response2 = client.PutAsync(requestURl, httpContent).Result;
}
Note: The filename should be in the Content-Dispostion header. Here I deploy the continuous webjob, if you want to deploy trigger webjob, you could change continuouswebjobs to triggeredwebjobs.
3.Test from the local.
4.Check the published result on azure.
I'm receiving the following error when I try connecting to the emulated storage queue service:
The MAC signature found in the HTTP request '...' is not the same as any computed signature.
Make sure the value of Authorization header is formed correctly including the signature.
This is the approach I'm using to connect to the azure storage:
var storageAccount = 'devstoreaccount1'
var accessKey= 'Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw=='
var azure = require('azure-storage');
var queueSvc = azure.createQueueService(storageAccount,accessKey);
queueSvc.createMessage('myqueue', "Hello world!", function(error, results, response){
if(!error){
// Message inserted
}
});
I also tryed using the following connection strings, without success:
UseDevelopmentStorage=true
and
DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;
AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;
BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;
TableEndpoint=http://127.0.0.1:10002/devstoreaccount1;
QueueEndpoint=http://127.0.0.1:10001/devstoreaccount1;
Everything is working properly in production environments, the issue is only related to the emulated service and specifically for the queues (emulated blobs are working as expected).
Any idea?
I have reproduced your error after testing your code with SDK 2.8.1.
I get detailed log from console by using queueSvc.logger.level = azure.Logger.LogLevels.DEBUG;.
The request uri generated by this method is https://devstoreaccount1.queue.core.windows.net:443/myqueue/messages, which is used to access storage account online called devstoreaccount1.
To access storage emulator:
var azure = require('azure-storage');
var devStoreCreds = azure.generateDevelopmentStorageCredentials();
var queueSvc = azure.createQueueService(devStoreCreds);
I have deployed a web job to Azure and also have a request/response graphs associated with it (insight). I want to extract the graph information and display it on another web page (as an embedded graph). Please see the attached screenshot for my azure dashboard which have the graph.
Is there a RESTApi or any SDK available to get graph(app insight) from azure and display it in the web page?
(hope this is not related to powerbi)
If you want get the requests and AverageResponseTime metrics We could use the Monitor API or Microsoft.Azure.Management.Monitor to do that. I do a demo to get the Azure WebApp request metrics demo. Other supported metrics supported metrics please refer to Azure Monitor Metrics List.
Preparetion:
Registry Azure Active Directory application and assign Role
Then we can get the tenantId,clientId,clientSecretKey
var azureTenantId = "tenantId";
var azureSecretKey = "clientSecretKey";
var azureAppId = "clientId";
var subscriptionId = "subscriptionId";
var resourceGroup = "ResourceGroup";
var webAppName = "webAppname";
var serviceCreds = ApplicationTokenProvider.LoginSilentAsync(azureTenantId, azureAppId, azureSecretKey).Result;
var client = new MonitorClient(serviceCreds) {SubscriptionId = subscriptionId};
var resourceUrl = $"subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Web/sites/{webAppName}";
var result = client.Metrics.ListWithHttpMessagesAsync(resourceUrl,metric: "Requests",interval:new TimeSpan(0,0,5,0),timespan: "2018-02-19T06:57:56Z/2018-02-20T07:57:56Z").Result;