How to upload files(images, pdf) using Azure blob storage with standard account? - nestjs

I am using azure blob storage standard account. When I try to upload files, it throws the error 'BlobTypeNotSupported'. I know standard account doesn't support 'Block Blobs'. Is there any other way to upload files using standard account?

I tried in my environment and got below results:
I created a standard storage account through azure portal :
I used the below code to upload images using nest.js
Code:
import { BlobServiceClient, BlockBlobClient } from '#azure/storage-blob';
import { Injectable } from '#nestjs/common';
#Injectable()
export class AppService {
azureConnection = "< Connection string >";
containerName = "test";
getBlobClient(imageName:string):BlockBlobClient{
const blobClientService = BlobServiceClient.fromConnectionString(this.azureConnection);
const containerClient = blobClientService.getContainerClient(this.containerName);
const blobClient = containerClient.getBlockBlobClient(imageName);
return blobClient;
}
async upload(file:Express.Multer.File){
const blobClient = this.getBlobClient(file.originalname);
await blobClient.uploadData(file.buffer);
}
}
console:
Postman:
You can use the post request to upload image via postman.
Portal:
The above request worked successfully and uploaded file through azure storage standard account using nest.js with Block-blob type.
Reference:
NestJS API File Operations Using Azure Blob Storage (learmoreseekmore.com)
by Naveen.

Related

Azure Blob Storage and NestJs: The requested URI does not represent any resource on the server

Issue
When sending request to create blob files, I got this error message:
The requested URI does not represent any resource on the server.
Background
I have Azure Blob Storage set up and have the following data:
Account name
Access key
Connection string
Connection URL
Container. Example: Container name: "TestContainer"
The above was also used to configure Azure Blob Storage connection in my NestJS project.
I want to create files inside the "TestContainer".
Currently, I'm using ContainerClient from #azure/storage-blob to create blob files.
Here is the code
import { ContainerClient } from '#azure/storage-blob';
#Injectable()
export class BlobStorageService {
constructor(
#Inject('CONTAINER_CLIENT') private readonly containerClient: ContainerClient
) {}
async create(key: string, message: any) {
const content = JSON.stringify(message);
try {
const blobName = key;
const appendBlobClient = this.containerClient.getAppendBlobClient(blobName);
await appendBlobClient.createIfNotExists();
await appendBlobClient.appendBlock(content, content.length);
} catch (error) {
throw error
}
}
}
Please help if you have experienced this before.
Thank you!
The requested URI does not represent any resource on the server.
The above error occurs either you are passing Invalid URI or incorrect URI.
As a workaround to upload files to azure container.I tried in my environment with blobserviceclient and Blockblobclient using following Document. by Naveen.
Code:
import { BlobServiceClient, BlockBlobClient } from '#azure/storage-blob';
import { Injectable } from '#nestjs/common';
#Injectable()
export class AppService {
azureConnection = "< Connection string >";
containerName = "test";
getBlobClient(imageName:string):BlockBlobClient{
const blobClientService = BlobServiceClient.fromConnectionString(this.azureConnection);
const containerClient = blobClientService.getContainerClient(this.containerName);
const blobClient = containerClient.getBlockBlobClient(imageName);
return blobClient;
}
async upload(file:Express.Multer.File){
const blobClient = this.getBlobClient(file.originalname);
await blobClient.uploadData(file.buffer);
}
}
Console:
Postman:
Portal:
Reference:
How to Upload Files to Azure using NestJS and typeORM with MySQL (freecodecamp.org) by Destiny Erhabor.

How to check existence of soft deleted file in Azure blob container with node js?

I have file which was stored in some Azure blob directory "folder1/folder2/file.txt". This file was soft deleted - I can see it in Azure web console. I need to have function which checks this file existence.
I tried library "azure-storage". It perfectly works with NOT removed files:
const blobService = azure.createBlobService(connectingString);
blobService.doesBlobExist(container, blobPath, callback)
May be anyone knows how use same approach with soft removed files?
I tied with lib "#azure/storage-blob".
But I stuck with endless entities there (BlobServiceClient, ContainerItem, BlobClient, ContainerClient, etc) and couldn't find way to see particular file in particular blob directory.
Following this MSDOC, I got to restore the Soft deleted blobs and their names with the below code snippet.
const { BlobServiceClient } = require('#azure/storage-blob');
const connstring = "DefaultEndpointsProtocol=https;AccountName=kvpstorageaccount;AccountKey=<Storage_Account_Key>;EndpointSuffix=core.windows.net"
if (!connstring) throw Error('Azure Storage Connection string not found');
const blobServiceClient = BlobServiceClient.fromConnectionString(connstring);
async function main(){
const containerName = 'kpjohncontainer';
const blobName = 'TextFile05.txt';
const containerClient = blobServiceClient.getContainerClient(containerName);
undeleteBlob(containerClient, blobName)
}
main()
.then(() => console.log(`done`))
.catch((ex) => console.log(ex.message));
async function undeleteBlob(containerClient, blobName){
const blockBlobClient = await containerClient.getBlockBlobClient(blobName);
await blockBlobClient.undelete(); //to restore the deleted blob
console.log(`undeleted blob ${blobName}`);
}
Output:
To check if the blob exists and if exists but in Soft-deleted state, I found the relevant code but it’s in C# provided by #Gaurav Mantri. To achieve the same in NodeJS refer here.

Azure Data Storage: Unable to upload file (Not authorized to perform this operation using this permission)

I'm trying to follow the example to upload a file to Azure Data Storage as mentioned in the documentation : https://learn.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-dotnet?tabs=visual-studio%2Cmanaged-identity%2Croles-azure-portal%2Csign-in-azure-cli%2Cidentity-visual-studio
Following is my code:
using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;
using System;
using System.IO;
using Azure.Identity;
// TODO: Replace <storage-account-name> with your actual storage account name
var blobServiceClient = new BlobServiceClient(
new Uri("https://[some azure storage]"),
new DefaultAzureCredential());
// Set container name
string containerName = "data";
// Get container
BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient(containerName);
// Create a local file in the ./data/ directory for uploading and downloading
string localPath = "data";
Directory.CreateDirectory(localPath);
string fileName = "testupload" + Guid.NewGuid().ToString() + ".txt";
string localFilePath = Path.Combine(localPath, fileName);
// Write text to the file
await File.WriteAllTextAsync(localFilePath, "Hello, World!");
// Get a reference to a blob
BlobClient blobClient = containerClient.GetBlobClient(fileName);
Console.WriteLine("Uploading to Blob storage as blob:\n\t {0}\n", blobClient.Uri);
// Upload data from the local file
await blobClient.UploadAsync(localFilePath, true);
But I'm getting an error message that the request is not authorized.
Error message:
Azure.RequestFailedException: 'This request is not authorized to perform this operation using this permission.
I have Contributor role (which based on description is Grant full access to manage all resources ....), is this role still not enough to perform the operation?
I tried in my environment and got below results:
Initially I tried same code in my environment and got same error
Console:
Azure.RequestFailedException: 'This request is not authorized to perform this operation using this permission.
The above error occurs when your principal doesn't has access to azure blob storage.
For accessing blob storage through identity, Gaurav Mantri comment it is correct, you need a role to access blob storage,
The roles are
Storage-blob-contributor(or)
Storage-blob-owner
Go to portal -> storage accounts -> Access Control (IAM) ->Add -> Add role assignments -> storage-blob-contributor or storage-blob-owner role to the storage account.
Portal:
After assigning role to my storage account, I executed same code and it successfully uploaded file in azure blob storage.
Code:
using Azure.Storage.Blobs;
using System;
using System.IO;
using Azure.Identity;
// TODO: Replace <storage-account-name> with your actual storage account name
var blobServiceClient = new BlobServiceClient(
new Uri("https://[some azure storage]"),
new DefaultAzureCredential());
// Set container name
string containerName = "test";
// Get container
BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient(containerName);
// Create a local file in the ./data/ directory for uploading and downloading
string localPath = "data";
Directory.CreateDirectory(localPath);
string fileName = "testupload" + Guid.NewGuid().ToString() + ".txt";
string localFilePath = Path.Combine(localPath, fileName);
// Write text to the file
await File.WriteAllTextAsync(localFilePath, "Hello, World!");
// Get a reference to a blob
BlobClient blobClient = containerClient.GetBlobClient(fileName);
Console.WriteLine("Uploading to Blob storage as blob:\n\t {0}\n", blobClient.Uri);
// Upload data from the local file
await blobClient.UploadAsync(localFilePath, true);
Console:
Portal:
Make sure you have to change the Network Access to Enable Public to All, if you're not using VPN or dedicated Network to access Azure Environment.

(Azure Storage - nodeJS) Getting SAS policies that are applied on blob container and queue

I'm trying to get the expiration date of the SAS policies that are applied on blob container and queue.
I'm able to get the information via powershell with the Get-AzStorageQueueStoredAccessPolicy and Get-AzStorageContainerStoredAccessPolicy but I cannot find a way to look a way to do the same via nodeJS.
I've went trough the MS node sdk for storage, i was able to find a way to setup the SAS policy but not to retrieve an existing one.
Do I need to go trough the ms graph?
Thank you for your help.
To get the access policies for a blob container, the method you would want to use is getAccessPolicy() which is in ContainerClient class.
import {BlobServiceClient} from '#azure/storage-blob';
const connectionString = "your-storage-account-connection-string";
const containerName = "your-container-name";
const blobServiceClient = BlobServiceClient.fromConnectionString(connectionString);
const containerClient = blobServiceClient.getContainerClient(containerName);
const accessPolicyResult = await containerClient.getAccessPolicy();
console.log(accessPolicyResult.signedIdentifiers);//access policies are defined in "signedIdentifiers"
Similarly to get the access policies for a queue, the method you would to use is getAccessPolicy() which is in QueueClient class.
import {QueueServiceClient} from '#azure/storage-queue';
const connectionString = "your-storage-account-connection-string";
const queueName = "your-queue-name";
const queueServiceClient = QueueServiceClient.fromConnectionString(connectionString);
const queueClient = queueServiceClient.getQueueClient(queueName);
const accessPolicyResult = await queueClient.getAccessPolicy();
console.log(accessPolicyResult.signedIdentifiers);//access policies are defined in "signedIdentifiers"

NodeJS: Azure functions with a `serviceBusTrigger` is giving `InvalidResourceName` error when trying to list blobs with `listBlobsFlat`

I have created two Azure functions, one with HTTPTrigger and another with ServiceBusTrigger. When I try to list blobs from HTTPTrigger function, with listBlobsFlat method of Azure Javascript SDK, I was able to list without any problem. When I try the same thing with ServiceBusTrigger function, I have an error saying Error in servicebus trigger RestError: <?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidResourceName</Code><Message>The specifed resource name contains invalid characters. I can see the queue message that triggered azure function. It is fine. The ServiceBusTrigger function works fine with other activities like uploading a Storage Account, etc.
My container name is datasources. It has only one file named twitter.csv. What am I missing?
Edit: Both the Functions use same function from a helper. I am using "#azure/service-bus": "^7.0.0-preview.2" and "#azure/storage-blob": "^12.1.2".
export const listAllBlobs = async (container: string = containerName): Promise<any> => {
const blobServiceClient = await BlobServiceClient.fromConnectionString(creds.connectionStr)
const containerClient = await blobServiceClient.getContainerClient(container);
const blobs = []
try {
for await (const blob of containerClient.listBlobsFlat()) {
blobs.push(blob)
}
return blobs
} catch(err) {
throw err
}
}

Resources