I have the following Azure function triggered when a file is uploaded to Blob Storage
[FunctionName("ImageAnalysis")]
public static async void Run(
[BlobTrigger("imageanalysis/{name}", Connection = "AzureWebJobsStorage")] Stream myBlob,
string name,
TraceWriter log)
{
log.Info($"C# Blob trigger function Processed blob\n Name:{name} \n Size: {myBlob.Length} Bytes");
}
I want to process the Blob that has been uploaded so ideally I would like it as a CloudBlockBlob instead of a Stream. Then I can just do some work and then delete the blob.
myBlob.DeleteIfExists()
Is there an easy way to cast or convert my Stream to CloudBlockBlob or do I need to use input/output bindings or something else?
Looking through the docs I see examples which use CloudBlockBlob but I can't seem to get it to work so think I am missing something?
Use this syntax for the binding. The trick is specifying FileAccess.ReadWrite in the attribute. The docs rather confusingly refer to this as "inout" for some reason.
[Blob("imageanalysis/{name}", FileAccess.ReadWrite, Connection = "AzureWebJobsStorage")] CloudBlockBlob blob, string name
Related
I'm working on pulling files via azure search in C#.
I have everything set up, but if I upload a new file, it takes 5 minutes for the file to be indexed to show up in the list of upload files after I pass in a specific filter.
https://learn.microsoft.com/en-us/azure/search/search-howto-schedule-indexers
Is there anyway around this?
Is there a way to run the indexer immediately after upload?
Thanks.
You can combine Event Grid / Azure functions with Blob Storage Binding to trigger your indexer.
Run indexer REST API:
https://learn.microsoft.com/en-us/rest/api/searchservice/run-indexer
[FunctionName("BlobTriggerCSharp")]
public static void Run([BlobTrigger("samples-workitems/{name}")] Stream myBlob, string name, ILogger log)
{
log.LogInformation($"C# Blob trigger function Processed blob\n Name:{name} \n Size: {myBlob.Length} Bytes");
//call the indexer from previous link
}
https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-storage-blob-trigger?tabs=csharp
I have created a blob trigger azure function which uses connection string in the code at the moment.
local.settings.json
public static class BlobTrigger_Fun
{
[FunctionName("BlobTrigger_Fun")]
public static void Run([BlobTrigger("democontainerazure/{name}", Connection = "AzureWebJobsStorage")]Stream myBlob, string name, ILogger log)
{
log.LogInformation($"C# Blob trigger function Processed blob\n Name:{name} \n Size: {myBlob.Length} Bytes");
}
}
I want to use managed identity to avoid use of connection string in the code.
No, you can't.
The MSI(managed identity) is not for such usage, it is just used for authenticating to azure services that support Azure AD authentication, the AzureWebJobsStorage is used for azure function runtime, in the function app, the property must be specified as an app setting in the site configuration.
I have two blob triggers which I want to trigger on. One works and one does not!
I use Azure Storage Explorer to make sure blobs are uploaded to each blob, scanFiles will never trigger, and scanExports seem to always trigger.
Question: What can cause some blobs to not trigger an Azure function?
[FunctionName("scanFiles")]
public static async Task FilesScan([BlobTrigger("files/{newBlobName}", Connection = "BlobConnectionstring")]CloudBlockBlob newBlob, string newBlobName, ILogger log, ExecutionContext context)
{
await VirusScan(newBlob, newBlobName, log, context);
}
[FunctionName("scanExports")]
public static async Task ExportsScan([BlobTrigger("exports/{newBlobName}", Connection = "BlobConnectionstring")]CloudBlockBlob newBlob, string newBlobName, ILogger log, ExecutionContext context)
{
await VirusScan(newBlob, newBlobName, log, context);
}
I found that the issue was because one of the blobs were considered High scale, which triggers this bug and also makes the host process kill itself when run locally (see this issue)
Creating Azure Functions targeting .Net Standard 2.0 using Visual Studio 2017.
Using the Add New Azure Function wizard, a blob trigger method is successfully created with the following method signature.
public static void Run([BlobTrigger("attachments-collection/{name}")] Stream myBlob, string name, ILogger log)
This method compiles and works fine.
However, we want to be able access the metadata connected to the CloudBlockBlob being saved to storage, which as far as I know is not possible using a stream. Other answers on this site such as (Azure Function Blob Trigger CloudBlockBlob binding) suggest you can bind to a CloudBlockBlob instead of a Stream and access the metadata that way. But the suggested solution does not compile in latest version of Azure Functions.
Microsoft's online documentation (https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-storage-blob#trigger---usage) also seems to confirm that it is possible to bind the trigger to a CloudBlockBlob rather than a Stream, but gives no example of the syntax.
Could someone please clarify the exact syntax required to enable Azure Function Blob storage trigger to bind to a CloudBlockBlob instead of the standard Stream?
Thanks
Thanks to Jerry Liu's s insights, this problem has been solved.
Method:
Use the default storage package for Azure Storage that is installed when you create a new Function App
Microsoft.Azure.WebJobs.Extensions.Storage (3.0.1)
This installs dependency
WindowsAzure.Storage (9.3.1)
Then both of the following method signatures will run correctly
public static async Task Run([BlobTrigger("samples-workitems/{name}")]Stream myBlob, string name, ILogger log)
and
public static async Task Run([BlobTrigger("samples-workitems/{name}")]CloudBlockBlob myBlob, string name, ILogger log)
Actually CloudBlockBlob does work, we don't need FileAccess.ReadWrite as it's BlobTrigger instead of Blob input or output.
public static Task Run([BlobTrigger("samples-workitems/{name}")]CloudBlockBlob blob, string name, ILogger log)
Update for Can't bind BlobTrigger to CloudBlockBlob
There's an issue tracking here, Function SDK has some problem integrating with WindowsAzure.Storge >=v9.3.2. So just remove any WindowsAzure.Storage package reference, Function SDK references v9.3.1 internally by default.
I want to create a blob storage trigger that takes any files put into blob storage (a fast process) and transfers them to Data Lake storage (NOT to another Blob Storage location).
Can this be done?
Can it be done using JavaScript, or does it require C#?
Does sample code exist showing how to do this? If so, would you be so kind as to point me to it?
Note: we've created a pipeline that will go from Blob Storage to Data lake storage. That's not what I'm asking about here.
You could potentially use an Azure Function or Azure Logic App to detect new files on Blob Storage and either call your webhook to trigger the pipeline or do the move itself.
Can this be done?
As jamesbascle mentioned that we could use Azure function to do that.
Can it be done using JavaScript, or does it require C#?
It can be done with javascript or C#.
Does sample code exist showing how to do this? If so, would you be so kind as to point me to it?
How to create a Blob storage triggered function, please refer to this document. We also could get the C#/javascript demo code from this document.
JavaScript code
module.exports = function(context) {
context.log('Node.js Blob trigger function processed', context.bindings.myBlob);
context.done();
};
C# code
[FunctionName("BlobTriggerCSharp")]
public static void Run([BlobTrigger("samples-workitems/{name}")] Stream myBlob, string name, TraceWriter log)
{
log.Info($"C# Blob trigger function Processed blob\n Name:{name} \n Size: {myBlob.Length} Bytes");
}