I am trying to send a file using external file protocol and FTP api connection. The configuration and code is straight forwards and the app runs successfully however no data is sent to the FTP and I cannot see any trace that the function even tried to send data using ftp.... What is wrong? and more important; Where can i monitor the progress of the external file api?
My code follows (Note: I have tried Stream and string as input and output)
run.csx
public static void Run(Stream myBlobInput, string name, out Stream
myFTPOutput, TraceWriter log)
{
myFTPOutput = myBlobInput;
//log.Info($"C# Blob trigger function Processed blob\n Name:{name} \n Content:{myBlob}");
log.Info($"C# Blob trigger function Processed blob\n Name:{name} \n Size:{myBlobInput.Length} \n Content:{myBlobInput.ToString()}");
}
function.json
"bindings": [
{
"name": "myBlobInput",
"type": "blobTrigger",
"direction": "in",
"path": "input/{name}",
"connection": "blob_STORAGE"
},
{
"name": "myFTPOutput",
"type": "apiHubFile",
"direction": "out",
"path": "/output/{name}",
"connection": "ftp_FTP"
}
],
"disabled": false
}
I could make it working :
If we want to have same file content in the output FTP server and same file name ,then here is the code and function.json
public static void Run(string myBlob, string name, TraceWriter log , out string outputFile )
{
log.Info($"2..C# Blob trigger function Processed blob\n Name:{name} \n Size:{myBlob.Length} \n Content:{myBlob.ToString()}");
outputFile=myBlob;
}
Also here is the function.json
{
"bindings": [
{
"name": "myBlob",
"type": "blobTrigger",
"direction": "in",
"path": "myblobcontainer/{name}",
"connection": "AzureWebJobsDashboard"
},
{
"type": "apiHubFile",
"name": "outputFile",
"path": "LogFiles/{name}",
"connection": "ftp_FTP",
"direction": "out"
}
],
"disabled": false
}
the input binding should have a valid container name as in blob account here--:
blob container structure as path
Also in output binding for FTP, the path should be any folder in Root of FTP , what you see in FTP login UI/console and then filename , which in this case {name} ,which allows us to keep the same output file name as input blob name.
Ok, so I changed the ftp Connection to some other server and it work like a charm. That means that there were some firewall refusal from the Azure Function that triggered. The sad thing is that no error messages triggers that i can spot. Thanks for all support
Related
I have a simple HTTP trigger Azure Function with multiple Service Bus output bindings. All the bindings are pointing to the same Topic, but they have different Subscriptions. If I was to set this function app through function.json it is pretty straightforward:
{
"bindings": [
{
"authLevel": "function",
"name": "req",
"type": "httpTrigger",
"direction": "in",
"methods": [
"get",
"post"
]
},
{
"name": "$return",
"type": "http",
"direction": "out"
},
{
"type": "serviceBus",
"connection": "SERVICEBUS",
"name": "output",
"topicName": "outtopic",
"subscriptionName": "sub",
"direction": "out"
},
{
"type": "serviceBus",
"connection": "SERVICEBUS",
"name": "output",
"topicName": "outtopic",
"subscriptionName": "sub2",
"direction": "out"
}
],
"disabled": false
}
But I publish my functions via Visual Studio and therefore my Azure Functions are read only in the portal and function.json is automatically generated by VS upon publishing.
The problem is that I cannot figure out how do I setup multiple output bindings pointing to different subscriptions. Currently I have something like this:
[FunctionName("Function2")]
public static async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)] HttpRequest req,
[ServiceBus("outtopic", entityType:EntityType.Topic)] IAsyncCollector<string> output,
[ServiceBus("outtopic", entityType: EntityType.Topic)] IAsyncCollector<string> output2,
ILogger log)
{
log.LogInformation("C# HTTP trigger function processed a request.");
string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
await output.AddAsync(requestBody);
return new OkObjectResult("OK");
}
As you can see output and output2 are pointing to the same Topic, but there is no option to specify Subscription.
At this point I am pretty confident that this has not been implemented yet. But I hope there is a workaround maybe?
Try this, add Connection property in the definition, per this example - if by subscription you mean on the azure subscription:
public static void Run([BlobTrigger("inputvideo/{customername}/{date}/{filename}", Connection = "AzureWebJobsStorage")]Stream myBlob,
string customername,
string date,
string filename,
[ServiceBus("detectobjectsqueue",EntityType.Queue, Connection="ServiceBusConnectionString")] IAsyncCollector<string> output,
ILogger log)
Update
Per your comment, I understood that by subscription you mean on topic subscription. In that case, the idea of topic is that all subscriptions receive the message. So you have one publisher, and whoever subscribes to the topic it will receive the message. If you would like to make sure that a specific subscriber receives the message, either implement message filtering(per type for example) on the receiving endpoint or use dedicated queue per subscriber.
Also, conceptually, the publisher should not know who are the subscribers, and the subscriber should not know who is the publisher. If you know who is the subscriber, why not use the REST call for example to the receiving endpoint?
It is not possible to directly put messages into a Topic Subscription, rather every message has to come through a Topic.
To make sure only a particular subscription receives a message, you need to configure the Topic Subscription rule. You can read more about rules in the blog post here.
I can't quite seem to get the output bindings to enable a file to be saved to blob storage. I have created an Azure Function using Python, that uses a CosmosDB Change Feed trigger. I need to save that document to blob storage.
I've set-up the function.json file as follows:
{
"scriptFile": "__init__.py",
"bindings": [
{
"type": "cosmosDBTrigger",
"name": "documents",
"direction": "in",
"leaseCollectionName": "leases",
"connectionStringSetting": "cosmos_dev",
"databaseName": "MyDatabase",
"collectionName": "MyCollection",
"createLeaseCollectionIfNotExists": "true"
},
{
"type": "blob",
"direction": "out",
"name": "outputBlob",
"path": "raw/changefeedOutput/{blobname}",
"connection": "blobStorageConnection"
}
]
}
So the trigger will get a documents like the following:
{ "id": "documentId-12345",
other sections here
"entity": "customer"
}
In the init.py file I have the base code of
def main(documents: func.DocumentList) -> func.Document:
logging.info(f"CosmosDB trigger executed!")
for doc in documents:
blobName = doc['id'] + '.json'
blobFolder= doc['entity']
blobData = doc.to_json()
I think i need to add in the def something like 'outputBlob: func.Out' but unsure how to proceed
Looking at the examples on github
https://github.com/yokawasa/azure-functions-python-samples/tree/master/v2functions/blob-trigger-watermark-blob-out-binding
it look like i have to
outputBlob.set(something)
So i'm looking for how to set up the def part and send the blob to the location that i've set from the data in the cosmosdb document.
I have tried the following:
def main(documents: func.DocumentList, outputBlob: func.Out[str] ) -> func.Document:
logging.info(f"CosmosDB trigger executed!")
for doc in documents:
blobName = doc['id'] + '.json'
outputBlob.set(blobName)
and get the result:
CosmosDB trigger executed!
Executed 'Functions.CosmosTrigger_py' (Failed, Id=XXXXX)
System.Private.CoreLib: Exception while executing function: Functions.CosmosTrigger_py. Microsoft.Azure.WebJobs.Host: No value for named parameter 'blobname'.
I could just call the connection stuff from the os.enviro, and get the connection string that way, I think and use the standard create_blob_from_text, with location, name and blob data,
block_blob_service.create_blob_from_text(blobLocation, blobName, formattedBlob)
Any pointers would be great
I have a activity function that should store message in Blob storage.I can overwrite a file in blob storage but i need to store data in different name.how to do that? Azure function doesn't support dynamic binding in nodejs.
Find one workaround, see whether it's useful.
Along with blob output binding, there's an activity trigger to receive message msg, we can put self-defined blob name in msg for blob binding path to consume.
In your orchestrator function which calls Activity function
yield context.df.callActivity("YourActivity", {'body':'messagecontent','blobName':'myblob'});
Then Activity function code should be modified
context.bindings.myOutputBlob = context.bindings.msg.body;
And its function.json can use blobName as expected
{
"bindings": [
{
"name": "msg",
"type": "activityTrigger",
"direction": "in"
},
{
"name":"myOutputBlob",
"direction": "out",
"type": "blob",
"connection": "AzureWebJobsStorage",
"path": "azureblob/{blobName}"
}
],
"disabled": false
}
Is it possible to invoke cosmos db trigger in azure pipeline? Pipeline is just copy data from azrue storage to cosmos db collection and it is necessary to invoke pre trigger. How to specify trigger id for copy activity?
From what you are saying, you might solve this by using Azure Functions with a Blob Trigger and a DocumentDB output binding.
With a functions.json similar to:
{
"disabled": false,
"bindings": [
{
"name": "myBlob",
"type": "blobTrigger",
"direction": "in",
"path": "<name-of-the-folder-where-files-get-uploaded>",
"connection":"MyStorageAccount"
},
{
"name": "documentToSave",
"type": "documentDB",
"databaseName": "MyDatabase",
"collectionName": "MyCollection",
"createIfNotExists": true,
"connection": "MyAccount_COSMOSDB",
"direction": "out"
}
]
}
And the function body could be something like:
// Blob trigger binding to a CloudBlockBlob
#r "Microsoft.WindowsAzure.Storage"
using Microsoft.WindowsAzure.Storage.Blob;
public static void Run(CloudBlockBlob myBlob, out object documentToSave, TraceWriter log)
{
// some logic to read the blob and parse it
documentToSave = new {
id = "some value",
.. other properties here
};
}
Trying to figure out how to use SendGrid from an azure function. Not much docs to find, but this is what I've tried:
#r "SendGrid"
using SendGrid.Helpers.Mail;
using System;
public static void Run(string myQueueItem, out Mail message, TraceWriter log)
{
log.Info($"C# Queue trigger function processed: {myQueueItem}");
message=new Mail();
}
I've hardcoded to, from, subject and body, together with api key in the integrate output section. This is my function.json:
{
"bindings": [
{
"name": "myQueueItem",
"type": "queueTrigger",
"direction": "in",
"queueName": "send-email-request",
"connection": "myStorage"
},
{
"type": "sendGrid",
"name": "message",
"apiKey": "SG.xxxxxxxxxxx",
"direction": "out",
"to": "me#gmail.com",
"from": "me#gmail.no",
"subject": "hardcoded",
"text": "test213"
}
],
"disabled": false
}
I get the following error:
Function ($SendEmailOnQueueTriggered) Error: Microsoft.Azure.WebJobs.Host: Error indexing method 'Functions.SendEmailOnQueueTriggered'. Microsoft.Azure.WebJobs.Host: 'SG.xxxxxx' does not resolve to a value.
Session Id: 9426f2d7e4374c7ba7e0612ea5dc1814
Timestamp: 2017-01-07T12:18:01.930Z
I've granted the Apikey full access in SendGrid. Any ideas what I've missed?
Larsi
The ApiKey field is not the actual ApiKey, it should instead be the name of an AppSettings key defined in the "functions app settings".
Adding "AzureWebJobsSendGridApiKey" in your AppSettings via local.settings.json and in Configuration -> Application Setting via Azure Portal will work.