azure data factory BlobEventsTrigger : set "advanced filter" programatically - azure

I am trying to set "advanced filter" of BlobEventsTrigger programmatically.
They are reset at each deployment. I need only 3 and having 7 of them are causing the job to start twice. It is super annoying to delete them manually after each deployment.
I have tried to add a field "advancedFilters" or "blobType" to the trigger json file without success.
"typeProperties": {
"blobPathBeginsWith": "/bingofile/blobs/",
"blobPathEndsWith": "/_SUCCESS",
"ignoreEmptyBlobs": false,
"scope": "/subscriptions/bingofilesup/resourceGroups/bingofilesup/providers/Microsoft.Storage/storageAccounts/bingofilesup",
"events": [
"Microsoft.Storage.BlobCreated"
]
I've also tried az eventgrid system-topic event-subscription update but this library does not work when it comes to updating advanced filter. It asks an endpoint (which is normally a facultative argument) , and when provided the existing data factory endpoint, it fails reaching it.
I have checked the documentation about this endpoint and it is said to be the webhook endpoint .
Endpoint where EventGrid should deliver events matching this event
subscription. For webhook endpoint type, this should be the
corresponding webhook URL. For other endpoint types, this should be
the Azure resource identifier of the endpoint. It is expected that the
destination endpoint to be already created and available for use
before executing any Event Grid command.
But it does not work .
Deployment failed. Correlation ID:
95e4fab5-163e-48ab-8cb2-b23432516e53. Webhook validation handshake
failed for [webwook end point provided in the topic]. Http POST
request failed with response code Unknown. For troublehooting, visit
https://aka.ms/esvalidation.
Any observation or suggestion would be great, thanks in advance !

According to my test, the endpoint https://pmeastasia.svc.datafactory.azure.com:4443/triggerevent/BlobEventsTrigger/<> is juts a base URL. When the events are sent to data factory or update subscription, azure will generate an endpoint with the base URL to do auth. So if you want to update the subscription with other tools, I think you need to use fildder to catch the request to get the whole endpoint at first.

Related

Webhook validation handshake failed

i'm new in azure and wanted to create an Event subscription to push any changes (here resource write success) from my blob storage to my Logic App via HTTP. In my Logic app i have a When a HTTP request is received, which would run when i send a request.
just wanted to demonstrate how i'm trying to create it.
NAME: 'i dont think this matters'
...
TOPIC NAME: 'just gave it a Random Name, should this refer to sth?'
Source Resource:'My Storage Account'
ENDPOINT:'i got the URL from the Overview page of my logic app (Workflow URL
)'
URL looks like this:
https://LOGICAPPNAME.azurewebsites.net:443/api/APPNAME/triggers/manual/invoke?api-version=2022-05-01&sp=%2Ftriggers%2Fmanual%2Frun&sv=1.0&sig=RjVKZbs-0CV559hZYlFfhM0k22W39lS5
when i copy and paste this to my browser i can trigger my Logic App. i think that act as a GET and i'm not sure if it sends it as POST or etc. it would make any difference.
and i got this error.
Deployment has failed with the following error: {"code":"Url validation","message":"Webhook validation handshake failed for https://LOGICAPPNAME.azurewebsites.net/api/APPNAME/triggers/manual/invoke. Http POST request retuned 2XX response with response body . When a validation request is accepted without validation code in the response body, Http GET is expected on the validation url included in the validation event(within 10 minutes). For troublehooting, visit https://aka.ms/esvalidation. Activity id:ID, timestamp: DATE TIME"}
If there's still confusion on how i'm doing this, i'm trying to follow THIS EXAMPLE. How can i fix this Error?
After reproducing from my end, I could able to achieve this following the link you have provided using the below details for creation of my logic app.
The reason you are receiving that error is that you need to use Logic App HTTP Trigger Request URL as Web Hook's endpoint and not the Logicapp's URL.
NOTE: You don't really have to call the Trigger again, whenever you make any changes to your storage account the logic apps gets triggered.
RESULTS:
When Blob is Created
When a Blob is Deleted
REFERENCES: "When blob is added or modified" trigger will not be fired on subfolder" answered by SamaraSoucy-MSFT - Microsoft Q&A

How to pass in client Certificate parameter when creating service fabric connection azure devops REST API

I am attempting to create a service fabric connection via the azure devops rest API, as documented here:
https://learn.microsoft.com/en-us/rest/api/azure/devops/serviceendpoint/endpoints/create?view=azure-devops-rest-6.0
When attempting to define the authorization parameters, I have the following fields in my request:
"authorization": {
"parameters":{
"certlookup":"Thumbprint",
"servercertthumbprint": "{{certificateThumbprint}}",
"certificate":"{{certname}}",
"certificatepassword":null,
},
"scheme":"certificate"
},
This will create a service connection to the cluster, however it does not look like the 'client certificate' parameter (as shown in screenshot bellow) is parsed in anyhwhere. I also cant seem to find anywhere in the documentation that says how to do this.
How can I pass in the "Client Certificate" Value when using the REST API?
Below is the detailed view of the parameters which you can use for creating Service Fabric Service Connection:
Pls try to make your request like below (un tested) to make it work:
"authorization": {
"parameters":{
"certlookup":"Thumbprint",
"servercertthumbprint": "{{certificateThumbprint}}",
"clientcertificatedata":"{{certname}}",
"password":null,
},
"scheme":"certificate"
},
Please refer to EndpointAuthorizationParameters Class for details about the parameters to use.
The correct parameter to specify this value is actually just "certificate".
The original request posted is correct, I was entering the name of the certificate rather than the encoding of it.

Is there a way to get the status of an Event Grid trigger for azure function (Complete /Pending or Running)

by Httptrigger Azure function, if you send a POST request you receive something like this as a response:
{
"id": "66ee5d08196874aeb99c9e62ddc7b190",
"statusQueryGetUri": "https://asynchttpfunction.azurewebsites.net/runtime/webhooks/durabletask/instances/66ee5d08196945aeb44c9e62ddc7b190?taskHub=Orchestration&connection=Storage&code=FSVfJyGODSeKHPO0cM8Po9e1jMT7MghVMGuJqTaGTN56E1RUHnlVJg==",
"sendEventPostUri": "https://asynchttpfunction.azurewebsites.net/runtime/webhooks/durabletask/instances/66ee5d08196945aeb44c9e62ddc7b190/raiseEvent/{eventName}?taskHub=Orchestration&connection=Storage&code=FSVfJyGODSeKHPO0cM8Po9e1jMT7MghVMGuJqTaGTN56E1RUHnlVJg==",
"terminatePostUri": "https://asynchttpfunction.azurewebsites.net/runtime/webhooks/durabletask/instances/66ee5d08196945aeb44c9e62ddc7b190/terminate?reason={text}&taskHub=Orchestration&connection=Storage&code=FSVfJyGODSeKHPO0cM8Po9e1jMT7MghVMGuJqTaGTN56E1RUHnlVJg==",
"rewindPostUri": "https://asynchttpfunction.azurewebsites.net/runtime/webhooks/durabletask/instances/66ee5d08196945aeb44c9e62ddc7b190/rewind?reason={text}&taskHub=Orchestration&connection=Storage&code=FSVfJyGODSeKHPO0cM8Po9e1jMT7MghVMGuJqTaGTN56E1RUHnlVJg==",
"purgeHistoryDeleteUri": "https://asynchttpfunction.azurewebsites.net/runtime/webhooks/durabletask/instances/66ee5d08196945aeb44c9e62ddc7b190?taskHub=Orchestration&connection=Storage&code=FSVfJyGODSeKHPO0cM8Po9e1jMT7MghVMGuJqTaGTN56E1RUHnlVJg=="
}
The statusQueryGetUri provides information of the long running orchestration instance. If you follow this link you will receive a suitable runtimeStatus that describes the status of the orchestration instance along with some other useful information.here
My question is now:
actually we don't send a POST request to an Event grid Azure function trigger, Is there any way to get the status of the Azure function? Complete or is still running?
The Azure Event Grid is an eventing Pub/Sub model where the interest of source is distributed to the subscribed event handler endpoint or resource in the reliable manner with a retry policy and dead-lettering option. The AEG is waiting for delivery response processing max. 60 seconds.
There is no built-in the features what you are asking in the AEG, however you can use the REST API for metrics of the specific subscription to obtain its counters value:
MatchedEventCount,
DeliveryAttemptFailCount,
DeliverySuccessCount,
DroppedEventCount,
DeadLetteredCount
The following GET is an example for getting a subscription metrics:
https://management.azure.com/subscriptions/mysubId/resourceGroups/mygroup/providers/Microsoft.EventGrid/topics/mytester/providers/Microsoft.EventGrid/eventSubscriptions/mysubscription/providers/Microsoft.Insights/metrics?api-version=2018-01-01&interval=PT5M&metricnames=MatchedEventCount,DeliveryAttemptFailCount,DeliverySuccessCount,DroppedEventCount,DeadLetteredCount
Note, that the authorization header with a bearer token is required for this call.
More details about the monitoring an event message delivery can be found here.

azure api management retrieving password from shell script ,saving it and deploying it

I had seen multiple examples how to use Azure API manager using powershell to retreive password for SCM but unable to find any example using it without powershell i.e. something execution in shell command line. i am also looking for example for saving and deploying my api
You can always inspect any Azure service API in action by doing what you need in Azure portal and seeing what requests get sent. Correlating them with documentation helps. So to get access token for SCM endpoint in APIM you need:
With any Azure credentials make a GET call to https://management.azure.com/subscriptions/.../resourceGroups/.../providers/Microsoft.ApiManagement/service/.../tenant/access/git?api-version=2018-01-01
in response you will get a payload similar to:
{
"id": "XXX",
...
}
Take "id" from that payload and make a POST call to https://management.azure.com/subscriptions/.../resourceGroups/.../providers/Microsoft.ApiManagement/service/.../users/XXX/token?api-version=2018-01-01
you will get your token:
{
"value":"..."
}

Building SOAP Listener with Azure Functions

I am using Azure Functions to build some integrations between various systems. I new requirement is to respond to record updates in Salesforce. Some quick research yielded what seems like a good solution from the Salesforce side. Use Outbound messaging which can send SOAP requests on record modifications.
How to create Salesforce application that will send record to external web service when record created/changed(https://salesforce.stackexchange.com/questions/73425/how-to-create-salesforce-application-that-will-send-record-to-external-web-servi)
The challenge now is to be able create a SOAP listener in Azure Function. I have created basic HTTP Triggers for my other listeners. Is there anything "built-in" to Azure Functions that would allow me to easily consume the incoming SOAP request?
Salesforce has the basics for a solution based on a more traditional web service and an ASMX file but I am not sure if or how that can be applied in Azure Functions. (https://developer.salesforce.com/docs/atlas.en-us.api.meta/api/sforce_api_om_outboundmessaging_listener.htm)
That notification is just a SOAP request that is made over HTTP, so really not too different than a regular HTTP trigger request.
Although you could just treat that as a plain request and parse the contents yourself, Azure Functions does expose the great WebHook support we get from ASP.NET WebHooks, and luckily, there is a Salesforce receiver that significantly simplifies this task.
DISCLAIMER: It's worth noting that although the receiver is technically enabled in Azure Functions, there's no official support for it yet, so you won't find a lot of documentation and help will be limited to what you get on SO and Forums. Official support to this and other receivers will hopefully be coming soon, which means documentation, templates and UI support will become available.
To get started, you need the following:
Create a new function, selecting the GenericWebHook - CSharp template (this works for node as well, but I'll focus on C# here.
Follow the steps outlined on the ASP.NET WebHooks integration with Salesforce post in order to create the outbound message. Here you want to use the Function Url given to you by the portal WITHOUT THE CODE QUERY STRING (having the code there wouldn't hurt, but the receiver does not use that information).
IMPORTANT: Get your Salesforce Organization ID, which will be used for authentication and is located under Administer > Company Profile > Company Information > Salesforce.com Organization ID and back in the Azure Functions portal, open the Keys panel, delete your default function key (not host key) and create a new key, named default (this name is important) using the Organization ID value you got from Salesforce.
Go to Integrate
On the integration page, select Advanced Editor on the upper right (as mentioned, there's no official support, so the UI does not expose this. We're putting our explorer hats on and venturing into a more advanced workflow here :) )
Change the webHookType property value to sfsoap and save the configuration. Your function.json config should look like the following:
function.json:
{
"bindings": [
{
"type": "httpTrigger",
"direction": "in",
"webHookType": "sfsoap",
"name": "req"
},
{
"type": "http",
"direction": "out",
"name": "res"
}
],
"disabled": false
}
Switch to the Develop tab. We're ready to write our code.
This is where the ASP.NET WebHooks receiver shines! It will parse the notification for you, exposing strong typed objects you can work with. All you need to do is modify the method/function signature you get withe template to use the SalesforceNotifications type, making sure you're referencing the required assembly (Microsoft.AspNet.WebHooks.Receivers.Salesforce, which is made available to you, so no need for package reference) and namespace reference (Microsoft.AspNet.WebHooks).
Here is a full sample of a function that will receive the request and log the Organization ID, Action ID, grab the first notification and log all of its properties:
#r "Microsoft.AspNet.WebHooks.Receivers.Salesforce"
using Microsoft.AspNet.WebHooks;
public static void Run(SalesforceNotifications req, TraceWriter log)
{
log.Info($"Salesforce webhook was triggered!");
log.Info(req.OrganizationId);
log.Info(req.ActionId);
var notification = req.Notifications.First();
foreach (var p in notification.Keys)
{
log.Info($"{p} = {notification[p]}");
}
}
This process will be a lot smoother when the receiver is officially supported, but even with the added steps, this still beats having to parse the SOAP messages yourself :)
I hope this helps!

Resources