i'm new in azure and wanted to create an Event subscription to push any changes (here resource write success) from my blob storage to my Logic App via HTTP. In my Logic app i have a When a HTTP request is received, which would run when i send a request.
just wanted to demonstrate how i'm trying to create it.
NAME: 'i dont think this matters'
...
TOPIC NAME: 'just gave it a Random Name, should this refer to sth?'
Source Resource:'My Storage Account'
ENDPOINT:'i got the URL from the Overview page of my logic app (Workflow URL
)'
URL looks like this:
https://LOGICAPPNAME.azurewebsites.net:443/api/APPNAME/triggers/manual/invoke?api-version=2022-05-01&sp=%2Ftriggers%2Fmanual%2Frun&sv=1.0&sig=RjVKZbs-0CV559hZYlFfhM0k22W39lS5
when i copy and paste this to my browser i can trigger my Logic App. i think that act as a GET and i'm not sure if it sends it as POST or etc. it would make any difference.
and i got this error.
Deployment has failed with the following error: {"code":"Url validation","message":"Webhook validation handshake failed for https://LOGICAPPNAME.azurewebsites.net/api/APPNAME/triggers/manual/invoke. Http POST request retuned 2XX response with response body . When a validation request is accepted without validation code in the response body, Http GET is expected on the validation url included in the validation event(within 10 minutes). For troublehooting, visit https://aka.ms/esvalidation. Activity id:ID, timestamp: DATE TIME"}
If there's still confusion on how i'm doing this, i'm trying to follow THIS EXAMPLE. How can i fix this Error?
After reproducing from my end, I could able to achieve this following the link you have provided using the below details for creation of my logic app.
The reason you are receiving that error is that you need to use Logic App HTTP Trigger Request URL as Web Hook's endpoint and not the Logicapp's URL.
NOTE: You don't really have to call the Trigger again, whenever you make any changes to your storage account the logic apps gets triggered.
RESULTS:
When Blob is Created
When a Blob is Deleted
REFERENCES: "When blob is added or modified" trigger will not be fired on subfolder" answered by SamaraSoucy-MSFT - Microsoft Q&A
Related
I am trying to call API endpoint as a POST and Content-Type as from-data using azure data factory web activity. Tried different way of passing Body parameters but it failed.
Here is the Postman Request.
Here is the Azure Data Factory Web Activity configurations.(use the body as a json and tried different combinations but all didn't work)
And above is the error message.
Any help would be highly appreciated.
Since your request is seen successfully executing from postman, try copying the entire body from that and use in web activity.
The format for passing body for a POST request from a web activity is shown here.
Also make sure you have entered valid “url–Target endpoint and path”. This is usually seen as Activity requires Public end point but you may have used Private vnet where this is not allowed. Web Activity is supported for invoking URLs that are hosted in a private virtual network as well by leveraging self-hosted integration runtime. The integration runtime should have a line of sight to the URL endpoint.
Note: The activity will timeout at 1 minute with an error if it does not receive a response from the endpoint.
Further going through some similar scenarios it is learnt that;
Mostly the header is passed as string in WebActivity whereas Postman
it is integer/long
In case your API tries redirecting, it seems that the web activity
in Azure Data Factory does not currently support following
redirects, meanwhile Postman and other tools and libraries usually
follow redirects by default or include a option for handling them.
Checkout the supported authentication types in the web activity. If you are trying to authorize your from, try set the following.
URL: https://login.microsoftonline.com/<<tenantid>>/oauth2/token
Headers: Content-Type - "application/x-www-form-urlencoded"
Body: grant_type=client_credentials&client_id=<<clientid>>f&client_secret=<<secret>>&resource=https%3A%2F%2Fmicrosoft.onmicrosoft.com%2F<<resourceId>>
Error code: 2108:
Message: Error calling the endpoint '%url;'. Response status code: '%code;'
Cause: The request failed due to an underlying issue such as network connectivity, a DNS failure, a server certificate validation, or a timeout.
Workaround: Make the API call using Powershell, and call that Powershell script from within Data Factory.
I am trying to set "advanced filter" of BlobEventsTrigger programmatically.
They are reset at each deployment. I need only 3 and having 7 of them are causing the job to start twice. It is super annoying to delete them manually after each deployment.
I have tried to add a field "advancedFilters" or "blobType" to the trigger json file without success.
"typeProperties": {
"blobPathBeginsWith": "/bingofile/blobs/",
"blobPathEndsWith": "/_SUCCESS",
"ignoreEmptyBlobs": false,
"scope": "/subscriptions/bingofilesup/resourceGroups/bingofilesup/providers/Microsoft.Storage/storageAccounts/bingofilesup",
"events": [
"Microsoft.Storage.BlobCreated"
]
I've also tried az eventgrid system-topic event-subscription update but this library does not work when it comes to updating advanced filter. It asks an endpoint (which is normally a facultative argument) , and when provided the existing data factory endpoint, it fails reaching it.
I have checked the documentation about this endpoint and it is said to be the webhook endpoint .
Endpoint where EventGrid should deliver events matching this event
subscription. For webhook endpoint type, this should be the
corresponding webhook URL. For other endpoint types, this should be
the Azure resource identifier of the endpoint. It is expected that the
destination endpoint to be already created and available for use
before executing any Event Grid command.
But it does not work .
Deployment failed. Correlation ID:
95e4fab5-163e-48ab-8cb2-b23432516e53. Webhook validation handshake
failed for [webwook end point provided in the topic]. Http POST
request failed with response code Unknown. For troublehooting, visit
https://aka.ms/esvalidation.
Any observation or suggestion would be great, thanks in advance !
According to my test, the endpoint https://pmeastasia.svc.datafactory.azure.com:4443/triggerevent/BlobEventsTrigger/<> is juts a base URL. When the events are sent to data factory or update subscription, azure will generate an endpoint with the base URL to do auth. So if you want to update the subscription with other tools, I think you need to use fildder to catch the request to get the whole endpoint at first.
I'm trying to fetch data from REST API and save the json string it into DataLake and I'm getting an error. I've followed the steps mentioned here
https://learn.microsoft.com/en-us/azure/data-factory/connector-rest & https://www.alexvolok.com/2019/adfv2-rest-api-part1-oauth2/
The API which I'm trying to connect uses OAuth2 so I need to first get the access token and then do a get request to get actual data.
Below are the steps which I'm following
Creating a Web HTTP request in the pipeline and passing the client_ID, client secret, username, password and grant type in the body of the request. When I debug the pipline I do get the Access_token which I need in step 2.
In Step two I have a copy activity which uses the output(access_token) from web to authenticate the second REST GET request but this is where I'm facing a lot of issues. The code which I'm using is "#concat('Bearer ', activity('GetAccessToken').output.access_token)"
In step 3 I have two datasets and 2 Linked services, Dataset 1 is a REST dataset which has the base url and relative url which is linked to the REST linked service and secondly the sink dataset is connected to AZURE datalake storage.
In Source Dataset I'm passing additional header Authorization = #concat('Bearer ', activity('GetAccessToken').output.access_token) and ideally since the API which I want to call will return empty if no parameters are send so I pass in the parameters inside the "Request Body" is that even correct? the Request body would look something like this "start_date=2020/07/17&end_date=2020/07/18".
The sink is a simple Json dataset stored in DataLake.
When I try to debug I get the error as below
But I'm getting the below error
{
"errorCode": "2200",
"message": "Failure happened on 'Source' side. ErrorCode=UserErrorHttpStatusCodeIndicatingFailure,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The HttpStatusCode 401 indicates failure. { \"Error\": { \"Message\":\"Authentication failed: Invalid headers\", \"Server-Time\":\"2020-07-27T06:59:24\", \"Id\":\"6AAF87BC-5634-4C28-8626-810A19B86BFF\" } },Source=Microsoft.DataTransfer.ClientLibrary,'",
"failureType": "UserError",
"target": "CopyDataFromAPI",
"details": []
}
Please advise if I'm doing anything wrong.
I Knew this was a simple issue,
So for people who are looking for answers.
Please make sure the REST Source URL starts with HTTPS:// instead of HTTP:// I Guess Azure does not pass headers to url which starts with HTTP:// which is strange because POSTMAN and python script has no problem sending the headers.
After developing in the sandbox, we got our api key approved and promoted to a live account.
Since then we've been getting the following response -
response: {
"errorCode": "ACCOUNT_LACKS_PERMISSIONS",
"message": "This Account lacks sufficient permissions."
}
http code: 401
exeucted at: 2017-05-17 15:03:59
Based on my research and according to ACCOUNT_LACKS_PERMISSIONS error when creating envelope
A setting needs to be switched on the backend at Docusign. The user mentions -
"They changed a setting called In Session to Enabled in API section near limiter that only the account manager or tier 2 support can change. All is well."
The account ID is 30953035
API username bcbffa28-a316-473e-b2b7-48d964d909a7
The API request is below. This was working just fine under a Demo account. I've even upgraded to the Intermediate API in the hopes that it will resolve my issues but no dice.
Support says that I need to post here...
This is caused by a bad account baseUrl that's being used in the request. When your integration performs authentication for a given user, if you are using Legacy auth (X-DocuSign-Authentication header) then you need to point to the following /login_information endpoint for the live system:
https://www.docusign.net/restapi/v2/login_information
When you get the response you then need to parse the baseUrl value that was returned and use that sub-domain for subsequent API requests. (Note that there are multiple sub-domains in the live system such as NA1, NA2, EU, etc)
The baseUrl that's returned will look something like:
https://na2.docusign.net/restapi/v2/accounts/12345/envelopes
Make sure you configure your code to read this sub-domain and use in subsequent requests, otherwise you if you simply use www for instance you will not be hitting the correct account endpoint and you'll receive the "Account lacks permissions" error you're receiving.
Ergin's answer seems to work; however, he does not state which part of the baseUrl to keep after parsing. In his example the baseUrl = "https://na2.docusign.net/restapi/v2/accounts/12345/envelopes" In all subsequent calls after authApi.Login(); use "https://na2.docusign.net/restapi" as the URL and that should eliminate the error message.
I'm trying to make some calls to the new Azure Scheduler API. However, all my requests come back with this error:
<Error xmlns="http://schemas.microsoft.com/windowsazure" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
<Code>AuthenticationFailed</Code>
<Message>The server failed to authenticate the request. Verify that the certificate is valid and is associated with this subscription.</Message>
</Error>
I'm pretty sure that I have everything setup correct because I can make calls using the same code and certificate to the Azure Service Management API.
The code I'm using to attach the certificate to the web request is from the MSDN Sample. The Scheduler API calls that I've tried to make are the Check Name Availability, Create Cloud Service, and Create Job Collection.
I've also verified that my subscription is Active for the preview of the Scheduler.
Here is an example of a request I've tried:
Create Cloud Service
Request A cloud service is created by submitting an HTTP PUT operation
to the CloudServices OData collection of the Service Management API
Tenant.Replace with your subscription ID and
with your cloud service ID.
So for this I create a web request pointing to:
https://management.core.windows.net/[MySubId]/cloudServices/[MyNewServiceName]
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(requestUri);
// Define the requred headers to specify the API version and operation type.
request.Headers.Add("x-ms-version", "2012-03-01");
request.Method = "PUT";
request.ContentType = "application/xml";
Next I add the request body as specified in the documentation:
<CloudService xmlns:i='http://www.w3.org/2001/XMLSchema-instance' xmlns='http://schemas.microsoft.com/windowsazure'>
<Label>[MyServiceName]</Label>
<Description>testing</Description>
<GeoRegion>uswest</GeoRegion>
</CloudService>
And finally I add the certificate that I use with my subscription to the account.
// Attach the certificate to the request.
request.ClientCertificates.Add(certificate);
I try to get the response and instead I get the error shown above.
BTW - I've also tried different regions thinking maybe it was a region issue since the scheduler isn't supported in all regions, but I still get the same response.
You need to register the scheduler in your application first by calling (PUT):
<subscription id>/services?service=scheduler.JobCollections&action=register
If you want to do this in .NET you can use the new Management libraries:
var schedulerServiceClient = new SchedulerManagementClient(credentials);
var result = schedulerServiceClient.RegisterResourceProvider();
Console.WriteLine(result.RequestId);
Console.WriteLine(result.StatusCode);
Console.ReadLine();
More detail: http://fabriccontroller.net/blog/posts/a-complete-overview-to-get-started-with-the-windows-azure-scheduler/