I am working to pull some objects from an Rest API endpoint where the response is restricted to send only 500 items in one call.
I am doing it in azure synapse and unable to get the specific header value which need to be passed into copy activity to continue the loop until all the data is retrieved.
Can someone help in this ?
Related
I am using Web Activity to get the response from an API. The API takes a Csv file as its body parameter along with some string. I tried using this link How can i pass file as parameter in post request (Web Activity) Azure Data Factory? but could not succeed, I was getting the below error
The other help says to use logic apps for this , has anyone tried using web activity for posting form-data.
I am trying to trigger my Azure Pipeline using an HTTP request. I am doing this using the Runs API provided by Azure DevOps,
https://learn.microsoft.com/en-us/rest/api/azure/devops/pipelines/runs/run-pipeline?view=azure-devops-rest-6.0
I want to be able to access the message body sent along with the POST request, within my pipeline.
I know that it is possible to do the following,
Add parameters to my pipeline similar to what has been shown below,
parameters:
- name: id
type: string
Send the message body of my request to the Runs API within templateParameters to be able to access the values as parameters.
The issue that I am facing is, my pipeline will actually be triggered by a Webhook and as a result, I don't have control over the structure of the message body. I cannot put the data within templateParameters for the values to be accessible through the parameters.
Is there any other way that I can access the values of the message body? Is there some other solution to my problem?
As #Thomas mentioned, the body of the request sent will need to be as defined in the DevOps API, therefore, I had to use an Azure Function as an intermediary to take my initial request, put it inside of templateParameters and re-route it to Azure DevOps.
My InvokeRESTAPI displays the following if executed:
I can see a Response field printed that is empty.
Can I somehow use this to get a value back from the API reply and use it to call another template with it?
On the task itself I am using this parameter:
successCriteria: eq(root["status"], "successful")
So I would assume that if there is some custom success criterion then I would be able to use a value from the response in order to define it.
Based on your screenshot, you are using the API to post the json content to Azure Logic App workflow.
So you need to add the Response Action in Azure Logic App to add response content to the work flow.
Here are the steps:
In Logic App:
Then when you run the API in Invoke Rest API task, it will show the customer response.
For more detailed info, you could refer to this doc: Construct the response
I am trying to use Azure Logic Apps to read data from one of the Salesforce objects which has huge number of records. The Salesforce connector fetches the record and returns the pagination link as #odata.nextLink. But when I try to use a JSON parser to read the value it’s coming as null.
I tried to access the nextlink in a browser but it requires authentication. What authentication do we pass here ?
I would like to use an until action to iterate till I get a next link . So how do I check the condition for the until loop ?
nextLink doesn't look like core Salesforce thing, it might be the OData connector preprocessing the results for you. You'd have to consult the documentation for the connector (if any).
Salesforce's REST API will return field with nextRecordsUrl if there's next page of results, you'd call that in loop until the field disappears. You'd call it like any other REST API resource available after login, by passing Authorization: Bearer <sessionId also known as accessToken here>. Again - probably the connector abstracts this away from you. Don't think you can send headers like that in browser, you'd need curl, Postman, SoapUI or similar http client.
If you don't get better answer and documentation is scarce - consider using raw REST API. Or Azure Data Factory has an almost-decent Salesforce connector?
https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/dome_query.htm
I have created an API in Azure API Management to get data from a backend API.
The result from the backend API is paged, including a 'next' url in the response which must be used for the next request.
How to handle this 'next' url, in order to concatenate all paged responses and return the total of all paginated responses as one single response through Azure APIM?
That would be quite complex policy, but could be done by combining retry and send-request. In short keep retrying request every time with next url and concatenating results as long as there is next link in response.