Call SharePoint Rest API from Azure Data Factory Pipeline Web - azure

I am trying to use Azure Data Factory to call the SharePoint Rest API to access sharepoint list data. I need to use the rest api because the SharePoint list has some complex data types (multi-select lookups).
In my pipleline I have 2 Web activities, one (called GetAToken) to get an access token and one (called GetListData) to get the data using the access token.
The GetAToken web activity does a post to https://accounts.accesscontrol.windows.net/ee780cdf-8eab-4e6f-a740-c31a2aede2c0/tokens/OAuth/2
with the body:
grant_type=client_credentials&client_id=016dfe90-1b12-4938-8a9a-dee02a43d013#ee780cdf-8eab-4e6f-a740-c31a2aede2c0&client_secret={mysecrethere}&resource=00000003-0000-0ff1-ce00-000000000000/russellwgove.sharepoint.com.sharepoint.com#ee780cdf-8eab-4e6f-a740-c31a2aede2c0
When I debug, This runs successfully and returns this object containing the token:
{
"token_type": "Bearer",
"expires_in": "86399",
"not_before": "1667487425",
"expires_on": "1667574125",
"resource": "00000003-0000-0ff1-ce00-000000000000/russellwgove.sharepoint.com.sharepoint.com#ee780cdf-8eab-4e6f-a740-c31a2aede2c0",
"access_token": "{the token is here}",
"ADFWebActivityResponseHeaders": {
"Pragma": "no-cache",
"Strict-Transport-Security": "max-age=31536000; includeSubDomains",
"X-Content-Type-Options": "nosniff",
"x-ms-request-id": "32926e95-5588-42ec-a8b3-349fef942a00",
"x-ms-ests-server": "2.1.14059.10 - EUS ProdSlices",
"X-XSS-Protection": "0",
"Cache-Control": "no-store, no-cache",
"P3P": "CP="DSP CUR OTPi IND OTRi ONL FIN"",
"Set-Cookie": "fpc=AgaNWKRvGG9IkIzP4suqH9H9VI5sAQAAAO3O9doOAAAA; expires=Sat, 03-Dec-2022 15:02:05 GMT; path=/; secure; HttpOnly; SameSite=None;esctx=AQABAAAAAAD--DLA3VO7QrddgJg7Wevrrm262sBqfXG85d6ros_Ik0DCezx_KFOnktHyOEHyz8OREffRB2ad9hCwezrn2CFOzyOu5ttnkxBd1-vfkQL6eU3a6-qGSd_SaxdNREiiyLIMJIyBPp8qQU5zM1ySi6c7LVde0JFJ3jRtgH7p5QOVD4glxgUfLkK273i69DivI0G_x3wQqkpy6FLpZa2XXOjSYwZPPm1tHc_wPVBp1zd_vF3IpLLIa_pEbA_ZsTr0HscgAA; domain=.accounts.accesscontrol.windows.net; path=/; secure; HttpOnly; SameSite=None;x-ms-gateway-slice=estsfd; path=/; secure; httponly;stsservicecookie=estsfd; path=/; secure; httponly",
"Date": "Thu, 03 Nov 2022 15:02:05 GMT",
"Content-Length": "1467",
"Content-Type": "application/json; charset=utf-8",
"Expires": "-1"
},
"effectiveIntegrationRuntime": "AutoResolveIntegrationRuntime (Central US)",
"executionDuration": 0,
"durationInQueue": {
"integrationRuntimeQueue": 0
},
"billingReference": {
"activityType": "ExternalActivity",
"billableDuration": [
{
"meterType": "AzureIR",
"duration": 0.016666666666666666,
"unit": "Hours"
}
]
}
}
The GetListData activity is triggered by the OnSuccess trigger from the GetAToken Activity. It makes a GET request to https://russellwgove.sharepoint.com/sites/tr/US/_api/Web/Lists(guid'233dd303-1ae3-422b-8960-ad53661e8ef6')/Items to get the list data. It has an Authorization header set to :
#{concat('Authorization: Bearer ', activity('GetAToken').output.access_token)}
When I debug,the GetListData activity fails with an error 2108. And a message :
Error calling the endpoint 'https://russellwgove.sharepoint.com'. Response status code: 'NA - Unknown'. More details: Exception message: 'NA - Unknown [ClientSideException] The format of value 'Authorization: Bearer {myTokenhere}' is invalid.'.
Request didn't reach the server from the client. This could happen because of an underlying issue such as network connectivity, a DNS failure, a server certificate validation or a timeout.
I have no idea what that message means 'Request didn't reach the server from the client. This could happen because of an underlying issue such as network connectivity, a DNS failure, a server certificate validation or a timeout.' What is the server and the client is this context?
Anyone has any ideas how to make this work?

Related

Can I change virtual machine auto shut down time after azure virtual machine created using Rest Api?

This Api on this URL can be helpfull ?
https://learn.microsoft.com/en-us/rest/api/dtl/virtualmachineschedules/get
Please Help !
No, the API you provided is for the VM in Azure DevTest Labs, if you want to change the auto shut downtime for a normal azure VM, use the one below.
To acquire an access token, see this link.
Request URL:
PUT https://management.azure.com/subscriptions/<subscription-id>/resourceGroups/<resource-group-name>/providers/microsoft.devtestlab/schedules/shutdown-computevm-<VMname>?api-version=2018-09-15
Request header:
Content-Type: application/json; charset=utf-8
Authorization: Bearer eyJ0exxxxx6dyJ9
Request body(1900 means 19:00 i.e. 7:00:00 PM):
{
"properties": {
"taskType": "ComputeVmShutdownTask",
"timeZoneId": "China Standard Time",
"dailyRecurrence": {
"time": "1900"
},
"targetResourceId": "/subscriptions/<subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.Compute/virtualMachines/<VMname>",
"status": "Enabled",
"notificationSettings": {
"timeInMinutes": 30,
"status": "Disabled"
}
},
"location": "eastus"
}

SFTP trigger that isn't triggering in Logic APP

I have also an SFTP trigger that isn't triggering. I added a file to the SFTP folder and nothing happens. I think I had done this successfully in the past. I have it set to to check every 3 minutes.
Edit: I found that if I clicked on the "skipped" entry that I could see the input and output links. I noticed X-Frame-Options is deny. can anyone help me to solve this issue
output is
{
"statusCode": 202,
"headers": {
"Pragma": "no-cache",
"Retry-After": "15",
"x-ms-request-id": "1eb3db25-ce9d-4387-91fc-612e035017f7",
"Strict-Transport-Security": "max-age=31536000; includeSubDomains",
"X-Content-Type-Options": "nosniff",
"X-Frame-Options": "DENY",
"Timing-Allow-Origin": "*",
"x-ms-apihub-cached-response": "true",
"Cache-Control": "no-store, no-cache",
"Date": "Thu, 03 Oct 2019 16:23:08 GMT",
"Location": "https://logic-apis-eastus2.azure-apim.net/apim/sftp/ddd3328a306446e4b5d438eb72ca1fba/datasets/default/triggers/batch/onupdatedfile?folderId=L2N5Z2RyaXZlL2MvVXBsb2FkRW5jb3VudGVyc0ZpbGU%3d&maxFileCount=1&triggerstate=eyJGaWxlSWQiOiIiLCJTdGF0dXMiOjEsIldpbmRvd1N0YXJ0VGltZSI6IjIwMTktMTAtMDNUMTY6MjI6MTZaIiwiV2luZG93RW5kVGltZSI6IjIwMTktMTAtMDNUMTY6MjI6MTZaIiwiTGFzdFByb2Nlc3NlZEZpbGVUaW1lIjoiMjAxOS0xMC0wM1QxNjoyMjowNy4wMDAwMDAxWiIsIkxhc3RDb25mbGljdFRpbWUiOiIyMDE5LTEwLTAzVDE2OjIyOjE5LjU4Mzg0NDNaIn0%3d",
"Set-Cookie": "ARRAffinity=a703f806325944e1ff92f5b6964fbc06b4fdf5647d48c76a29966c392ea989b9;Path=/;HttpOnly;Domain=sftp-eus2.azconn-eus2.p.azurewebsites.net",
"Content-Length": "0",
"Expires": "-1"
}
}
There could be multiple reasons of this issue as mentioned below:
1) Check your Logic App is enabled or not.
2) The SFTP connector handles only files that are 50 MB or smaller and doesn't support message chunking.
3) Do you have the correct SFTP folder specified in your SFTP connector? Are you uploading your files to the same folder?

In azure logic App how to pass Cookie to Http request

In azure logic app by calling first Http endpoint we need to get the cookie and pass to the second endpoint. But while passing second API, I am passing in header and it is sending but the response is wrong(text/html).
But When I checked by using postman it worked fine got response in JSON(application/json).
The header getting after calling second API or endpoint through logic app ,I am getting is:
{
"statusCode": 200,
"headers": {
"Cache-Control": "no-store, must-revalidate, no-cache",
"X-Powered-By": "Undertow/1",
"Set-Cookie": "JSESSIONID=Ntu9NMCoekk56JveKceina3FCwgNJptU4EK0GOIz.s192-169-141-210; path=/StLightControl",
"Server": "WildFly/10",
"Pragma": "no-cache,no-cache",
"Date": "Tue, 12 Jun 2018 14:25:45 GMT",
"Connection": "keep-alive",
"Content-Length": "2585",
"Expires": "0,0",
"Content-Type": "text/html; charset=UTF-8"
}
}
The “Add new parameter” dropdown allows you to enable a checkbox that says “Cookie” that enables you to enter the HTTP cookie (from dynamic content if you want to). You cannot enter the value straight in the dropdown, so close it down after you selected the checkbox and see a magic Cookie field appear underneath the Authentication field.
More information to be found here: HowTo: Call an API with cookie authentication from a Logic App

Azure Logic Apps Cannot send HTTP request for wns/raw notification

I have a small logic app which is meant to fetch data from somewhere and store it on blob. I then would like to send the URL to all devices via push notification. I wish to send the URL as a raw notification, so on the app/background task, i can do some processiong.
The problem is when i use logic app to create a http POST request to send a notification, i get a 400 error. The same header with authentication and etc, with the payload and URL works fine on a POSTMAN or REST API CLIENT. THe following are the inputs and outputs. Please help. Brain dead already.
This is the input.
{
"uri": "https://xxoppoc.servicebus.windows.net/xxopPOC/messages/?api-version=2015-01",
"method": "POST",
"headers": {
"Authorization": "SharedAccessSignature sr=sb%3a%2f%2fxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxt%2f&sig=qEnxxxxxxxxxxxx&skn=DefaultFullSharedAccessSignature",
"Content-Type": "application/octet-stream",
"ServiceBusNotification-Format": "windows",
"ServiceBusNotification-Tags": "MyTag",
"X-WNS-Type": "wns/raw"
},
"body": "Some Raw Information. Hello World"
}
This is the output:
{
"statusCode": 400,
"headers": {
"transfer-Encoding": "chunked",
"date": "Wed, 30 Mar 2016 14:10:41 GMT",
"server": "Microsoft-HTTPAPI/2.0"
},
"body": {
"$content-type": "application/xml; charset=utf-8",
"$content": "PEVycm9yPjxDb2RlPjQwMDwvQ29kZT48RGV0YWlsPlRoZSBjb250ZW50IHR5cGUgZm9yIGEgJ3ducy9yYXcnIG5vdGlmaWNhdGlvbiBtdXN0IGJlICdhcHBsaWNhdGlvbi9vY3RldC1zdHJlYW0nLlRyYWNraW5nSWQ6NTNmNjhlMGItNDc1MC00ZDRkLWJiNTAtMzJjNTBmOGIyNDk1X0czLFRpbWVTdGFtcDozLzMwLzIwMTYgMjoxMDo0MSBQTTwvRGV0YWlsPjwvRXJyb3I+"
}
}
Let me mention again, I got the authentication correct as it works on Advanced REST Client on chrome and POSTMAN. The above logic app also works if i send a wns/toast notification with xml as its content-type. I however need it to be a wns/raw notification. Please help. Thank you
EDIT/PROGRESS
Thanks to MichaelB, We figured out that the content-type is being modified. I sent the request to a different URL to look at the header. The following was observed:
Content-Type: application/octet-stream; charset=utf-8
If I use the above Content-Type on POSTMAN. It actually fails as well. So this could be 1 step. Why is Azure Logic Apps adding charset-utf-8 to the end of my content type. Can I stop it?

Envelope Recipient not matching when requesting view

When creating envelopes which are used later to generate a view URL for the signer to use instead of receiving an email, we are running into a problem where the user we are specifying isn't valid.
For example, here is an envelope with one recipient, "Peter Hughes Jr."
Request URL: https://na2.docusign.net/restapi/v2/accounts/4476693/envelopes/f3a45945-f1ff-4731-a407-963a32378d77/recipients
------
{"IntegratorKey":"***","Password":"***","Username":"onboarding#impellam.com"}
Request headers:
X-DocuSign-Authentication : {"IntegratorKey":"***","Password":"***","Username":"onboarding#impellam.com"}
Host : na2.docusign.net
Connection : Keep-Alive
------
Request body:
------
Response headers:
Strict-Transport-Security : max-age=7776000; includeSubDomains
Content-Length : 696
Cache-Control : no-cache
Content-Type : application/json; charset=utf-8
Date : Thu, 18 Dec 2014 15:26:42 GMT
------
Response body:
{
"signers": [
{
"name": "Peter Hughes Jr.",
"email": "pwhughes#verizon.net",
"recipientId": "1",
"recipientIdGuid": "2461be9a-69d3-420d-b5f3-2c327af2f5f6",
"requireIdLookup": "false",
"userId": "0d5035dc-b70f-4026-8283-5c8d8903385a",
"clientUserId": "1",
"routingOrder": "1",
"note": "",
"roleName": "Applicant",
"status": "sent",
"templateLocked": "false",
"templateRequired": "false"
}
],
"agents": [],
"editors": [],
"intermediaries": [],
"carbonCopies": [],
"certifiedDeliveries": [],
"inPersonSigners": [],
"recipientCount": "1",
"currentRoutingOrder": "1"
}
When we attempt to set up a view for this user, the following exception occurs:
Request URL: https://na2.docusign.net/restapi/v2/accounts/4476693/envelopes/f3a45945-f1ff-4731-a407-963a32378d77/views/recipient
------
{"IntegratorKey":"***","Password":"***","Username":"onboarding#impellam.com"}
------
EXCEPTION
System.Net.WebException: The remote server returned an error: (400) Bad Request.
at System.Net.HttpWebRequest.GetResponse()
at DocusignView.Request.Send[T](String body) in C:\Users\johnathank\Desktop\docusigntest\DocusignView.cs:line 293
------
Request headers:
X-DocuSign-Authentication : {"IntegratorKey":"***","Password":"***","Username":"onboarding#impellam.com"}
Content-Type : application/json
Host : na2.docusign.net
Content-Length : 143
Expect : 100-continue
------
Request body:
{"authenticationMethod":"email","clientUserId":"1","email":"pwhughes#verizon.net","returnUrl":"www.impellam.com","userName":"Peter Hughes Jr."}
------
Response headers:
Strict-Transport-Security : max-age=7776000; includeSubDomains
Content-Length : 152
Cache-Control : no-cache
Content-Type : application/json; charset=utf-8
Date : Thu, 18 Dec 2014 15:28:31 GMT
------
Response body:
{
"errorCode": "UNKNOWN_ENVELOPE_RECIPIENT",
"message": "The recipient you have identified is not a valid recipient of the specified envelope."
}
We've actually identified the problem, but haven't found a good way to implement a solution yet. When inspecting the envelope within Docusign, I can see that this user has adopted the name "Peter W. Hughes Jr".
So I attempted sending another view request with the name "Peter W. Hughes Jr." instead, and it worked...
However this poses a problem. How are we able to find out what the adopted name of the user is? It is not in the original envelope data, and from browsing the API at iodocs.docusign.com, none of the methods there seem to give me the user's adopted name.
I have read an alternate solution that says we can provide a userId in the view request instead of the userName, but when I attempt to do that, I receive an error indicating that userName is required.
Request URL: https://na2.docusign.net/restapi/v2/accounts/4476693/envelopes/f3a45945-f1ff-4731-a407-963a32378d77/views/recipient
------
{"IntegratorKey":"***","Password":"***","Username":"onboarding#impellam.com"}
------
EXCEPTION
System.Net.WebException: The remote server returned an error: (400) Bad Request.
at System.Net.HttpWebRequest.GetResponse()
at DocusignView.Request.Send[T](String body) in C:\Users\johnathank\Desktop\docusigntest\DocusignView.cs:line 295
------
Request headers:
X-DocuSign-Authentication : {"IntegratorKey":"***","Password":"***","Username":"onboarding#impellam.com"}
Content-Type : application/json
Host : na2.docusign.net
Content-Length : 191
Expect : 100-continue
------
Request body:
{"authenticationMethod":"email","clientUserId":"1","email":"pwhughes#verizon.net","returnUrl":"www.impellam.com","userId":"2461be9a-69d3-420d-b5f3-2c327af2f5f6","userName":"Peter Hughes Jr."}
------
Response headers:
Strict-Transport-Security : max-age=7776000; includeSubDomains
Content-Length : 167
Cache-Control : no-cache
Content-Type : application/json; charset=utf-8
Date : Thu, 18 Dec 2014 15:52:06 GMT
------
Response body:
{
"errorCode": "INVALID_REQUEST_PARAMETER",
"message": "The request contained at least one invalid parameter. A value was not found for parameter 'userName'."
}
This is only an issue when multiple envelopes are sent in flight to the same recipient name, email and clientUserId combination.
Once you send the recipienttoken and they sign with a different name, using the same request for the second envelope will fail because of the name change.
Best practice would be to generate a new clientUserId for each envelope or prevent the recipient from changing their name within the DocuSign Account Features. The feature is called "Signature Adoption Configuration".

Resources