submitting a form through node as if connected to a site - node.js

I'm trying to create an automated tool that will submit a form so that it can detect changes in the response over time.
Normally, to submit this form I log in to a site and submit the form.
When trying to send the exact same request through Postman or a small node app I made, I get status 302 (while resending the request through firefox debugger works fine).
I even tried to use the HAR exported from the browser.
this is the code I use:
request({
har:{
"bodySize": 320,
"method": "POST",
"url": "https://***.com/url-to-send-the-form",
"httpVersion": "HTTP/2.0",
"headers": [
{
"name": "Host",
"value": "www.***.com"
},
{
"name": "User-Agent",
"value": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0"
},
{
"name": "Accept",
"value": "*/*"
},
{
"name": "Accept-Language",
"value": "en-US,en;q=0.5"
},
{
"name": "Accept-Encoding",
"value": "gzip, deflate, br"
},
{
"name": "Referer",
"value": "https://***.com/page-where-the-form-is/"
},
{
"name": "Content-Type",
"value": "application/x-www-form-urlencoded; charset=UTF-8"
},
{
"name": "X-Requested-With",
"value": "XMLHttpRequest"
},
{
"name": "Content-Length",
"value": "320"
},
{
"name": "Cookie",
"value": "visid_incap_1156372=***; _pk_id.7.32fa=***; _pk_ref.7.32fa=***; ASP.NET_SessionId=***; __RequestVerificationToken=***; BIGipServerMFT-Frontends=***; incap_ses_1051_1156372=***; _pk_ses.7.32fa=***; windID_1b5aa482-1126-4c17-a65f-dbac1e05cfb9=***; .AspNet.ApplicationCookie=***; family_Composition=***"
},
{
"name": "Connection",
"value": "keep-alive"
}
],
"cookies": [
{
"name": "visid_incap_1156372",
"value": "***"
},
{
"name": "_pk_id.7.32fa",
"value": "***"
},
{
"name": "_pk_ref.7.32fa",
"value": "***"
},
{
"name": "ASP.NET_SessionId",
"value": "***"
},
{
"name": "__RequestVerificationToken",
"value": "***"
},
{
"name": "BIGipServerMFT-Frontends",
"value": "***"
},
{
"name": "incap_ses_1051_1156372",
"value": "***"
},
{
"name": "_pk_ses.7.32fa",
"value": "***"
},
{
"name": "windID_1b5aa482-1126-4c17-a65f-dbac1e05cfb9",
"value": "***"
},
{
"name": ".AspNet.ApplicationCookie",
"value": "***"
},
{
"name": "family_Composition",
"value": "***"
}
],
"queryString": [],
"headersSize": 1717,
"postData": {
"mimeType": "application/x-www-form-urlencoded",
"params": [
{
"name": "key",
"value": "{val}"
}
],
"text": "key=val-url-encoded"
}
}
}, function (error, response, body) {
console.log(response.statusCode);//returns 302
});
Help will be very much appreciated

Related

Azure Logic App not able create client certificate authentication with converted base64 encoded pfx

I want to get the token information for ADP Client through Azure Logic App. I have the Client Certificate from ADP so I decided to use HTTP trigger from Logic App and selected authentication type "Client Certificate".
Since I cant directly use certificate in Logic app so I converted certificate into base64Encoded .pfx format, and certificate is not having any password.
below is the sample code for the request
{
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {},
"contentVersion": "1.0.0.0",
"outputs": {},
"triggers": {
"HTTP": {
"inputs": {
"authentication": {
"pfx": "convertedbase64string",
"type": "ClientCertificate"
},
"body": "grant_type=client_credentials&client_id=ClientId&client_secret=client_secret",
"headers": {
"content-type": "application/x-www-form-urlencoded"
},
"method": "POST",
"uri": "https://accounts.adp.com/auth/oauth/v2/token"
},
"recurrence": {
"frequency": "Month",
"interval": 15
},
"type": "Http"
}
}
},
"kind": "Stateful"
}
above request returns me bad request, can anyone help me what is going wrong here?
For converting into base64 I used below steps in power shell
$pfx_cert = get-content 'C:\sample\adpcertificate.pfx' -Encoding Byte
$output =[Convert]::ToBase64String($pfx_cert)
$output
I tried same request with client certificate using postman which is working fine, but not able to get succeed with Logic App.
Any help is much appreciated.
There are only few differences between the headers sent from Postman and the Logic App. The main difference is that Postman also sends the accept-header: "Accept": "*/*" and leaves out alle the x-ms-* headers from the logic app.
I created a Logic App with http-trigger, which I post to from Postman and Logic App to inspect the changes:
With Postman
{
"headers": {
"Connection": "keep-alive",
"Accept": "*/*",
"Accept-Encoding": "br,gzip,deflate",
"Host": "....westeurope.logic.azure.com:443",
"User-Agent": "PostmanRuntime/7.28.4",
"Postman-Token": "...-baea-4e89-9bf6-490a63968b5d",
"Content-Length": "76",
"Content-Type": "application/x-www-form-urlencoded"
},
"body": {
"$content-type": "application/x-www-form-urlencoded",
"$content": "Z3JhbnRfdHlwZT1jbGllbnRfY3JlZGVudGlhbHMmY2xpZW50X2lkPUNsaWVudElkJmNsaWVudF9zZWNyZXQ9Y2xpZW50X3NlY3JldA==",
"$formdata": [
{
"key": "grant_type",
"value": "client_credentials"
},
{
"key": "client_id",
"value": "ClientId"
},
{
"key": "client_secret",
"value": "client_secret"
}
]
}
}
With Logic App
{
"headers": {
"Connection": "Keep-Alive",
"Accept-Encoding": "gzip,deflate",
"Accept-Language": "en",
"Host": "...westeurope.logic.azure.com",
"User-Agent": "azure-logic-apps/1.0,(workflow ...; version ...)",
"x-ms-trigger-callback-url": "https://....westeurope.logic.azure.com/ <...>",
"x-ms-trigger-type": "Http",
"x-ms-workflow-id": "...",
"x-ms-workflow-version": "...",
"x-ms-workflow-name": "myworkflowname",
"x-ms-workflow-system-id": "/locations/westeurope/scaleunits/...",
"x-ms-workflow-run-id": "...",
"x-ms-workflow-operation-name": "HTTP",
"x-ms-execution-location": "westeurope",
"x-ms-workflow-subscription-id": "...",
"x-ms-workflow-resourcegroup-name": "..",
"x-ms-tracking-id": "...",
"x-ms-correlation-id": "...",
"x-ms-client-request-id": "...",
"x-ms-activity-vector": "...",
"Content-Length": "76",
"Content-Type": "application/x-www-form-urlencoded"
},
"body": {
"$content-type": "application/x-www-form-urlencoded",
"$content": "Z3JhbnRfdHlwZT1jbGllbnRfY3JlZGVudGlhbHMmY2xpZW50X2lkPUNsaWVudElkJmNsaWVudF9zZWNyZXQ9Y2xpZW50X3NlY3JldA==",
"$formdata": [
{
"key": "grant_type",
"value": "client_credentials"
},
{
"key": "client_id",
"value": "ClientId"
},
{
"key": "client_secret",
"value": "client_secret"
}
]
}
}
Solution
My solution would be to manually add the Accept-Header in the post request in the Logic App.
"headers": {
"Accept": "*/*",
// ...
},
I sadly don't have an ADP account to verify this, but I've seen other APIs break when no accept header is sent.

Azure AD B2C REST API : Create Bulk Local Accounts

Is it possible to create users in bulk via the REST API. Same as we do for single user in the below URL.
https://graph.windows.net/{MYADB2C}.onmicrosoft.com/users?api-version=1.6
We have the provision via the Azure portal but could'nt find anything with REST API.
UPDATED
Sample request for Batch processing
POST https://graph.windows.net/{}.onmicrosoft.com/$batch?api-version=1.6
Headers :
Authorization : {token}
Content-Type : multipart/mixed; boundary=changeset_***********
Body :
{
"requests": [
{
"id": "1",
"method": "POST",
"url": "/users",
"body": {
"accountEnabled": true,
"creationType": "LocalAccount",
"displayName": "test1#gamil.com",
"passwordPolicies": "DisablePasswordExpiration, DisableStrongPassword",
"passwordProfile": {
"password": "***",
"forceChangePasswordNextLogin": false
},
"signInNames": [
{
"type": "emailAddress",
"value": "test1#gamil.com"
}
]
},
"headers": {
"Content-Type": "application/json"
}
},
{
"id": "2",
"method": "POST",
"url": "/users",
"body": {
"accountEnabled": true,
"creationType": "LocalAccount",
"displayName": "test2#gmail.com",
"passwordPolicies": "DisablePasswordExpiration, DisableStrongPassword",
"passwordProfile": {
"password": "***",
"forceChangePasswordNextLogin": false
},
"signInNames": [
{
"type": "emailAddress",
"value": "test1#gamil.com"
}
]
},
"headers": {
"Content-Type": "application/json"
}
}
]
}
Yes. You can batch operations by referring to Batch processing | Graph API concepts.
But we recommend you use Microsoft Graph API JSON Batching instead of Azure AD Graph Batch processing because Azure AD Graph content is no longer updated.
An example using Microsoft Graph API here:
POST https://graph.microsoft.com/v1.0/$batch
Accept: application/json
Content-Type: application/json
{
"requests": [
{
"id": "1",
"method": "POST",
"url": "/users",
"body": {
"accountEnabled": true,
"displayName": "at1",
"mailNickname": "at1",
"userPrincipalName": "at1#**.onmicrosoft.com",
"passwordProfile" : {
"forceChangePasswordNextSignIn": true,
"password": "password-value"
}
},
"headers": {
"Content-Type": "application/json"
}
},
{
"id": "2",
"method": "POST",
"url": "/users",
"body": {
"accountEnabled": true,
"displayName": "at2",
"mailNickname": "at2",
"userPrincipalName": "at2#**.onmicrosoft.com",
"passwordProfile" : {
"forceChangePasswordNextSignIn": true,
"password": "password-value"
}
},
"headers": {
"Content-Type": "application/json"
}
},
{
"id": "3",
"method": "POST",
"url": "/users",
"body": {
"accountEnabled": true,
"displayName": "at3",
"mailNickname": "at3",
"userPrincipalName": "at3#**.onmicrosoft.com",
"passwordProfile" : {
"forceChangePasswordNextSignIn": true,
"password": "password-value"
}
},
"headers": {
"Content-Type": "application/json"
}
}
]
}
You can have a quick test in Microsoft Graph Explorer.

Azure Logic Apps, no content from queue

I have the following trigger:
"triggers": {
"When_there_are_messages_in_a_queue": {
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['azurequeues']['connectionId']"
}
},
"method": "get",
"path": "/#{encodeURIComponent('drugconsortium-positive-result')}/message_trigger"
},
"recurrence": {
"frequency": "Minute",
"interval": 1
},
"splitOn": "#triggerBody()?['QueueMessagesList']?['QueueMessage']",
"type": "ApiConnection"
}
}
then I try to get messages from queue:
"Get_messages": {
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['azurequeues']['connectionId']"
}
},
"method": "get",
"path": "/#{encodeURIComponent('drugconsortium-positive-result')}/messages",
"queries": {
"visibilitytimeout": "30"
}
},
"runAfter": {},
"type": "ApiConnection"
}
and then try to send email with queue content:
"For_each_2": {
"actions": {
"Send_email_(V4)_3": {
"inputs": {
"body": {
"from": "email#zzz.com",
"ishtml": true,
"subject": "message",
"text": "<p>want!!!#{items('For_each_2')?['MessageText']}</p>",
"to": "receiver#gmail.com"
},
"host": {
"connection": {
"name": "#parameters('$connections')['sendgrid']['connectionId']"
}
},
"method": "post",
"path": "/v4/mail/send"
},
"runAfter": {},
"type": "ApiConnection"
}
},
"foreach": "#body('Get_messages')?['QueueMessagesList']?['QueueMessage']",
"runAfter": {
"Get_messages": [
"Succeeded"
]
},
"type": "Foreach"
}
After execution I see a message for the trigger:
{
"MessageId": "83aac220-9ee5-440c-9a34-391abaa0e464",
"InsertionTime": "Thu, 12 Mar 2020 20:21:51 GMT",
"ExpirationTime": "Thu, 19 Mar 2020 20:21:51 GMT",
"PopReceipt": "AgAAAAMAAAAAAAAA+JoXDKz41QE=",
"TimeNextVisible": "Thu, 12 Mar 2020 20:23:09 GMT",
"DequeueCount": "1",
"MessageText": "{\"DriverId\":-1,\"DriverName\":\"John Smith\",\"DriverSSN\":\"111-11-1111\",\"CarrierName\":\"Carrier Name\",\"DER\":{\"Name\":\"Der name\",\"Email\":\"werew#zzzz.com\"}}"
}
but Get_messages has in result:
QueueMessage : []
and I don't receive any email. What is wrong?
This is because you don't need Get_messages, check the trigger definition When there are messages in a queue, it will return the Messages type, means you already get the message.
Cause the queue is already processed by the trigger the it will be removed from the queue so you could not get it again with the action.
So after the trigger you could get the message content with the dynamic content.

How to parse content of BLOB-file in Azure Logic App

I'm working on an Azure Logic App which is triggered everytime a JSON-file is added to a BLOB-storage. The JSON-file contains a CustomerId and based on this Id I want to sent the contents of the JSON-file to a different endpoint using an HTTP-request.
My Azure Logic App currently looks like this;
I've been researching and trying a lot of things for the entire morning, but I can't get my head around this. I've tried things like;
json(body('Get_blob_content_using_path'))
and
decodeBase64(body('Get_blob_content_using_path'))
and just the default option like visible in the screenshot. But I can't figure out how to do this. All I want is go left or right based on the CustomerId.
So for clarity. The problem lies within the condition-step of the Logic App. I can retrieve the BLOB-file from the storage, but the issue is with parsing the CustomerId from the JSON so I can validate it within the condition. Does anybody has an idea on how I can fix this?
In the end I was able to solve the issue by adding a compose step before the condition and after the steps which will get me the content of the BLOB-file. The compose get's the content of the blob-file which I then can validate against the CustomerId I want. This topic got me in the right direction for my solution.
UPDATE:
The final logic app looks like this;
Which is begin created with the following logic app code;
{
"$connections": {
"value": {
"azureblob": {
"connectionId": "<snip>",
"connectionName": "azureblob",
"id": "<snip>"
},
"slack": {
"connectionId": "<snip>",
"connectionName": "slack",
"id": "<snip>"
}
}
},
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {
"Compose": {
"inputs": "#base64ToString(body('Get_blob_content').$content)",
"runAfter": {
"Get_blob_content": [
"Succeeded"
]
},
"type": "Compose"
},
"Condition_2": {
"actions": {
"Condition_3": {
"actions": {
"Delete_blob_3": {
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['azureblob']['connectionId']"
}
},
"method": "delete",
"path": "/datasets/default/files/#{encodeURIComponent(encodeURIComponent(triggerBody()?['Id']))}"
},
"runAfter": {},
"type": "ApiConnection"
}
},
"else": {
"actions": {
"Copy_blob_2": {
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['azureblob']['connectionId']"
}
},
"method": "post",
"path": "/datasets/default/copyFile",
"queries": {
"destination": "/<some-blob-container>/#{triggerBody()?['Name']}",
"overwrite": false,
"queryParametersSingleEncoded": true,
"source": "#triggerBody()?['Path']"
}
},
"runAfter": {
"Post_message_2": [
"Succeeded"
]
},
"type": "ApiConnection"
},
"Delete_blob_4": {
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['azureblob']['connectionId']"
}
},
"method": "delete",
"path": "/datasets/default/files/#{encodeURIComponent(encodeURIComponent(triggerBody()?['Id']))}"
},
"runAfter": {
"Copy_blob_2": [
"Succeeded"
]
},
"type": "ApiConnection"
},
"Post_message_2": {
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['slack']['connectionId']"
}
},
"method": "post",
"path": "/chat.postMessage",
"queries": {
"channel": "<snip>",
"text": "<some-message>"
}
},
"runAfter": {},
"type": "ApiConnection"
}
}
},
"expression": {
"or": [
{
"equals": [
"#outputs('HTTP_2')['statusCode']",
200
]
},
{
"equals": [
"#outputs('HTTP_2')['statusCode']",
202
]
}
]
},
"runAfter": {
"HTTP_2": [
"Succeeded",
"Failed"
]
},
"type": "If"
},
"HTTP_2": {
"inputs": {
"authentication": {
"password": "<some-password>",
"type": "Basic",
"username": "<some-username>"
},
"body": "#outputs('Compose')",
"headers": {
"Content-Type": "application/json"
},
"method": "POST",
"uri": "<some-url>"
},
"runAfter": {},
"type": "Http"
}
},
"else": {
"actions": {
"Condition_4": {
"actions": {
"Delete_blob_5": {
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['azureblob']['connectionId']"
}
},
"method": "delete",
"path": "/datasets/default/files/#{encodeURIComponent(encodeURIComponent(triggerBody()?['Id']))}"
},
"runAfter": {},
"type": "ApiConnection"
}
},
"else": {
"actions": {
"Copy_blob_3": {
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['azureblob']['connectionId']"
}
},
"method": "post",
"path": "/datasets/default/copyFile",
"queries": {
"destination": "/<some-blob-container>/#{triggerBody()?['Name']}",
"overwrite": false,
"queryParametersSingleEncoded": true,
"source": "#triggerBody()?['Path']"
}
},
"runAfter": {
"Post_message_3": [
"Succeeded"
]
},
"type": "ApiConnection"
},
"Delete_blob_6": {
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['azureblob']['connectionId']"
}
},
"method": "delete",
"path": "/datasets/default/files/#{encodeURIComponent(encodeURIComponent(triggerBody()?['Id']))}"
},
"runAfter": {
"Copy_blob_3": [
"Succeeded"
]
},
"type": "ApiConnection"
},
"Post_message_3": {
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['slack']['connectionId']"
}
},
"method": "post",
"path": "/chat.postMessage",
"queries": {
"channel": "<snip>",
"text": "<some-message>"
}
},
"runAfter": {},
"type": "ApiConnection"
}
}
},
"expression": {
"or": [
{
"equals": [
"#outputs('HTTP_3')['statusCode']",
200
]
},
{
"equals": [
"#outputs('HTTP_3')['statusCode']",
202
]
}
]
},
"runAfter": {
"HTTP_3": [
"Succeeded"
]
},
"type": "If"
},
"HTTP_3": {
"inputs": {
"authentication": {
"password": "<some-password>",
"type": "Basic",
"username": "<some-username>"
},
"body": "#outputs('Compose')",
"headers": {
"Content-Type": "application/json"
},
"method": "POST",
"uri": "<some-url>"
},
"runAfter": {},
"type": "Http"
}
}
},
"expression": {
"and": [
{
"contains": [
"#outputs('Compose')",
"\"CustomerId\":\"00000000-0000-0000-0000-000000000000\""
]
}
]
},
"runAfter": {
"Compose": [
"Succeeded"
]
},
"type": "If"
},
"Get_blob_content": {
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['azureblob']['connectionId']"
}
},
"method": "get",
"path": "/datasets/default/files/#{encodeURIComponent(encodeURIComponent(triggerBody()?['Path']))}/content",
"queries": {
"inferContentType": true
}
},
"runAfter": {},
"type": "ApiConnection"
}
},
"contentVersion": "1.0.0.0",
"outputs": {},
"parameters": {
"$connections": {
"defaultValue": {},
"type": "Object"
}
},
"triggers": {
"When_a_blob_is_added_or_modified_(properties_only)": {
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['azureblob']['connectionId']"
}
},
"method": "get",
"path": "/datasets/default/triggers/batch/onupdatedfile",
"queries": {
"folderId": "<some-generated-folderid>",
"maxFileCount": 100
}
},
"metadata": {
"<some-generated-folderid>": "/<some-blob-container>"
},
"recurrence": {
"frequency": "Minute",
"interval": 1
},
"splitOn": "#triggerBody()",
"type": "ApiConnection"
}
}
}
}

How to fix "it doesn't support that" when I query Alexa retrievable property

I have this endpoint with powerState and connectivity properties both retrievable:
{
"endpointId": "123",
"manufacturerName": "abc",
"friendlyName": "lamp",
"description": "lamp",
"displayCategories": [
"LIGHT"
],
"capabilities": [
{
"type": "AlexaInterface",
"interface": "Alexa.PowerController",
"version": "3",
"properties": {
"supported": [{
"name": "powerState"
}],
"retrievable": true
}
},
{
"type": "AlexaInterface",
"interface": "Alexa.EndpointHealth",
"version": "3",
"properties": {
"supported": [{
"name": "connectivity"
}],
"retrievable": true
}
}
],
"cookie": {}
}
I respond to ReportState request by sending the state of both properties:
{
"context": {
"properties": [
{
"namespace": "Alexa.PowerController",
"name": "powerState",
"value": "ON",
"timeOfSample": "2019-01-10T10:17:19.99Z",
"uncertaintyInMilliseconds": 50
},
{
"namespace": "Alexa.EndpointHealth",
"name": "connectivity",
"value": {
"value": "OK"
},
"timeOfSample": "2019-01-10T10:17:19.90Z",
"uncertaintyInMilliseconds": 50
}
]
},
"event": {
"header": {
"namespace": "Alexa",
"name": "StateReport",
"payloadVersion": "3",
"messageId": "cde",
"correlationToken": "efg123"
},
"endpoint": {
"scope": {
"type": "BearerToken",
"token": "ab123"
},
"endpointId": "123",
"cookie": {}
},
"payload": {}
}
}
And I send the same context in the PowerController response.
In the Alexa app I can see my device, turn it on and off and view its state rightly. I can ask Alexa to turn on and off my device and the response in "OK", but if I try to ask:** "Is my device turned on?" ** Alexa answers that this device doesn't support that.
The skill language is Italian and PowerController documentation https://developer.amazon.com/it/docs/device-apis/alexa-powercontroller.html says that query is supported.
What am I missing?

Resources