I discovered that axios is returning a string instead of a valid json.
headers:Object {date: "Tue, 02 Jun 2020 08:44:06 GMT", server: "Apache", connection: "close", …}
connection:"close"
content-type:"text/html; charset=UTF-8"
date:"Tue, 02 Jun 2020 08:44:06 GMT"
server:"Apache"
transfer-encoding:"chunked"
how do I change the content-type to application/json in NestJs application?
I tried this but didnt not work
const meterInfo = await this.httpService.get(url, { headers: { "Content-Type": "application/json" } }).toPromise();
Here is the invalid json returned.
"{"status":"00","message":"OK","access_token":"2347682423567","customer":{"name":"John Doe","address":"Mr. John Doe 34 Tokai, leaflet. 7999.","util":"Demo Utility","minimumAmount":"13897"},"response_hash":"c43c9d74480f340f55156f6r5c56487v8w"}"
Instead of sending a Content-Type header, you should send an Accept header with the same MIME type. This tells the server what you are expecting to receive, and if Content Negotiation is set up properly, it will allow you to get a JSON back instead of that weird string.
this.httpService.get(
url,
{
headers: {
'Accept': 'application/json',
},
},
).toPromise();
If that doesn't work, you'll need to provide your own serializer to take the string from that wonky format to JSON, or get in touch with the server admins and see if they can provide you better documentation about how to consume their API.
Related
[FINAL UPDATE]
Fixed per this thread: https://github.com/docusign/docusign-esign-node-client/issues/295
Apparently related to security additions made to DocuSign's API.
I haven't encountered this issue before today. I gave a demo to a client just last week and the same code was working fine. Today I get the following:
{
"status": 444,
"response": {
"req": {
"method": "GET",
"url": "https://demo.docusign.net/restapi...",
"data": {},
"headers": {
"user-agent": "node-superagent/3.8.2",
"x-docusign-sdk": "Node",
"authorization": "Bearer ABC...",
"content-type": "application/json",
"accept": "application/json"
}
},
"header": {
"content-type": "text/html",
"x-docusign-tracetoken": "2eeb8caa-8865-4898-bef9-d3611bfaa3f7",
"x-docusign-node": "DA2DFE5",
"date": "Fri, 17 Jun 2022 01:02:02 GMT",
"content-length": "54",
"connection": "close",
"strict-transport-security": "max-age=31536000; includeSubDomains"
},
"status": 444,
"text": "The custom error module does not recognize this error."
}
}
Through the node SDK I fetch a token using requestJWTUserToken, and set the apiclient's auth header, then I new-up an EnvelopesAPI instance. When I make a call to getEnvelope() or listStatusChanges(), then I get the error above. None of this code has changed in months, and I'm using the same integration key, account, private key, everything. I've demo'd it all a few times now - no issues.
An interesting observation: the error above gives me a URL and token. The token is valid, and if I make a request to the URL (the envelopes endpoint) via Postman using said token, then the request succeeds. So my calls through the SDK seem to be failing for some reason.
I can't seem to crack this one, and now I can't get around it given a couple demo systems that worked just last week.
I'm using the docusign-esign 5.17 module - upgraded from 5.15 in an attempt to fix the issue. No luck.
Where is this error coming from?
[Update 1]
I'm running my node app that is making requests through the DocuSign Node SDK against a proxy so I can see what the failing request actually look like:
They fail the same way.
HTTP/1.1 444
Content-Type: text/html
X-DocuSign-TraceToken: 338534c6-c8c3-4b01-9b66-35d697cd0053
X-DocuSign-Node: DA1DFE4
Date: Fri, 17 Jun 2022 03:55:07 GMT
Content-Length: 54
Vary: Accept-Encoding
Connection: close
Strict-Transport-Security: max-age=31536000; includeSubDomains
The custom error module does not recognize this error.
I'm using Proxyman to catch the request, and like Chrome or Firefox it will let you copy a request as a cURL command. If I copy the failing request as cURL, then run in at the terminal, it succeeds.
[MacBookPro0020]~/source/docusign/jwt-smoke-test:0 (master)
$ curl 'https://demo.docusign.net/restapi/v2.1/accounts/a0a4c81f-.../envelopes?envelope_ids=e750526f-...&envelope_ids=a38b794b...&envelope_ids=a5d8c586-...' \
-H 'Host: demo.docusign.net' \
-H 'User-Agent: node-superagent/3.8.2' \
-H 'X-DocuSign-SDK: Node' \
-H 'Node-Ver: v14.18.3' \
-H 'Authorization: Bearer ABCD...' \
-H 'Accept: application/json' \
-H 'Connection: close' \
-H 'Content-Type: application/json' \
--proxy http://localhost:9090
{"resultSetSize":"1","startPosition":"0","endPosition":"0","totalSetSize":"1","nextUri":"","previousUri":"","envelopes":[{"status":"created","documentsUri":"/envelopes/d97565c8...purgeState":"unpurged","envelopeIdStamping":"true","autoNavigation":"true","isSignatureProviderEnvelope":"false","allowComments":"true","anySigner":null,"envelopeLocation":"current_site"}]}
I'm using a JWT auth token, so again, I'm getting a valid token. Calls through the SDK consistently fail, but cURL'ing and manually requesting through Postman both succeed.
I'm at a loss.
Additional details: I see this same issue on MacOS and Windows (i.e., node app hosting docusign-esign). I'm using auth code grant to send envelopes and query envelope statuses and that works fine. I've used JWT Grant without issue up until this week (just demo'd automated functionality last week and it worked.) I haven't made any code changes to my DocuSign functionality, nor have my colleagues, at least according to the repo's history.
I don't recall ever encountering the error above before. I'd love to know why cURL'ing the same request succeeds. I'd rather not ditch the SDK and roll my own requests, but it wouldn't be difficult.
[Update 2]
Here's a simple repro - it's a quick and dirty copy of the QuickStart demo project for the node SDK. I'm using only docusign-esign.
Exact. same. issue.
Again, I can take that token and drop it into cURL or postman and the request will succeed. There's not a lot of code here. The token is valid.
async function main() {
// Data used
// dsConfig.dsClientId
// dsConfig.impersonatedUserGuid
// dsConfig.privateKey
// dsConfig.dsOauthServer
let dsConfig = dsConfig_customer; // defined globally
const jwtLifeSec = 10 * 60, // requested lifetime for the JWT is 10 min
dsApi = new docusign.ApiClient();
dsApi.setOAuthBasePath(dsConfig.dsOauthServer.replace('https://', '')); // it should be domain only.
const results = await dsApi.requestJWTUserToken(dsConfig.dsClientId,
dsConfig.impersonatedUserGuid, 'signature impersonation', dsConfig.privateKey,
jwtLifeSec);
console.log( results.body.access_token );
const userInfo = await dsApi.getUserInfo(results.body.access_token);
dsApi.setBasePath(userInfo.accounts[0].baseUri + '/restapi');
dsApi.addDefaultHeader( 'Authorization', 'Bearer ' + results.body.access_token );
const envelopesAPI = new docusign.EnvelopesApi(dsApi);
const res = await envelopesAPI.getEnvelope( dsConfig.accountID, 'e1917111-2900-48e8-9054-799169379c8a', null );
console.log(res);
return {
accessToken: results.body.access_token,
tokenExpirationTimestamp: expiresAt,
userInfo,
account: userInfo.accounts[0]
};
}
main().then(result => console.log(result)).catch(err=>console.error(err));
...
header: {
'content-type': 'text/html',
'x-docusign-tracetoken': '685b6226-a0d3-4547-94c7-df0216d884a3',
'x-docusign-node': 'DA2DFE188',
date: 'Fri, 17 Jun 2022 05:20:12 GMT',
'content-length': '54',
vary: 'Accept-Encoding',
connection: 'close',
'strict-transport-security': 'max-age=31536000; includeSubDomains'
},
statusCode: 444,
status: 444,
statusType: 4,
info: false,
ok: false,
redirect: false,
clientError: true,
serverError: false,
error: Error: cannot GET /restapi/v2.1/accounts/49754554-ABCD-.../envelopes/e1917111-2900-48e8-9054-799169379c8a (444)
...
I finally manage to resolve the issue by downgrading my Docusign SDK NodeJS version from 5.15.0 to 5.7.0 in my package.json file.
An issue relative to this can be find there, I really hope this issue will be resolved anytime soon.
Edit :
The Docusign support actually took action on this (the github's issue from above is now closed), it might be interesting to test again with the latest version of Docusign SDK NodeJS (or the one you were using previously).
I'm creating a simple Lambda function to serve a POST in AWS Gateway:
import json
import boto3
from datetime import datetime
def lambda_handler(event,context):
wf = event['WF']
if wf == 'start1':
body = 'Suceeded'
return {
"isBase64Encoded": False,
'statusCode': 200,
'body': json.dumps(body),
'headers': {
'Content-Type': 'application/json'
}
}
else:
body = 'Failed'
return {
"isBase64Encoded": False,
'statusCode': 400,
'body': json.dumps(body),
'headers': {
'Content-Type': 'application/json'
}
}
If I send a POST with this body is successful:
{
"WF": "start1",
"Context": "OK"
}
Returns:
Test Event Name
test
Response
{
"isBase64Encoded": false,
"statusCode": 200,
"body": "\"Suceeded\"",
"headers": {
"Content-Type": "application/json"
}
}
Function Logs
START RequestId: 91663a9d-d8b6-4aad-b8b6-eb2f15ff0354 Version: $LATEST
END RequestId: 91663a9d-d8b6-4aad-b8b6-eb2f15ff0354
REPORT RequestId: 91663a9d-d8b6-4aad-b8b6-eb2f15ff0354 Duration: 1.74 ms Billed Duration: 2 ms Memory Size: 128 MB Max Memory Used: 66 MB Init Duration: 300.73 ms
Request ID
91663a9d-d8b6-4aad-b8b6-eb2f15ff0354
But if I run a test in the API Gateway service in AWS I get:
Fri Jun 04 13:45:22 UTC 2021 : Sending request to https://XXXXXXX
Fri Jun 04 13:45:22 UTC 2021 : Received response. Status: 200, Integration latency: 15 ms
Fri Jun 04 13:45:22 UTC 2021 : Endpoint response headers: {Date=Fri, 04 Jun 2021 13:45:22 GMT, Content-Type=application/json, Content-Length=159, Connection=keep-alive, x-amzn-RequestId=d479fdac-6737-42fa-96a3-9e991056b48d, X-Amz-Function-Error=Unhandled, x-amzn-Remapped-Content-Length=0, X-Amz-Executed-Version=$LATEST, X-Amzn-Trace-Id=root=1-60ba2e72-ff62f051febbd21d6cc9d114;sampled=0}
Fri Jun 04 13:45:22 UTC 2021 : Endpoint response body before transformations: {"errorMessage": "'WF'", "errorType": "KeyError", "stackTrace": [" File \"/var/task/lambda_function.py\", line 7, in lambda_handler\n wf = event['WF']\n"]}
Fri Jun 04 13:45:22 UTC 2021 : Lambda execution failed with status 200 due to customer function error: 'WF'. Lambda request id: d479fdac-6737-42fa-96a3-9e991056b48d
Fri Jun 04 13:45:22 UTC 2021 : Method completed with status: 502
Why line 7 is wrong?
Is there another way to parse the body of the POST?
The solution was found by disabling Lambda Proxy Integration in the POST - Integration Request.
It looks like your API Gateway invocation is not populating the event parameter of your Lambda function. When you create a test event your lambda function's page, you get to create your own test event. API Gateway will not use this test event, it will instead send an empty event (probably null).
I.e.
import json
import boto3
from datetime import datetime
def lambda_handler(event,context): # Event is null here
wf = event['WF'] # Attempting to read the attribute 'WF' of null results in an error
In my instance, I had received this error when invoking my Lambda function via API gateway, it was also completely fine when invoking directly the lambda function expecting a simple JSON body.
my expected payload:
{"Email": "foo3#foo.com"}
It turns out Lambda Proxy Integration in the POST in the configuration API gateway changes the payload structure before passing into Lambda, and I was getting the error missing Key attribute Email. I noticed that line Endpoint request body after transformations: in cloudwatch. I learned that it was changing the payload.
How I debug this:
Invoke Lambda using test events ensuring the both the error scenario and success scenarios are working.
Ensure there are no permission issues via API gateway by making sure that your API gateway can invoke the Lambda and have appropriate IAM roles/policies
Ensure your lambda has the API gateway as the trigger
I tried removing all the security token first via API gateway to verify I am able to invoke the API
I tried both HTTP methods POST And GET, seeing the same error (Email was missing) was the key to figuring this out as I was already passing the payload in the request body in the POST method.
I'm trying to fetch data from a ticketing system with logic app, using the built-in HTTP module.
When testing with postman, I get the following response:
GET: https://ticketsystem/api/ticket/{{number}}
{
"tickets": [
{
"links": {
"data1": {
"id": 4
},
"data2": {
"id": 3
},
"data3": {
"id": 969
}
...
},
"data1Id": 4,
"data2Id": 3,
"data3Id": 969,
"att1": 1,
"att1": 2,
"att1": 3,
"att1": 4
....
}
]}
But, when trying through the HTTP logic app module, this is the response:
{
"data1Id": 4,
"data2Id": 3,
"data3Id": 969,
"att1": 1,
"att1": 2,
"att1": 3,
"att1": 4
...
}
Everything else is the same, I have even tried in a new logic app and a totally different azure account. It is still the same.
I've looked through the http header response, and there are some differences.
Postman:
Cache-Control: no-cache
Pragma: no-cache
Content-Type: application/vnd.api+json; charset=utf-8
Content-Encoding: gzip
Expires: -1
Vary: Accept-Encoding
Server: Microsoft-IIS/10.0
X-PS-ActionTime: 00:00:00.0022451
X-Frame-Options: deny
X-XSS-Protection: 1; mode=block
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
Date: Wed, 16 Jun 2021 09:41:50 GMT
Content-Length: 819
Azure HTTP:
"Pragma": "no-cache",
"Vary": "Accept-Encoding",
"X-PS-ActionTime": "00:00:00.0022021",
"X-Frame-Options": "deny",
"X-XSS-Protection": "1; mode=block",
"Strict-Transport-Security": "max-age=31536000; includeSubDomains; preload",
"Cache-Control": "no-cache",
"Date": "Wed, 16 Jun 2021 09:43:27 GMT",
"Server": "Microsoft-IIS/10.0",
"Content-Type": "application/json; charset=utf-8",
"Expires": "-1",
"Content-Length": "1733"
It looks like the "Content-Encoding: gzip" is missing from logic app, but I do not know why this is affecting the overall response structure. Also how to fix this issue.
I have tried to enable "Allow chunking", without any luck.
I understand that I might create an Azure Function to go around this, but I'm trying to avoid that for now.
Any advice?
EDIT
I tested with powershell Invoke-WebRequest, and I see that this is behaving the same as the Logic app HTTP action.
From powershell, the header is also the same (missing Content-Encoding: gzip) and the "Content-Type" = "application/json; charset=utf-8"
But, when testing with python (3.9) with the request module, then it's spitting out the same data as postman.
Content-Type: application/vnd.api+json; charset=utf-8
Content-Encoding: gzip
I am really trying to understand the difference here on the header level, as this is the only difference between the responses, and also what application/vnd.api+json and Content-Encoding: gzip does here.
Did you check this :
https://learn.microsoft.com/en-us/azure/connectors/connectors-native-http#omitted-http-headers
I see a feedback for this too: https://feedback.azure.com/forums/287593-logic-apps/suggestions/42578674-enable-support-for-content-type-header-in-http-get
I've sovled it.
I simply put this as a header on the HTTP Action:
"Accept": "application/vnd.api+json; charset=utf-8"
And the response message was the same as in Postman.
This still does not answer why it is behaving differently, since none of the request headers had this value in all of the metodes I tried.
On the command line, I can do a request like: curl -I -H "Fastly-Debug: 1"
and it will return a lot of helpful information from the CDN serving that URL, in this case, Fastly:
cache-control: public, max-age=0, must-revalidate
last-modified: Tue, 20 Apr 2021 21:17:46 GMT
etag: "4c5cb3eb0ddb001584dad329b8727a9a"
content-type: text/html
server: AmazonS3
surrogate-key: /v3.6/tutorial/nav/alerts-and-monitoring/
accept-ranges: bytes
date: Fri, 30 Apr 2021 20:50:15 GMT
via: 1.1 varnish
age: 0
fastly-debug-path: (D cache-lga21923-LGA 1619815815) (F cache-lga21940-LGA 1619815815)
fastly-debug-ttl: (M cache-lga21923-LGA - - 0)
fastly-debug-digest: 04c3e527819b6a877de6577f7461e132b97665100a63ca8f667d87d049092233
x-served-by: cache-lga21923-LGA
x-cache: MISS
x-cache-hits: 0
x-timer: S1619815815.944515,VS0,VE136
vary: Accept-Encoding
content-length: 65489
How can I do this in node?
This is my attempt:
const headers = {
'Fastly-Key': environment.getFastlyToken(),
'Accept': 'application/json',
'Content-Type': 'application/json',
'Fastly-Debug': 1
};
async retrieveSurrogateKey(url) {
console.log("this is the url: ", url)
try {
request({
method: `GET`,
url: url,
headers: headers,
}, function(err, response, body) {
if (err){
console.trace(err)
}
console.log(request.headers)
})
} catch (error) {
console.log("error in retrieval: ", error)
}
}
Is there a way for me to pass in the -I and -H flags?
The -H (header) flag allows you to specify a custom header in cURL. Your code already does this - great! All that's left is emulating the -I (head) flag.
From cURL manpage for the -I option:
Fetch the headers only! HTTP-servers feature the command HEAD which this uses to get nothing but the header of a document.
To use the HEAD method, you need to specify it instead of GET:
method: `HEAD`
And finally, the headers returned by the server in the response can be obtained from response.headers.
Etag is supported in SDK: https://sap.github.io/cloud-sdk/docs/java/features/odata/use-typed-odata-v4-client-in-sap-cloud-sdk-for-java/#handling-of-etags
So experimenting it by using BusinessPartner entity in S/4 EX.
But seems there's no If-Match header:
How come the header doesn't show up - any prerequisite with etag?
(entering on behalf of the implementation partner team)
I checked the VersionIdentifier of the response and it was not set to a value.
I also checked the response's JSON __metadeta and header, but there were no values that appeared to correspond to the ETag value.
[Code]
BusinessPartner bp1 = new DefaultBusinessPartnerService().getBusinessPartnerByKey(bpId).execute(dest);
log.debug("get 1: {}", bp1);
log.debug("get 1 VersionIdentifier: {}", bp1.getVersionIdentifier());
bp1.setOrganizationBPName1("SCP Update 1st:" + System.currentTimeMillis());
ODataUpdateResult result1 = new DefaultBusinessPartnerService().updateBusinessPartner(bp1).execute(dest);
log.debug("Update1 Http Status: {}", result1.getHttpStatusCode());
bp1.setOrganizationBPName1("SCP Update 2nd:" + System.currentTimeMillis());
bp1.setVersionIdentifier("dummy");
ODataUpdateResult result2 = new DefaultBusinessPartnerService().updateBusinessPartner(bp1).execute(dest);
log.debug("Update2 Http Status: {}", result2.getHttpStatusCode());
[Log]
get 1: BusinessPartner(super=VdmObject(customFields={}, changedOriginal...
get 1 VersionIdentifier: None
Update1 Http Status: 204
Update2 Http Status: 204
[GET Response JSON(__metadata) / Response Header]
(It has masked the IP address.)
"__metadata": {
"id": "https://xxx.xxx.xxx.xxx:xxxxxx/sap/opu/odata/sap/API_BUSINESS_PARTNER/A_BusinessPartner('1000001')",
"uri": "https://xxx.xxx.xxx.xxx:xxxxxx/sap/opu/odata/sap/API_BUSINESS_PARTNER/A_BusinessPartner('1000001')",
"type": "API_BUSINESS_PARTNER.A_BusinessPartnerType"
},
HTTP/1.1 200 OK
content-type: application/json; charset=utf-8
content-length: 3152
dataserviceversion: 2.0
sap-metadata-last-modified: Thu, 14 May 2020 23:58:07 GMT
cache-control: no-store, no-cache
sap-processing-info: ODataBEP=,crp=,RAL=,st=,MedCacheHub=SHM,codeployed=X,softstate=
sap-server: true
sap-perf-fesrec: 243070.000000
I tried setting the VersionIdentifier to a meaningless value in my test code (2nd update).
The update process seems to be successful, although the request header now has "If-Match" added to it.
(I was expecting the update to fail because the values never match, so I was hoping the update would fail.)
[2nd Update(setVersionIdenfifier)]
(It has masked some of the values.)
PATCH http://xxx.xxx.xxx.xxx:xxxxxx/sap/opu/odata/sap/API_BUSINESS_PARTNER/A_BusinessPartner(BusinessPartner='1000001') HTTP/1.1
x-csrf-token: xxx
Content-Type: application/json
Accept: application/json
If-Match: dummy
Authorization: Basic xxx
SAP-Connectivity-SCC-Location_ID: xxx
Proxy-Authorization: Bearer xxx
sap-language: en
sap-client: xxx
Content-Length: 55
If ETags are not part of OData service responses, then you should approach the IT/administrators who maintain the S/4 backend. The SAP Cloud SDK is only consuming the OData service. Unfortunately it can't leverage ETag support if it's disabled.