Nodejs request: HPE_INVALID_HEADER_TOKEN - node.js

I receive HPE_INVALID_HEADER_TOKEN on a certain page using request module. From what I've found on Google, this is caused by an incorrect / malformed server response, however the latter is not under my control. Can I configure request to ignore invalid headers or just give me the whole raw response for processing?

Node v12 has a new parser that is more strict with header validation, which can cause the same errors, especially with sites using Imperva/Incapsula that include an invalid header in HTTP 1.1 responses.
A temporary solution is to use the --http-parser=legacy option in node's command line, or in the NODE_OPTIONS environment variable.
Since v12.15.0 and v10.19.0 you can do this:
http.request(url, { insecureHTTPParser: true })
Additional information can be found here:
https://github.com/nodejs/node/issues/27711#issuecomment-584621376
Edit: --http-parser=legacy is no longer supported in recent versions (including minor versions of v12). A solution for newer node versions is to use --insecure-http-parser instead.

The solution is this library: https://www.npmjs.com/package/http-parser-js
So to fix your problem:
npm install http-parser-js
Add this code before require('request')
process.binding('http_parser').HTTPParser = require('http-parser-js').HTTPParser;

I had the same issue, running the code like this worked for me
nodemon --http-parser=legacy

Had the same error and provided solutions are not working anymore in node v14 and v16.
I ended up using node-libcurl (with node v16.15.1)
const { Curl } = require('node-libcurl');
const curl = new Curl();
var api_URL = "https://www.......";
curl.setOpt('URL', api_URL);
curl.setOpt('FOLLOWLOCATION', true);
curl.setOpt(Curl.option.HTTPHEADER, ['Content-Type: application/json', 'Authorization: Bearer XYZ']);
curl.on('end', function(statusCode, data, headers) {
// do something with the data you get..
// .....
this.close();
});
curl.on('error', curl.close.bind(curl));
curl.perform();
If you like to see how your Header looks like, add this option:
curl.setOpt(Curl.option.VERBOSE, true);

I was able to find what was the problem (which header was causing this) by inspecting headers in the Chrome Dev-Tools in RAW mode.
Turns out I had "Content-Type:" without a type.

Related

How to fix Sandbox bluesnap unsafe legacy renegotiation disabled error when using node.js 17

Node 17 (all versions) return the following error:
https://sandbox.bluesnap.com/services/2/payment-fields-tokens failed, reason: write EPROTO 00DE371301000000:error:0A000152:SSL routines:final_renegotiate:unsafe legacy renegotiation disabled
I don't have the same issue when using Node 16.
I tried setting
fetch( // node-fetch
'https://sandbox.bluesnap.com/services/2/payment-fields-tokens',
{
agent: new https.Agent({ // fix attempt
rejectUnauthorized: false,
}),
body, // defined somewhere else
}
);
I also set NODE_TLS_REJECT_UNAUTHORIZED=0 in my .env file
and NODE_OPTIONS=--openssl-legacy-provider
and the same issue happens
I know this is probably a problem with bluesnap but a workaround would help while they resolve their issues and I can keep using the latest node version
The new version/endpoints for Bluesnap API, are OK with the latest version of NodeJS.
https://sandpay.bluesnap.com
https://pay.bluesnap.com
I believe if you force the TLS version to TLS1.3, "secure renegotiation" will no longer be an issue (it is not supported with TLSv1.3). You can do that by setting the maxVersion to 'TLSv1.3' within your node client. This should allow you to use ws.bluesnap.com domain which support TLS1.3
If you're using nvm try running this command from the root directory:
nvm install --lts

How to configure node-fetch to use the company proxy?

I am executing a standalone nodejs script (no web server involved) that needs to fetch result from a third party api. The program uses 'node-fetch' to do the fetch(url) and I run it using node .\test.js from command line.
It fails when I am connected to our company network but works fine on direct internet. I have configured the proxy settings in npm and could see that npm config ls shows the correct values for proxy and https-proxy.
So the questions are:
1. Does running the test.js via node not pick the proxy config from npm?
2. How to make sure that the fetch(url) call goes through our proxy?
Thanks in advance
This worked for me, try using this :
https://github.com/TooTallNate/node-http-proxy-agent
The request formed will be similar to this:
fetch('accessUrl', {agent: new HttpsProxyAgent('proxyHost:proxyPort')})
.then(function (res) {
})

npm ERR! code UNABLE_TO_VERIFY_LEAF_SIGNATURE [duplicate]

I'm using node.js request.js to reach an api. I'm getting this error
[Error: UNABLE_TO_VERIFY_LEAF_SIGNATURE]
All of my credentials are accurate and valid, and the server's fine. I made the same request with postman.
request({
"url": domain+"/api/orders/originator/"+id,
"method": "GET",
"headers":{
"X-API-VERSION": 1,
"X-API-KEY": key
},
}, function(err, response, body){
console.log(err);
console.log(response);
console.log(body);
});
This code is just running in an executable script ex. node ./run_file.js, Is that why? Does it need to run on a server?
Note: the following is dangerous, and will allow API content to be intercepted and modified between the client and the server.
This also worked
process.env['NODE_TLS_REJECT_UNAUTHORIZED'] = '0';
It's not an issue with the application, but with the certificate which is signed by an intermediary CA.
If you accept that fact and still want to proceed, add the following to request options:
rejectUnauthorized: false
Full request:
request({
"rejectUnauthorized": false,
"url": domain+"/api/orders/originator/"+id,
"method": "GET",
"headers":{
"X-API-VERSION": 1,
"X-API-KEY": key
},
}, function(err, response, body){
console.log(err);
console.log(response);
console.log(body);
});
The Secure Solution
Rather than turning off security you can add the necessary certificates to the chain. First install ssl-root-cas package from npm:
npm install ssl-root-cas
This package contains many intermediary certificates that browsers trust but node doesn't.
var sslRootCAs = require('ssl-root-cas/latest')
sslRootCAs.inject()
Will add the missing certificates. See here for more info:
https://git.coolaj86.com/coolaj86/ssl-root-cas.js
CoolAJ86's solution is correct and it does not compromise your security like disabling all checks using rejectUnauthorized or NODE_TLS_REJECT_UNAUTHORIZED. Still, you may need to inject an additional CA's certificate explicitly.
I tried first the root CAs included by the ssl-root-cas module:
require('ssl-root-cas/latest')
.inject();
I still ended up with the UNABLE_TO_VERIFY_LEAF_SIGNATURE error. Then I found out who issued the certificate for the web site I was connecting to by the COMODO SSL Analyzer, downloaded the certificate of that authority and tried to add only that one:
require('ssl-root-cas/latest')
.addFile(__dirname + '/comodohigh-assurancesecureserverca.crt');
I ended up with another error: CERT_UNTRUSTED. Finally, I injected the additional root CAs and included "my" (apparently intermediary) CA, which worked:
require('ssl-root-cas/latest')
.inject()
.addFile(__dirname + '/comodohigh-assurancesecureserverca.crt');
For Create React App (where this error occurs too and this question is the #1 Google result), you are probably using HTTPS=true npm start and a proxy (in package.json) which goes to some HTTPS API which itself is self-signed, when in development.
If that's the case, consider changing proxy like this:
"proxy": {
"/api": {
"target": "https://localhost:5001",
"secure": false
}
}
secure decides whether the WebPack proxy checks the certificate chain or not and disabling that ensures the API self-signed certificate is not verified so that you get your data.
It may be very tempting to do rejectUnauthorized: false or process.env['NODE_TLS_REJECT_UNAUTHORIZED'] = '0'; but don't do it! It exposes you to man in the middle attacks.
The other answers are correct in that the issue lies in the fact that your cert is "signed by an intermediary CA." There is an easy solution to this, one which does not require a third party library like ssl-root-cas or injecting any additional CAs into node.
Most https clients in node support options that allow you to specify a CA per request, which will resolve UNABLE_TO_VERIFY_LEAF_SIGNATURE. Here's a simple example using node's built-int https module.
import https from 'https';
const options = {
host: '<your host>',
defaultPort: 443,
path: '<your path>',
// assuming the bundle file is co-located with this file
ca: readFileSync(__dirname + '/<your bundle file>.ca-bundle'),
headers: {
'content-type': 'application/json',
}
};
https.get(options, res => {
// do whatever you need to do
})
If, however, you can configure the ssl settings in your hosting server, the best solution would be to add the intermediate certificates to your hosting provider. That way the client requester doesn't need to specify a CA, since it's included in the server itself. I personally use namecheap + heroku. The trick for me was to create one .crt file with cat yourcertificate.crt bundle.ca-bundle > server.crt. I then opened up this file and added a newline after the first certificate. You can read more at
https://www.namecheap.com/support/knowledgebase/article.aspx/10050/33/installing-an-ssl-certificate-on-heroku-ssl
You can also try by setting strictSSL to false, like this:
{
url: "https://...",
method: "POST",
headers: {
"Content-Type": "application/json"},
strictSSL: false
}
I had the same issues. I have followed #ThomasReggi and #CoolAJ86 solution and worked well but I'm not satisfied with the solution.
Because "UNABLE_TO_VERIFY_LEAF_SIGNATURE" issue is happened due to certification configuration level.
I accept #thirdender solution but its partial solution.As per the nginx official website, they clearly mentioned certificate should be combination of The server certificate and chained certificates.
Just putting this here in case it helps someone, my case was different and a bit of an odd mix. I was getting this on a request that was accessed via superagent - the problem had nothing to do with certificates (which were setup properly) and all to do with the fact that I was then passing the superagent result through the async module's waterfall callback. To fix: Instead of passing the entire result, just pass result.body through the waterfall's callback.
Following commands worked for me :
> npm config set strict-ssl false
> npm cache clean --force
The problem is that you are attempting to install a module from a repository with a bad or untrusted SSL[Secure Sockets Layer] certificate. Once you clean the cache, this problem will be resolved.You might need to turn it to true later on.
Another approach to solving this securely is to use the following module.
node_extra_ca_certs_mozilla_bundle
This module can work without any code modification by generating a PEM file that includes all root and intermediate certificates trusted by Mozilla. You can use the following environment variable (Works with Nodejs v7.3+),
NODE_EXTRA_CA_CERTS
To generate the PEM file to use with the above environment variable. You can install the module using:
npm install --save node_extra_ca_certs_mozilla_bundle
and then launch your node script with an environment variable.
NODE_EXTRA_CA_CERTS=node_modules/node_extra_ca_certs_mozilla_bundle/ca_bundle/ca_intermediate_root_bundle.pem node your_script.js
Other ways to use the generated PEM file are available at:
https://github.com/arvind-agarwal/node_extra_ca_certs_mozilla_bundle
NOTE: I am the author of the above module.
I had an issue with my Apache configuration after installing a GoDaddy certificate on a subdomain. I originally thought it might be an issue with Node not sending a Server Name Indicator (SNI), but that wasn't the case. Analyzing the subdomain's SSL certificate with https://www.ssllabs.com/ssltest/ returned the error Chain issues: Incomplete.
After adding the GoDaddy provided gd_bundle-g2-g1.crt file via the SSLCertificateChainFile Apache directive, Node was able to connect over HTTPS and the error went away.
If you come to this thread because you're using the node postgres / pg module, there is a better solution than setting NODE_TLS_REJECT_UNAUTHORIZED or rejectUnauthorized, which will lead to insecure connections.
Instead, configure the "ssl" option to match the parameters for tls.connect:
{
ca: fs.readFileSync('/path/to/server-ca.pem').toString(),
cert: fs.readFileSync('/path/to/client-cert.pem').toString(),
key: fs.readFileSync('/path/to/client-key.pem').toString(),
servername: 'my-server-name' // e.g. my-project-id/my-sql-instance-id for Google SQL
}
I've written a module to help with parsing these options from environment variables like PGSSLROOTCERT, PGSSLCERT, and PGSSLKEY:
https://github.com/programmarchy/pg-ssl
Hello just a small adition to this subject since in my case the
require('ssl-root-cas/latest')
.inject()
.addFile(__dirname + '/comodohigh-assurancesecureserverca.crt');
didn't work out for me it kept returning error that the file could not be downloaded i had been a couple of hours into the reasearch of this particular error when I ran into this response https://stackoverflow.com/a/65442604
Since in my application we do have a proxy to proxy some of our requests as a security requirement of some of our users I found that in the case you are consulting an API that has this issue and if you can access the API url throught your browser you can proxy your request and it might fix the [Error: UNABLE_TO_VERIFY_LEAF_SIGNATURE] issue.
An example of how i use my proxy
await axios.get(url, {
timeout: TIME_OUT,
headers: {
'User-Agent': 'My app'
},
params: params,
proxy: {
protocol: _proxy.protocol,
host: _proxy.hostname,
port: _proxy.port,
auth: {
username: _proxy_username,
password: _proxy_password
}
}
});
I had the same problem and I am able to fix it the following way,
Use the full-chain or just the chain certificate instead of just the certificate.
That is all.
This same error can be received when trying to install a local git shared repo from npm.
The error will read: npm ERR! code UNABLE_TO_VERIFY_LEAF_SIGNATURE
Apparently there is an issue with the certificate, however what worked for me was change the link to my shared repo in the package.json file from:
"shared-frontend": "https://myreposerver"
to:
"shared-frontend": "git+https://myreposerver"
In short, just adding git+ to the link solved it.
Another reason node could print that error is because a backend connection/service is misconfigured.
Unfortunately, the node error doesn't say which certificate it was unable to verify [feature request !]
Your server may have a perfectly good certificate chain installed for clients to connect and even show a nice padlock in the browser's URL bar, but when the server tries to connect to a backend database using a different misconfigured certificate, then it could raise an identical error.
I had this issue in some vendor code for some time. Changing a backend database connection from self-signed to an actual certificate resolved it.
You have to include the Intermediate certificate in your server. This solves the [Error: UNABLE_TO_VERIFY_LEAF_SIGNATURE]

nodejs http.getagent on v0.10.6 can't work

I'm a newbie to nodejs ,recently i tried upgrade nodejs from 0.4.7 to 0.10.6,but my app.js got a error message ,
"http.getAgent: TypeError: Object# has no method 'getAgent' "
then checked codes from app.js found
" var somevar = http.getAgent('somedomain' , 80 ); "
so which class or method can replace getAgent with 0.10.6 ?
Thanks a lot !
I actually can't find any information on the removal of getAgent -- even in the changelog. It seems that you would only need to use the http agent for hyper-technical reasons. If you really need to use the agent, it seems that you can use http.globalAgent (property, not function) instead.

Problems directly using PhantomJS in node.js

I'm attempting to use PhantomJS, and I've installed it via NPM.
I can't seem to run any of the of the examples, in fact I can't even run:
var page = require('webpage').create();
I get the error:
Error: Cannot find module 'webpage'
Is there anything i'm missing? I'm using a few other modules that I've installed via NPM in the same directory with no issues
PhantomJS is not for Node.js. You are likely running the examples through node binary.
Read the Getting Started documentation carefully and you'll see that every single PhantomJS example need to be invoked like:
phantomjs hello.js
Note that there is a bridge between Node.js and PhantomJS. In that case, you need to follow the given examples for that particular bridge (there are a few different ones).
You can use something like this:
var page = new WebPage();
Example of code :
var page = new WebPage();
page.open('http://example.com', function(status) {
console.log("Status: " + status);
if(status === "success") {
page.render('example.png');
}
phantom.exit();
});

Resources