We are currently building a desktop application in node-webkit and we need to send http requests to a remote server. For this we decided to use request, a http wrapper module for node.
This works fine on all but one of our machines. The code for the download looks a bit like this:
var options = {
url: url
};
request.post(options
, function (error, response, body)
{
if (!error && response.statusCode == 200)
{
cb && cb(null, body);
}
}
).on('error', function (err)
{
}).pipe(writeStream);
So with this the result we get on my machine is this:
On our network here the proxy server is 172.24.8.14 and my address is 172.24.9.130. Node sent the request through the proxy server wich contacted the target server. The result that is sent back is a 301 which is expected.
...And on the other machine:
This time Node attempted to send the request directly to the target server. This resulted in the proxy blocking the request completely.
The strange thing is that we do not specify a proxy in our code however the requests do seem to go through the proxy...but not on the other machine.
Is there some reason for this? How is node somehow detecting the proxy and sending the request to the proxy?
The reason for this turned out to be that our network was using an NTLM proxy which required ISA client to be running on our machines but it was not running on the other machine. Installing ISA client on that machine allowed traffic to go through the proxy as normal.
Related
I have a application in nodejs that fetch requests from third party API and update data in my MySQL database.
exports.saveDeposits = async (req, res, next) => {
const ep = await epicPayKeys.getKeys(); // It is a database request
ep.map( async (key) => {
...
// Some transactions here
// fetch() request to third party API
// .catch() which returns res.status(500).send('Error saveDeposits');
}
res.status(202).send('Save Deposits');
}
When I run it in my localhost, everything works just fine. I moved the project to production and when I run it, I got an error:
HTTP Error 502.3 - Bad Gateway
The operation timed out
The CGI application did not return a valid set of HTTP errors.
A server acting as a proxy or gateway was unable to process the request due to an error in a parent gateway.
This error occurs when a CGI application does not return a valid set of HTTP headers, or when a proxy or gateway was unable to send the request to a parent gateway. You may need to get a network trace or contact the proxy server administrator, if it is not a CGI problem.
Any idea how can I fix it?
Thanks
I have two nodejs files. One sends a post request (using axios) from an AWS EC2 instance and one receives it (using express) on my PC. For some reason, my PC nodejs server isn't receiving the post requests.
I have this code for my receiver:
app.post('/', function (req, res) {
console.log(req.body)
res.end("Request received!")
}
And nothing is logged.
The code is probably not the issue: in all the other cases the receiver logs the request body and the sender logs the response, and the receiving server handles it correctly (putting it in a mongodb).
The cases when it works are:
Sending from my PC, Receiving in my PC
Sending from AWS, Receiving in AWS (different instances)
Sending from my PC, Receiving in AWS
I thought it might be blocked by one of the two ends, so I used security groups and opened all the ports (incoming and outcoming) on the AWS instances, and I went on the firewall on my computer and made rules that opened the needed ports, and it still doesn't work.
I also added these headers to the axios request:
const headers = {
"Access-Control-Allow-Origin": "*",
"Access-Control-Allow-Headers": "X-Requested-With",
"Content-Type":"application/json"
}
axios.post('http://(myip):3001', {arr:inf}, {headers:header}).then((res)=>{...
but it still doesn't work. Any suggestions?
EDIT: 20 or so seconds after I send the request, the node server that sends it times out, returning ETIMEDOUT
You mentioned below three cases :
Sending from my PC, Receiving in my PC - fine
Sending from AWS, Receiving in AWS (different instances) - fine
Sending from my PC, Receiving in AWS - Fine, as you must be using Public IP of the EC2 instance to connection from your PC
But sending from EC2 to your PC wont work until and unless your PC has a public IP. How will request from EC2 intance reach your PC ?
I am developing bots for telegram, I am from Iran and telegram url is blocked in my country and I am forced to use VPN/Proxy servers to access telegram api from my local dev machine.
But I have other apps running on my system that won't work throw a VPN, So I am forced to use proxifier, I can define rules for the apps that I need to go throw a proxy.
But node.exe is ignoring this rules for some reason, I can see in NetLimiter that the connection is coming from C:\Program Files (x86)\nodejs\node.exe, But adding this path to proxifier's rules has no effect, other apps like telegram itself and firefox and ... works fine with these rules ...
So has anyone managed to force node.exe to go throw proxifier?
I also tried to setup a proxcy with php in my host, but none of the proxy scripts I found was able to handle the file uploads
My last hope is to install some modules for apache and use it as a proxy or just install nginx ...
I also tried https://github.com/krisives/proxysocket and https://github.com/TooTallNate/node-https-proxy-agent with no success, its just keeps throwing errors :(
Ok, after hours of trying finally got this to work with proxifier.
https://github.com/TooTallNate/node-https-proxy-agent
new HttpsProxyAgent('http://username:password#127.0.0.1:8080')
Update :
This approach had its problems so I created a small personal proxy server with node-http-proxy on my server and connected to it:
process.env["NODE_TLS_REJECT_UNAUTHORIZED"] = 0;
const debug = require('debug')('app');
const http = require('http');
const httpProxy = require('http-proxy');
const proxy = httpProxy.createProxyServer({
secure : false
});
proxy.on('error', function (e) {
debug(e);
});
const server = http.createServer(function(req, res) {
// You can define here your custom logic to handle the request
// and then proxy the request.
proxy.web(req, res, { target: 'https://api.telegram.org', });
});
server.listen(3333);
And simply just redirected all the request to this server.
I have a node.js app, which test uptime of websites.
I'm using HTTP client library 'request' (https://www.npmjs.com/package/request) to scan my differents websites but from one client of mine, I frequently get error :
ECONNRESET
All other domains (hundreds of them) are working fine. This client is using incapsula CDN.
The app is running in a docker EC2 cluster in AWS. I tried to switch today to another HTTP client library 'Axios' (https://www.npmjs.com/package/axios) but still the same error.
Any idea on how can I get a solution ?
You've to manage network error:
devices down
host unreachable
service on host unreachable
service on host restarted (TCP connexion broken)
Here is a part of a code to do this:
Connexion.on('error', (err) => {
console.log(" Disconnected. Velbus reusedSocket:",Connexion.reusedSocket, " err.code:", err.code)
if (Connexion.reusedSocket && err.code === 'ECONNRESET') {
// retriableRequest();
}
});
Late answer but if that could help others...
I have a very simple piece of code written in node.js (see below example) which runs on Kubernetes on GCP and AWS. The app just does GET request to import data from an external application. Sometimes that GET request could take ~30 mins or more.
var http = require("http");
var request = require('request');
var options = {
url: "my-external-service",
gzip: true,
// (1) forever: true,
// (2) agent: new http.Agent({ keepAlive: true })
};
request.get(options, (error, response, body) => {
console.log('error:', error);
console.log('statusCode:', response && response.statusCode);
console.log('body:', body);
});
The problem is that my node.js app never receives the response even if it was returned by the external service. It seems that the connection was dropped somewhere between the node app and external service.
I've tried to use:
request.js's forever: true option
setting keepAlive agent
It didn't help. My node application would run forever if I don't use the timeout option. When timeout is set and exceeded I got ESOCKETTIMEDOUT error.
What is interesting:
the same code works perfectly fine on my local machine
it also works fine on Linux virtual machine which works behind NAT
works fine from Kubernetes machine when I repeat the request using CURL
Is it posiible Kubernetes drops my connection? And why?
How does CURL set the keepalive settings and why CURL request work from Kubernetes?
PS. Instead of request.js, I tried to use other libs like node-fetch or axios but results were the same.