So I have been building a cli/gui app using electron for work. I have a progress bar that needs an interval to be run every so often to refresh the progress bar. Recently we wanted to run this in jenkins so having progress bars refresh ever 80 ms would be super anoying. I added some code to clear the interval if a certain environment variable was set and then for some reason the app started to hang after sending http requests. The server would never receive the requests and the app would just sit there. After a lot of debugging i have found that putting a setInterval(() => { }, 80); (one that does anything or nothing for any amount of time) anywhere in the code fixes the problem. Has anyone seen this before? i feel like I'm going crazy!
edit:
setting a timeout on the request will make the request fail after the timeout.
edit:
so I can't show you the exact code but here is the jist of one of the request calls.
return new Promise((resolve, reject) => {
request.put({
url: this.buildUrl(armada, '/some-path'),
body: vars,
json: true
}, (err, resp, body) => {
logger.comment('err "%s"', JSON.stringify(err));
logger.comment('resp "%s"', JSON.stringify(resp));
logger.comment('body "%s"', JSON.stringify(body));
let e = this.handleError(err, resp, body, 'Error getting stuff');
if (e) { return reject(e); }
logger.comment('got back body "%s"', body);
resolve(resp);
});
});
and then if I have an interval somewhere it doesn't hang. and if I don't it does.
i can paste this code anywhere in my code and everything starts working
setInterval(() => { }, 80);
Now it's not a specific request made as the app makes a lot of different requests. any of the requests it makes can hang. And they don't always hang. about 1 in 10 times everything will work just fine for an individual request.
Related
I am trying to use different Axios calls to get some data from a remote server. One by one the calls are working but as soons as I call them directly after each other its throwing the error message about the headers. I did some research already and I guess it has sth to do that there the headers of the first call gets in the way of the second call. That is probably a very simplematic description of the problem but I am new to node js and the way those axios calls are working.
This is an example of one of my Api calls:
app.get('/api/ssh/feedback', function(req, res){
conn.on('ready', function(){
try {
let allData = {}
var command = 'docker ps --filter status=running --format "{{.Names}}"'
conn.exec(command, function(err, stream){
if (err) throw console.log(err)
stream.on('data', function(data){
allData = data.toString('utf8').split('\n').filter(e=>e)
return res.json({status: true, info: allData})
})
stream.on('close', function(code){
console.log('Process closed with: ' + code)
conn.end()
})
stream.on('error', function(err){
console.log('Error: ' + err)
conn.end()
})
})
} catch (err) {
console.error('failed with: ' + err)
}
}).connect(connSet)
})
I am using express js as a middleware and the shh2 package to get the connection with the remote server. How I mentioned before the call is working but crashes if it is not the first call. I am able to use the api again after I restart the express server.
This is how I am calling the api through axios in my node js frontend:
getNetworkStatus(e){
e.preventDefault()
axios.get('/api/ssh/network').then(res =>{
if(res.data.status){
this.setState({network_info: 'Running'})
this.setState({network: res.data.info})
} else {
this.setState({network_info: 'No Network Running'})
this.setState({network: 'No Network detected'})
}
}).catch(err => {
alert(err)
})
}
I would be really grateful for any help or advice how to solve this problem. Thanks to everyone who spends some time to help me out.
There are two issues in the code you've provided:
You are making assumptions about 'data' events. In general, never assume the size of the chunks you receive in 'data' events. You might get one byte or you might get 1000 bytes. The event can be called multiple times as chunks are received and this is most likely what is causing the error. Side note: if the command is only outputting text, then you are better off using stream.setEncoding('utf8') (instead of manually calling data.toString('utf8')) as it will take care of multi-byte characters that may be split across chunks.
You are reusing the same connection object. This is a problem because you will continue to add more and more event handlers every time that HTTP endpoint is reached. Move your const conn = ... inside the endpoint handler instead. This could also be causing the error you're getting.
When I start up my application, it sends "App Started" twice, and if I write in Slack (Which is where the app "lives" (sorry for being unable to state this accurately, but you get the point), #saltbot dadjoke everything works but it happens twice. I am using the slackbots library.
This one only gets executed twice (the console log only logs once), but the handleMessage(data) then executes the dadjoke function, which also only has one console.log output. Regardless, the result is that it sends two messages (four total because of 2 per dadjoke()).
bot.on('message', (data) => {
if(data.type !== 'message') {
return;
}
console.log("Message received")
handleMessage(data);
})
Function that runs twice, but only consoles logs once
const dadJokes = (userId) => {
console.log("Userid: ", userId)
axios({
"method":"GET",
"url":"https://dad-jokes.p.rapidapi.com/random/jokes",
"headers":{
"content-type":"application/octet-stream",
"x-rapidapi-host":"dad-jokes.p.rapidapi.com",
"x-rapidapi-key":"ab6f131638mshd9ccda499375f86p1a3471jsnb7698b37e834",
"useQueryString":true
}
})
.then((response)=>{
bot.postMessage(userId, response.data.setup)
setTimeout(() => {
bot.postMessage(userId, response.data.punchline)
}, 7000)
})
.catch((error)=>{
console.log(error)
})
}
Here is the full code: https://github.com/hagenek/saltbot
So.. Not sure if I should delete question or not, but the solution: closed everything and restarted my computer. Loaded up my zsh terminal and ran the app. Now it works.
I am developing a script on Node.js that sends a lot of requests to an API. After several requests (more than 380 requests), we receive the following error message : Error: socket hang up (code:ECONNRESET). This is a big issue for our script since we would like to send around 10000 requests.
This is not an issue with the rate limit of the API because we are already handling this.
Our script is running on OVH server, and we send our requests using the package request-promise. Our version of Node.js is v 9.9.0.
Here is the function where the error is thrown :
const pollSession = async (sessionUrl) => {
let session;
try {
session = await rp.get({ url: sessionUrl, json: true }, (err, res, body) => {
if (err) {
console.log('Err: ', err);
} else {
DEBUG && console.log("Status code: ",res && res.statusCode);
DEBUG && console.log("Status: ",res && res.body && res.body.Status);
statusCode = res && res.statusCode;
status = res && res.body && res.body.Status;
}
});
} catch (e) {
console.log ("----- pollSession : in catch with return value :"+e);
return e;
}
return session;
}
When the request is working, we are calling this function few times in order to get the full response (because the response is huge).
When the error "Err: { Error: socket hang up" is thrown, we are calling the function again and it returns this error again. We can't afford to give up on those requests so we would like to know how to work around this error. Maybe it is possible to increase the max number of sockets (I saw it was possible with http agent, but we are using request-promise package) ?
Let me know if you need further information
After a lot of tests, I find out that this is related to the API I am sending requests to, Skyscanner for the record. Some flights I am searching are too long to be retrieved and lead to this error. Fixed this issue by catching the error.
I am creating an app and using http://c9.io environment to develop it. It is a NodeJS app, which provides some REST endpoints for the client side application to query. Till now, everything was running fine, and today what I observe is that for 1 call sent by the browser to the REST API, 2 requests are being shown as received, and the request handler is being called 2 times. This has slowed the response time for one request.
In Chrome developer tools, it shows only one request sent, however, I am using app.use() to log incoming requests in Express and it prints the same 2 times for each request. Also, the handler is called twice.
This is happening intermittently, not every time. I am behind a corporate network. As I have sent a lot of requests in the day for testing, is there any chance that a monitoring program is sending the requests since it finds it suspicious? I have not edited the code that handles the requests.
Edit: Adding the code for handlers as suggested.
app.get('/suggestions/:keyword', function(r, s) {
sug_db.retrieveSuggestions(r.params.keyword, function(data) {
s.writeHead(200, {'content-type': 'text/html'});
s.write(renderSugg({data: data}))
s.end();
});
});
app.get('/search/:query', function(r, s) {
esc_db.search(r.params.query, function(data) {
s.send(renderResults({query: r.params.query, results:data}));
});
});
As you can see, they do nothing but get some data from a database and return the result as HTTP response. The templating engine I am using is Pug (formerly Jade)
It doesn't look like that code that you included in the question can be guilty of running twice. But maybe some code in sug_db.retrieveSuggestions or esc_db.search does that.
What I would do is this:
Add some logging inside the code that you provided, both before calling the functions and inside the callback:
app.get('/suggestions/:keyword', function(r, s) {
console.log('*** GET /suggestions/:keyword handler');
sug_db.retrieveSuggestions(r.params.keyword, function(data) {
console.log('GET /suggestions/:keyword callback');
s.writeHead(200, {'content-type': 'text/html'});
s.write(renderSugg({data: data}))
s.end();
});
});
app.get('/search/:query', function(r, s) {
console.log('*** GET /search/:query handler');
esc_db.search(r.params.query, function(data) {
console.log('GET /search/:query callback');
s.send(renderResults({query: r.params.query, results:data}));
});
});
(or change console.log to whatever method of logging you use).
I would see what is actually called twice - the handlers themselves, or the callbacks, or none. Next would be examination of the functions that are actually called by the handlers:
sug_db.retrieveSuggestions()
esc_db.search()
renderSugg()
renderResults()
It's important to see what is actually called twice and then examine why it can be happening. But it can happen if, for example, you do something like:
function badFunction(data, callback) {
if (something) {
callback('error');
}
callback('ok');
}
instead of:
function goodFunction(data, callback) {
if (something) {
callback('error');
} else {
callback('ok');
}
}
I would expect that the functions that are called from the handlers could do something like that to call the callback twice - and maybe the condition or error that they checking didn't happen before but happens now, causing the change in behavior.
I have a web app that is published via ExpressJS on NodeJS, of course. It uses CouchDB as it's data source. I implemented long polling to keep the app in sync at all times between all users. To accomplish this I use the following logic:
User logs into app and an initial long poll request is made to Node via an Express route.
Node in turn makes a long poll request to CouchDB.
When Couch is updated it responds to the request from Node.
Lastly Node responds to the browser.
Simple. What is happening, though, is that when I refresh the browser it freezes up on every fifth refresh. Huh? very wierd. But I can reproduce it over and over, even in my test environment. Every fifth refresh without fail freezes up Node and causes the app to freeze. Rebooting Node fixes the issue.
After much hair pulling I THOUGHT I solved it by changing this:
app.get('/_changes/:since*', security, routes.changes);
To this:
app.get('/_changes/:since*', security, function () { routes.changes });
However, after further testing this is just failing to run routes.changes. So no actual solution. Any ideas why long polling CouchDb from Node would do such a strange thing? On the fifth refresh I can have a break point in node on the first line of my routing code and it never get's hit. However, in the browser I can break on the request to node for long polling and it seems to go out. It's like Node is not accepting the connection for some reason...
Should I be approaching long polling from Node to CouchDB in a different way? I'm using feed=longpoll, should I maybe be doing feed=continuous? If I turn down the changes_timeout in couchdb to 5 seconds it doesn't get rid of the issue, but it does make it easier to cope with since the freezes only last 5 seconds tops. So this would seem to indicate that node can't handle having several outstanding requests to couch. Maybe I will try a continuous feed and see what happens.
self.getChanges = function (since) {
Browser:
$.ajax({
url: "/_changes/" + since,
type: "GET", dataType: "json", cache: false,
success: function (data) {
try {
self.processChanges(data.results);
self.lastSeq(data.last_seq);
self.getChanges(self.lastSeq());
self.longPollErrorCount(0);
} catch (e) {
self.longPollErrorCount(self.longPollErrorCount() + 1);
if (self.longPollErrorCount() < 10) {
setTimeout(function () {
self.getChanges(self.lastSeq());
}, 3000);
} else {
alert("You have lost contact with the server. Please refresh your browser.");
}
}
},
error: function (data) {
self.longPollErrorCount(self.longPollErrorCount() + 1);
if (self.longPollErrorCount() < 10) {
setTimeout(function () {
self.getChanges(self.lastSeq());
}, 3000);
} else {
alert("You have lost contact with the server. Please refresh your browser.");
}
}
});
}
Node:
Routing:
exports.changes = function (req, res) {
var args = {};
args.since = req.params.since;
db.changes(args, function (err, body, headers) {
if (err) {
console.log("Error retrieving changes feed: "+err);
res.send(err.status_code);
} else {
//send my response... code removed here
}
})
}
Database long poll calls:
self.changes = function (args, callback) {
console.log("changes");
if (args.since == 0) {
request(self.url + '/work_orders/_changes?descending=true&limit=1', function (err, res, headers) {
var body = JSON.parse(res.body);
var since = body.last_seq;
console.log("Since change: "+since);
self.longPoll(since, callback);
});
} else {
self.longPoll(args.since, callback);
}
}
self.longPoll = function (since, callback) {
console.log("about to request with: "+since);
request(self.url + '/work_orders/_changes?feed=continuous&include_docs=true&since=' + since,
function (err, res, headers) {
console.log("finished request.")
if (err) { console.log("Error starting long poll: "+err.reason); return; } //if err send it back
callback(err, res.body);
});
}
Socket.io will automatically fall back to long polling, and doesn't have a problem like the one you are having. So just use that. Also for CouchDB changes use this https://github.com/iriscouch/follow or maybe this one https://npmjs.org/package/changes like the other guy suggested.
Its very bad practice to reinvent things when we have popular modules that already do what you need. There are currently more than 52,000 node modules on https://npmjs.org/. People make a big deal about copying and pasting code. In my mind reinventing basic stuff is even worse than that.
I know with so many modules its hard to know about all of them, so I'm not saying you can never solve the same problem as someone else. But take a look at npmjs.org first and also sites like http://node-modules.com/ which may give better search results.