Problem:
I am doubting that something in my network system is slow which is why responses from NodeJS server arrives late. I've noticed that the server is ready with the response fairly fast however, it takes time to reach the web browser.
Question:
What are the ways I can confirm whether this is infact happening or not? Basically to capture the time when the response left from my NodeJS server and then from the physical server and then compare it with the time that it arrived on the client web browser. How can I do this?
What I tried:
I tried putting a console.log after ctx.body to identify the point of time when the response left the server. But is this the correct point or can I go further down? I am a little unsure. Please advise.
FYI, I am using Koa.js.
As we cannot always assume that the time on the client is exactly the same as on the server, what you theoretically can do is:
client side: before sending a request to the server, get the current time with
const startClient = Date.now()
on the server side you implement a small request time (koa) middleware like this:
app.use(function *(next){
const startServer = new Date.now();
await next;
const ms = (new Date.now()) - startServer;
ctx.set('X-Server-Time', ms);
});
Place this middleware before defining your routes. This should return a time the server took to complete its task to the client (in the X-Server-Time header). So this is basically the time from when the server receives the request till he is ready and sends results back.
as soon as the client gets back any information from the server, get again the current time and calculate time spend overall:
myHeaders = response.headers;
if (myHeaders[X-Server-Time]) {
const msServer = parseInt(myHeaders[X-Server-Time]);
const endClient = Date.now();
const msOverall = endClient - startClient;
// output time consumed
console.log('Overall time in ms to complete : ' + msOverall);
console.log('Server time in ms to complete : ' + msServer);
console.log('Network time in ms to complete : ' + msOverall - msServer);
}
Code not testet, but I hope this gives an idea how to measure timing...
Try Koa.js in debug mode and add a logger middleware to identify server side issues.
Use the network inspector in Chrome or any equivalent in other browsers to identify client side issues.
If none of the above helps, you can try debugging on the network level with tcpdump and Wireshark. This helps identifying protocol and connection issues.
Related
I'm new to backend development so have just made my first server using NodeJS and the http module. This is my code so far:
const http = require("http");
let count = 0;
const server = http.createServer((req, res) => {
res.write(count.toString());
//tells the server that all of the headers and body have been sent, so the message is complete
res.end();
count += 1;
});
server.listen(3000);
I understand how almost all of this code works. However, whenever I refresh the page on my local environment (send a new request) I would expect the displayed response to increment by 1, however, it increments by 2. The only reason I can think this would happen is that the request event listener is being fired twice on each page reload, however, I cannot find anything to help me with this issue so any help would greatly be appreciated.
If i understand what this website says https://www.w3schools.com/nodejs/met_http_createserver.asp
The requestListener is called every time a request is sent to the server.
You can check in your Navigator console (in the Network sub-tab) the different requests sent to your server.
Maybe there are multiple requests sent to your server on page reload.
I hope this helped, otherwise, you can try to log in your requestListener the req object to know what triggers it, and where does it come from.
I'm at a little standstill here.
I'm creating an express app that have to export a huge amount of data from a Shopify store with a single request.
The issue is that when I reach the 2min mark the request is fired again ( since the default timeout is 2min ), so I increased the server.setTimeout() to take in consideration the time that it needs.
So this fixed the fire of the request on the 2min mark, but once my request finish, once again for some reason a second request is made.
Here is a bare bone example of the issue:
const express = require('express');
function sleep(ms){
return new Promise(resolve=>{
setTimeout(resolve,ms)
})
}
app.get('/export', async (req, res) => {
// Sleep for 2:03min
await sleep(123000);
console.log('Starting request');
res.status(200).send('Finish');
})
const server = app.listen(3000, () => {
console.log(`Example app listening on port ${port}`)
})
// Set timeout to 2:04min
server.setTimeout(124000);
If you open http://localhost:3000/export the result form the above code returns:
Starting request <- at the beginning
Starting request <- at the 2:04 mark
Is this some issue because of the async/await, since it seems that res.send() never fires ?
Can someone clarify "why" is this happening and "how" to prevent it?
PS: I can't use a flag or something to check if the request was already made, since the APP needs to work for users exporting the data at the same time, and the second request comes as a brand new one.
Well I lost around 4 hours on this today and the conclusion is that my tunel service ( http://serveo.net/ ) was firing another request at some point perfectly timed when my export is done. ( as for why, I don't have an answer to that ) I'm still not sure if this is the correct conclusion, but switching to a different option ( or using the direct localhost ) didn't show any issues.
I moved to OpenVPN and all of my problems were gone. Ngrok was OK as well, but since the free version didn't have a fixed URL ( and it's a pain to change all of the end points in the App setup each day I start the service ) I went with OpenVPN.
The root of the problem was that res.status(200).send('Finish') was never firing for some specific reason or if it was it sure didn't seems so.
Thanks for all of the help.
I am currently developing my own desktop clock app and after successfully receiving the current date and time via custom API and locally, I've come to the point where
serious complications may occur in the future.
With the current implementation of (local-time), the time is updated locally - every minute per app instance.
It would be unnecessary silly if I try to achieve the same for the (server-time) -> to send a GET request each minute to the Server from every existing app instance...
So, here comes my question..
What are the more efficient alternatives?
P.S. The server environment is Node.js.
The received time is in the form of a JSON.
you can try this code
const serverTime = new Date(); // lets say this is time from server
const timer = (time) =>{
setTimeout(()=>{
time.setSeconds(time.getSeconds() + 1);
console.log(time)
timer(time);
},1000)
}
timer(serverTime);
I'm working on a project that uses the binance api to create an interface to make day trading cryptos easier.
The call to their api looks like this:
binance.websockets.candlesticks(['BNBBTC'], "1m", function(candlesticks) {
let { e:eventType, E:eventTime, s:symbol, k:ticks } = candlesticks;
let { o:open, h:high, l:low, c:close, v:volume, n:trades, i:interval, x:isFinal, q:quoteVolume, V:buyVolume, Q:quoteBuyVolume } = ticks;
console.log(symbol+" "+interval+" candlestick update");
console.log("open: "+open);
console.log("high: "+high);
console.log("low: "+low);
console.log("close: "+close);
console.log("volume: "+volume);
console.log("isFinal: "+isFinal);
});
It seems to be returning data at a fixed interval, so I'm skeptical as to whether it's actually real time, but regardless, I'm wondering how to send this data to the front end as it comes in.
Currently, I'm doing this with the static data:
router.get('/interface', function(req,res) {
binance.candlesticks("BNBBTC", "5m", function(ticks, symbol) {
console.log("candlesticks()", ticks);
let last_tick = ticks[ticks.length - 1];
let [time, open, high, low, close, volume, closeTime, assetVolume, trades, buyBaseVolume, buyAssetVolume, ignored] = last_tick;
console.log(symbol+" last close: "+close);
res.render('interface', {ticks:ticks});
});
});
I've messed with socket.io in the past, but am unsure how to utilize it. Any help would be much appreciated! And please hmu if you're interested in cryptos. We are putting together a group in discord to share our research, and trading strategies.
To initiate the data sending process from the backend, (instead of frontend requesting data), you should use websockets (socketIO as you have mentioned).
To do that, first, you should start a socketio server in your express app, by wrapping the http/https server or express app.
Then, from the frontend, you should initiate a socketio-client.
Next, your frontend client should establish a connection with the server using the connect method of the socketio-client. It will fire an event in the server, with the socket connection.
Finally, the server can use that socket connection, to send any amount of data to the client. (You might need to save the connection for latter use).
i'm trying to do basically the same thing, what discord group you talking about?
I have app hosted on heroku + heroku postgres instance.
My REST api fetches some data from DATABASE and returns data back to client.
I am using Sequelize.js as ORM and restify.js as server.
My API function looks like this:
app.get('/test', function(req,res,next){
t1 = new Date()
db.getSomeData.done(function(err,result){
t2 = new Date()
console.log((t2.getTime()-t1.getTime())+'ms')
res.json({}) // it is not mistake, I return empty json to make sure that amount of data transfered back to client is not causing dely
next()
})
})
For tests I always fetch exactly same data from DB. console.log says that DB query takes 400-500ms. However when I am testing ajax requests (in chrome) and I can see that average request time is 500-800ms, but from time to time (it happens around 2-3 times per 10 tries) response is received after 3-5 seconds. This is strange because db query time is still normal (400ms for example) and I am not sending any data back to client. API is used only by me so there is no load on heroku server. I though that it might be caused by my internet connection so I made second test with following handler:
app.get('/test', function(req,res,next){
res.json({}) // it is not mistake, I return empty json to make sure that amount of data transfered back to client is not causing dely
next()
})
And for 100 times ther eis no delay - all requeststs are completed in 73-89 ms.
Do you know what might be a problem? It seems that request sometimes takes more time just because db query was done, no matter how much data is sent back. (despite that db query itself does not take time longer than normal)...
That is super-strange because I tried something like this:
app.get('/test', function(req,res,next){
setTimeout(function(){
res.json({}) // it is not mistake, I return empty json to make sure that amount of data transfered back to client is not causing dely
next()
},500)
})
And problem still occurs: 3-4 time per 10 tries response is received after 1.5 - 3.5 seconds where average is 800-900ms.
Also Response-Time header set by restify is correct despite the delay. For example:
Chrome says: 3.5s
Response-Time: 600s
It means that lag is caused by Heroku?
Resolved. It was a Heroku platform issue.
https://status.heroku.com/incidents/649
Investigating
Our automated systems have detected potential platform errors.
We are investigating.
Posted Jul 12, 2014 23:38 UTC