We are trying to achieve a faster response time for a server call, and we opted to move the HTTP functionality to Websockets through socket.io.
Strangely enough, even though the general opinion is that Websockets should be faster especially since there's no handshake and any other connection setup functionality, our results don't show this, and actually HTTP edges out better since consecutive requests get back faster.
The backend is in Node.js with compression enabled (app.use(compression())), while the socket.io server is initalized with the deflate extension, as:
const io = new Server(server, {
serveClient: false,
pingInterval: 5000,
pingTimeout: 10000,
allowEIO3: true,
cors: {
origin: config.cors.origins,
credentials: true,
},
perMessageDeflate: {
threshold: 2048, // defaults to 1024
zlibDeflateOptions: {
chunkSize: 8 * 1024, // defaults to 16 * 1024
},
zlibInflateOptions: {
windowBits: 14, // defaults to 15
memLevel: 7, // defaults to 8
},
clientNoContextTakeover: true, // defaults to negotiated value.
serverNoContextTakeover: true, // defaults to negotiated value.
serverMaxWindowBits: 10, // defaults to negotiated value.
concurrencyLimit: 20, // defaults to 10
},
});
The socket listens like this:
io.use(async (socket, next) => {
try {
const userId = await socketAuthentication(socket.request);
socket.on('search', async ({ data }) => {
const response = await app.locals.searchController.search(data, userId);
socket.emit('search', { ...response });
});
next();
} catch (err) {
next(err);
}
});
While express listens for the respective HTTP request normally, through a router which then calls the same controller function (await app.locals.searchController.search(data, userId)), so no difference there.
The server is in AWS, on an EC2 instance behind nginx, and with SSL.
Below are the results for the same request, done sequentially 32 times:
Are we missing anything? I thought that the deflate configuration could have something to do, since there is a general opinion of not having it enabled (socket.io from version 3 and up disables it).
Related
In the code below:
The client is supposed to start emitting images as long Base64 Strings to the server, one after another at close intervals.
As soon as the first emit with a Base64 String takes place, the socket is disconnected. I tried with hardcoded 2-character strings and this doesn't happen.
I believe it's either the size of the data that is the problem or the fact that socket hasn't finished emitting the first Base64 string when the 2nd and 3rd etc.. emit comes.
const io = new Server(httpServer, {
cors: {
credentials: true,
origin: 'http://localhost:4200',
methods: ["GET", "POST"]
}
})
io.on('connection', function(socket) {
console.log('connected')
socket.on('newDataFromClient', async (newDataFromTheClient) => {
// will work only after I refresh the page after a login
})
socket.on('disconnect', (reason) => {
// the reason is "transport error"
console.log('disconnected due to = ', reason)
})
})
httpServer.listen(4200)
This is the client-side code:
let socket
// this is only executed once, on page load
function setup() {
fetch('/setup')
.then((response) => response.json())
.then((setup) => {
doSomething(setup)
socket = io('http://localhost:4200', {
withCredentials: true,
transports: ['websocket']
})
})
}
// this is executed repeatedly at close intervals
async function sendToServer(imageAsBase64) {
socket.emit('newDataFromClient', imageAsBase64)
}
What am I doing wrong?
The problem was socket.io's maxHttpBufferSize limit, set to 1Mb by default.
Each Base64 string I'm sending is around 2.5Mb.
I had to update my server-side code to
const io = new Server(httpServer, {
cors: {
origin: 'http://localhost:4200',
methods: ["GET", "POST"]
},
maxHttpBufferSize: 4e6 // 4Mb
})
And now everything works. I found the answer here.
I have created an internet speed test app using client speed, location, and ISP. Problem is that when this code is executed it pulls the speed, ISP and location of where the data centre is wherever it is deployed(in my example an AWS server in Virginia via Heroku). My thought is that I need this code to execute on the client's browser side instead of on the server. Is this possible using Express, Mongoose, and EJS?
Relevant code posted below. This is from my controller. I didn't include the rest because it's just logic for something separate/render code.
function index(req, res, next) {
let speedtest = new FastSpeedtest({
token: hidden, // required
verbose: false, // default: false
timeout: 10000, // default: 5000
https: true, // default: true
urlCount: 5, // default: 5
bufferSize: 8, // default: 8
unit: FastSpeedtest.UNITS.Mbps, // default: Bps
proxy: "http://optional:auth#my-proxy:123", // default: undefined
});
speedtest.getSpeed().then((s) => {
fetch(ipApiToken).then(function (response) {
response.json().then((jsonData) => {
res.render("testSpeed", { s, jsonData });
});
});
})
.catch((e) => {
console.error(e.message);
});
}
async function create(req, res, next) {
let userIsp = req.body.isp
let userSpeed = req.body.speed
let userLocation = req.body.location
let newSpeedTest = await importSpeed.speedModel.create({speed: Math.round(userSpeed),location: userLocation, isp: '', isp_id: ''});
it's not possible since you send the request from the server.
If you want to obtain data such as ping / download speed etc. from a computer, you have to send the request from this very computer.
The API I'm pulling JSON info from (ipapi) can pull data using an IP address. Luckily I found that Express has a built-in feature in 'req' for pulling the client's IP address. I still have the issue of getting accurate speed tests but this might help other people in the future.
var ip = req.headers['x-forwarded-for'] || req.connection.remoteAddress;
This pulls the client IP address in a controller function.
I just changed it to:
speedtest.getSpeed().then((s) => {
fetch('https://ipapi.co/' + ip + '/json/?key=' + ipApiToken).then(function (response) {
response.json().then((jsonData) => {
res.render("testSpeed", { s, jsonData });
});
});
})
The full code with these changes:
function index(req, res, next) {
var ip = req.headers['x-forwarded-for'] || req.connection.remoteAddress;
let speedtest = new FastSpeedtest({
token: 'YXNkZmFzZGxmbnNkYWZoYXNkZmhrYWxm', // required
verbose: false, // default: false
timeout: 10000, // default: 5000
https: true, // default: true
urlCount: 5, // default: 5
bufferSize: 8, // default: 8
unit: FastSpeedtest.UNITS.Mbps, // default: Bps
proxy: "http://optional:auth#my-proxy:123", // default: undefined
});
speedtest.getSpeed().then((s) => {
fetch('https://ipapi.co/' + ip + '/json/?key=' + ipApiToken).then(function (response) {
response.json().then((jsonData) => {
res.render("testSpeed", { s, jsonData });
});
});
})
.catch((e) => {
console.error(e.message);
});
}
I have a Koajs node app in a docker container on an EC2 instance. The app is behind an AWS Application Load Balancer.
The app simply takes a POSTed file and responds with a stream that the client can view events on.
So my server is doing the right thing (sending file data), and my client is doing the right thing (receiving file data and sending back progress), but the ALB is timing out. I don't understand why it's timing out. Both client and server are sending and receiving data to/from each other, so I would think that would qualify as keep alive traffic.
Here's the code that each is running.
Client:
const request = require('request-promise');
const fs = require('fs');
const filePath = './1Gfile.txt';
const file = fs.createReadStream(filePath);
(async () => {
// PUT File
request.put({
uri: `http://server/test`,
formData: { file },
headers: { Connection: 'keep-alive' },
timeout: 200000,
})
.on('data', (data) => {
const progressString = data.toString();
console.log({ progressString });
});
})();
Server:
const { Readable } = require('stream');
const Koa = require('koa');
const router = require('koa-router')();
(async () => {
const app = module.exports = new Koa();
router.get('/healthcheck', async (ctx) => {
ctx.status = 200;
});
router.put('/test', test);
async function test(ctx) {
const read = new Readable({
objectMode: true,
read() { },
});
ctx.body = read;
let i = 1;
setInterval(() => {
read.push(`${process.hrtime()}, ${i}`);
ctx.res.write('a');
i++;
}, 3000);
}
app.use(router.routes());
app.use(router.allowedMethods());
app.listen(3000, (err) => {
if (err) throw err;
console.info(`App started on port 3000 with environment localhost`);
});
})();
Both server and client are logging the correct things, but the ALB just times out at whatever I set it's idle timeout to. Is there some trick to tell the ALB that traffic is really flowing?
Thanks so much for any light you can shed on it.
Just a quick guess, you need to enable keepAlive when using the request-promise. add forever: true in options. Try this:
request.put({
uri: `http://server/test`,
formData: { file },
headers: { Connection: 'keep-alive' },
timeout: 200000,
forever: true,
})
We have a similar issue about timeout when using request-promise-native. We fixed by adding this option. Hopfully it works out for you.
I'm attempting to wait for the result of a stream with my Apollo Server. My resolver looks like this.
async currentSubs() {
try {
const stream = gateway.subscription.search(search => {
search.status().is(braintree.Subscription.Status.Active);
});
const data = await stream.pipe(new CollectObjects()).collect();
return data;
} catch (e) {
console.log(e);
throw new Meteor.Error('issue', e.message);
}
},
This resolver works just fine when the data stream being returned is small, but when the data coming in is larger, I'm getting a 503 (Service Unavailable). I looks like the timeout is happening around 30 seconds. I've tried increasing the timeout of my Express server with graphQLServer.timeout = 240000; but that hasn't made a difference.
How can I troubleshoot this & where is the 30 second timeout coming from? It only fails when the results take longer.
I'm using https://github.com/mrdaniellewis/node-stream-collect to collect the results from the stream.
Error coming in from the try catch:
I20180128-13:09:26.872(-7)? { proxy:
I20180128-13:09:26.872(-7)? { error: 'Post http://127.0.0.1:26474/graphql: net/http: request canceled (Client.Timeout exceeded while awaiting headers)',
I20180128-13:09:26.872(-7)? level: 'error',
I20180128-13:09:26.873(-7)? msg: 'Error sending request to origin.',
I20180128-13:09:26.873(-7)? time: '2018-01-28T13:09:26-07:00',
I20180128-13:09:26.873(-7)? url: 'http://127.0.0.1:26474/graphql' } }
Had this same problem and was a pretty simple solution. My calls were lasting a bit over 30 seconds and the default timeout was returning 503s as well so I increased that.
Assuming you're using apollo-engine (this may be true for some other forms of Apollo), you can set your engine configs like so:
export function startApolloEngine() {
const engine = new Engine({
engineConfig: {
stores: [
{
name: "publicResponseCache",
memcache: {
url: [environmentSettings.memcache.server],
keyPrefix: environmentSettings.memcache.keyPrefix
}
}
],
queryCache: {
publicFullQueryStore: "publicResponseCache"
},
reporting: {
disabled: true
}
},
// GraphQL port
graphqlPort: 9001,
origin: {
requestTimeout: "50s"
},
// GraphQL endpoint suffix - '/graphql' by default
endpoint: "/my_api_graphql",
// Debug configuration that logs traffic between Proxy and GraphQL server
dumpTraffic: true
});
engine.start();
app.use(engine.expressMiddleware());
}
Notice the part where I specify
origin: {
requestTimeout: "50s"
}
That alone is what fixed it for me. Hope this helps!
You can find more information about that here
I'm using axios to make HTTP requests.
axios offers the option to specify new http.Agent() in the request config when used in Node.
The http Agent options are:
const agentOptions = {
keepAlive: true, // Keep sockets around even when there are no outstanding requests, so they can be used for future requests without having to reestablish a TCP connection. Defaults to false
keepAliveMsecs: 1000, // When using the keepAlive option, specifies the initial delay for TCP Keep-Alive packets. Ignored when the keepAlive option is false or undefined. Defaults to 1000.
maxSockets: Infinity, // Maximum number of sockets to allow per host. Defaults to Infinity.
maxFreeSockets: 256 // Maximum number of sockets to leave open in a free state. Only relevant if keepAlive is set to true. Defaults to 256.
}
There is also the HTTP header with timeout and max options.
The options don't seem to line up (keepAliveMsecs, maxSockets and maxFreeSockets vs timeout and max), which suggests they are different.
What's the difference between Node's new Agent({ keepAlive: true }) and the HTTP header Keep-Alive: timeout=5, max=1000..?
This is based on what I've gathered myself.
The HTTP header Keep-Alive: timeout=5, max=1000 is just a header sent with HTTP requests. See it as a way to communicate between two hosts (client and server). The host says 'hey keep the connection alive please'. This is automatic for modern browsers and servers might implement it or not. The keepAlive: true of the agent is as the documentation says
Not to be confused with the keep-alive value of the Connection header.
What that means is that keepAlive: false != Connection: close. It doesn't really have anything to do with the header. The agent will take care of things at the TCP level with sockets and such on the HTTP client.
keepAlive boolean Keep sockets around even when there are no outstanding requests, so they can be used for future requests without having to reestablish a TCP connection
As soon as you use an agent for your HTTP client, the Connection: Keep-Alive will be used. Unless keepAlive is set to false and maxSockets to Infinity.
const options = {
port: 3000,
agent: new http.Agent({
keepAlive: false ,
maxSockets: Infinity,
})
};//----> Connection: close
What exactly is an agent?
An Agent is responsible for managing connection persistence and reuse for HTTP clients. It maintains a queue of pending requests for a given host and port, reusing a single socket connection for each until the queue is empty, at which time the socket is either destroyed or put into a pool where it is kept to be used again for requests to the same host and port. Whether it is destroyed or pooled depends on the keepAlive option.
Pooled connections have TCP Keep-Alive enabled for them, but servers may still close idle connections, in which case they will be removed from the pool and a new connection will be made when a new HTTP request is made for that host and port. Servers may also refuse to allow multiple requests over the same connection, in which case the connection will have to be remade for every request and cannot be pooled. The Agent will still make the requests to that server, but each one will occur over a new connection.
Regarding timeout and max, as far as I know, these are set (automatically?) when adding config for Apache
#
# KeepAlive: Whether or not to allow persistent connections (more than
# one request per connection). Set to "Off" to deactivate.
#
KeepAlive On
#
# MaxKeepAliveRequests: The maximum number of requests to allow
# during a persistent connection. Set to 0 to allow an unlimited amount.
# We recommend you leave this number high, for maximum performance.
#
MaxKeepAliveRequests 100
#
# KeepAliveTimeout: Number of seconds to wait for the next request from the
# same client on the same connection.
#
KeepAliveTimeout 5
which gives
Connection:Keep-Alive
Keep-Alive:timeout=5, max=100
But these are irrelevant for NodeJS? I'll let more experimented people answer this. Anyway, the agent won't set these and won't modify Connection: Keep-Alive unless setting keepAlive to false and maxSockets to Infinity as said above.
However, for the agent config to have any meaning, Connection must be set to Keep-Alive.
Okay, now for a little experiment to see the agent at work!
I've set up a client for testing (since axios use http.agent for the agent anyway, I just use http).
const http = require('http');
const options = {
port: 3000,
agent: new http.Agent({
keepAlive: true,
maxSockets: 2,
}),
// headers: {
// 'Connection': 'close'
// }
};
var i = 0;
function request() {
console.log(`${++i} - making a request`);
const req = http.request(options, (res) => {
console.log(`STATUS: ${res.statusCode}`);
console.log(`HEADERS: ${JSON.stringify(res.headers)}`);
res.setEncoding('utf8');
res.on('data', (chunk) => {
console.log(`BODY: ${chunk}`);
});
res.on('end', () => {
console.log('No more data in response.');
});
});
req.on('error', (e) => {
console.error(`problem with request: ${e.message}`);
});
req.end();
}
setInterval(function(){ request(); }, 3000); // send a request every 3 seconds
And the server is an express application (I'll skip the details)
server.on('connection', function(socket) {
socket.id = shortid.generate();
//socket.setTimeout(500)
console.log("A new connection was made by a client." + ` SOCKET ${ socket.id }`);
socket.on('end', function() {
console.log(`SOCKET ${ socket.id } END: other end of the socket sends a FIN packet`);
});
socket.on('timeout', function() {
console.log(`SOCKET ${ socket.id } TIMEOUT`);
});
socket.on('error', function(error) {
console.log(`SOCKET ${ socket.id } ERROR: ` + JSON.stringify(error));
});
socket.on('close', function(had_error) {
console.log(`SOCKET ${ socket.id } CLOSED. IT WAS ERROR: ` + had_error);
});
});
To make you see that keepAlive: false != Connection: close, let set keepAlive to false and see what happens server-side.
agent: new http.Agent({
keepAlive: false,
maxSockets: 20
})
Server
Client
As you can see, I've not set maxSockets to Infinity so even though keepAlive in the agent was set to false, the Connection header was set to Keep-Alive. However, each time a request was sent to the server, the socket on the server was immediately closed after each request. Let's see what happens when we set keepAlive to true.
Server
Client
This time around, only one socket have been used. There was a persistent connection between the client and the server that persisted beyond a single request.
One thing I've learned, thanks to this great article is that on Firefox, you can have as many as 6 concurrent persistent connections at a time. And you can reproduce this with the agent by setting maxSockets to 6. For testing purposes, I'll set this to 2. And also, I won't return anything from the server so the connection will be left hanging.
agent: new http.Agent({
keepAlive: true,
maxSockets: 2,
}),
//res.send('response from the server');
Server
Client
The client keeps sending requests but only two have been received by the server. Yet after two minutes, see http_server_timeout
The number of milliseconds of inactivity before a socket is presumed to have timed out.
two new requests are accepted. Actually, the client has queued the subsequent requests and once the server freed the sockets, the client was able to send two new requests from the queue.
So, I hope this helps.