NodeJS - express server, pm2 cluser and nginx balancing - multiple threads - node.js

Question

I'm trying to set up express server to listen on multiple threads. I've used pm2 to set up application on ports 3000, 3001, 3002, 3003 but requests are still waiting for each other...
express application index.js:
const express = require('express')
const axios = require('axios')
const app = express()
app.get('/', async (req, res) => {
console.log('-----> GOT REQUEST -> ' + (new Date()).getTime());
let resp = await axios.get("here some correct http get");
res.send("Hello world!")
console.log(' Response time: ' + (new Date()).getTime());
})
let instance = +process.env.NODE_APP_INSTANCE || 0;
let port = (+process.env.PORT || 3000) + instance;
app.listen(port, () => console.log('Example app listening on port ' + port))
so every instance is on another port. Now it's time for nginx:
upstream test_upstream {
least_conn;
server 127.0.0.1:3000;
server 127.0.0.1:3001;
server 127.0.0.1:3002;
server 127.0.0.1:3003;
}
server {
listen 8000;
location / {
proxy_hide_header Access-Control-Allow-Origin;
add_header Access-Control-Allow-Origin * always;
proxy_hide_header Access-Control-Allow-Methods;
add_header Access-Control-Allow-Methods "GET,POST,DELETE,PUT,OPTIONS" always;
proxy_hide_header Access-Control-Allow-Headers;
add_header Access-Control-Allow-Headers "Authorization, X-Requested-With, Content-Type" always;
proxy_hide_header Access-Control-Allow-Credentials;
add_header Access-Control-Allow-Credentials "true" always;
if ($request_method = OPTIONS ) { # Allow CORS
add_header Access-Control-Allow-Origin *;
add_header Access-Control-Allow-Methods "GET,POST,DELETE,PUT,OPTIONS";
add_header Access-Control-Allow-Headers "Authorization, X-Requested-With, Content-Type";
add_header Access-Control-Allow-Credentials "true" always;
add_header Content-Length 0;
add_header Content-Type text/plain;
add_header Allow GET,POST,DELETE,PUT,OPTIONS;
return 200;
}
proxy_pass http://test_upstream;
}
}
So far so good. My environment:
node v 10.3.0
cpu cores 8, but i'm using only 4 instances
Ok, started application:
┌──────────┬────┬─────────┬───────┬────────┬─────────┬────────┬─────┬───────────┬───────────────┬──────────┐
│ App name │ id │ mode │ pid │ status │ restart │ uptime │ cpu │ mem │ user │ watching │
├──────────┼────┼─────────┼───────┼────────┼─────────┼────────┼─────┼───────────┼───────────────┼──────────┤
│ index │ 0 │ cluster │ 57069 │ online │ 6 │ 17m │ 0% │ 39.7 MB │ administrator │ disabled │
│ index │ 1 │ cluster │ 57074 │ online │ 6 │ 17m │ 0% │ 39.0 MB │ administrator │ disabled │
│ index │ 2 │ cluster │ 57091 │ online │ 6 │ 17m │ 0% │ 37.5 MB │ administrator │ disabled │
│ index │ 3 │ cluster │ 57097 │ online │ 6 │ 17m │ 0% │ 38.8 MB │ administrator │ disabled │
└──────────┴────┴─────────┴───────┴────────┴─────────┴────────┴─────┴───────────┴───────────────┴──────────┘
Now it's time to invoke it. I want to send multiple requests at the same time:
async sendRequest() {
const startTime = performance.now();
console.log("Sending request");
const els = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9];
const promises = els.map(() => axios.get("http://my_server_with_nginx:8000"));
let results = await Promise.all(promises);
console.log(results);
const stopTime = performance.now();
console.log("Time of request: " + (stopTime - startTime));
}
And test looks like this:
And finally node app log:
0|index | -----> GOT REQUEST -> 1527796135425
0|index | Response time: 1527796135572
1|index | -----> GOT REQUEST -> 1527796135595
1|index | Response time: 1527796135741
2|index | -----> GOT REQUEST -> 1527796135766
2|index | Response time: 1527796136354
3|index | -----> GOT REQUEST -> 1527796136381
3|index | Response time: 1527796136522
0|index | -----> GOT REQUEST -> 1527796136547
0|index | Response time: 1527796136678
1|index | -----> GOT REQUEST -> 1527796136702
1|index | Response time: 1527796136844
2|index | -----> GOT REQUEST -> 1527796136868
2|index | Response time: 1527796137026
3|index | -----> GOT REQUEST -> 1527796137098
3|index | Response time: 1527796137238
0|index | -----> GOT REQUEST -> 1527796137263
0|index | Response time: 1527796137395
1|index | -----> GOT REQUEST -> 1527796137419
1|index | Response time: 1527796137560
As we can see, it's correctly balancing requests to nodes, but somewhere has stalled. How to force it to run in parallel?

Answer 1

It turns out that everything works just great.
Problem was in the browser. When browser send the same http get requests they are queued. To change that i had to change invocation:
const els = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9];
const promises = els.map(() => axios.get("http://my_server_with_nginx:8000"));
to this:
const els = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9];
const promises = els.map((el) => axios.get(`http://my_server_with_nginx:8000?number=${el}`));

Answer 2

Are you sure that the requests are really on all cores? I mean to test this you will need to use a synchronous function. If you use async methods, the requests will be still accomplished asynchronously even on one thread.