Request are not distributed across their worker process - node.js

I was just experimenting worker process hence try this:
const http = require("http");
const cluster = require("cluster");
const CPUs = require("os").cpus();
const numCPUs = CPUs.length;
if (cluster.isMaster) {
console.log("This is the master process: ", process.pid);
for (let i = 0; i < numCPUs; i++) {
cluster.fork();
}
cluster.on("exit", (worker) => {
console.log(`worker process ${process.pid} has died`);
console.log(`Only ${Object.keys(cluster.workers).length} remaining...`);
});
} else {
http
.createServer((req, res) => {
res.end(`process: ${process.pid}`);
if (req.url === "/kill") {
process.exit();
}
console.log(`serving from ${process.pid}`);
})
.listen(3000);
}
I use loadtest to check "Are Request distributed across their worker process?" But I got same process.pid
This is the master process: 6984
serving from 13108
serving from 13108
serving from 13108
serving from 13108
serving from 13108
...
Even when I kill one of them, I get the same process.pid
worker process 6984 has died
Only 3 remaining...
serving from 5636
worker process 6984 has died
Only 2 remaining...
worker process 6984 has died
Only 1 remaining...
How I am getting same process.pid when I killed that? And Why my requests are not distributed across their worker process?
Even when I use pm2 to test cluster mood using:
$ pm2 start app.js -i 3
[PM2] Starting app.js in cluster_mode (3 instances)
[PM2] Done.
┌────┬────────────────────┬──────────┬──────┬───────────┬──────────┬──────────┐
│ id │ name │ mode │ ↺ │ status │ cpu │ memory │
├────┼────────────────────┼──────────┼──────┼───────────┼──────────┼──────────┤
│ 0 │ app │ cluster │ 0 │ online │ 0% │ 31.9mb │
│ 1 │ app │ cluster │ 0 │ online │ 0% │ 31.8mb │
│ 2 │ app │ cluster │ 0 │ online │ 0% │ 31.8mb │
└────┴────────────────────┴──────────┴──────┴───────────┴──────────┴──────────┘
for loadtest -n 50000 http://localhost:3000 I check pm2 monit:
$ pm2 monit
┌─ Process List ───────────────────────────────────────────────────┐┌── app Logs ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│[ 0] app Mem: 43 MB CPU: 34 % online ││ │
│[ 1] app Mem: 28 MB CPU: 0 % online ││ │
│[ 2] app Mem: 27 MB CPU: 0 % online ││ │
│ ││ │
│ ││ │
│ ││ │
│ ││ │
│ ││ │
│ ││ │
│ ││ │
│ ││ │
│ ││ │
│ ││ │
│ ││ │
│ ││ │
└──────────────────────────────────────────────────────────────────┘└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
┌─ Custom Metrics ─────────────────────────────────────────────────┐┌─ Metadata ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Heap Size 20.81 MiB ││ App Name app │
│ Heap Usage 45.62 % ││ Namespace default │
│ Used Heap Size 9.49 MiB ││ Version N/A │
│ Active requests 0 ││ Restarts 0 │
└──────────────────────────────────────────────────────────────────┘└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
left/right: switch boards | up/down/mouse: scroll | Ctrl-C: exit To go further check out https://pm2.io/
But surprisingly, app1 and app2 never hit any request as well as it didn't show any app log.
Update 1
I still couldn't figure out any solution. If any further query need please ask for that. I faced that issue first time. That's why maybe I was unable to represent where the exact problem occurring.
Update 2
After getting some answer I try to test it again with a simple node server:
Using pm2 without any config:
Using config suggested from #Naor Tedgi's answer:
Now the server is not running at all.

I got this
Probably it is OS related,I am on Ubuntu 20.04.

you don't have cluster mode enabled if you want to use pm2 as load balancer you need to add
exec_mode cluster
add this config file name it config.js
module.exports = {
apps : [{
script : "app.js",
instances : "max",
exec_mode : "cluster"
}]
}
and run pm2 start config.js
then all CPU usage will divide equally
tasted on
os macOS Catalina 10.15.7
node v14.15.4

Not sure why, but it seems that for whatever reason cluster doesn't behave on your machine the way it should.
In lieu of using node.js for balancing you can go for nginx then. On nginx part it's fairly easy if one of the available strategies is enough for you: http://nginx.org/en/docs/http/load_balancing.html
Then you need to make sure that your node processes are assigned different ports. In pm2 you can either use https://pm2.keymetrics.io/docs/usage/environment/ to either manually increment port based on the instance id or delegate it fully to pm2.
Needless to say, you'll have to send your requests to nginx in this case.

Related

How can I print Cypress base Url in the `Git bash CLI console`

How can I print Cypress base Url in the Git bash CLI console while running the Cypress test ? Could someone please advise ?
When running headless, the browser logs don't show in the terminal, but those from the Cypress node process (aka cy.task()) do show up.
I assume the baseUrl is changing during the test, here is an example that does that and logs the current value using a task.
The configured value is http://localhost:3000, the test changes it to http://localhost:3001.
cypress.config.js
const { defineConfig } = require('cypress')
module.exports = defineConfig({
e2e: {
setupNodeEvents(on, config) {
console.log('Printing baseUrl - during setup', config.baseUrl)
on('task', {
logBaseUrl(baseUrl) {
console.log('Printing baseUrl - from task', baseUrl)
return null
}
})
},
baseUrl: 'http://localhost:3000'
},
})
test
it('logs baseUrl to the terminal in run mode', () => {
console.log('Printing baseUrl - directly from test', Cypress.config('baseUrl')) // no show
Cypress.config('baseUrl', 'http://localhost:3001')
cy.task('logBaseUrl', Cypress.config('baseUrl'))
})
terminal
Printing baseUrl - during setup http://localhost:3000
====================================================================================================
Running: log-baseurl.cy.js (1 of 1)
Printing baseUrl - from task http://localhost:3001
√ logs baseUrl to the terminal in run mode (73ms)
1 passing (115ms)
(Results)
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Tests: 1 │
│ Passing: 1 │
│ Failing: 0 │
│ Pending: 0 │
│ Skipped: 0 │
│ Screenshots: 0 │
│ Video: true │
│ Duration: 0 seconds │
│ Spec Ran: log-baseurl.cy.js │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
====================================================================================================
(Run Finished)
Spec Tests Passing Failing Pending Skipped
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ √ log-baseurl.cy.js 108ms 1 1 - - - │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
√ All specs passed! 108ms 1 1 - - -
Done in 18.49s.
You can use a nodejs command console.log(Cypress.config().baseUrl).
That does not require Git Bash though, only nodejs installed on Your Windows.
Or you can add in your tests cy.config().baseUrl.

The package import path is different for dynamic codegen and static codegen

Here is the structure for src directory of my project:
.
├── config.ts
├── protos
│ ├── index.proto
│ ├── index.ts
│ ├── share
│ │ ├── topic.proto
│ │ ├── topic_pb.d.ts
│ │ ├── user.proto
│ │ └── user_pb.d.ts
│ ├── topic
│ │ ├── service.proto
│ │ ├── service_grpc_pb.d.ts
│ │ ├── service_pb.d.ts
│ │ ├── topic.integration.test.ts
│ │ ├── topic.proto
│ │ ├── topicServiceImpl.ts
│ │ ├── topicServiceImplDynamic.ts
│ │ └── topic_pb.d.ts
│ └── user
│ ├── service.proto
│ ├── service_grpc_pb.d.ts
│ ├── service_pb.d.ts
│ ├── user.proto
│ ├── userServiceImpl.ts
│ └── user_pb.d.ts
└── server.ts
share/user.proto:
syntax = "proto3";
package share;
message UserBase {
string loginname = 1;
string avatar_url = 2;
}
topic/topic.proto:
syntax = "proto3";
package topic;
import "share/user.proto";
enum Tab {
share = 0;
ask = 1;
good = 2;
job = 3;
}
message Topic {
string id = 1;
string author_id = 2;
Tab tab = 3;
string title = 4;
string content = 5;
share.UserBase author = 6;
bool good = 7;
bool top = 8;
int32 reply_count = 9;
int32 visit_count = 10;
string create_at = 11;
string last_reply_at = 12;
}
As you can see, I try to import share package and use UserBase message type in Topic message type. When I try to start the server, got error:
no such Type or Enum 'share.UserBase' in Type .topic.Topic
But when I changed the package import path to a relative path import "../share/user.proto";. It works fine and got server logs: Server is listening on http://localhost:3000.
Above is the usage of dynamic codegen.
Now, I switch to using static codegen, here is the shell script for generating the codes:
protoc \
--plugin=protoc-gen-ts=./node_modules/.bin/protoc-gen-ts \
--ts_out=./src/protos \
-I ./src/protos \
./src/protos/**/*.proto
It seems protocol buffer compiler doesn't support relative path, got error:
../share/user.proto: Backslashes, consecutive slashes, ".", or ".." are not allowed in the virtual path
And, I changed the the package import path back to import "share/user.proto";. It generated code correctly, but when I try to start my server, got same error:
no such Type or Enum 'share.UserBase' in Type .topic.Topic
It's weird.
Package versions:
"grpc-tools": "^1.6.6",
"grpc_tools_node_protoc_ts": "^4.1.3",
protoc --version
libprotoc 3.10.0
UPDATE:
repo: https://github.com/mrdulin/nodejs-grpc/tree/master/src
Your dynamic codegen is failing because you are not specifying the paths to search for imported .proto files. You can do this using the includeDirs option when calling protoLoader.loadSync, which works in a very similar way to the -I option you pass to protoc. In this case, you are loading the proto files from the src/protos directory, so it should be sufficient to pass the option includeDirs: [__dirname]. Then the import paths in your .proto files should be relative to that directory, just like when you use protoc.
You are probably seeing the same error when you try to use the static code generation because it is actually the dynamic codegen error; you don't appear to be removing the dynamic codegen code when trying to use the statically generated code.
However, the main problem you will face with the statically generated code is that you are only generating the TypeScript type definition files. You also need to generate JavaScript files to actually run it. The official Node gRPC plugin for proto is distributed in the grpc-tools package. It comes with a binary called grpc_tools_node_protoc, which should be used in place of protoc and automatically includes the plugin. You will still need to pass a --js_out flag to generate that code.

Getting error when running Postman Script via Newman

I am trying to run the following Postman Script via Newman to write the response to file:
//verify http response code
pm.test("Report Generated", function () {
pm.response.to.have.status(200);
});
var fs = require('fs');
var outputFilename = 'C:/Users/archit.goyal/Downloads/spaceReport.csv';
fs.writeFileSync(outputFilename, pm.response.text());
The request gives a response but getting the following error when writing to file:
1? TypeError in test-script
┌─────────────────────────┬──────────┬──────────┐
│ │ executed │ failed │
├─────────────────────────┼──────────┼──────────┤
│ iterations │ 1 │ 0 │
├─────────────────────────┼──────────┼──────────┤
│ requests │ 20 │ 0 │
├─────────────────────────┼──────────┼──────────┤
│ test-scripts │ 20 │ 1 │
├─────────────────────────┼──────────┼──────────┤
│ prerequest-scripts │ 0 │ 0 │
├─────────────────────────┼──────────┼──────────┤
│ assertions │ 2 │ 0 │
├─────────────────────────┴──────────┴──────────┤
│ total run duration: 1m 48.3s │
├───────────────────────────────────────────────┤
│ total data received: 1.24MB (approx) │
├───────────────────────────────────────────────┤
│ average response time: 5.3s │
└───────────────────────────────────────────────┘
# failure detail
1. TypeError fs.writeFileSync is not a function
at test-script
inside "3i_BMS_Amortization_Schedule / GetReport"
Please help
Postman itself cannot execute scripts like that. To save your responses of all the api requests, you can create a nodeJS server which will call the api through newman and then save the response to a local file. Here is an example -
var fs = require('fs'),
newman = require('newman'),
allResponse=[],
outputFilename = 'C:/Users/archit.goyal/Downloads/spaceReport.csv';
newman.run({
collection: '//your_collection_name.json',
iterationCount : 1
})
.on('request', function (err, args) {
if (!err) {
//console.log(args); // --> args contain ALL the data newman provides to this script.
var responseBody = args.response.stream,
response = responseBody.toString();
allResponse.push(JSON.parse(response));
}
})
.on('done', function (err, summary) {
fs.writeFile(outputFilename,"");
for(var i =0;i<allResponse.length;i++)
{
fs.appendFileSync(outputFilename,JSON.stringify(allResponse[i],null,5));
}
});
Note that, the above code will save only the responses. Other data like request , or URl can be extracted in a similar manner. To run this, install newman in the same directory as the script, then run the script using -
node file_name.js

How can we execute messages at a particular time daily in nodejs?

I want to execute the message daily at 10'o clock 5 minutes 25 seconds but I tried the program I got an error:
ERROR OCCURED:
throw patterns[5] + ' is a invalid expression for week day';
^
is a invalid expression for week day
Code:
var cron = require('node-cron');
cron.schedule('25 05 10 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31 January,February,March,April,May,June,July,August,September,October,November,December Monday,Tuesday,Wednesday,Thursday,Friday,Saturday,Sunday ', function(){
console.log('Time for breakfast');
});
The cron syntax is as described in the node-cron library page:
┌────────────── second (optional)
│ ┌──────────── minute
│ │ ┌────────── hour
│ │ │ ┌──────── day of month
│ │ │ │ ┌────── month
│ │ │ │ │ ┌──── day of week
│ │ │ │ │ │
│ │ │ │ │ │
* * * * * *
So, for executing the task every day at 10:05:25, try
25 5 10 * * *
cron.schedule('25 5 10 * * *', function() {
console.log('Time for breakfast');
});

how to set a variable automatically zero for each midnight in node js

how to set a variable automatically zero for each midnight in node js is it possible to use node-schedule-npm ? if so please help me how to do it
initially var count= 0; when action is performed, it will get incremented throughout the day, for next day it should be automatically set to zero.
You can use node-cron from npm.
$ npm install --save node-cron
Here is an example code
var cron = require('node-cron');
cron.schedule('* * * * *', function(){
console.log('running a task every minute');
});
# ┌────────────── second (optional)
# │ ┌──────────── minute
# │ │ ┌────────── hour
# │ │ │ ┌──────── day of month
# │ │ │ │ ┌────── month
# │ │ │ │ │ ┌──── day of week
# │ │ │ │ │ │
# │ │ │ │ │ │
# * * * * * *
you can try this
var cron = require('node-schedule');
cron.scheduleJob('00 00 12 * * 1-7',function setVotesToZero(req, res, next) {
db.users.find().toArray(function(err, vote) {
if (err) return next(err);
db.users.update({}, {
$set: {
count : 0
},
}, function(err, result) {
if (err) return next(err);
res.send({
status: 'success',
});
});
})
})

Resources