I am looking for a way (a npm module or a lib would be really useful) to run a javascript function in a new process. However, I don't want to define this function in a different file. I am looking for something like the POSIX fork mechanism.
how can I achieve this in node?
You can fork new processes using child_process.fork. It accepts a separate module, but if you must have it all contained in one file you could give your script a mode parameter:
var child_process = require('child_process');
var mode = process.argv[2] ? process.argv[2] : 'default';
var modes = {
default: function() {
console.log('I am the default, I will fork a child');
child_process.fork(__filename, ['child']);
},
child: function() {
console.log('I am the child!');
}
};
modes[mode]();
Related
I want to update my env variable in node js, but i am not able to update its env variable, i tried with console.log(process.env.DB_HOST) but still i am getting old value, here i have added my whole code, can anyone please look in to it, and help me to resolve this issue,
function exec_api() {
return new Promise(async function (resolve) {
const execSync = require('child_process').exec;
//let child_process_obj = execSync('DB_HOST='+process.env.UNITTEST_DB_HOST+' DB_DATABASE='+process.env.UNITTEST_DB_DATABASE+' DB_USERNAME='+process.env.UNITTEST_DB_USERNAME+' DB_PASSWORD='+process.env.UNITTEST_DB_PASSWORD+' PORT='+process.env.UNITTEST_SERVICE_PORT+' ./node_modules/.bin/nodemon main.js');
await execSync('export DB_HOST=' + process.env.UNITTEST_DB_HOST);
await execSync('export DB_DATABASE=' + process.env.UNITTEST_DB_DATABASE);
await execSync('export DB_USERNAME=' + process.env.UNITTEST_DB_USERNAME);
await execSync('export DB_PASSWORD=' + process.env.UNITTEST_DB_PASSWORD);
await execSync('export PORT=' + process.env.UNITTEST_API_BACKEND_PORT);
let child_process_obj = await execSync('node main.js');
unittest_api_backend_process_id = child_process_obj.pid;
resolve(true);
});
}
TLDR: Just change process.env
To change, add or delete environment variables, use process.env. The following is test code showing how this works:
In main.js:
console.log(process.env.DB_DATABASE);
In exec.js:
const execSync = require ('child_process').execSync;
process.env.DB_DATABASE = 'foo'; // this is ALL you need to do
console.log(execSync('node main.js').toString('utf8'));
With the two files above, if you run node exec.js you will see foo printed out in the console. This is printed from main.js which inherits the environment from exec.js.
So all you need to do in your code is:
I want to update my env variable in node js, but i am not able to update its env variable, i tried with console.log(process.env.DB_HOST) but still i am getting old value, here i have added my whole code, can anyone please look in to it, and help me to resolve this issue,
function exec_api() {
return new Promise(function (resolve) {
const exec = require('child_process').exec;
// The following is node.js equivalent of bash "export":
process.env.DB_HOST = process.env.UNITTEST_DB_HOST;
process.env.DB_DATABASE = process.env.UNITTEST_DB_DATABASE;
process.env.DB_USERNAME = process.env.UNITTEST_DB_USERNAME;
process.env.DB_PASSWORD = process.env.UNITTEST_DB_PASSWORD;
process.env.PORT = process.env.UNITTEST_SERVICE_PORT;
let child_process_obj = exec('node main.js', {
stdio: ['inherit', 'inherit', 'inherit']
});
unittest_api_backend_process_id = child_process_obj.pid;
resolve(true);
});
}
Note that if you want the promise to return when the main.js ends you need to do:
function exec_api() {
return new Promise(function (resolve) {
const exec = require('child_process').exec;
// The following is node.js equivalent of bash "export":
process.env.DB_HOST = process.env.UNITTEST_DB_HOST;
process.env.DB_DATABASE = process.env.UNITTEST_DB_DATABASE;
process.env.DB_USERNAME = process.env.UNITTEST_DB_USERNAME;
process.env.DB_PASSWORD = process.env.UNITTEST_DB_PASSWORD;
process.env.PORT = process.env.UNITTEST_SERVICE_PORT;
let child_process_obj = exec('node main.js', {
stdio: ['inherit', 'inherit', 'inherit']
});
unittest_api_backend_process_id = child_process_obj.pid;
child_process_obj.on('exit', () => resolve(true));
// ^^^ Cannot use `await` as the API is not promise based
// but event based instead.
});
}
Long story: The full explanation of why export doesn't work
On unixen, environment variables, and indeed, the entire environment including current working directory, root directory (which can be changed via chroot) etc. are not features of shells. They are features of processes.
We may be familiar with the export syntax of some shells to set environment variables for child processes but that is the shell's syntax. It has nothing to do with environment variables themselves. C/C++ for example don't use export instead uses the setenv() function do set environment variables (indeed, internally that's what bash/sh/ksh etc do when implementing export).
In node.js, the mechanism for reading and setting environment variables is via process.env.
Why asking a shell to do it don't work
This is not merely a node.js issue. It also won't work in bash:
In exporter.sh:
#! /bin/bash
export DB_DATABASE=$1
In exec.sh:
#! /bin/bash
./exporter.sh foo
echo $DB_DATABASE ;# does not print "foo"
This is a core security feature of unixen: other users should not be allowed to mess with your process. The way this policy is enforced in the case of the environment is that only a parent process can set the environment of the child process. A child process is not allowed to set the environment of the parent process. The assumption is that the child process belongs to the parent process so you should be allowed to do what you want to a program - but since the parent process (you) don't belong to the child process the child is not allowed to mess with the parent's environment.
That's why your attempt to use export doesn't work. It actually works (the variables are indeed created in the subshell) but is not allowed to change the environment of it's parent (the node.js process)
When you use export in a terminal, it instructs the shell to set environment variables.
When you call exec from your code, you are not running such a shell, with the reason being that it would become a challenge to extract the output of every command.
This makes export an ignored command.
You can solve this by passing an option object to execSync:
execSync('node main.js', {
env: {
DB_HOST: 'localhost',
// More envs...
}
}
I have read the details on NodeJS site : https://nodejs.org/api/modules.html. I don't understand how modules work, and what are the minimal steps for creating a module, and how npm can help me.
How can I create a module?
How do I use a module?
What does putting it on npm mean?
Note: this is a self answered question, with the purpose of sharing knowledge as a canonical.
You can create a NodeJS module using one line of code:
//mymodule.js
module.exports = 3;
Then you can load the module, by using require:
//app.js
require('./mymodule.js')
I added './' because it is a module of one file. We will cover it later.
Now if you do for example:
var mymodule = require('./mymodule.js');
console.log(mymodule); // 3
You can replace the number 3, with a function, for example:
//mymodule.js:
module.exports = function () {
console.log('function inside the module');
};
Then you can use it:
var mymodule = require('./mymodule.js');
mymodule();
Private variables:
Every variable you define inside A module will be defined only inside it:
//mymodule.js
var myPrivateVariable = 3;
publicVariable = 5; // Never user global variables in modules
//It's bad-pracrtice. Always add: var.
module.exports = function() {
// Every function of the module can use the private variables
return myPrivateVariable++
};
//app.js
var mymodule = require('./mymodule.js');
console.log(mymodule()); // return 3
console.log(mymodule()); // return 4
Reuse modules:
One more thing you need to know about NodeJS modules, is that if you use the same module twice(require it), it will return the same instance, it will not run in twice.
for example:
//app.js
var mymodule1 = require('./mymodule.js');
var mymodule2 = require('./mymodule.js');
console.log(mymodule1()); //return 3
console.log(mymodule2()); //return 4 (not 3)
console.log(mymodule1()); //return 5
As you see in the example below, that private variable is shared between all the instances of the module.
A module package
If your module contain more than one file, or you want to share the module with others, you have to create the module in separate folder, and create a package.json file for the module.
npm init will create package.json file for you.
For modules, there are 3 required parts:
package.json
{
"name" : "You module name",
"version" : "0.0.3"
}
Now, you can publish the module, using npm publish. I recommend you publish all your modules to github as well, then the module will be connected to your github page.
What you publish to NPM will be accessible by everyone. So never publish modules that contain private data. For that you can use private npm modules.
Next steps
Modules can return more than one function or one variable. See this samples in which we return an object.
module.exports.a = function() {
// ..
};
module.exports.b = function() {
// ..
};
// OR
myObj = {
a:3,
b:function() {
return this.a;
}
};
module.exports = myObj;
More info:
Read about package.json files
Versioning in you modules best practice
More best practive for NodeJS modules
Private modules, using private npm
Related Questions:
What is the purpose of Node.js module.exports and how do you use it?
module.exports vs exports in Node.js
Creating module in node.js is pretty simple!!!
You may consider module as a set of functionalities you can use in other code by simply just requiring it.
for eg:Consider a file functional.js having the content:
function display(){
console.log('i am in a display function');
}
module.exports = display;
Now just require it in any other module like:
var display = require('./functional');
display()
Output:i am in a display function
Similarly you can do:
var exports = module.exports = {};
exports.display = function(){
console.log('i am in the display function');
}
or you do the same for objects like:
var funObj = {
hello:function(){
console.log('hello function');
},
display:function(){
console.log('display function');
}
};
module.exports = funObj;
There are two main ways for wiring modules. One of them is using hard coded dependencies, explicitly loading one module into another using a require call. The other method is to use a dependency injection pattern, where we pass the components as a parameter or we have a global container (known as IoC, or Inversion of Control container), which centralizes the management of the modules.
We can allow Node.js to manage the modules life cycle by using hard coded module loading. It organizes your packages in an intuitive way, which makes understanding and debugging easy.
Dependency Injection is rarely used in a Node.js environment, although it is a useful concept. The DI pattern can result in an improved decoupling of the modules. Instead of explicitly defining dependencies for a module, they are received from the outside. Therefore they can be easily replaced with modules having the same interfaces.
Let’s see an example for DI modules using the factory pattern:
class Car {
constructor (options) {
this.engine = options.engine
}
start () {
this.engine.start()
}
}
function create (options) {
return new Car(options)
}
module.exports = create
I'm trying to create a Node script that will ask the user a few questions, save the input and using the answers run a mvn archetype:generate command to install their environment.
I got as far to where I get the Maven command running. But when Maven asks for user input for values such as groupId and `` I can enter the values, give an [enter] and that's where it stops.
It doesn't take input and process them. All it does is display it, as the CLI does, but doesn't accept them.
Here's a snippet of code with the values for user input pre-filled:
var spawn = require('child_process').spawn;
var answerCollection = {
"name": "nameOfMyArchetype", //answer of inquiry
"version": "1.2.3.4" //answer of inquiry
};
var cmd = "mvn";
var args = [
"archetype:generate",
"-DarchetypeArtifactId=" + answerCollection.name,
"-DarchetypeGroupId=com.backbase.expert.tools",
"-DarchetypeVersion=" + answerCollection.version
];
var runCmd = function(cmd, args, callback) {
var child = spawn(cmd, args);
child.stdin.pipe(process.stdin);
child.stdout.pipe(process.stdout);
child.stdout.on('end', function(res) {
console.log("stdout:end");
callback(res);
});
child.stderr.on('data', function(text) {
console.log("stderr:data");
console.log(data);
});
child.stderr.on('exit', function(data) {
console.log("stderr:exit");
console.log(data);
});
};
So far I've tried the above code with child_process and spawn = require('child_process').spawn('bash').
Question: Is there any other way to make sure I can trigger a script and if that returns with a prompt and asks for input I can type and enter and the script will continue?
From Facebook I got this tip to use cross-spawn, instead of child_process:
From Robert Haritonov:
Use cross-spawn: spawn('bower', bowerCommand, {stdio:'inherit'}).on('close', function () {});
This works perfectly well and provides exactly the behaviour I need.
I have the following NodeJS code:
var spawn = require('child_process').spawn;
var Unzipper = {
unzip: function(src, dest, callback) {
var self = this;
if (!fs.existsSync(dest)) {
fs.mkdir(dest);
}
var unzip = spawn('unzip', [ src, '-d', dest ]);
unzip.stdout.on('data', function (data) {
self.stdout(data);
});
unzip.stderr.on('data', function (data) {
self.stderr(data);
callback({message: "There was an error executing an unzip process"});
});
unzip.on('close', function() {
callback();
});
}
};
I have a NodeUnit test that executes successfully. Using phpStorm to debug the test the var unzip is assigned correctly
However if I run the same code as part of a web service, the spawn call doesn't return properly and the server crashes on trying to attach an on handler to the nonexistent stdout property of the unzip var.
I've tried running the program outside of phpStorm, however it crashes on the command line as well for the same reason. I'm suspecting it's a permissions issue that the tests don't have to deal with. A web server spawning processes could cause chaos in a production environment, therefore some extra permissions might be needed, but I haven't been able to find (or I've missed) documentation to support my hypothesis.
I'm running v0.10.3 on OSX Snow Leopard (via MacPorts).
Why can't I spawn the child process correctly?
UPDATES
For #jonathan-wiepert
I'm using Prototypical inheritance so when I create an "instance" of Unzipper I set stdout and stderr ie:
var unzipper = Unzipper.spawn({
stdout: function(data) { util.puts(data); },
stderr: function(data) { util.puts(data); }
});
This is similar to the concept of "constructor injection". As for your other points, thanks for the tips.
The error I'm getting is:
project/src/Unzipper.js:15
unzip.stdout.on('data', function (data) {
^
TypeError: Cannot call method 'on' of undefined
As per my debugging screenshots, the object that is returned from the spawn call is different under different circumstances. My test passes (it checks that a ZIP can be unzipped correctly) so the problem occurs when running this code as a web service.
The problem was that the spawn method created on the Object prototype (see this article on Protypical inheritance) was causing the child_process.spawn function to be replaced, so the wrong function was being called.
I saved child_process.spawn into a property on the Unzipper "class" before it gets clobbered and use that property instead.
Let's say I have a file "/tmp/sample.txt" and I want to move it to "/var/www/mysite/sample.txt" which is in a different volume.
How can i move the file in node.js?
I read that fs.rename only works inside the same volume and util.pump is already deprecated.
What is the proper way to do it? I read about stream.pipe, but I couldn't get it to work. A simple sample code would be very helpful.
Use the mv module:
var mv = require('mv');
mv('source', 'dest', function(err) {
// handle the error
});
If on Windows and don't have 'mv' module, can we do like
var fs = require("fs"),
source = fs.createReadStream("c:/sample.txt"),
destination = fs.createWriteStream("d:/sample.txt");
source.pipe(destination, { end: false });
source.on("end", function(){
fs.unlinkSync("C:/move.txt");
});
The mv module, like jbowes stated, is probably the right way to go, but you can use the child process API and use the built-in OS tools as an alternative. If you're in Linux use the "mv" command. If you're in Windows, use the "move" command.
var exec = require('child_process').exec;
exec('mv /temp/sample.txt /var/www/mysite/sample.txt',
function(err, stdout, stderr) {
// stdout is a string containing the output of the command.
});
You can also use spawn if exec doesn't work properly.
var spawn = require("child_process").spawn;
var child = spawn("mv", ["data.csv","./done/"]);
child.stdout.on("end", function () {
return next(null,"finished")
});
Hope this helps you out.