Redirecting stdout to file nodejs - node.js

I've created:
var access = fs.createWriteStream('/var/log/node/api.access.log', { flags: 'w' });
Then piped:
process.stdout.pipe(access);
Then tried:
console.log("test");
And nothing has appeared in /var/log/node/api.access.log. However this way is working:
process.stdout.pipe(access).write('test');
Could someone explain what am I doing wrong ?

I solved this problem the following way:
var access = fs.createWriteStream('/var/log/node/api.access.log');
process.stdout.write = process.stderr.write = access.write.bind(access);
Of course you can also separate stdout and stderr if you want.
I also would strongly recommend to handle uncaught exceptions:
process.on('uncaughtException', function(err) {
console.error((err && err.stack) ? err.stack : err);
});
This will cover the following situations:
process.stdout.write
process.stderr.write
console.log
console.dir
console.error
someStream.pipe(process.stdout);
throw new Error('Crash');
throw 'never do this';
throw undefined;

Checkout console.Console, the parent class of the normal console.
var myLogFileStream = fs.createWriteStream(pathToMyLogFile);
var myConsole = new console.Console(myLogFileStream, myLogFileStream);
You can then you use myConsole.log, myConsole.error, myConsole.dir, etc. and write directly to your file.
You can also monkey patch process.stdout.write as follows:
var fn = process.stdout.write;
function write() {
fn.apply(process.stdout, arguments);
myLogFileStream.write.apply(myLogFileStream, arguments);
}
process.stdout.write = write;
there are also other options for overwriting console._stdout depending on the motivation for logging the stdout to a file.

process.stdout is a Writable. pipe is a method of Readable(Cf StreamAPI documentation : https://nodejs.org/api/stream.html
You can see the documentation of process.stdout here : https://nodejs.org/api/process.html#process_process_stdout
It's surprising that you can do process.stdout.pipe(...); without any error. But i suppose this call just do nothing. Except returning a new Writable stream binded to stdout (or maybe it returns process.stdout itself. There's no specification for that in the documentation).
If you want to redirect stdout to a file, you have many solutions :
Just use your command line to do that. Windows style : node myfile.js > api.access.log.
Replace the console object by your own object. And you can rewrite console methods.
I'm not sure, but it may be possible to replace process.stdout with your own stream (and you can do whatever you want with this)

#user3173842
for the reply on
I solved this problem the following way:
var access = fs.createWriteStream('/var/log/node/api.access.log');
process.stdout.write = process.stderr.write = access.write.bind(access);
you do understand that process.stdout continues after process.on('exit') and therefore the fs.WriteStream closes after with process.stdout, according to
https://github.com/nodejs/node/issues/7606
so now the question remains, if the developer desired to have the fs.Writestream.write() return to its normal functionality and when fs.Writestream.end is called the writestream closes. How would the developer go about doing this I did
a_l = asyncify_listener
p_std_stream_m is a process stream manager object
p_std_stream_m.std_info.p_stdout_write = process.stdout.write
process.stdout.write = w_stream.write.bind(w_stream)
process.once('beforeExit', a_l( p_std_stream_m.handler,process.stdout,w_stream ) )
where in the 'beforeExit' event listener I did
process.stdout.write = p_std_stream_m.std_info.p_stdout_write
w_stream.end()
It works and you use the once method because the process.stdout seems to do a lot of work
at this time.
Is this good practice, would you do this or what would you do in this situation
anyone can feel free to reply.

Originally based on #Anatol-user3173842 answer
But in my case I needed to hook the stdout & stderr and also write into a file.
So for those who need to keep the normal stdout behaviour in addition to writing into the file. Use the following.
For non-errors:
// stdout logging hook
const stdoutWrite0 = process.stdout.write;
process.stdout.write = (args) => { // On stdout write
CustomLogger.writeToLogFile('log', args); // Write to local log file
args = Array.isArray(args) ? args : [args]; // Pass only as array to prevent internal TypeError for arguments
return stdoutWrite0.apply(process.stdout, args);
};
For errors:
// stderr logging hook
const stderrWrite0 = process.stderr.write;
process.stderr.write = (args) => { // On stderr write
CustomLogger.writeToLogFile('error', args); // Write to local error file
args = Array.isArray(args) ? args : [args]; // Pass only as array to prevent internal TypeError for arguments
return stderrWrite0.apply(process.stderr, args);
};
// uncaught exceptions
process.on('uncaughtException', (err) => {
CustomLogger.writeToLogFile('error', ((err && err.stack) ? err.stack : err));
});
Here is the CustomLogger code, where I also separate the log files by date:
export class CustomLogger {
static LOGS_DIR = 'location-of-my-log-files';
private static logDailyName(prefix: string): string {
const date = new Date().toLocaleDateString().replace(/\//g, '_');
return `${CustomLogger.LOGS_DIR}/${prefix}_${date}.log`;
}
private static writeToLogFile(prefix, originalMsg) {
const timestamp = Date.now();
const fileName = this.logDailyName(prefix);
const logMsg = prepareForLogFile(originalMsg);
fs.appendFileSync(fileName, `${timestamp}\t${logMsg}\n\n`);
return originalMsg;
}
}

Here's a quick example of a logger class that redirects stdout, stderr and exceptions to a file, while still writting everything to the console:
class Logger {
#log_stream
#stdout_write
#stderr_write
constructor(path) {
this.#log_stream = fs.createWriteStream(path, { flags: 'a' })
this.#stdout_write = process.stdout.write.bind(process.stdout)
this.#stderr_write = process.stderr.write.bind(process.stderr)
process.stdout.write = this.stdout_write.bind(this)
process.stderr.write = this.stderr_write.bind(this)
process.on('uncaughtException', function(err) {
console.error((err && err.stack) ? err.stack : err)
})
}
stdout_write(buffer) {
this.#log_stream.write(buffer)
this.#stdout_write(buffer)
}
stderr_write(buffer) {
this.#log_stream.write(buffer)
this.#stderr_write(buffer)
}
}
const logger = new Logger('example.log')

Related

Send NodeJS child process stdout to Express response object

I currently pipe the stdout of a child process to an (Express) response object like this:
try{
let proc = spawn(exe, args);
proc.stdout.pipe(response);
let exitCode = await ending.completion(proc);
}
finally {
// my own clean up goes here (e.g. releasing locks)
}
where ending.completion is defined as
async function(process){
return new Promise(function(resolve,reject){
let done = false;
process.on('exit', (code, signal)=>{
if(!done){
done = true;
if(code !== null){
resolve(code);
} else {
reject(signal);
}
}
});
process.on('error', (err)=>{
if(!done){
done = true;
reject(err);
}
});
});
};
This generally works fine except for the fact that proc internally does a lot of waiting (on the download/transfer) before it can complete/return. I would prefer allowing it to dump its output into a buffer, which then trickles into the response object as the download proceeds. How can I introduce such a buffer?
(A second (and I think related) problem is that if the download is aborted client-side, proc never completes but keeps waiting on its stdout.)
Looks like exec method does exactly what you need.
Spawns a shell then executes the command within that shell, buffering
any generated output.
Use it instead of spawn. Note that unlike spawn, exec expects the whole command (with all the args) as a single string. Here is example from an officials documentation:
const util = require('util');
const exec = util.promisify(require('child_process').exec;
async function lsExample() {
const { stdout, stderr } = await exec('ls');
console.log('stdout:', stdout);
console.error('stderr:', stderr);
}
lsExample();

How to get console.log line numbers shown in Nodejs?

Got an old application, that prints out quite a lot of messages using console.log, but I just can not find in which files and lines console.log is called.
Is there a way to hook into the app and show file name and line numbers?
Having full stack trace for each call is a bit noisy. I've just improved the #noppa's solution to print only the initiator:
['log', 'warn', 'error'].forEach((methodName) => {
const originalMethod = console[methodName];
console[methodName] = (...args) => {
let initiator = 'unknown place';
try {
throw new Error();
} catch (e) {
if (typeof e.stack === 'string') {
let isFirst = true;
for (const line of e.stack.split('\n')) {
const matches = line.match(/^\s+at\s+(.*)/);
if (matches) {
if (!isFirst) { // first line - current function
// second line - caller (what we are looking for)
initiator = matches[1];
break;
}
isFirst = false;
}
}
}
}
originalMethod.apply(console, [...args, '\n', ` at ${initiator}`]);
};
});
It also patches other methods (useful for Nodejs, since warn and error don't come with a stack trace as in Chrome).
So your console would look something like:
Loading settings.json
at fs.readdirSync.filter.forEach (.../settings.js:21:13)
Server is running on http://localhost:3000 or http://127.0.0.1:3000
at Server.app.listen (.../index.js:67:11)
For a temporary hack to find the log statements that you want to get rid of, it's not too difficult to override console.log yourself.
var log = console.log;
console.log = function() {
log.apply(console, arguments);
// Print the stack trace
console.trace();
};
// Somewhere else...
function foo(){
console.log('Foobar');
}
foo();
That will print something like
Foobar
Trace
at Console.console.log (index.js:4:13)
at foo (index.js:10:13)
at Object.<anonymous> (index.js:12:1)
...
A lot of noise in there but the second line in the call stack, at foo (index.js:10:13), should point you to the right place.
All solutions to this question so far rely on splitting and matching the stack trace as a string, which will break in (the unlikely) case the format of that string is changed in the future. Inspired by this gist on GitHub and the other answers here, I want to provide my own solution:
'use strict';
const path = require('path');
['debug', 'log', 'warn', 'error'].forEach((methodName) => {
const originalLoggingMethod = console[methodName];
console[methodName] = (firstArgument, ...otherArguments) => {
const originalPrepareStackTrace = Error.prepareStackTrace;
Error.prepareStackTrace = (_, stack) => stack;
const callee = new Error().stack[1];
Error.prepareStackTrace = originalPrepareStackTrace;
const relativeFileName = path.relative(process.cwd(), callee.getFileName());
const prefix = `${relativeFileName}:${callee.getLineNumber()}:`;
if (typeof firstArgument === 'string') {
originalLoggingMethod(prefix + ' ' + firstArgument, ...otherArguments);
} else {
originalLoggingMethod(prefix, firstArgument, ...otherArguments);
}
};
});
// Tests:
console.log('%s %d', 'hi', 42);
console.log({ a: 'foo', b: 'bar'});
Unlike the other solutions, this script
outputs no additional lines and
handles string substitutions correctly.
You can color the prefix with chalk or color.js, but I didn't want to introduce dependencies for this here.
The above script uses the V8 API to customize stack traces. The callee is a CallSite object with the following methods in case you want to customize the prefix:
getThis: returns the value of this
getTypeName: returns the type of this as a string. This is the name of the function stored in the constructor field of this, if available, otherwise the object’s [[Class]] internal property.
getFunction: returns the current function
getFunctionName: returns the name of the current function, typically its name property. If a name property is not available an attempt is made to infer a name from the function’s context.
getMethodName: returns the name of the property of this or one of its prototypes that holds the current function
getFileName: if this function was defined in a script returns the name of the script
getLineNumber: if this function was defined in a script returns the current line number
getColumnNumber: if this function was defined in a script returns the current column number
getEvalOrigin: if this function was created using a call to eval returns a string representing the location where eval was called
isToplevel: is this a top-level invocation, that is, is this the global object?
isEval: does this call take place in code defined by a call to eval?
isNative: is this call in native V8 code?
isConstructor: is this a constructor call?
isAsync: is this an async call (i.e. await or Promise.all())?
isPromiseAll: is this an async call to Promise.all()?
getPromiseIndex: returns the index of the promise element that was followed in Promise.all() for async stack traces, or null if the CallSite is not a Promise.all() call.
This answer is a cross-post of an answer I just gave to a similar question as more people might find this page.
I found Dmitry Druganov's answer really nice, but I tried it on Windows 10 (with Node 8.9.4) and it didn't work well. It was printing the full path, something like:
Loading settings.json
at fs.readdirSync.filter.forEach (D:\Users\Piyin\Projects\test\settings.js:21:13)
Server is running on http://localhost:3000 or http://127.0.0.1:3000
at Server.app.listen (D:\Users\Piyin\Projects\test\index.js:67:11)
So I took said answer and made these improvements (from my point of view):
Assume the important line of the stack trace is the third one (the first one is the word Error and the second one is where you place this script)
Remove the current script folder path (given by __dirname, which in my case is D:\Users\Piyin\Projects\test). Note: For this to work well, the script should be on the project's main Javascript
Remove the starting at
Place the file information before the actual log
Format the information as Class.method at path/to/file:line:column
Here it is:
['log','warn','error'].forEach((methodName) => {
const originalMethod = console[methodName];
console[methodName] = (...args) => {
try {
throw new Error();
} catch (error) {
originalMethod.apply(
console,
[
(
error
.stack // Grabs the stack trace
.split('\n')[2] // Grabs third line
.trim() // Removes spaces
.substring(3) // Removes three first characters ("at ")
.replace(__dirname, '') // Removes script folder path
.replace(/\s\(./, ' at ') // Removes first parentheses and replaces it with " at "
.replace(/\)/, '') // Removes last parentheses
),
'\n',
...args
]
);
}
};
});
And here's the new output:
fs.readdirSync.filter.forEach at settings.js:21:13
Loading settings.json
Server.app.listen at index.js:67:11
Server is running on http://localhost:3000 or http://127.0.0.1:3000
Here's the minified-by-hand code (240 bytes):
['log','warn','error'].forEach(a=>{let b=console[a];console[a]=(...c)=>{try{throw new Error}catch(d){b.apply(console,[d.stack.split('\n')[2].trim().substring(3).replace(__dirname,'').replace(/\s\(./,' at ').replace(/\)/,''),'\n',...c])}}});
Slightly modified version of noppa's answer, this version will output something like:
/file/in-which/console/is/called.js:75:23
The stuff you want to log.
This is clean and convenient (especially for use in VSCode - which will turn the file path into a link).
const { log } = console;
function proxiedLog(...args) {
const line = (((new Error('log'))
.stack.split('\n')[2] || '…')
.match(/\(([^)]+)\)/) || [, 'not found'])[1];
log.call(console, `${line}\n`, ...args);
}
console.info = proxiedLog;
console.log = proxiedLog;
// test
console.log('Hello!');
The snippet will only work well in a NodeJS environment…
Appends the line number to the end of the log
const stackTrace = function () {
let obj = {}
Error.captureStackTrace(obj, stackTrace)
return obj.stack
}
const getLine = function (stack) {
let matchResult = stack.match(/\(.*?\)|\s.+/g) || []
let arr = matchResult.map((it) => {
return it.split(' ').pop().replace(/\(|\)/g, '')
})
return arr[1] ?? ''
}
const log = function (...args) {
let stack = stackTrace() || ''
let matchResult = getLine(stack)
let line = matchResult
for (var i in arguments) {
if (typeof arguments[i] == 'object') {
// util.inspect(arguments[i], false, 2, false)
arguments[i] = JSON.stringify(arguments[i])
}
}
arguments[i] += ' ' + line
console.log.apply(console, arguments)
}
log("test")
Simple & exhaustive solution if you want to temporarily find the origin of logs:
{
const logOriginal = process.stdout.write
// #ts-ignore
const log = (msg) => logOriginal.call(process.stdout, msg + '\n')
;['stdout', 'stderr'].forEach((stdName) => {
// #ts-ignore
var methodOriginal = process[stdName].write
// #ts-ignore
process[stdName].write = function (...args) {
log('LOG:')
// #ts-ignore
methodOriginal.apply(process[stdName], args)
// #ts-ignore
log(new Error().stack.replace(/^Error/, 'LOGGED FROM:'))
}
})
Error.stackTraceLimit = Infinity
}
const showName = (name) => {
return
console.log(name)
}
showName(“Crush”)

Retrieve stdout to variable

I’m trying to run child process in next code:
run = function (cmd, callback) {
var spawn = require('child_process').spawn;
var command = spawn(cmd);
var result = '';
command.stdout.on('data', function (data) {
result += data.toString();
});
command.on('exit', function () {
callback(result);
});
}
execQuery = function (cmd) {
var result = {
errnum: 0,
error: 'No errors.',
body: ''
};
run(cmd, function (message) {
result.body = message;
console.log(message);
});
return result;
}
After execution execQuery('ls') result.body is always empty, but console.log is contain value.
I ran a quick test and the command's exit event is firing before all of stdouts data is drained. I at least got the output captured and printed if I changed your exit handler to look for command.stdout's end event.
command.stdout.on('end', function () {
callback(result);
});
That should help a bit. Note there are existing libraries you might want to use for this and a truly correct implementation would be significantly more involved than what you have, but my change should address your current roadblock problem.
Random tip: it is the node convention to always reserve the first argument of callback functions for an error and your snippet is inconsistent with that convention. You probably should adjust to match the convention.
Oh sorry, let me address your question about result.body. The run function is ASYNCHRONOUS! That means that your return result; line of code executes BEFORE the run callback body where result.body = message; is. You can't use return values like that anywhere in node when you have I/O involved. You have to use a callback.

Node.js synchronous prompt

I'm using the prompt library for Node.js and I have this code:
var fs = require('fs'),
prompt = require('prompt'),
toCreate = toCreate.toLowerCase(),
stats = fs.lstatSync('./' + toCreate);
if(stats.isDirectory()){
prompt.start();
var property = {
name: 'yesno',
message: 'Directory esistente vuoi continuare lo stesso? (y/n)',
validator: /y[es]*|n[o]?/,
warning: 'Must respond yes or no',
default: 'no'
};
prompt.get(property, function(err, result) {
if(result === 'no'){
console.log('Annullato!');
process.exit(0);
}
});
}
console.log("creating ", toCreate);
console.log('\nAll done, exiting'.green.inverse);
If the prompt is show it seems that it doesn't block code execution but the execution continues and the last two messages by the console are shown while I still have to answer the question.
Is there a way to make it blocking?
With flatiron's prompt library, unfortunately, there is no way to have the code blocking. However, I might suggest my own sync-prompt library. Like the name implies, it allows you to synchronously prompt users for input.
With it, you'd simply issue a function call, and get back the user's command line input:
var prompt = require('sync-prompt').prompt;
var name = prompt('What is your name? ');
// User enters "Mike".
console.log('Hello, ' + name + '!');
// -> Hello, Mike!
var hidden = true;
var password = prompt('Password: ', hidden);
// User enters a password, but nothing will be written to the screen.
So give it a try, if you'd like.
Bear in mind: DO NOT use this on web applications. It should only be used on command line applications.
Update: DO NOT USE THIS LIBRARY AT ALL. IT IS A TOTAL JOKE, TO BE PERFECTLY FRANK.
Since Node.js 8, you can do the following using async/await:
const readline = require('readline');
const rl = readline.createInterface({
input: process.stdin,
output: process.stdout
});
function readLineAsync(message) {
return new Promise((resolve, reject) => {
rl.question(message, (answer) => {
resolve(answer);
});
});
}
// Leverages Node.js' awesome async/await functionality
async function demoSynchronousPrompt() {
var promptInput = await readLineAsync("Give me some input >");
console.log("Won't be executed until promptInput is received", promptInput);
rl.close();
}
Since IO in Node doesn't block, you're not going to find an easy way to make something like this synchronous. Instead, you should move the code into the callback:
...
prompt.get(property, function (err, result) {
if(result === 'no'){
console.log('Annullato!');
process.exit(0);
}
console.log("creating ", toCreate);
console.log('\nAll done, exiting'.green.inverse);
});
or else extract it and call the extracted function:
...
prompt.get(property, function (err, result) {
if(result === 'no'){
console.log('Annullato!');
process.exit(0);
} else {
doCreate();
}
});
...
function doCreate() {
console.log("creating ", toCreate);
console.log('\nAll done, exiting'.green.inverse);
}
Old question, I know, but I just found the perfect tool for this. readline-sync gives you a synchronous way to collect user input in a node script.
It's dead simple to use and it doesn't require any dependencies (I couldn't use sync-prompt because of gyp issues).
From the github readme:
var readlineSync = require('readline-sync');
// Wait for user's response.
var userName = readlineSync.question('May I have your name? ');
console.log('Hi ' + userName + '!');
I'm not affiliated with the project in any way, but it just made my day, so I had to share.
I've come across this thread and all the solutions either:
Don't actually provide a syncronous prompt solution
Are outdated and don't work with new versions of node.
And for that reason I have created syncprompt
. Install it with npm i --save syncprompt and then just add:
var prompt = require('syncprompt');
For example, this will allow you to do:
var name = prompt("Please enter your name? ");
It also supports prompting for passwords:
var topSecretPassword = prompt("Please enter password: ", true);
Vorpal.js is a library I made that has just recently been released. It provides synchronous command execution with an interactive prompt, like you are asking. The below code will do what you are asking:
var vorpal = require('vorpal')();
vorpal.command('do sync')
.action(function (args) {
return 'i have done sync';
});
With the above, the prompt will come back after a second is up (only after callback is called).
This is dependency free, synchronous and works on Windows, Linux and OSX:
// Synchronously prompt for input
function prompt(message)
{
// Write message
process.stdout.write(message);
// Work out shell command to prompt for a string and echo it to stdout
let cmd;
let args;
if (os.platform() == "win32")
{
cmd = 'cmd';
args = [ '/V:ON', '/C', 'set /p response= && echo !response!' ];
}
else
{
cmd = 'bash';
args = [ '-c', 'read response; echo "$response"' ];
}
// Pipe stdout back to self so we can read the echoed value
let opts = {
stdio: [ 'inherit', 'pipe', 'inherit' ],
shell: false,
};
// Run it
return child_process.spawnSync(cmd, args, opts).stdout.toString().trim();
}
const buffer = Buffer.alloc(1024);
require("fs").readSync(process.stdin.fd, buffer);
console.log(buffer.toString());
You can use prompt-sync
const prompt = require('prompt-sync')()
const ans = prompt('How many more times? ') // get input from the user.
P.S. prompt-sync acts weird, if prompt message contains new line character, so if you need multiline prompt just use console.log():
const prompt = require('prompt-sync')()
console.log('How many more times?\n')
const ans = prompt('') // get input from the user.

Catching console.log in node.js?

Is there a way that I can catch eventual console output caused by console.log(...) within node.js to prevent cloggering the terminal whilst unit testing a module?
Thanks
A better way could be to directly hook up the output you to need to catch data of, because with Linus method if some module write directly to stdout with process.stdout.write('foo') for example, it wont be caught.
var logs = [],
hook_stream = function(_stream, fn) {
// Reference default write method
var old_write = _stream.write;
// _stream now write with our shiny function
_stream.write = fn;
return function() {
// reset to the default write method
_stream.write = old_write;
};
},
// hook up standard output
unhook_stdout = hook_stream(process.stdout, function(string, encoding, fd) {
logs.push(string);
});
// goes to our custom write method
console.log('foo');
console.log('bar');
unhook_stdout();
console.log('Not hooked anymore.');
// Now do what you want with logs stored by the hook
logs.forEach(function(_log) {
console.log('logged: ' + _log);
});
EDIT
console.log() ends its output with a newline, you may want to strip it so you'd better write:
_stream.write = function(string, encoding, fd) {
var new_str = string.replace(/\n$/, '');
fn(new_str, encoding, fd);
};
EDIT
Improved, generic way to do this on any method of any object with async support See the gist.
module.js:
module.exports = function() {
console.log("foo");
}
program:
console.log = function() {};
mod = require("./module");
mod();
// Look ma no output!
Edit: Obviously you can collect the log messages for later if you wish:
var log = [];
console.log = function() {
log.push([].slice.call(arguments));
};
capture-console solves this problem nicely.
var capcon = require('capture-console');
var stderr = capcon.captureStderr(function scope() {
// whatever is done in here has stderr captured,
// the return value is a string containing stderr
});
var stdout = capcon.captureStdout(function scope() {
// whatever is done in here has stdout captured,
// the return value is a string containing stdout
});
and later
Intercepting
You should be aware that all capture functions will still pass the values through to the main stdio write() functions, so logging will still go to your standard IO devices.
If this is not desirable, you can use the intercept functions. These functions are literally s/capture/intercept when compared to those shown above, and the only difference is that calls aren't forwarded through to the base implementation.
Simply add the following snippet to your code will let you catch the logs and still print it in the console:
var log = [];
console.log = function(d) {
log.push(d);
process.stdout.write(d + '\n');
};

Resources