I am trying to use Pino with Sequelize's options.logging:
A function that gets executed every time Sequelize would log something. Function may receive multiple parameters but only first one is printed by console.log. To print all values use (...msg) => console.log(msg)
Here's what I've tried:
const pino = require('pino')
const logger = pino({ level: 'debug', prettyPrint: true })
const Sequelize = require('sequelize')
sequelize = new Sequelize({
dialect: 'sqlite',
storage: '../db.sqlite3',
logging: logger.debug()
})
But nothing is printed to the console. I know logging is working, as I logger.debug('test') works when called elsewhere in the code.
I found this library (from this issue) but I am not really sure how to use it with Sequelize.
You do not need to call your function, you just need to pass it to Sequelize.
So basically you should write logging: msg => logger.info(msg), for example. Don't worry about losing some other parameters, console.log only uses the first one (as described in the documentation).
Simple working example:
{
// ...
logging: sql => logger.info(sql),
// ...
}
Full (or almost full) clone of console.log behavior:
{
// ...
logging: (sql, timing) => logger.info(sql, typeof timing === 'number' ? `Elapsed time: ${timing}ms` : ''),
// ...
}
Tip: You can use the logging option for each of your queries and they will obviously work the same way.
Tip #2: You can also use logging: logger.info.bind(logger). But you will probably search for another workaround if you choose this one :)
Related
How can I manually set the level of a log when using Pino?
Here's some sample code:
const baseLogger = pino(loggerOptions);
const activityLogger = baseLogger.child({ name: "activity" });
const workerLogger = baseLogger.child({ name: "worker" });
Runtime.install({
logger: new DefaultLogger("INFO", (entry) => {
workerLogger.error({
level: entry.level.toLowerCase(),
message: entry.message,
timestamp: Number(entry.timestampNanos / BigInt(1000000)),
...entry.meta,
})
}
)
})
which produces logs like the following:
{"level":"error","time":1674573001943,"pid":95258,"name":"worker","level":"info","message":"Workflow bundle created","timestamp":1674573001943,"size":"0.70MB"}
Note that level appears twice. Ideally I'd like to invoke workerLogger.log and manually pass a level field but it seems pino does not make this easy. Is there a way to log with pino but not use one of the default level functions like .info, .debug, etc...?
I have a logging module that I use in many of my projects, which generally exports a single Winston logger, so all I did was define a logger and it's transports, then export it:
module.exports = logger;
when importing using const logger = require('mylogger.js') I then use the various levels built in (logger.info logger.debug etc).
I've now decided that I want to create a second logging function that will write logs to a different file, so I need to create and export a new transport. Thing is, if I switch to module.exports = {logger, mynewlogger}, that will change the way I import and call the functions, and I have that in many places.
Besides creating second file and importing both, is there any other way to add a second export without having to change my code everywhere else?
It's either new module that re-exports both:
logger-and-mynewlogger.js
module.exports = {logger, mynewlogger}
Or a separate module:
mynewlogger.js
module.exports = mynewlogger
Or using existing function as module object:
logger.mynewlogger = ...
module.exports = logger;
The first two options are preferable because they result in reasonably designed modules, while the last one is a quick and dirty fix.
Yes, you can define multiple transports for a single exported logger. When creating your Winston log, the 'transports' property is an array which allows you to define multiple outputs.
Here's an example of one I have that has two transports. Firstly, console and the second a daily rotating log.
const winston = require('winston');
const Rotate = require('winston-daily-rotate-file');
const tsFormat = () => (new Date()).toLocaleTimeString();
const logger = new (winston.Logger)({
transports: [
// colorize the output to the console
new (winston.transports.Console)({
timestamp: tsFormat,
colorize: true,
level: 'info',
}),
new (Rotate)({
filename: `${logDir}/${logName}-app.log`,
timestamp: tsFormat,
datePattern: 'YYYY-MM-DD',
prepend: true,
level: env === 'development' ? 'verbose' : 'info',
}),
],
});
module.exports = logger;
I'd like to override some values at test-time, specifically setting my retries for an http service to 1 (immediate failure, no retries). Our project uses node-config. According to the docs I can override with NODE_CONFIG env variable:
node myapp.js --NODE_CONFIG='{"Customer":{"dbConfig":{"host":"customerdb.prod"}}}'
Well I would prefer to do this in my test, but not for all tests. The code says that you can allow config mutations by setting ALLOW_CONFIG_MUTATIONS.
process.env.ALLOW_CONFIG_MUTATIONS = "true";
const importFresh = require('import-fresh');
importFresh("config");
process.env.NODE_CONFIG = JSON.stringify({httpServices:{integration:{enrich: {retryInterval: 1, retries: 1}}}});
expect(process.env.NODE_CONFIG, 'NODE_CONFIG not set').to.exist();
expect(process.env.NODE_CONFIG, 'NODE_CONFIG not set').to.match(/retryInterval/);
expect(process.env.ALLOW_CONFIG_MUTATIONS, 'ALLOW_CONFIG_MUTATIONS not set').to.equal("true");
const testConfig = require("config");
console.dir(testConfig.get("httpServices.integration.enrich"));
expect(testConfig.get("httpServices.integration.enrich.retryInterval"), 'config value not set to 1').to.equal(1);
Result:
{ url: 'https://internal-**********',
retryInterval: 5000,
retries: 5 }
`Error: config value not set to 1: Expected 5000 to equal specified value: 1`
How do I get this override to work?
(expect is from Hapi.js Code library)
I'm one of the maintainers of node-config. Your bug is that you used require the second time when you should have used importFresh again.
Your first use of "importFresh()" does nothing different than require() would, because it is the first use of require().
After setting some variables, you call require(), which will return the copy of config already generated and cached, ignoring the effects of the environment variables set.
You only needed to use importFresh() once, where you currently use require(). This will cause a "fresh" copy of the config object to be returned, as you expected.
Simply changing config's property worked for me.
For example:
const config = require( 'config' );
config.httpServices.integration.enrich.retryInterval = 1;
// Do your tests...
UPD: Make sure that overrides are done before anyone calls the first config.get(), because the config object is made immutable as soon as any client uses the values via get().
Joining late, but other answers did not fit with the testing standard in my project, so here is what I came up with
TL;DR
Use mocks..
Detailed Answer
node-config uses a function get to get the configuration values.
By mocking the function get you can easily modify any configuration you see fit..
My personal favorite library is sinon
Here is an implementation of a mock with sinon
const config = require('config');
const sinon = require('sinon');
class MockConfig {
constructor () {
this.params = {};
this.sandbox = sinon.sandbox.create();
}
withConfValue (confKey, confValue) {
this.params.confValues[confKey] = confValue;
return this;
}
reset () {
this.params.confValues: {};
return this;
}
restore() {
this.sandbox.restore();
}
apply () {
this.restore(); // avoid duplicate wrapping
this.sandbox.stub(config, 'get').callsFake((configKey) => {
if (this.params.confValues.hasOwnProperty(configKey)) {
return this.params.confValues[configKey];
}
// not ideal.. however `wrappedMethod` approach did not work for me
// https://stackoverflow.com/a/57017971/1068746
return configKey
.split('.')
.reduce((result, item) => result[item], config)
});
}
}
const instance = new MockConfig();
MockConfig.instance = () => instance;
module.exports = MockConfig;
Usage would be
const mockConfig = require('./mock_config').instance();
...
beforeEach(function () {
mockConfig.reset().apply();
})
afterEach(function () {
mockConfig.reset().clear();
})
it('should do something') {
mockConfig.withConfValue('some_topic.some_field.property', someValue);
... rest of the test ...
}
Assumptions
The only assumption this approach makes is that you adhere to node-config way of reading the configuration (using the get function) and not bypass it by accessing fields directly.
It's better to create a development.json, production.json et test.json in your config folder node-config will use it your app configuration.
you just net to set your NODE_ENV to use the specific file.
Hope it helps :)
I am new to knex migrations and for the past 2 days I have been struggling to get it working but nothing happen. I am trying to run my migrations programmatically using the knex.migration object.
First using the cli, I create a migration file in the migrations directory. Here is its content:
exports.up = function(knex, Promise) {
return Promise.all([
knex.schema.createTable('users', function (table) {
table.increments('id').primary();
table.string('username');
table.string('password');
table.string('email');
table.string('name');
table.timestamp('date');
}),
]);
};
exports.down = function(knex, Promise) {
};
Then from my code I initialize the Knex object:
var knex = Knex({
client:'sqlite3',
connection:{
filename: './knex.sqlite'
}
});
Then I execute the migration:
knex.migrate.latest().then(()=>{
// console.log()
}).catch(err =>{
//
});
But absolutely nothing happens. My migration file is never executed and there is no error or warning message. So I don't know where to look at to start searching for the problem. When I look at my sqlite database, I can see that tables knex_migrations, knex_migrations_lock and sqlite_sequence have been created.
So what I am doing wrong here? Is there something I am missing?
Thanks for any suggestion
There's no requirement to use the CLI tools. Sometimes it's not possible to use it due to its limitations and in this case it's indeed possible to use the migration API directly, like so:
const knex = require('knex')({
// Here goes the Knex config
});
const migrationConfig = {
directory: __dirname + './migrations',
}
console.info('Running migrations in: ' + migrationConfig.directory);
knex.migrate.latest(migrationConfig).then(([batchNo, log]) => {
if (!log.length) {
console.info('Database is already up to date');
} else {
console.info('Ran migrations: ' + log.join(', '));
}
// Important to destroy the database, otherwise Node script won't exit
// because Knex keeps open handles.
knex.destroy();
});
There was two issues in the original question:
The migration directory was not specified - in this case Knex is not smart and will simply not do anything instead of throwing an error. Most likely the default used by Knex is not right so it's best to specify it.
knex.destroy() was missing. In this case, the script will never exit because Knex keeps open handles on the database, so it just looks like it's stuck doing nothing.
The above script also outputs more log info to see what's exactly happening.
Knex migrations are supposed to run by Knex CLI，FYI: https://knexjs.org/#Migrations
As your code described, I found a strange issue:
knex.migrate is actually undefined, it's not a property of knex.
Is there an equivalent of log.IsDebugEnabled in Winston?
I want to use this to skip expensive logging code in a production environment but have it execute in development.
For example:
if(winston.isDebugEnabled){
// Call to expensive dump routine here
dump();
}
Checking winston.debug just checks whether the method is defined, not whether it is enabled.
Many thanks!
Edit: Added code example.
I've added a method to my logger to achieve just that:
logger.isLevelEnabled = function(level) {
return _.any(this.transports, function(transport) {
return (transport.level && this.levels[transport.level] <= this.levels[level])
|| (!transport.level && this.levels[this.level] <= this.levels[level]);
}, this);
};
This goes through each of your logger's transports and checks whether it 'wants' to log the specified level.
Note _.any is lodash, you can replace with for loop.
I'm sure you'd be able to get that directly from winston, but if you want to have different logging levels for different environments, you should pass these in when you're creating the winston.logger.
For example:
// default log file level is info (which is the lowest by default)
var logFileLevel = 'info';
if (process.env.NODE_ENV == 'production') {
// only write logs with a level of 'error' or above when in production
logFileLevel = 'error';
}
var logger = new (winston.Logger)({
transports: [
new (winston.transports.File)({
filename: '/var/log/node-logger.log',
level: logFileLevel
})
]
});
It's also useful to switch out the transports. You might want to use the Console transport whilst in development, and the File transport when in production for example.
More documentation on all this on the winston readme.
Try
if ( logger.levels[logger.level] >= logger.levels['debug'] ) {
// expensive calculation here
logger.debug(...)
}
Since Winston 3.1.0 (PR), you can use the Logger functions isLevelEnabled(string) & isXXXEnabled() for this.