node.js module self talk back to js that requires it - node.js

im having problems understanding how to talk 'up and down' between app.js and modules...
I think its with a callback but i've also seen things like self._send(), this.send() and module.exports.emit
I'm quite confused.
I recently installed pdfkit from npm (quite good 6/10 :p) I want to learn by improving it slightly though by adding a done event/callback for doc.write().
I know its not that important but i've been looking through my installed modules and that is probably the easiest example of code that wouldn't hurt to have a 'DONE' I also figured this function would be good to learn from as it uses fs.writeFile which has a function(){} that fires when its finished writing so the fact that i can see where in the code it ends makes it an easy learning tool.
I've modified the code a few times tried to compare modules to see where similar things have been done but i just keep breaking it with errors, i don't feel like i'm getting anywhere:
inside the pdfkit module document.js i've made changes:
var EventEmitter = require('events').EventEmitter;//ben
module.exports = new EventEmitter();//ben
PDFDocument.prototype.write = function(filename, fn, callback) {//ben added callback
return this.output(function(out) {
return fs.writeFile(filename, out, 'binary', fn, function(){//ben added finished function
//module.exports.emit('pdf:saved');//ben
callback();//ben
});
});
};
in my app.js:
doc.write('public_html/img/'+_.c+'_'+_.propertyid+'.pdf',function(){console.log('pdf:saved');});
//doc.on('pdf:saved',function(){console.log('pdf:saved');});
I'm also not really sure what i'm querying on google, please can someone help me?

EventEmitter is required to create objects with event storage, and emit/catch events.
While what you are using in your example is called 'callback'.
It is two different ways, and can be sort of cross used. Sometimes it is good to use events, but sometimes just callback is enough.
The best way to have head around it: play with callbacks (forget about events for now). Try to think of different uses of callbacks and maybe even have callback function and pass it around. Then come to callback chains. And only after start playing with EventEmitter. Remember that EventEmitter is different thing from callbacks, with sometimes compatible use cases, but generally is used in different cases.
Here is your code, simplified with same functionality as you have/need atm:
PDFDocument.prototype.write = function(filename, callback) {
this.output(function(out) {
fs.writeFile(filename, out, 'binary', callback);
});
};
And use it the way you already do.
Do not try to generate garbage of code that will only complicate everything - it is better to study speficic areas seperatelly, and then switch to next one. Otherwise will mess up in mind.

FIXED
fn is the callback function!
PDFDocument.prototype.write = function(filename, fn) {
return this.output(function(out) {
return fs.writeFile(filename, out, 'binary', fn);
});
};
and by naming my callback function to fn it works!
doc.write('public_html/img/'+_.c+'_'+_.propertyid+'.pdf',function fn(){console.log('pdf:saved');});
for me that is a massive learning mountain climbed!!

Related

How can I split my code, and keep it all asynchronous?

I've been coding just as a side project for a bit, piecing together bits that other people have written (it's for a simple discord bot). I want to split my code to make it easier to problem solve and read, however whenever I try to use the code it comes up with an error saying 'SyntaxError: await is only valid in async function'.
I've tried supposedly loading the code asynchronously, loading it with require() and then making a single command asynchronous, making the entire code in the file asynchronous (it's not just one command I want to load, but a whole file. Also I'm not sure if I tried it correctly or not), using the npm async-require, and maybe some others that have been around on the internet.
//one of the solutions I've tried. This is just copy pasted from the
//answer
//file2.js
var fs = require('fs');
module.exports = function (callback) {
fs.readFile('/etc/passwd', function (err, data) {
callback(err, data);
});
};
//file1.js
require('./passwords')(function (err, passwords) {
// This code runs once the passwords have been loaded.
});
In the first file before I split it, I started it with client.on('message', async message => { and it made me able to use the await function in every command. I want to still be able to do that, but just have it a bit neater and easier to use by splitting it.
I'm trying to get this done so I can move on to a different question I asked and give one of the answers a tick. Any help would be greatly appreciated <3
Fix those awaits so that they are not inside async functions. This is a lexical issue that can be solved just by looking at the location were the error occurs. Just look for the nearest containing function to where the await is and mark it async. Repeat until the error goes away.

Why node 10 has made it mandatory to pass callback on fs.writeFile()?

Suddenly, I started getting this error on my application when the node engine was upgraded to 10.7.0
TypeError [ERR_INVALID_CALLBACK]: Callback must be a function
Code which was working with node 4.5: fs.writeFile(target, content);
After a bit of debugging I found this in node_internal/fs.js:
function writeFile(path, data, options, callback) {
callback = maybeCallback(callback || options);
...
}
function maybeCallback(cb) {
if (typeof cb === 'function')
return cb;
throw new ERR_INVALID_CALLBACK();
}
Certainly, if do not pass a third/fourth argument here, my code will fail. I want to know is there any way to mitigate this problem. Or if not, what could be the motivation behind such a breaking change. After all, fs.writeFile() is such a basic operation, issues such as these are really a pain while upgrading.
Node.js has documented the purpose for this change: https://github.com/nodejs/node/blob/master/doc/api/deprecations.md#dep0013-fs-asynchronous-function-without-callback
There is a lot more discussion here: https://github.com/nodejs/node/pull/12562#issuecomment-300734746
In fact it seems like some developers agree with you, however the decision has been made and the callback is now required.
There is no mitigation per se; you will just have to add a callback. Even an empty one will work okay:
fs.writeFile(target, content, () => {});
I understand this may require a lot of changes for currently working code, but in fact it might be a good opportunity for you to add error handling as well.

To async, or not to async in node.js?

I'm still learning the node.js ropes and am just trying to get my head around what I should be deferring, and what I should just be executing.
I know there are other questions relating to this subject generally, but I'm afraid without a more relatable example I'm struggling to 'get it'.
My general understanding is that if the code being executed is non-trivial, then it's probably a good idea to async it, as to avoid it holding up someone else's session. There's clearly more to it than that, and callbacks get mentioned a lot, and I'm not 100% on why you wouldn't just synch everything. I've got some ways to go.
So here's some basic code I've put together in an express.js app:
app.get('/directory', function(req, res) {
process.nextTick(function() {
Item.
find().
sort( 'date-modified' ).
exec( function ( err, items ){
if ( err ) {
return next( err );
}
res.render('directory.ejs', {
items : items
});
});
});
});
Am I right to be using process.nextTick() here? My reasoning is that as it's a database call then some actual work is having to be done, and it's the kind of thing that could slow down active sessions. Or is that wrong?
Secondly, I have a feeling that if I'm deferring the database query then it should be in a callback, and I should have the actual page rendering happening synchronously, on condition of receiving the callback response. I'm only assuming this because it seems like a more common format from some of the examples I've seen - if it's a correct assumption can anyone explain why that's the case?
Thanks!
You are using it wrong in this case, because .exec() is already asynchronous (You can tell by the fact that is accepts a callback as a parameter).
To be fair, most of what needs to be asynchronous in nodejs already is.
As for page rendering, if you require the results from the database to render the page, and those arrive asynchronously, you can't really render the page synchronously.
Generally speaking it's best practice to make everything you can asynchronous rather than relying on synchronous functions ... in most cases that would be something like readFile vs. readFileSync. In your example, you're not doing anything synchronously with i/o. The only synchronous code you have is the logic of your program (which requires CPU and thus has to be synchronous in node) but these are tiny little things by comparison.
I'm not sure what Item is, but if I had to guess what .find().sort() does is build a query string internally to the system. It does not actually run the query (talk to the DB) until .exec is called. .exec takes a callback, so it will communicate with the DB asynchronously. When that communication is done, the callback is called.
Using process.nextTick does nothing in this case. That would just delay the calling of its code until the next event loop which there is no need to do. It has no effect on synchronicity or not.
I don't really understand your second question, but if the rendering of the page depends on the result of the query, you have to defer rendering of the page until the query completes -- you are doing this by rendering in the callback. The rendering itself res.render may not be entirely synchronous either. It depends on the internal mechanism of the library that defines the render function.
In your example, next is not defined. Instead your code should probably look like:
app.get('/directory', function(req, res) {
Item.
find().
sort( 'date-modified' ).
exec(function (err, items) {
if (err) {
console.error(err);
res.status(500).end("Database error");
}
else {
res.render('directory.ejs', {
items : items
});
}
});
});
});

synchronous sqlite transactions node

I installed sqlite3 with npm
npm install sqlite3 --save
I have written some basic functions that I would like to be performed synchronously rather then with async.
for instance I would like to get column names in one function and the row count in another.
I would like to simply return these values. currently I am using a callback like so
d.cinfo = function(table, callback){
var o = {};
db.each("PRAGMA table_info(" + table + ")", function(err, col){
o[col.name] = col.type;
}, function(){
if(typeof callback == 'function') callback(o);
});
}
d is an object which is later exposed
however Id like to return the values
d.cinfo = function(table, callback){
var o = {};
db.each("PRAGMA table_info(" + table + ")", function(err, col){
o[col.name] = col.type;
}, function(){
return o;
});
}
is there a way I can achieve this. I found documentation saying it was possible but then I learned that it was outdated and im not sure its meant for the same api
I have implemented bluebird however Promise.promisifyAll(middleware) returns and error "Object # has no method .then() anyone know what im doing wrong
I've come across a problem like this and many others while using Express and it really drives me crazy where I just can't avoid callback hells.
Some of the possible solutions can be:
Using the same old event driven style. Instead of calling nested functions, emit events for them. But I'm assuming you wouldn't like that very much, but that atleast gets rid of the nested callbacks.
There's a npm module called bluebird that allows you to extend existing modules with promises. (Using promisifyAll). So promises will still trim down the code and you'll have much cleaner code than you had before using promises.
If you don't mind switching frameworks, you can try KoaJS that allows you to yield(fun stuff from ECS 6) functions using generator functions(another cool feature). You can have more reading about this here.
So the above code can be re-written like:
var rows = yield db.each() //or something like that
this is an amazing feature but the drawback is that you have to use unstable versions of node (0.11.x). We also faced some issues setting up https using 0.11.x and had to downgrade to 0.10.x (we're not using koa either)
If there are only a few places where you need to do such kind of stuff, I would recommend using bluebird, but if you're tired of having lots of callbacks in your code, better go for Koa, although I've already mentioned you the tradeoffs.

Node.js // A module to manage a sqlite database

I managed to create a module to handle all the database call. It uses this lib: https://github.com/developmentseed/node-sqlite3
My issues are the following.
Everytime I make a call, I need to make sure the database exist, and if not to create it.
Plus, as all the calls are asynchronous, I end up having loads of functions in functions in callbacks ... etc.
It pretty much looks like this:
getUsers : function (callback){
var _aUsers = [];
var that = this;
this._setupDb(function(){
var db = that.db;
db.all("SELECT * FROM t_client", function(err, rows) {
rows.forEach(function (row) {
_aUsers.push({"cli_id":row.id,"cli_name":row.cli_name,"cli_path":row.cli_path});
});
callback(_aUsers);
});
});
},
So, is there any way I can export my module only when the database is ready and fully created if it does not exist yet?
Does anyone see a way around the "asynchronous" issue?
You could also try using promises or fibers ...
I don't think so. If you make it synchronous, you are taking away the advantage. Javascript functions are meant to be that way. Such a situation is referred to as callback hell. If you are facing problems managing callbacks then you can use these libraries :
underscore
async
See these guides to understand basics of asynchronous programming
Node.js: Style and structure
Using underscore.js managing-callback-spaghetti-in-nodejs
Using async.js node-js-async-programming

Resources