Ok, say I have two listeners with callbacks, and the code in one callback depends on a variable (UIDfromOnEndFunction) from the other callback.
For example:
//using andris9/mailparser on github
var mailparser = new MailParser({
streamAttachments: true
}
// OnEnd Function
mailparser.on("end", function(objMail){
**UIDfromOnEndFuntion** = objMail.UID;
saveToDB("mail" + "1234", objMail);
});
mailparser.on("attachment", function(attachment){
var output = fs.createWriteStream("attachments/"
+ **UIDfromOnEndFuntion** + "/" + attachment.generatedFileName);
// need UIDfromOnEndFunction here
attachment.stream.pipe(output);
});
How do I cause the callback in mailparser.on("attachment" to get the variable UIDfromOnEndFunction.
Does this involve promises? How do you do this?
You can do this via a closure: just access a variable from outside.
//using andris9/mailparser on github
var UIDfromOnEndFunction;
var mailparser = new MailParser({
streamAttachments: true
}
// OnEnd Function
mailparser.on("end", function(objMail){
UIDfromOnEndFuntion = objMail.UID;
saveToDB("mail" + "1234", objMail);
});
mailparser.on("attachment", function(attachment){
var output = fs.createWriteStream("attachments/" + UIDfromOnEndFuntion + "/" + attachment.generatedFileName);
attachment.stream.pipe(output);
});
Please note my comment about ensuring end is called before attachment. If they do not fire in this way, this is fundamentally impossible.
OK I came up with a solution. It's based on closures from #Brenden Ashworth 's suggestion. It's untested but I'm fairly certain it would work, and I wanted to post this before I moved on as I found I didn't need to do what the original question described to get my project working.
However, I still think it is useful to have a solution to this type of problem as the need could arise, and I don't know a better solution.
Here's my solution:
//using andris9/mailparser on github
var mailparser = new MailParser({
streamAttachments: true
}
var UIDfromOnEndFuntion;
var myAttachment;
var intNumberOfEmitsToEndAndAttachment = 0;
var funcBothEndAndAttachmentEmitted = function () {
var output = fs.createWriteStream("attachments/"
+ UIDfromOnEndFuntion + "/" + myAttachment.generatedFileName);
//UIDfromEndFunction should be garaunteed to be
//populated by .once("end",...)
myAttachment.stream.pipe(output);
//myAttachment should be gauranteed to be populated
//by .once("attachment",...)
}
mailparser.once("end", function(objMail){
UIDfromOnEndFuntion = objMail.UID;
saveToDB("mail" + "1234", objMail);
intNumberOfEmitsToEndAndAttachment++;
if (intNumberOfEmitsToEndAndAttachment == 2) {
funcBothEndAndAttachmentEmitted();
}
});
mailparser.once("attachment", function(attachment){
myAttachment = attachment;
intNumberOfEmitsToEndAndAttachment++;
if (intNumberOfEmitsToEndAndAttachment == 2) {
funcBothEndAndAttachmentEmitted();
}
});
Now this would only work for a single emitted "end" and a single emitted "attachment".
You could get more creative with how the tracking is done to handle multiple attachments. For example, instead of using an integer to track the total number of calls, an array of objects could be used like [{"attachment",attachment_args1},{"attachment",attachment_args2},{"end",end_args2}] to do the tracking of calls (this would mean attachment has been called twice so far, and "end" once, for example, and you could trigger a function based on that knowledge like I do by calling funcBothEndAndAttachmentEmitted()).
I think this needs to be cleaned up and made into a library, unless there is a better way to do it that's not apparent. (Please comment if you know a better solution or I might go ahead and write a library for this solution.)
Another solution I thought of that might work is putting mailparser.once("attachment"...) inside of the callback for mailparser.once("end"...) but I suspect that wouldn't work if "attachment" is emitted first, and this solution seems a bit cludgey compared to a library-based solution if you're working with many different emitted events for some reason or different objects emitting different events.
Related
I've written a very tiny script in node.js to check out how links can be passed to a function using loop.
I could see that I can do the same in two ways but can't figure out which way I should stick to, meaning which one is ideal and why?
One way: for (link in links) {}
var request = require('request');
var links = ['https://stackoverflow.com/questions/ask', 'https://github.com/request/request'];
for (link in links) {
(function(url) {
request(url, function() {
console.log(url);
});
})(links[link]);
}
The other way: for (const link of links) {}
var request = require('request');
var links = ['https://stackoverflow.com/questions/ask', 'https://github.com/request/request'];
for (const link of links) {
(function(url) {
request(url, function() {
console.log(url);
});
})(link);
}
There is no ideal way or at least an universal ideal way of doing this, So i will point out the difference between these two.
In first for loop you are iterating the array as an object (in javascript array is an object which can traverse with indexes). But will create a global variable called link after the execution. So an unwanted variable and memory location is created.
Try console.log(link) after the execution.
The second for loop is introduced with ECMA Script 6 and won't create a global variable and is recommended. Because of the readability and the more control over your data, and link can be defined as const if you want. So it won't be modified inside the loop.
For node.js I guess second one may be perfect for most scenarios. But in javascript, the first one may be higher in performance wise, if you are compiling it from ES6 to ES5 and it is the case for most scenarios.
I'm experiencing a problem with node. I'm trying to use a language detection algorithm, but I'm having trouble with scopes.
After saving the response to "langVastus" and then extracting the language to "keel", I get the right result inside the Algorithmia function, but not on the outside.
The console logs print out
Inside: en
Outside:
And the code looks like this:
var langVastus = "";
var keel = "";
Algorithmia.client("simpIVxv0Ex5Xen1bVCLVXnxYpr1")
.algo("nlp/LanguageIdentification/1.0.0")
.pipe(input)
.then(function(response) {
langVastus = response.get();
keel = langVastus[0].language;
console.log("Inside: " + keel);
});
console.log("Outside: " + keel);
res.render("lang", {keel: keel});
What am I doing wrong?
The problem was that I initialized the variable inside a route. Taking it outside the route fixed my problem :)
It seems like the block that you're calling "Inside", runs inside of a promise callback.
In that case the log from the outside will run before the callback is called.
TL;DR: is there a way to wait for a module import with async functionality to complete before continuing with execution in the calling module in order to keep module functionality contained?
I'm working on a personal node project that I've been structuring in a modular/OOP way as the codebase has continued to grow. One requirement has been to enable logging across modules / objects, where different logfiles can be logged to at different times. I thought that I had solved the problem in a pretty clean way by creating a Logger.js file with an init function that I could use at any time by simply importing the Logger.js file in any module that I needed. Here is the stripped down code to illustrate this:
Logger.js
module.exports.init = function(location) {
var logFileBaseName = basePath + fullDatePathName;
var studentLogFile = fs.createWriteStream(logFileBaseName + '-student.log', {flags : 'a'});
var teacherLogFile = fs.createWriteStream(logFileBaseName + '-teacher.log', {flags : 'a'});
this.studentLog = function () {
arguments[0] = '[' + Utils.getFullDate() + '] ' + arguments[0].toString();
studentLogFile.write(util.format.apply(null, arguments) + '\n');
}
this.teacherBookLog = function () {
arguments[0] = '[' + Utils.getFullDate() + '] ' + arguments[0].toString();
teacherLogFile.write(util.format.apply(null, arguments) + '\n');
}
}
This seemed great, because in my main entrypoint I could simply do:
Main.js
const Logger = require('./utils/Logger');
Logger.init(path);
Logger.studentLog('test from Main');
// all my other code and more logging here
And in my other dozens of files I could do even less:
AnotherFile.js
const Logger = require('./utils/Logger');
Logger.studentLog('test from AnotherFile')
Then the requirement came to log not only to a file for the 'student logs', but to Discord (a chat client) as well. Seemed easy, I had this Logger file and I could just initialize Discord and log to Discord alongside the 'student logs', something like this:
Logger.js
module.exports.init = function(location) {
// code we've already seen above
var client = new Discord.Client();
client.login('my_login_string');
channels = client.channels;
this.studentLog = function () {
arguments[0] = '[' + Utils.getFullDate() + '] ' + arguments[0].toString();
var message = util.format.apply(null, arguments) + '\n';
studentLogFile.write(message);
channels.get('the_channel_to_log_to').send(message)
}
// more code we've already seen above
}
The problem is that if you were to re-run Main.js again, the studentLog would fail because the .login() function is asynchronous, it returns a Promise. The login has not completed and channels would be an empty Collection by the time we try to call Logger.studentLog('test from Main');
I've tried using a Promise in Logger.js, but of course execution of Main.js continues before the promise returns in Logger.js. I would love it if Main.js could simply wait until the Discord login was complete.
My question is, what is the best way to make this work while keeping with the pattern I've been using? I know that I could wrap my entire main function in a promise.then() that waits for Discord login to complete, but that seems a bit absurd to me. I'm trying to keep functionality contained into modules and would not like for this kind of Logger code / logic to spill out into my other modules. I want to keep it to a simple Logger import as I've been doing.
Any advice would be great!!
If the result of some async function is awaited and then used in the same caller function, the result is resolved first, then used. If the result is used in another function or module (e.g. the result is assigned to a global variable), it is not resolved. In your case, if client.login() assigns a value to client.channels asynchronously, that assignment is not awaited, and channels = client.channels assignment will assign undefined to channels.
To resolve this issue, you must use a callback or return a promise from client.login(), as stated in the comments.
You can refer to this article.
Let me offer my solution to the "asynchronously initialised logger" problem. Note that this only deals with logging and most likely cannot be generalised.
Basically, all messages are appended to a queue that is only sent to the remote location once a flag inidicating that the connection is ready is set.
Example:
//Logger.js
module.exports = {
_ready: false,
_queue: [],
init(): {
return connectToRemote().then(()=>{this._ready = true})
},
log(message): {
console.log(message);
_queue.push(message)
if (this._ready) {
let messagesToSend = this._queue;
this._queue = [];
this._ready = false;
sendToRemote(messagesToSend).then(()=>this._ready = true);
}
}
}
You can require the logger in any file and use the log funciton right away. The logs will be sent only after the init funciton that you can call anytime is resolved.
This is a very bare bones example, you may also want to limit the queue size and/or only send the logs in bulk in certain time intervals, but you get the idea.
First of all, I'm trying to test the second time a function being called returns the correct value. To be more specific, I'm trying to test the second time an event is received by socket.on() returns the correct value.
I know that Sinon.spy() can detect whether a function being called or not. But it seems not working with socket.io events.
I'm trying to test for example,
var socketio = io.connect(someUrl);
socketio.on(eventName, cb);
if the 'eventName' is called. I tried
var spy = sinon.spy(socketio, 'on');
assert(spy.withArgs(eventName).called);
But it says it's never being called.
Furthermore, what if I'd like to test the data in the callback function is right or not, for example
var socketio = io.connect(someUrl);
socketio.on(eventName, function(data) {
data.should.equal(something);
});
Is it possible to test that?
Update:
I solved the problem by adding a count in callback
var count = 0;
socketio.on(eventName, function(data) {
count++;
if(count === 2) {
data.should.equal(something)
}
}
But I think this is a hacky way, is there any standard/smart way to do it(if possible using sinon)?
You don't want to spy on on -- that's only called once, at the time you set up the listener. (Unless you actually want to spy to see if the listeners are being set up at all, and don't care if they're actually used. That doesn't appear to be what you want here.)
Instead, you want to spy on the cb callback function, to see if and how it runs.
var callbackSpy = sinon.spy(cb);
Then, pass that spy in as the callback:
socketio.on(eventName, callbackSpy);
And test if the callback was ever run with the desired argument value:
assert(callbackSpy.calledWith(something));
Is there a way to remove the listeners on a passed in named callback function that are wrapped in an anonymous function?
UPDATE. More complete code examples below.
Here are the details.
I've a function that gets passed in a named callback.
Before
function read (message, named_callback ) {
var named_callback = named_callback || default_callback
, message = message || "Choose: ";
stdout.write(message);
stdin.resume();
stdin.setEncoding('utf8');
stdin.on('data', named_callback);
});
};
All the named_callbacks take and prepare a passed in user input (answer).
answer = answer.trim().toLowerCase();
I end up repeating the trimming and lowercasing line everywhere! I wanted to move this step into one place, so tried to prepare the answer before it got passed into the callback.
I wrote this:
After
function read (message, named_callback ) {
var named_callback = named_callback || default_callback
, message = message || "Choose: ";
stdout.write(message);
stdin.resume();
stdin.setEncoding('utf8');
stdin.on('data', function (answer) {
answer = answer.trim().toLowerCase();
named_callback(answer);
});
};
However, this results in event listeners not being removed, and they just pile up until the program crashes with too many listeners.
Thank you.
the problem is probably not where you think it is. with the info you give i would expect you just call the read method everytime, and thats where the tomanylisteners comes into place, because you just everytime append a new 'data' listener. if you change that 'on' to 'once' your application shouldnt crash anymore:
stdin.once('data'...
this of course isnt the solution to your problem, it is just to illustrate where your problem is (its not the trim/tolowercase.
if you show as a little bit more of your code maybe we are able to help you better, but probably your read-method is just unecessary overhead...