Hope you are well.
I need your help to understand how to logically organize a program in Node JS to avoid repetition of code given its asynchronous property (as a beginner ..). Let's take an example to make it easier to explain.
One has some data in a mongo database (let's say a list of name). This list of name can be access thanks to the function readData as below
function readData(criteriaRead,callback) {
mongodb.stuff(..)
callback('data read on mongodb')
}
I have two actions in my program: one is to print out the list of name, the other is to check if a name is in the list.
For the first case, it's simple, I just need to have a function like this
function printout(data) {console.log(data)}
and to do this
readData(criteriaRead,printout)
In the second case, let's say I have a function like this
checkIfInIt(array,dataToCheck) {//stuff to check console.log(results)}
Now, I have an issue because if I doreadData(criteriaRead,checkIfInIt) it won't work as checkIfInIt requires two parameters.
I would need a function like this
function readDataBis(criteriaRead,dataToCheck,callback) {
mongodb.stuff(..)
callback('data read on Mongodb','dataToCheck')
}
and then readDataBis(criteriaRead,dataToCheck,checkIfInIt) would work but I have a huge repetition in my code.
How to avoid that?
There are several solutions for this type of issue, but here's an easy one for your case
Declare your function with the three parameters as such
function readData(callback, criteriaRead, dataToCheck) { ...
Inside, check if dataToCheck is undefined, and continue with the flow of the second function you had if that's the case. (Otherwise just do the read function)
Call them like so
readData(callback, criteriaRead); // Third parameter missing, will be undefined
readData(callback, criteriaRead, dataToCheck);
You could also pass in an object for your parameters like this, if it would make it simpler
function readData(callback, params) { ...
And call like this
readData(callback, { criteriaRead: criteriaRead, dataToCheck: dataToCheck });
Related
I've got a little app i've written with node.js and puppeteer. I'm trying to require a function from a different file into my evaluate callback, however the function never fires and causes evaluate to fail. Here is a pretty simple example, and maybe somebody can see if i'm just doing something stupid here.
Evaluate is called from File A
product = await page.evaluate( source.getProductInformation )
source.getProductInformation is defined in File B, this function fails when I call a function I require from within File B
const priceSavePercent = calculateSavePercentage(priceWasNum, priceCurrentNum)
calculateSavePercentage is simply required at the top of File B const { calculateSavePercentage } = require('../modules/helpers')
I try to console log everywhere and don't get any output to my console, and my evaluate callback doesn't return the object it's suppose to. Is there a different way i'm suppose to require dependencies into File B? I have an npm package and a constant also required in File B and both don't cause issues. Any help is greatly appreciated. Let me know if you need any more info.
The problem is that when you evaluate, you are accessing the page you are scraping javascript, not your own so that function isn't defined you could try something like this.
await page.evaluate((source) => {
source.getProductInformation();
}, source);
As I understand it, the node-netstat package parses the output of the netstat command and it calls the callback I supply, once per line of data it parses.
I could do with knowing when it's made its last call, so I know to callback the function that was supplied to my function by elsewhere, but I'm not really sure how to do this..
this.myfunc = function(callback){
netstat(null, function(data){
//netstat will call this function X times. I'd like to accumulate data
});
callback( ..data from netstat.. );
}
If netstat's callback only fired once, with all the data, then I could probably have called callback at the end of function(data), but the multi-calls is confounding that. What do we do in situations like this? (Note also, it's a really prehistoric version of node: 0.10.24)
You can pass an option object to netstat(options, handler) function.
In option object, there is a done field which you can pass a callback function.
More information about option object can be found here
I am trying to understand the node.js documentation specifically for the https.get() method. https://nodejs.org/dist/latest-v8.x/docs/api/https.html#https_https_get_options_callback
What is unclear to me is the callback. The example in the document indicates the callback can take a res (response) object as its parameter but I am unsure if this is the only parameter it can take or more importantly where I can find the definition of the res object so I can know what properties and methods I can access on this object.
Is there a straightforward way to identify this?
I have read this thread: Trying to understand nodejs documentation. How to discover callback parameters and the answers seem to suggest that if there is a non-error argument that a callback can take it will be documented, but I am assuming that answer is outdated.
I've run into the same issue with many Node/NPM packages. Documentation sometimes does not describe the parameters well.
So, welcome to JavaScript in 2018! It's gotten a lot better, though, to be honest.
My go-to method is to try the methods and dump the information myself.
Try a console.dir(res) in your callback:
https.get('https://encrypted.google.com/', (res) => {
console.dir(res);
});
Alternatively, you can set a breakpoint in the callback and inspect it yourself. You can then probe the arguments object* to see what else, if anything, was passed as an argument, or do another console dump:
https.get('https://encrypted.google.com/', function (res) {
console.dir("args:", arguments);
console.dir("res:", res);
});
EDIT: Wait, apparently the arguments variable is not available to arrow functions, fixed the second example.
*From MDN:
The arguments object is not an Array. It is similar to an Array, but
does not have any Array properties except length.
From your link https://nodejs.org/dist/latest-v8.x/docs/api/https.html#https_https_get_options_callback, you can see that it works like the http version :
Like http.get() but for HTTPS.
With http.get() clickable.
On that page (https://nodejs.org/dist/latest-v8.x/docs/api/http.html#http_http_get_options_callback), we can see this :
The callback is invoked with a single argument that is an instance of http.IncomingMessage
With http.IncomingMessage clickable, linking this page :
https://nodejs.org/dist/latest-v8.x/docs/api/http.html#http_class_http_incomingmessage
I agree the Node documentation is not very clear about the callbacks in general, and that is a shame. You can still use IDEs with good intellisense (and JSDoc to identify the type of the function parameters), like VSCode.
Or you can use a debugger, always works :)
Edit: If you want to see all the parameters sent to a function, you can use the spread syntax like this :
function foo(...params) {
// Here params is an array containing all the parameters that were sent to the function
}
If you want the absolute truth, you can look at the implementation. Though that's fairly time consuming.
If you find that the documentation is wrong, or in this case could be improved by adding a sentence about the callback parameter to https.get(), please open an issue, or, better yet, a pull request. This is where the change needs to be made:
https://github.com/nodejs/node/blob/67790962daccb5ff19c977119d7231cbe175c206/doc/api/https.md
In mobile apps apps we can't (or should not) make network requests on the main thread. We normally get the results of the request back via a callback or a closure that is executed on the main thread when the result is available. Since the user may have moved on or the result may no longer be need, for example it may be an old request arriving out of order, we need to check that the action in the callback or closure should actually be executed based on the current state of the app.
In the case of iOS and swift I am planning on using closures so I am thinking of doing something like this for every request I make.
assume I have a method that looks something like this
func makeRequest(identifier: String, handler: (ident: String, result: ResultObject) -> Void) {
...
...
handler(identifier, result)
}
In addition to the handler that will be called when the result is available, I will pass in the value of an identifier, which in turn will be passed to the handler when it is called. The closure will capture a reference to the identifier when the request is created, so it be able to get the value that the reference holds at the time the handler is actually called. So it would look something like this, where ident is the value that commandIdentifier was when the request was made, and commandIdentifier inside the closure will be the value when the closure is actually executed.
commandIdentifer = "some unique identifier"
makeRequest(commandIdentifer) { ident, result in
if commandIdentifier == ident {
// do something
} else {
// do something else
}
}
I don't think there is anything special here, so my question is this:
Is this a general pattern, and if so where can I find any documentation on it?
I am particularly interested if there is some general way of creating the identifier and how to relate its reference in the main thread.
Also if I am total wrong and this not a good approach, I would like to hear that as well
I've used almost exactly that approach before. I use an integer identifier, and increment it when issuing a new request. That way if the pending request is superseded by a new one you can just drop the stale response on the floor.
I'm in a rather unique situation (one I've never found myself in before) and I can't find anything on how it should be handled, so I thought I'd ask here. (and perhaps start a good discussion on how it should be handled)
I'm writing a Node.js/Express application that does a series of database calls in various route handlers. I'm using the node-sqlite3 module to make the database calls. In this case the user is uploading a file, so I take that from the form POST data and save it to the filesystem (will be moved to blob storage later) and generate some xml files for other reasons. The way that I name these files is by the id in the database to facilitate routes like '/file/:id' to GET the file later on, as well as leveraging the database to ensure I don't have name collision issues.
I'm doing this in CoffeeScript so I'm wrapping this model in a class, so it uses this (or #) to access these helper methods. Before I was doing a last_insert_rowid() call to get the id of the thing I just inserted but this opens me up to potential race conditions. So as it turns out when you do an INSERT in the node-sqlite3 module the callback it calls when it's done saves the last inserted row id in the this object (so (err) -> fileid = #.lastID). Now for some code (names changed and simplified).
uploadFile: (fileData, cb) ->
#.openConnectionIfClosed()
#db.serialize =>
# preserve the # object to allow calls to class methods like #.saveFile
#db.run 'insert into thing (val=$val)', $val: 'some_val', (err) ->
if err
cb err
return
# # refers to the this object provided by #db.run
fileid = #.lastID
async.waterfall [
(async_cb) =>
# uh oh, the # object is the one from the db library, not my class
# earning me a 'method undefined' error
#.saveFile fileid, async_cb
# etc... you get the idea
...
], (err) -> cb err
#.closeConnection()
So I'm needing to retain the class this object to access instance methods, but I also need to get the lastID value out of the this object returned by default from the callback in the db call. Obviously if I change the callback on the db call to fat arrows I can access my class methods, but then the #.lastID returns undefined.
What's the "proper" way to achieve this? (in CoffeeScript or JavaScript, doesn't particularly matter) I think that the way I can solve this is by assigning a context variable like ctxt = # at the top of the method but obviously this isn't favorable. Any ideas?
EDIT: Oh, and I forgot to mention, I can't just name the file something else because I also need that id to update the database. The call to async.waterfall does: Save file -> generate xml metadata file & save it -> add the path of the xml file to the database entry retroactively. Though I could potentially use a statement to do that without completely isolated database calls, haven't investigated that yet.
In JavaScript, if you need to preserve a particular context, its standard practice to use the variable self, as such:
var self = this;
In CoffeeScript, the fat arrow is standard practice for cases like this, though using 'self' is not wrong either.