I've been trying to figure out how to use node-fibers to make my database code less messy in node.js, but I can't get it to work. I boiled the code down to this as a minimum test case:
var Future = require('fibers/future');
var pg=require('pg');
var connstr = "pg://not_the_real_user:or_password#localhost/db";
var pconnect = Future.wrap(pg.connect);
Fiber(function() {
var client = pconnect(connstr).wait();
console.log("called function");
}).run();
If I leave it as is, I get the following error:
pgfuture.js:10
}).run();
^
TypeError: undefined is not a function
at Object.PG.connect.pools.(anonymous function).genericPool.Pool.create (/home/erik/code/treehouse-node/node_modules/pg/lib/index.js:49:20)
at dispense (/home/erik/code/treehouse-node/node_modules/pg/node_modules/generic-pool/lib/generic-pool.js:223:17)
at Object.exports.Pool.me.acquire (/home/erik/code/treehouse-node/node_modules/pg/node_modules/generic-pool/lib/generic-pool.js:267:5)
at PG.connect (/home/erik/code/treehouse-node/node_modules/pg/lib/index.js:75:15)
at Future.wrap (/home/erik/code/treehouse-node/node_modules/fibers/future.js:30:6)
at /home/erik/code/treehouse-node/pgfuture.js:8:18
However, if I comment out the line that calls pconnect, I get the "called function" message on the console and no errors. The example on the github page has an almost identical structure, and it does work correctly on my system, but I'm stumped as to what I'm doing wrong here.
Edit: Additional details
I've managed to get the code to run after a fashion in two different ways that seem unrelated, but both have the same behavior. After the function finishes, node just hangs and I have to kill it with ctrl-c. Here are the two things I've done to get that result:
1) Wrap pg.connect in an anonymous function, and then wrap THAT with Future:
pconnect = Future.wrap(function(err,cb){pg.connect(err,cb);});
2) This one is a real mystery, but it appears to have the same result. Inside the fiber, I just directly call pg.connect before the call to pconnect, and everything seems to work out.
// add this line before call to pconnect
pg.connect(connstr, function(e,c){console.log("connected.");});
// and now the original call to pconnect
var client = pconnect(connstr).wait();
I can imagine a circumstance in which (1) would make sense if, for example, the pg.connect function has other optional arguments that are somehow interfering with the expected layout of the Future.wrap call. Another possibility is that an object is going out of scope and the "this" reference is undefined when the actual call to pconnect is made. I'm at a loss to understand why (2) has any effect though.
Edit: partial answer
Okay, so I answered at least part of the question. The thought I had about object scope turned out to be correct, and by using the bind() function I was able to eliminate the extra layer of callback wrapping:
var pconnect = Future.wrap(pg.connect.bind(pg));
It still hangs at the end of the execution for unknown reasons though.
Are you disconnecting from database at the end of execution?
If not, it prevents node.js program from exiting.
Adding another code of my own which leaks.
#Almad I'm disconnecting here with the provided callback, but still hangs:
var future = Future.task(function() {
var ret = Future.wrap (pg.connect.bind(pg), "array") (conString).wait ();
ret[1]();
}).detach();
Related
I am still quite new to Node.js and can't seem to find anything to help me around this.
I am having an issue of getting the query from my last record and adding it to my variable.
If I do it like below: -
let lastRecord = Application.find().sort({$natural:-1}).limit(1).then((result) => { result });
Then I get the value of the variable showing in console.log as : -
Promise { <pending> }
What would I need to do to output this correctly to my full data?
Here is it fixed:
Application.findOne().sort({$natural:-1}).exec().then((lastRecord) => {
console.log(lastRecord); // "lastRecord" is the result. You must use it here.
}, (err) => {
console.log(err); // This only runs if there was an error. "err" contains the data about the error.
});
Several things:
You are only getting one record, not many records, so you just use findOne instead of find. As a result you also don't need limit(1) anymore.
You need to call .exec() to actually run the query.
The result is returned to you inside the callback function, it must be used here.
exec() returns a Promise. A promise in JavaScript is basically just a container that holds a task that will be completed at some point in the future. It has the method then, which allows you to bind functions for it to call when it is complete.
Any time you go out to another server to get some data using JavaScript, the code does not stop and wait for the data. It actually continues executing onward without waiting. This is called "asynchronisity". Then it comes back to run the functions given by then when the data comes back.
Asynchronous is simply a word used to describe a function that will BEGIN executing when you call it, but the code will continue running onward without waiting for it to complete. This is why we need to provide some kind of function for it to come back and execute later when the data is back. This is called a "callback function".
This is a lot to explain from here, but please go do some research on JavaScript Promises and asynchronisity and this will make a lot more sense.
Edit:
If this is inside a function you can do this:
async function someFunc() {
let lastRecord = await Application.findOne().sort({$natural:-1}).exec();
}
Note the word async before the function. This must me there in order for await to work. However this method is a bit tricky to understand if you don't understand promises already. I'd recommend you start with my first suggestion and work your way up to the async/await syntax once you fully understand promises.
Instead of using .then(), you'll want to await the record. For example:
let lastRecord = await Application.find().sort({$natural:-1}).limit(1);
You can learn more about awaiting promises in the MDN entry for await, but the basics are that to use a response from a promise, you either use await or you put your logic into the .then statement.
I have the following code in NodeJS:
var targetDir = tmpDir + date;
try {
fs.statSync(targetDir);
}
catch (e) {
mkdirp.sync(targetDir, {mode: 755});
}
process.chdir(targetDir);
doStuffThatDependsOnBeingInTargetDir();
My understanding is that in NodeJS, functions such process.chdir are asynchronously executed. So if I need execute some code afterwards, how do I guarantee that I'm in the directory before I execute my subsequent function?
If process.chdir took a callback then I would do it in the callback. But it doesn't. This asynchronous paradigm is definitely confusing for a newcomer so I figured I would ask. This isn't the most practical consideration since the code seems to work anyways. But I feel like I'm constantly running into this and don't know how to handle these situations.
process.chdir() function is a synchronous function. As you said yourself, it does not have a callback function to tell if it succeeded or not. It does however throw an exception if something goes wrong so you would want to invoke it inside a try catch block.
You can check if the process successfully changed directory by process.cwd() function.
I'm learning node now and I'm confused about the err parameter.
I thought it's supposed to be the first argument of a callback function but I don't see it in many call back functions. Can anyone explain it to me? Thanks!
There's many different kinds of functions and callback functions in particular. The Node.js standard for callback functions is those of the form:
function(err, arg1, arg2, ...)
Where arg1 and so forth are only present if relevant but the err argument is always first. This is the reverse of a lot of historical JavaScript code where errors would be the last argument.
The Node.js method of forcing the error as the first argument even if there's no error makes ignoring errors harder, you rarely forget to declare that argument, and makes their location predictable.
Now this only applies in the case of a general-purpose callback. That is, there are occasions where calling a function will trigger a singular callback at some point in the future. You'll see them used like this:
doStuff(function(err, successValue) { ... });
There's also the style popularized by jQuery where one or more of your callbacks will be triggered depending on the outcome of the operation:
doStuff({
success: function(successValue) { ... },
error: function(err) { ... },
timeout: function() { ... }
});
Note that in this case you may have both the error and timeout callbacks being fired. You're not obligated to populate all of these, either.
The downside to this approach is the unpredictability of which ones get called and the risk of handling something twice inadvertently.
The error parameter is usually for asynchronous code.
node errors
Most asynchronous methods that accept a callback function will accept an Error object passed as the first argument to that function. If that first argument is not null and is an instance of Error, then an error occurred that should be handled.
app.get() sends get request and return an error like a 404
and you could do something like this res.status(404).render( in app.get()
Express error handling
error-handling functions have four arguments instead of three: (err, req, res, next)
The reason why some code uses err as the first parameter is because some code like fs.readFileis programmed to check if there was an error and to handle it. The author of the API specifically wrote code to check the first argument for an error and handle it.
That's why it is available to you for some methods an unavailable for other methods.
First: a callback is just a function. Different callbacks serve different purposes.
In general, a function that performs an asynchronous action and should "return" a value gets passed a callback function that will take (at least) two arguments: the first is used to pass errors (if any), the second (and following) are used to pass the value(s) that should be returned to the caller.
You noticed that net.createServer() will also take a callback function, but that function only has one argument.
That's because, in this case, the callback isn't used to pass errors and/or values. Instead, it's a function that gets called when a new connection is made to the server.
It's really a bit of a shortcut. This code:
var server = net.createServer(function(connection) {
...
});
Is short for this code:
var server = net.createServer();
server.on('connection', function(connection) {
...
});
server = require('http').createServer(function(request, response){
.. //writehead and stuff
myfunction.Division.getByName(callback, function(params) {
... // [code being executed defining the response]
myresponse = 'hello, world';
response.write(myresponse);
... // rest of the code
My question here is how to write the response when my code has been executed ?
server.on(something) perhaps ? Not sure.
EDIT:
Someone once said if it's stupid but it works it is not stupid.
The mess up was due the async call of this object. Nevertheless I am calling this entire object from external file. using fs
case 'current':
eval ( fs.readFileSync (__dirname+'/data/current/current.js') );
console.log(BB);
break;
I was trying to write my response in this file where my function was, but that's wrong since it never waited for that fail to execute. When I defined before myfunction.Division.getByName a variable called BB which is supposed to be my response (it could go even higher in my code) and modified it inside myfunction.Division.getByName the call after fs function executed perfectly.
So I can get the response.write after the fs call.
So far it's working fine, numerous attempts with timeouts and such, but the sync call from fs is fixing it perfectly. If there are bugs I will look for solution and update again.
My while loop is giving me an error of call stack size exceeded, so I tried to limit it in some ways that did not work and now, I have the following piece:
var hasnext = true;
while(hasnext) {
options.form['formBusca:dataScroller1'] = (++page).toString();
var request = client.post(options, function(error, response, body) {
var html = dom.load(body);
var buttons = html('td.rich-datascr-button');
if (some_length_condition_that_does_not_matter) {
hasnext = false;
}
});
process.stdout.write((page).toString() + '\r');
break;
}
If the logic I learned is still the same nowadays, this while should execute once and only once, since its last statement is a break without condition, so it should end after finising the first iteration.
However, this does not work. In some way, I exceed the call stack everytime. Why does it happen, and how can I prevent it from happening?
RangeError: Maximum call stack size exceeded
Additional information: client is the request module and dom is the cheerio module.
If you are getting an error about the call stack size being exceeded then it's likely a problem with recursion in a function you are calling, not the while loop. You are doing an asynchronous post() call within the loop and there is probably something recursing in there. You can test by commenting out the whole post() block to see if it works, and then commenting out just the code inside the callback. Find where it's actually happening.
The only other possibility is that your call stack is already very deep by the time this code is running and so running normal code like this is exceeding the maximum call stack size. That's easy to test though by just throwing an exception in the loop and seeing how deep your call stack is.
EDIT
It looks like you may be using the request module by mikeal. I use that all the time. I have no idea what your dom variable is referencing though. Depending on what options you are passing in your post() call and what dom.load() is doing, I could easily see your problem being either of those.