I a trying to solve a programming problem in node js that needs to a rad a file.
What is the right way to do this using qio.
Here is my program
var express = require('express')
var qfs = require('q-io/fs')
var q = require('q')
var fs = require('fs')
var app = express()
app.get('/books', function(req, res){
qfs.read(process.argv[3])
// .then( function(buf){res.json(JSON.parse(buf))})
// .done()
.then(res.send).done()
/* .then(null, function(abc, err){
res.json(err)
console.log("Error handler")
res.status(500)
})*/
})
app.listen(process.argv[2])
I know I can read file synchronously, also the following code works
qfs.read(process.argv[3])
.then( function(buf){res.json(JSON.parse(buf))})
.done()
but the main code gives error, I understand this is because the app object has gone out of scope because the request handler has probably already returned.
/home/ubuntu/mahesh/node_tries/node_modules/q/q.js:155
throw e;
^
TypeError: Cannot read property 'req' of undefined
at send (/home/ubuntu/mahesh/node_tries/node_modules/express/lib/response.js:103:17)
at _fulfilled (/home/ubuntu/mahesh/node_tries/node_modules/q/q.js:834:54)
at self.promiseDispatch.done (/home/ubuntu/mahesh/node_tries/node_modules/q/q.js:863:30)
at Promise.promise.promiseDispatch (/home/ubuntu/mahesh/node_tries/node_modules/q/q.js:796:13)
at /home/ubuntu/mahesh/node_tries/node_modules/q/q.js:604:44
at runSingle (/home/ubuntu/mahesh/node_tries/node_modules/q/q.js:137:13)
at flush (/home/ubuntu/mahesh/node_tries/node_modules/q/q.js:125:13)
What is happening here?? At what point does the handler for express.js return when res.end is called?
A little late, but I stumbled upon this problem myself and wanted to help anyone who encounters this in the future.
The line on which your program fails, in response.js, is this one:
var req = this.req;
When res.send() is invoked, the function send is invoked with the context of res (so "this" is res), and this.req is the request object.
However, when you passed the function as a variable to the promise, it lost the context and became just a function. So, when it was invoked, "this" was undefined and this.req became an error. So to fix your problem, replace
.then(res.send).done()
with
.then(books => res.send(books)).done()
Related
I'm using express API connected to MongoDB and when I query the DB to get the result, I can use them only within the scope of the callback and even I assign the value of the result to a globally declared variable, when using console.log(), the result is undefined
here is my code
const MongoClient = require('mongodb').MongoClient;
const express = require('express');
const assert = require('assert');
const app = express();
const PORT = process.env.PORT||13000;
app.get('/',(req,res) => {
let myIncomingData;
const url = 'mongodb://localhost:27017/';
MongoClient.connect(url,{useNewUrlParser:true},(err,db)=>{
assert.equal(err,null);
db.db('my_posts').collection('posts').find().toArray((err,result) =>{
assert.equal(err,null);
console.log(result); // here the result gets printed
myIncomingData = result;
}); // db.collection
db.close();
}); // MongoClient.connect
console.log(myIncomingData); // print undefined why ??
}) // app.get
app.listen(PORT,()=>console.log('server is running ...'))
node is asynchronous, so at the time that you're doing the console.log, myIncomingData is still undefined (you can even see this by the order of the console.logs -- first it will log that undefined from that second console.log and then it will log the console.log that's in the callback). If you wanted to see this in action, you could try doing something like setTimeout(() => console.log(myIncomingData), 5 * 1000) -- by delaying 5 seconds, you can be pretty sure that the callback code will have run, and if you want to do anything further with that value, you'll need to do it from within the callback (either inline or by calling another function)
async/await or Promises can make code like this easier to read and reason about. If you want to continue using callbacks, it'd be worth taking a look at a callback tutorial to make sure you understand what's going on with them.
I was trying to get my hands dirty on advanced NodeJS concepts by Stephen Grinder.
Trying to teach the mere basics of redis, Stephen did something like this
app.get('/api/blogs', requireLogin, async (req, res) => {
//This time we are setting
const redis = require('redis')
const redisURL = 'redis://127.0.0.1:6379';
const client = redis.createClient(redisURL);
const util = require('util')
client.get = util.promisify(client.get)
//We are checking if we have ever fetched any blogs related to the user with req.user.id
const cachedBlog = await client.get(req.user.id)
//if we have stored list of blogs, we will return those
if (cachedBlog) {
console.log(cachedBlog)
console.log("Serving from Cache")
return res.send(JSON.parse(cachedBlogs))
} //this is JSONIFIED as well so we need to convert it into list of arrays
console.log("serving from Mongoose")
//if no cache exsist
const blogs = await Blog.find({_user: req.user.id})
//blogs here is an object so we would need to stringfy it
res.send(blogs);
client.set(req.user.id, JSON.stringify(blogs))
})
And it works without any error but in last two lines, if we change the order
client.set(req.user.id, JSON.stringify(blogs))
res.send(blogs);
it does not display my blog.
Since inside the API, I am considering both of them to run asynchronously, I thought order won't matter.
Can anyone tell me what am I missing or unable to comprehend?
Since OP asks to understand the difference, not fix the code:
express runs the request handler function and catches synchronous errors (they become http500 errors). It doesn't do anything with the promise returned from the async function and doesn't use await internally, so you don't get error handling for async functions for free. All asynchronous errors need to be caught inside and passed to the next callback or handled in your code by sending an appropriate status code and error body.
When an error occurs, JS stops and doesn't execute any more lines in the function. So if an error is thrown from client.set placed before res.send, the line with send won't run and no response is sent. The browser should continue waiting for the response until timeout.
The other way around - you send response before the error, so you get the page, but the response doesn't end (I'd assume the connection remains open as if the backend was going to send more) but ever since early versions of Firefox browsers start rendering HTML as it's downloaded, so you see a page even though the browser is still waiting for the response to finish.
The order of these two lines doesn't matter but that res.send isn't called in case client.set goes first means that there's an error. If an error occurs in async function, this may result in UnhandledPromiseRejectionWarning warning that will be visible in console.
There are several problems with this snippet.
That the error occurs even when though client.set is asynchronous suggests that client.set causes synchronous error which wasn't caught.
client.set wasn't promisified but it should for correct control flow. That it wasn't provided with callback argument could be a reason why it caused an error.
As explained in this answer, Express doesn't support promises, all rejections should be explicitly handled for proper error handling.
All common code like require goes outside middleware function. It should be:
const redis = require('redis')
const redisURL = 'redis://127.0.0.1:6379';
const client = redis.createClient(redisURL);
const util = require('util')
client.get = util.promisify(client.get)
client.set = util.promisify(client.set)
app.get('/api/blogs', requireLogin, async (req, res, next) => {
try {
const cachedBlog = await client.get(req.user.id)
if (cachedBlog) {
return res.send(JSON.parse(cachedBlogs))
}
const blogs = await Blog.find({_user: req.user.id});
await client.set(req.user.id, JSON.stringify(blogs));
res.send(blogs);
} catch (err) {
next(err);
}
})
Most popular libraries have promise counterparts that allow to skip boilerplate promisification code, this applies to redis as well.
The two task will runs asynchronously but the order of execution matters.
client.set(req.user.id, JSON.stringify(blogs)) execution starts first, but as you are not using await, the promise will not be resolved but execution has already started.
After that res.send() will execute.
You are not getting the response implies that there is some error in the execution of client.set(req.user.id, JSON.stringify(blogs)).
Use Try catch block to trace this error (as mentioned in other answers).
You can also add these lines in your code to catch other "unhandledRejection" or "uncaughtException" error (if any).
process.on('unhandledRejection', (err) => {
logger.error('An unhandledRejection error occurred!');
logger.error(err.stack)
});
process.on('uncaughtException', function (err) {
logger.error('An uncaught error occurred!');
logger.error(err.stack);
});
I want to test the error in a request return. I'm using nock in my tests, how can I force Nock to provoke an error? I want to achieve 100% test coverage and need to test err branch for that
request('/foo', function(err, res) {
if(err) console.log('boom!');
});
Never enter in the if err branch. Even if hit err is a valid response, my Nock line in test looks like this
nock('http://localhost:3000').get('/foo').reply(400);
edit:
thanks to some comments:
I'm trying to mock an error in the request. From node manual:
https://nodejs.org/api/http.html#http_http_request_options_callback
If any error is encountered during the request (be that with DNS resolution, TCP level errors, or actual HTTP parse errors) an 'error' event is emitted on the returned request object
An error code (e.g. 4xx) doesn't define the err variable. I'm trying to mock exactly that, whatever error that defines the err variable and evaluates to true
Use replyWithError.
From the docs:
nock('http://www.google.com')
.get('/cat-poems')
.replyWithError('something awful happened');
When you initialise a http(s) request with request(url, callback), it returns an event emitter instance (along with some custom properties/methods).
As long as you can get your hands on this object (this might require some refactoring or perhaps it might not even be suitable for you) you can make this emitter to emit an error event, thus firing your callback with err being the error you emitted.
The following code snippet demonstrates this.
'use strict';
// Just importing the module
var request = require('request')
// google is now an event emitter that we can emit from!
, google = request('http://google.com', function (err, res) {
console.log(err) // Guess what this will be...?
})
// In the next tick, make the emitter emit an error event
// which will trigger the above callback with err being
// our Error object.
process.nextTick(function () {
google.emit('error', new Error('test'))
})
EDIT
The problem with this approach is that it, in most situations, requires a bit of refactoring. An alternative approach exploits the fact that Node's native modules are cached and reused across the whole application, thus we can modify the http module and Request will see our modifications. The trick is in monkey-patching the http.request() method and injecting our own bit of logic into it.
The following code snippet demonstrates this.
'use strict';
// Just importing the module
var request = require('request')
, http = require('http')
, httpRequest = http.request
// Monkey-patch the http.request method with
// our implementation
http.request = function (opts, cb) {
console.log('ping');
// Call the original implementation of http.request()
var req = httpRequest(opts, cb)
// In next tick, simulate an error in the http module
process.nextTick(function () {
req.emit('error', new Error('you shall not pass!'))
// Prevent Request from waiting for
// this request to finish
req.removeAllListeners('response')
// Properly close the current request
req.end()
})
// We must return this value to keep it
// consistent with original implementation
return req
}
request('http://google.com', function (err) {
console.log(err) // Guess what this will be...?
})
I suspect that Nock does something similar (replacing methods on the http module) so I recommend that you apply this monkey-patch after you have required (and perhaps also configured?) Nock.
Note that it will be your task to make sure you emit the error only when the correct URL is requested (inspecting the opts object) and to restore the original http.request() implementation so that future tests are not affected by your changes.
Posting an updated answer for using nock with request-promise.
Let's assume that your code calls request-promise like this:
require('request-promise')
.get({
url: 'https://google.com/'
})
.catch(res => {
console.error(res);
});
you can set up nock like this to simulate a 500 error:
nock('https://google.com')
.get('/')
.reply(500, 'FAILED!');
Your catch block would log a StatusCodeError object:
{
name: 'StatusCodeError',
statusCode: 500,
message: '500 - "FAILED!"',
error: 'FAILED!',
options: {...},
response: {
body: 'FAILED!',
...
}
}
Your test can then validate that error object.
Looks like you're looking for an exception on a nock request, this maybe can help you:
var nock = require('nock');
var google = nock('http://google.com')
.get('/')
.reply(200, 'Hello from Google!');
try{
google.done();
}
catch (e) {
console.log('boom! -> ' + e); // pass exception object to error handler
}
I am running node.js with express. I wrote a node module with methods in it so when you go to
http://bla.com/module_name/method_name
it will run the method.
The method follows the typical style of
exports.method_name(req, res, next);
my main app does something like this:
app.all("*", resSetup, controller, render);
and controller is the thing that will call the method based on the path.
it seems that if there is an undefined variable error in the method, express will just hang there and not throw any error. Nothing will appear in the console log either. I can put a console message right before and after where the error occurs and the before will appear in the log, and after will not.
I can wrap it in a try/catch and get this:
[ReferenceError: blabla is not defined]
but no line numbers or anything.
My guess is that express is somehow preventing the errors from coming up. When I put the error in the function called "controller" that is directly in the route, it shows that error correctly.
It might not matter too much, but here is the code I am working on:
https://github.com/RobKohr/quick-site/blob/master/index.js
Line 189 is where the method call happens.
Building on Ruairi's comment above, I had this same issue with when using 'q' (https://github.com/kriskowal/q) and promises with express - node would hang and no error was generated.
By adding a catch to the end of the promise 'callback' chain I was able to see the error and print it to console etc.
The code ends up looking like:
export function index(req, res) {
//Create the 'promise'
var request = req.body;
var queryJobID = req.query.jobId;
console.log('queryJobID: ' + queryJobID);
var jobStatusPromsie = jobManager.job.getStatus(queryJobID);
Q.all([jobStatusPromsie])
.then(
function (result) {
var responseData = {};
console.log('Job Status Response received');
if (result != null) {
//Without the .catch below an error here will be 'silent'
console.log('jobStatus result: ' + util.inspect(result, false, null));
responseData['status'] = 'OK';
responseData['progress'] = result.progress;
res.json(responseData);
} else {
console.log('jobStatus Error');
responseData['status'] = 'Error';
}
res.json(responseData);
console.log('jobStatus Response data sent');
},
function (error) {
console.log('Error while getting job status:', error);
res.json("Error");
})
.catch(function(err) {
//handle errors
console.log('Promise error while getting job status:', err);
});
}
Express heavily relies on Nodes asynchronous nature. Seeing errors thrown like on line 30 would give me the creeps if I was maintaining this. Try refactoring your code to only use the next(err) pattern.
The reason that you app is hanging is that Express hasn't finished the HTTP response (eg: res.send()). This means you have broken plumbing where an Error has bubbled up the call stack but not redirected into the Express middleware pipeline. Try registering some error middleware to see if it gets called with your error.
When creating a POST request with valid JSON, the bodyParser parses the body of the POST request correctly. However, if I submit an invalid JSON string as the body, I receive the error:
SyntaxError: Unexpected token ILLEGAL
at parse (native)
at IncomingMessage.<anonymous>(/home/.../middleware/bodyParser.js:69:15)
...
So, it appears that the body parser is failing during the parsing of the body. However, I would like to catch this failure and return an error. I'm unsure what I can do to catch it, so any help would be appreciated. Thanks.
This is in the connect.js bodyParser middleware. It DOES do a try/catch and then calls next(err). You should be able to catch this error and handle with additional custom code using the app.error() callback hook. http://expressjs.com/guide.html#error-handling
For some reason, when using express/connect, JSON.parse doesn't throw exceptions, which is why your error handler doesn't fire.
I've logged an issue with express to find out what's going on, but in the meantime you can use this workaround:
express.bodyParser.parse['application/json'] = function(data) {
var result = JSON.parse(data)
if (typeof result != 'object') {
throw new Error('Problems parsing JSON')
}
return result;
}
app.use(express.bodyParser());
update: this issue is not familiar to the author of express, so I am wondering if it's another library causing it. Will have to dismantle my code piece by piece to figure out where this behaviour is being introduced.
Try to put your
app.use(express.bodyParser());
after
app.use(express.errorHandler(...))
solved it for me.
You may also adapt the following code to manage the error
express.bodyParser.parse['application/json'] = function(req, options, fn){
var buf = '';
req.setEncoding('utf8');
req.on('data', function(chunk){ buf += chunk });
req.on('end', function(){
try {
req.body = buf.length
? JSON.parse(buf)
: {};
fn();
} catch (err){
fn(new Error('Problems parsing JSON'));
}
});
};
bodyParser must be above app.use(app.router), it doesn't matter relative location to error handler as Perki