I'm starting to use NodeJs recently and I'm trying to create a API that will get some information from web compile it and show to the user.
My question is the follow
router.get('/', function (req, res, next) {
https.get(pageUrl, function (res) {
res.on('data', function (responseBuffer) {
//Important info;
info = responseBuffer;
}
}
res.render('page', { important: info});
}
How can I wait until I have the "info" var and then send the res.render. Because right now if I try to wait it usually the program ends and don't wait the result.
Thanks.
Assuming your https.get call gives you a stream with an 'end' event [1], you can do the following:
router.get('/', function (req, res, next) {
https.get(pageUrl, function (res) {
var info;
res.on('data', function (responseBuffer) {
//Important info;
info = responseBuffer;
}
res.on('end', function() {
res.render('page', { important: info});
})
}
}
Note that the above code will not work because you shadowed the base res parameter with the res parameter from the https.get callback.
Also, note that the 'data' event may be emitted several times (again, assuming a standard stream implementation[1]), so you should accumulate the results inside your info variable.
[1] Could you please post more information about your code, such as where the https library comes from (is it the standard HTTPS lib?).
Personal thought: I highly suggest using the request module, disponible on NPM via npm install request, for HTTP(S) requests to external services. It's got a neat interface, is simple to use and handles a lot of situations for you (redirects are one example, JSON and Content-Type another).
Related
I have a website that will use EJS templating to display data from my database. Instead of creating a second API just to do this, I would like to use the pre-existing API that other developers can use. Is there a way to get data from my API in the most meta way possible.
main app route
app.get("/facts/random", async (req, res) => {
/* Make request to /api/random here and get JSON data */
});
API route
app.get("/api/random/", async (req, res) => {
let results = await Fact.find();
const randomResult = Math.floor(Math.random() * results.length);
return res.json(results[randomResult]);
})
I want to make a request from the first route to the second route using the most meta way possible.
If /api/random is in another microservice then you can use Axios to make an http call, but if it is in the same Express App then refactor the code and turn it into a function and call that function in both the controllers.
Usually, you don't have your own http server make a request to itself. Usually, you just factor out the functionality the other API is using into an exported function and you call the function directly.
It's definitely more efficient that way since there's really no reason to package it up into an http request, go through the network stack, parse it as an http request in your server, process it, create an http response, send that back through the network stack, parse the response, then process it when you could avoid 6 of those 8 steps by just calling a common function you factored out.
app.get("/facts/random", async (req, res) => {
let randomResult = await apiRandom();
...
});
async function apiRandom() {
let results = await Fact.find();
const randomResult = Math.floor(Math.random() * results.length);
return results[randomResult];
}
app.get("/api/random/", async (req, res) => {
let randomResult = await apiRandom();
res.json(randomResult);
});
If, for some reason, you do want to make requests to your own http server, then get a library like got() or axios() that makes that easy.
I'm quite new to Nodejs. In the following code I am getting json data from an API.
let data_json = ''; // global variable
app.get('/', (req, res) => {
request('http://my-api.com/data-export.json', (error, response, body) => {
data_json = JSON.parse(body);
console.log( data_json ); // data prints successfully
});
console.log(data_json, 'Data Test - outside request code'); // no data is printed
})
data_json is my global variable and I assign the data returned by the request function. Within that function the json data prints just fine. But I try printing the same data outside the request function and nothing prints out.
What mistake am I making?
Instead of waiting for request to resolve (get data from your API), Node.js will execute the code outside, and it will print nothing because there is still nothing at the moment of execution, and only after node gets data from your api (which will take a few milliseconds) will it execute the code inside the request. This is because nodejs is asynchronous and non-blocking language, meaning it will not block or halt the code until your api returns data, it will just keep going and finish later when it gets the response.
It's a good practice to do all of the data manipulation you want inside the callback function, unfortunately you can't rely on on the structure you have.
Here's an example of your code, just commented out the order of operations:
let data_json = ''; // global variable
app.get('/', (req, res) => {
//NodeJS STARTS executing this code
request('http://my-api.com/data-export.json', (error, response, body) => {
//NodeJS executes this code last, after the data is loaded from the server
data_json = JSON.parse(body);
console.log( data_json );
//You should do all of your data_json manipluation here
//Eg saving stuff to the database, processing data, just usual logic ya know
});
//NodeJS executes this code 2nd, before your server responds with data
//Because it doesn't want to block the entire code until it gets a response
console.log(data_json, 'Data Test - outside request code');
})
So let's say you want to make another request with the data from the first request - you will have to do something like this:
request('https://your-api.com/export-data.json', (err, res, body) => {
request('https://your-api.com/2nd-endpoint.json', (err, res, body) => {
//Process data and repeat
})
})
As you can see, that pattern can become very messy very quickly - this is called a callback hell, so to avoid having a lot of nested requests, there is a syntactic sugar to make this code look far more fancy and maintainable, it's called Async/Await pattern. Here's how it works:
let data_json = ''
app.get('/', async (req,res) => {
try{
let response = await request('https://your-api.com/endpoint')
data_json = response.body
} catch(error) {
//Handle error how you see fit
}
console.log(data_json) //It will work
})
This code does the same thing as the one you have, but the difference is that you can make as many await request(...) as you want one after another, and no nesting.
The only difference is that you have to declare that your function is asynchronous async (req, res) => {...} and that all of the let var = await request(...) need to be nested inside try-catch block. This is so you can catch your errors. You can have all of your requests inside catch block if you think that's necessary.
Hopefully this helped a bit :)
The console.log occurs before your request, check out ways to get asynchronous data: callback, promises or async-await. Nodejs APIs are async(most of them) so outer console.log will be executed before request API call completes.
let data_json = ''; // global variable
app.get('/', (req, res) => {
let pr = new Promise(function(resolve, reject) {
request('http://my-api.com/data-export.json', (error, response, body) => {
if (error) {
reject(error)
} else {
data_json = JSON.parse(body);
console.log(data_json); // data prints successfully
resolve(data_json)
}
});
})
pr.then(function(data) {
// data also will have data_json
// handle response here
console.log(data_json); // data prints successfully
}).catch(function(err) {
// handle error here
})
})
If you don't want to create a promise wrapper, you can use request-promise-native (uses native Promises) created by the Request module team.
Learn callbacks, promises and of course async-await.
I'm looking for a solution to waiting for an event to happen before sending a HTTP response.
Use Case
The idea is I call a function in one of my routes: zwave.connect("/dev/ttyACM5"); This function return immediately.
But there exists 2 events that notice about if it succeed or fail to connect the device:
zwave.on('driver ready', function(){...});
zwave.on('driver failed', function(){...});
In my route, I would like to know if the device succeed or fail to connect before sending the HTTP response.
My "solution"
When an event happen, I save the event in a database:
zwave.on('driver ready', function(){
//In the database, save the fact the event happened, here it's event "CONNECTED"
});
In my route, execute the connect function and wait for the event to
appear in the database:
router.get('/', function(request, response, next) {
zwave.connect("/dev/ttyACM5");
waitForEvent("CONNECTED", 5, null, function(){
response.redirect(/connected);
});
});
// The function use to wait for the event
waitForEvent: function(eventType, nbCallMax, nbCall, callback){
if(nbCall == null) nbCall = 1;
if(nbCallMax == null) nbCallMax = 1;
// Looking for event to happen (return true if event happened, false otherwise
event = findEventInDataBase(eventType);
if(event){
waitForEvent(eventType, nbCallMax, nbCall, callback);
}else{
setTimeout(waitForEvent(eventType, callback, nbCallMax, (nbCall+1)), 1500);
}
}
I don't think it is a good practice because it iterates calls over the database.
So what are your opinions/suggestions about it?
I've gone ahead and added the asynchronous and control-flow tags to your question because at the core of it, that is what you're asking about. (As an aside, if you're not using ES6 you should be able to translate the code below back to ES5.)
TL;DR
There are a lot of ways to handle async control flow in JavaScript (see also: What is the best control flow module for node.js?). You are looking for a structured way to handle it—likely Promises or the Reactive Extensions for JavaScript (a.k.a RxJS).
Example using a Promise
From MDN:
The Promise object is used for asynchronous computations. A Promise represents a value which may be available now, or in the future, or never.
The async computation in your case is the computation of a boolean value describing the success or failure to connect to the device. To do so, you can wrap the call to connect in a Promise object like so:
const p = new Promise((resolve) => {
// This assumes that the events are mutually exclusive
zwave.connect('/dev/ttyACM5');
zwave.on('driver ready', () => resolve(true));
zwave.on('driver failed', () => resolve(false));
});
Once you have a Promise representing the state of the connection, you can attach functions to its "future" value:
// Inside your route file
const p = /* ... */;
router.get('/', function(request, response, next) {
p.then(successful => {
if (successful) {
response.redirect('/connected');
}
else {
response.redirect('/failure');
}
});
});
You can learn more about Promises on MDN, or by reading one of many other resources on the topic (e.g. You're Missing the Point of Promises).
Have you tried this? From the look of it, your zwave probably have already implemented an EventEmmiter, you just need to attach a listener to it
router.get('/', function(request, response, next) {
zwave.connect("/dev/ttyACM5");
zwave.once('driver ready', function(){
response.redirect(/connected);
});
});
There is a npm sync module also. which is used for synchronize the process of executing the query.
When you want to run parallel queries in synchronous way then node restrict to do that because it never wait for response. and sync module is much perfect for that kind of solution.
Sample code
/*require sync module*/
var Sync = require('sync');
app.get('/',function(req,res,next){
story.find().exec(function(err,data){
var sync_function_data = find_user.sync(null, {name: "sanjeev"});
res.send({story:data,user:sync_function_data});
});
});
/*****sync function defined here *******/
function find_user(req_json, callback) {
process.nextTick(function () {
users.find(req_json,function (err,data)
{
if (!err) {
callback(null, data);
} else {
callback(null, err);
}
});
});
}
reference link: https://www.npmjs.com/package/sync
Background
Yes, there are a lot of different Node.js logging library winston, bunyan and console.log. It's easy to log down the information of the specific request when it has called and when and what information would be in response.
The problem
The problem begins with the sub function calls. When under one request your calling multiple functions which also uses the same logging, how would you pass the request meta - data to these log calls (function parameters seems to be one possible way but these are really messy) ?
Example
Small visual for coders:
// Middleware to set some request based information
app.use(function (req, res, next) {
req.rid = 'Random generated request id for tracking sub queries';
});
app.get('/', function (req, rest) {
async.series({
'users': async.apply(db.users.find),
'posts': async.apply(db.posts.find),
}, function (err, dbRes) {
console.log('API call made ', req.rid)
res.end(dbRes);
});
});
// Now the database functions are in other file but we also need to track down the request id in there
(db.js)
module.exports = {
users: {
find: function () {
console.log('Calling users listing ', req.rid); // ERROR this is not possible to access, not in this scope
// Make query and return result
}
},
posts: {
find: function () {
console.log('Calling post listing ', req.rid); // ERROR this is not possible to access, not in this scope
// Make query and return result
}
}
};
You can log your requests with simple conf in your app.js with;
app.use(function(req, res, next){
console.log('%s %s', req.method, req.url);
next();
});
However, you need to provide logs for specific functions in your controller.
I am using Node.js with Express and have code similar to this as part of my routes:
requireLogin: function(req, res, next) {
User.find(req.session.userId)
.on('success', function(user) {
req.addListener('data', function(chunk) {
console.log("DATA: " + chunk);
}
next()
}
}
I am using Sequelize and the User.find method is accessing the database. The trouble is, the request 'data' event that I bind to is never fired. It seems that the data event had already been triggered and handled by the time the user is returned from the database and it's too late to do anything with it. In the example above I could just move the req.addListener to outside the database callback, but in reality I am calling next() here which can't be moved.
All of the following route middleware that is called by next() then doesn't have access to the request data since these events have already been fired. Worse than that, they just hang waiting for the data event from req because it has already happened.
How can I somehow delay the data event so that it can be bound to from within the database callback? Or have I misunderstood something fundamental and need to change my way of going about this?
Thanks a lot.
Edit: I found a relevant discussion in the nodejs Google group which suggests there isn't a solution that will work for me.
var cache = new function () {
var arr = [],
cbs = [];
this.add = function(data) {
arr.push(data);
cbs.forEach(function(cb) {
cb(arr);
});
}
this.get = function(cb) {
cbs.push(arr);
if (arr.length > 0) {
cb(arr);
}
}
};
req.addListener('data', function(chunk) {
cache.add(chunk);
};
User.find(
req.session.userId
).on('success', function(user) {
cache.get(function(data) {
// stuff
next();
});
};
I presume what you actually want is some kind of message caching. Now this is a vague proof of concept. What you actually want depends on your code.
If you have any kind of deferred library / abstraction available then the code will become a lot smaller.