I have a problem with a routing function in express.
I call from firebase realtime database , the JSON has a lot of nested data and I would like to retrieve it with a for loop.
router.get('/admin_tavoli', async (req, res) => {
try {
var lista_tavoli = await firebase().ref('tavoli').once('value');
var lista_tavoli_val = lista_tavoli.val();
for(var i in lista_tavoli_val){
console.log(lista_tavoli_val[i].comanda.tavolo);
}
res.send('ok');
} catch (err) {
res.json({ message: err })
}
});
If I keep to the first level to JSON for example
for(var i in lista_tavoli_val){
console.log(lista_tavoli_val[i].comanda);
}
there are no problems.
But if I go deeper to JSON
for(var i in lista_tavoli_val){
console.log(lista_tavoli_val[i].comanda.tavolo);
}
the execution of the program goes in error, but the strange thing is that in the terminal I see the correct data.
Why does this happen?
thanks to all for the help
Related
I need to fetch two different MongoDB collections (db.stats and db.tables ) for the same request req.
Now, in the code below, I am nesting the queries within the callback function.
router.post('/', (req, res) => {
let season = String(req.body.year);
let resultData, resultTable;
db.stats.findOne({Year: season}, function (err, data) {
if (data) {
resultData = getResult(data);
db.tables.findOne({Year: season}, function (err, data) {
if (data) {
resultTable = getTable(data);
res.render('index.html', {
data:{
result : resultData,
message: "Working"}
});
} else {
console.log("Error in Tables");
}
});
} else {
console.log("Error in Stats");
}
});
});
This code works, but there a few things that don't seem right. So my question is:
How do I avoid this nested structure? Because it not only looks ugly but also, while I am processing these requests the client side is unresponsive and that is bad.
What you have right now is known as the callback hell in JavaScript. This is where Promises comes in handy.
Here's what you can do:
router.post('/', (req, res) => {
let season = String(req.body.year);
var queries = [
db.stats.findOne({ Year: season }),
db.tables.findOne({ Year: season })
];
Promise.all(queries)
.then(results => {
if (!results[0]) {
console.log("Error in Stats");
return; // bad response. a better way is to return status 500 here
} else if (!results[1]) {
console.log("Error in Tables");
return; // bad response. a better way is to return status 500 here
}
let resultData = getResult(results[0]);
let resultTable = getTable(results[1]);
res.render('index.html', { data: {
result : resultData,
message: "Working"
} });
})
.catch(err => {
console.log("Error in getting queries", err);
// bad response. a better way is to return status 500 here
});
});
It looks like you are using Mongoose as your ODM to access your mongo database. When you don't pass in a function as the second parameter, the value returned by the function call (e.g. db.stats.findOne({ Year: season })) will be a Promise. We will put all of these unresolved Promises in an array and call Promise.all to resolve them. By using Promise.all, you are waiting until all of your database queries get executed before moving on to render your index.html view. In this case, the results of your database function calls will be stored in the results array in the order of your queries array.
Also, I would recommend doing something like res.status(500).send("A descriptive error message here") whenever there is an error on the server side in addition to the console.log calls.
The above will solve your nested structure problem, but latter problem will still be there (i.e. client side is unresponsive when processing these requests). In order to solve this, you need to first identify your bottleneck. What function calls are taking up most of the time? Since you are using findOne, I do not think that will be the bottleneck unless the connection between your server and the database has latency issues.
I am going to assume that the POST request is not done through AJAX since you have res.render in it, so this problem shouldn't be caused by any client-sided code. I suspect that either one of getResult or getTable (or both) is taking up quite a significant amount of time, considering the fact that it causes the client side to be unresponsive. What's the size of the data when you query your database? If the size of it is so huge that it takes a significant amount of time to process, I would recommend changing the way how the request is made. You can use AJAX on the front-end to make a POST request to the back-end, which will then return the response as a JSON object. That way, the page on the browser would not need to reload, and you'll get a better user experience.
mongodb driver return a promise if you dont send a callback so you can use async await
router.post('/', async(req, res) => {
let season = String(req.body.year);
let resultData, resultTable;
try {
const [data1,data2] = await Promise.all([
db.stats.findOne({Year: season}),
db.tables.findOne({Year: season})
]);
if (data1 && data2) {
resultData = getResult(data1);
resultTable = getTable(data2);
return res.render('index.html', {
data: {
result: resultData,
message: "Working"
}
});
}
res.send('error');
console.log("Error");
} catch (err) {
res.send('error');
console.log("Error");
}
});
I have a big problem.
I want to iterate over collection a result set and for each set i want to find one result.
This looks like this:
router.get('/', function(req, res) {
var floors = [];
var rooms = [];
req.db.collection('floors').find().sort({_id: 1}).forEach(function(floor) {
floors.push(floor);
});
req.db.collection('rooms').find().sort({_id: 1}).forEach(function(room) {
req.db.collection('floors').findOne({_id: new ObjectID(room.floorId)}, function(error, floor) {
room.floor = floor;
rooms.push(room);
});
});
res.render('rooms', { floors: floors, rooms: rooms });
});
The Problem is that the page will be rendered before the iteration is complete.
I tried to use async and promises, but i didn't get it to run.
Basically you have to wait until all your queries are done before sending the rendering result. Unfortunately you don't use promises so this will get a bit messy.
It appears that you are using the native client and according to the docs there is a second callback that gets called when all iterations are done
http://mongodb.github.io/node-mongodb-native/2.2/api/Cursor.html#forEach
router.get('/', function(req, res, next) {
var floors = [];
var rooms = [];
function done(err){
if(err) {
return next(err);
}
res.render('rooms', { floors: floors, rooms: rooms });
}
function getRooms(err){
if(err){
return next(err);
}
req.db.collection('rooms').find().sort({_id: 1}).forEach(function(room) {
// you already have all the floors, no need to hit the db again
floors.find(floor => floor._id === room.floorId); // not sure about this 100% as _id could be an object
}, done);
}
req.db.collection('floors').find().sort({_id: 1}).forEach(function(floor) {
floors.push(floor);
}, getRooms);
});
to be noted that this request will get quite heavy when your db grows.
I am a nodejs beginner and I ran into some callback behaviour that I don't understand. I am using an Express router to write Mongoose objects to Mongodb using a POST request. In the body of the request I am passing in a nested json structure with two fields - jobDetails and examples. The data in jobDetails is used to create a Job Mongoose object and the data in examples is used to create several Example Mongoose objects. The Job and Example objects are linked in that a Job contains a list of Example objects in one of it's fields.
The way I tried to implement this was with callbacks in the following way. Basically I save the Job object to mongo first, then iterate over the examples - each time creating an Example object and linking it to the job via the .job field and also saving the Example object to mongo. Then in the callback to the Example object save function I updated the Job object with the new Example object and saved the updated version to mongo.
router.post('/jobs', function (req, res, next) {
var job = new Job(req.body.jobDetails);
var examples = req.body.examples;
console.log("JOB DETAILS");
console.log(req.body.jobDetails);
console.log("EXAMPLES");
console.log(req.body.examples);
//save job
job.save(function (err, job) {
console.log(err);
});
//save examples
for(i=0; i<examples.length;i++){
var eg = new Example({content: examples[i]});
eg.job=job;
eg.save(function (err, eg){
job.examples.push(eg);
job.save(function(err, job){
console.log(err);
});
console.log(err);
});
}
});
This did not perform as I would expect. Specifically, double the number of examples were actually saved to mongo with several duplicates and some missing. I understand that callbacks are asynchronous but to me that still doesn't explain why double the number of examples would be saved and some would be duplicated and some would be missing.
I eventually got it to work correctly without using callbacks at all in the following way.
router.post('/jobs', function (req, res, next) {
var job = new Job(req.body.jobDetails);
var examples = req.body.examples;
console.log("JOB DETAILS");
console.log(req.body.jobDetails);
console.log("EXAMPLES");
console.log(req.body.examples);
//save job
job.save(function (err, job) {
console.log(err);
});
//save examples
for(i=0; i<examples.length;i++){
var eg = new Example({content: examples[i]});
eg.job=job;
eg.save(function (err, eg){
console.log(err);
});
job.examples.push(eg);
job.save(function(err,job){
console.log(err);
});
}
});
And I'm not sure if this is the optimal solution either. But I would like to know why my original approach resulted in the unintended behaviour.
This should work..
router.post('/jobs', function(req, res, next) {
var job = new Job(req.body.jobDetails);
var examples = req.body.examples;
console.log("JOB DETAILS");
console.log(req.body.jobDetails);
console.log("EXAMPLES");
console.log(req.body.examples);
//save job
job.save(function(err, result) {
if (!err) {
//save examples
for (i = 0; i < examples.length; i++) {
var eg = new Example({
content: examples[i]
});
eg.job = job;
eg.save(function(err, eg) {
job.examples.push(eg);
job.save(function(err, job) {
if (!err)
job.examples = [];
});
console.log(err);
});
}
}
});
});
I would suggest you use a library like async to perform such save operations in a step by step manner. Follow this approach for better readability of code and better results
var async = require('async');
router.post('/jobs', function(req, res, next) {
var job = new Job(req.body.jobDetails);
var examples = req.body.examples;
var savedExamples = [];
console.log("JOB DETAILS");
console.log(req.body.jobDetails);
console.log("EXAMPLES");
console.log(req.body.examples);
async.eachSeries(examples, function iteratee(example, callback) {
var eg = new Example({
content: example
});
eg.job = job;
eg.save(function(err, savedEg) {
if(!err) {
savedExamples.push(savedEg);
}
callback(err)
});
}, function(err) {
if(err) {
//handle errors
}
job.examples = savedExamples;
job.save(function(err,job) {
if(err) {
//handle errors
}
//success callback
});
});
});
Using this approach you will have to call the save function for job only once after all other operations are completed. If an error is triggered at any point the whole flow is stopped. For more info regarding async library refer this!
Because of an error that I had in the client, my node / express API was getting the exact same PUT request twice simultaneously everytime that a form was sent.
In response, the server was always crashing with the following error:
Error: Can't set headers after they are sent
Now I fixed the client so server is not crashing anymore, but I'd like to fix the API properly. Anyone could kill my server by sending two PUT requests simultaneously.
So my question:
How to best handle the "can't set headers" error with Express?
What is the mistake in my server code that allowed for that error to happen in the first place?
Here is my server code:
server.js
router.route('/deals/:id').put(deal.update);
deal.js
var r = {};
exports.update = function(req, res) {
r = res;
var u = {
$set: {}
}
for(var x in req.body){
if (['name','stages'].indexOf(x)>=0)
u.$set[x] = req.body[x];
}
Pipeline // this is a mongoose object
.findOneAndUpdate({'_id.id':req.params.id},u,{new:true})
.exec(respond);
}
function respond(err, data) {
if (err) {
return r.send(err);
} else {
return r.json(data);
}
}
UPDATE:
The error came from how I declared a global r in order to have a single response callback outside for all of my methods.
I fixed it by having one respond callback for every method.
Still interested to know if I could get away with a bind(res) instead?
You missed a return, so that when there was an error, you called res.send twice. With the return, execution will stop after the first one.
I see what you are trying to do for your follow on question, but making a global variable r is definitely not a good way to go. The way I do patterns like this is to pass res to the follow on function - see below. There may be a better way, but this seems legitimate at least.
exports.update = function(req, res) {
var u = {
$set: {}
}
for(var x in req.body){
if (['name','stages'].indexOf(x)>=0)
u.$set[x] = req.body[x];
}
Pipeline // this is a mongoose object
.findOneAndUpdate({'_id.id':req.params.id},u,{new:true})
.exec(function(err, data) { respond(err, data, res) });
}
function respond(err, data, res) {
if (err) {
return res.send(err);
}
return res.json(data);
}
Ok, lets say I have two Models. Contract and CommLog. Both work find independently but I need many CommLog to relate to each Contract.
In the ContractSchema trying async
ContractSchema.methods.getCommLog = function getCommLog(){
var log = false;
async.parallel([
function(){
CommLog.find({commType:'contract',parent:this._id},function(err,comms){
log = comms;
});
}],
function(){return log;});
};
Where I am trying to use it
router.get('/:code', function(req, res, next) {
Contract.findOne({accessCode:req.params.code},function(err,contract){
if(err)
res.send(err);
var data;
if(contract != null){
var comms = contract.getCommLog();
data = {error:false,data:contract,commlog:comms}
}else{
data = {error:true,message:"No Contract"}
}
res.json(data);
});
});
Where it shows var comms = contract.getCommLog(); It is never returning anything because the getCommLog() is not executing async...
I think its my misunderstanding of mongoose querying, so if you understand what I am trying to accomplish, please let me know what I am doing wrong. I have tried without async which would always return false.
The find call can return all matching results with one query, so I don't think you need async here. The reason it is not populating correctly when you call res.json(data) is because you are not waiting for the method call to finish before you fire off your server response. You would be better off nesting an additional CommLogs.find call within the Contract.find call, and only sending your response once that finishes.
//pseudo code:
Contract.find({}, function(err, contract) {
if(err || !contract) {
//return error response
}
else {
CommLogs.find({contract: contract._id}, function(err, commlogs) {
if(err || !commlogs) {
//return error response 2
}
else {
res.json({errors: false, contract: contract, commlogs: commlogs});
}
});
}
}