Nothing important
My first question here on stackoverflow. I've used it for years to find answers, but now I need a bit of guidance. I'm new to node and express and the async way of structuring an app.
Goal - A REST interface with validation and neDB database
I got the following code working. POST a new user is the only route. It's based on many answers and tuts mixed together. I find it hard to scaffold out the logic, to get a structure you can build on.
I'm not sure at all whether this structure is crap or not. Any advice would be appreciated.
Main file is initializing the database, middleware validator, and starting the app.
// rest.js
var express = require('express'),
bodyParser = require('body-parser'),
validator = require('express-validator'),
db = require('./database/db'),
userRouter = require('./routers/users');
db.init();
var app = express();
app.use(bodyParser.urlencoded({ extended: true }));
app.use(bodyParser.json());
app.use(validator());
app.use('/api/users', userRouter);
var port = process.env.PORT || 8080;
app.listen(port);
Database
This question/answer made me create the small database module with alterations.
How do you pass objects around node express application?
It doesn't have much attention. Maybe because it's very obvious or maybe not a good answer.
The idea is that whem i get multiple collections, they all get initialized on startup, but I can request a single collection if that's all the module needs, or I can get the entire db object back if another module would require that.
// database/db.js
var nedb = require('nedb');
var db = {};
db.init = function() {
db.users = new nedb({ filename: './database/data/users', autoload: true });
db.users.ensureIndex({ fieldName: 'username', unique: true }, function (err) {});
db.users.ensureIndex({ fieldName: 'email', unique: true }, function (err) {});
};
db.get = function(collection) {
if (collection && db[collection])
return db[collection];
return db;
}
module.exports = db;
Router
I require the User Model here and use the express-validator and sanitizes the request before passing it on to the model, based on a minimalist key schema in the model. I don't have any controllers. If I had (or when I do), I would put the validation there. The router is supposed to send the response and status right?
// routers/users.js
var express = require('express'),
_ = require('lodash'),
User = require('../models/user');
var userRouter = express.Router();
userRouter.route('/')
.post(function(req, res) {
req.checkBody('username', 'Username must be 3-20 chars').len(3,20);
req.checkBody('email', 'Not valid email').isEmail();
req.checkBody('password', 'Password must be 6-20 chars').len(6,20);
var err = req.validationErrors();
if (err) {
res.status(422).send(err);
return;
}
var data = _.pick(req.body, _.keys(User.schema));
User.create(data, function (err, newData) {
if (err) {
res.status(409).send(err);
} else {
res.status(201).send(newData);
}
});
});
module.exports = userRouter;
Model
The model requires the database module and gets the "connection". Is this OK?
// models/user.js
var db = require('../database/db');
var User = function (data) {
this.data = data;
};
User.schema = {
_id: null,
username: null,
email: null,
password: null
};
User.create = function (data, callback) {
db.get('users').insert(data, callback);
};
module.exports = User;
Thanks for reading this far. Now, my question is:
Is there something fundamentally wrong with this setup, concerning the database usage and the validation logic. I know the model looks stupid :)
Related
I'm getting my callback from my dao and into my service and in my service i have iterated the list of object to be sent to my dao.
My service code
var dao = require('./dao');
var async = require('async');
exports.addUser = function(obj,callback) {
async.forEachOf(obj,function(value,key,callback){
dao.addUser(value,function(data){
callback(data);
})
})
}
this callback is not going to my control layer
Control layer
var express = require('express');
var app = express();
var service = require('./service');
app.use(express.urlencoded({ extended: false }));
app.use(express.json());
app.post('/addPerson',function(req,res){
var obj = req.body;
console.log(obj);
console.log("......",obj.name);
console.log("......",obj.age);
service.addUser(obj,function(data) {
console.log("---->",data);
res.json(data);
})
})
var server = app.listen(8080,function(){});
i need to send the data back to the browser
There are two problems with your existing code
1. By referring callback you are calling callback of the asyn.forEachOf
2. Trying to call ur original callback (which intern does res.json(data)). Here you are trying to send multiple response which is not possible.
Try using async.mapValues instead
exports.addUser = function(obj,callback) {
async.mapValues(obj,function(value,key,cb){
dao.addUser(value,function(data){
cb(null, data);
})
}, function(err, result) {
// result is now a map of results for each key
callback(result);
});
}
I've posted an approximation of my node web application below. The original problem I had is that I want to be able to let the client know on post to createentity whether the insert was successful and the id of the inserted entity. However, connection.query having a callback rather than running synchronously, I can't use the entityservice how I'd expect in another language, ie simply returning the result synchronously. There are several solutions, and I'm curious which is the best/common practice re node.js or if there is another I'm not thinking of.
Passing res down to the service, and responding within a callback; seems poor practice
Similarly, passing functions to execute after success/failure to the service; also seems poor practice
Returning a promise from the service and setting res based on resolution/failure; seems like services shouldn't return promises but I'm new to node
Some better method using appropriate features of node.js of which I'm unaware
trying to change the service such that it runs synchronously and just returning the result Other questions/answers have made me leery that this is possible or wise
structure the application some other way
something else?
//app.js
var express = require('express');
var bodyParser = require('body-parser')
var path = require('path');
var EntityService = require('./entityService.js');
var app = express();
var urlencodedParser = bodyParser.urlencoded({ extended: true })
app.post('/createentity', urlencodedParser, function(req, res){
EntityService.createEntity(req.body.name);
res.status(200).json(null);
});
app.listen(3000);
//entityService.js
var mysql = require('mysql');
EntityService = function(){
var connection = mysql.createConnection({
host : CONNECTION_IP,
user : 'root',
password : 'password',
database : 'entitydb'
});
connection.connect();
this.createEntity = function(name){
var record = {name: 'name'};
var query = connection.query('INSERT INTO entity set ?', record, function(error, results, fields ){
//want to return the results.insertId from the service
});
}
}
module.exports = new EntityService();
The correct approach here is option 3 - have your service return a Promise
this.createEntity = name => new Promise((resolve, reject) => {
const query = connection.query('...', { name }, (err, results) => {
if (err) return reject(err);
return resolve(results.map(r => r.insertId));
});
})
If you're on the latest version of Node, I'd go with the asynch/await syntax.
Your service returns a promise,then your calling code can do:
app.post('/createentity', urlencodedParser, async function(req, res){
const entity = await EntityService.createEntity(req.body.name);
res.status(200).json(entity);
});
I am trying to insert a sub document into all existing documents in a collection in db,in nodejs using express framework.Following is the code snippet:
updatedoc: function(update,options,cb)
{
return this.update({},update,options).exec(cb);
}
where parameters update and options are as follows :
const update = { $push: { "defaultads": content }};
const options = { multi: true};
It seems to run and gives the following output on the console :
{ n: 1, nmodified: 1, ok: 1 }
but no push takes place at all ,in any of the documents of the database.
I have checked :
1) whether i am pushing in the right db.
2) whether correct values are being passed
However I am not able to find where I am going wrong.
I am new to nodejs and would really appreciate guidance in solving this problem.
Thanks in advance.
I am giving a simple code with full fledged requirement of yours. First create a config.js using this file you will be connected to mongodb.Here is the code
module.exports = {
'secretKey': '12345-67890-09876-54321',
'mongoUrl' : 'mongodb://localhost:27017/product'
}
Next create a models folder . Keep this schema in this models folder . I named it as product.js. Here is the code
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
var imageSchema = new Schema({
imagepath:{
type:String
}
});
var nameSchema = new mongoose.Schema({
productName:{type: String},
productPrice:{type: Number},
imagePaths:[imageSchema]
});
module.exports = mongoose.model("product", nameSchema);
Next create a routes folder and keep this routes code in this folder I named it as route.js. Here is the code
var express = require('express');
var bodyParser = require('body-parser');
var mongoose = require('mongoose');
var Product = require('../models/product');
var app = express();
var Router = express.Router();
Router.use(bodyParser.json());
Router.get('/product',function(req,res){
Product.find({}, function (err, product) {
if (err) throw err;
res.json(product);
});
})
Router.post('/productData',function(req, res, next){
Product.create(req.body, function (err, product) {
if (err) throw err;
console.log('Product Data created!');
var id = product._id;
res.writeHead(200, {
'Content-Type': 'text/plain'
});
res.end('Added the product data with id: ' + id);
});
})
Router.post('/subdocument',function (req, res, next) {
Product.find({},function (err, result) {
if (err) throw err;
for(var i=0;i<result.length;i++){
result[i].imagePaths.push(req.body);
result[i].save(function (err, ans) {
if (err) throw err;
console.log('SubDocument created!');
});
}
res.send("Successfully added");
});
})
module.exports = Router;
Next server code I named it as app.js. Here is the code
var express = require('express');
var bodyParser = require('body-parser');
var Product = require('./models/product');
var mongoose = require('mongoose');
var config = require('./config');
mongoose.connect(config.mongoUrl);
var db = mongoose.connection;
db.on('error', console.error.bind(console, 'connection error:'));
db.once('open', function () {
console.log("Connected correctly to server");
});
var app = express();
var route=require('./routes/route');
app.use('/route',route);
app.listen(3000,function(){
console.log("Server listening on 3000");
});
Run the server as node app.js.
API's
Use GET method http://localhost:3000/route/product .This is for getting all the product information.
Use POST method http://localhost:3000/route/productData .This for creating document. Post data in json format through request body like
{
"productName" : "sweets",
"productPrice" : "33"
}
You will see the response like this.
First post some documents i.e., post 2 or 3 documents and then you can see the documents with get API as I mentioned above. Then you will see all the documents contains empty sub document. You will see the response like this
And now add the sub document to all using the below api.
Use POST method http://localhost:3000/route/subdocument . Using this you can add a sub document to all the documents like this you need to add sub document
{
"imagepath" : "desktop"
}
You can see the response like this
After this again when you run the get API you can see all the sub documents are added to all documents.
Hope this helps.
I am receiving an empty array with the following code:
var mongoose = require('mongoose');
var db = mongoose.createConnection('localhost', 'mytestapp');
var SurveySchema = require('../models/Survey.js').SurveySchema;
var Survey = mongoose.model('SurveySchema', SurveySchema, 'surveys');
var UserSchema = require('../models/Survey.js').User;
var User = mongoose.model('user', UserSchema, 'users');
exports.getSurveysForUser = function(User) {
return function (req, res) {
User
.findOne({_id: req.params.userId})
.populate('surveys')
.exec(function (err, user){
if (err) return res.json({error: err})
else {
var surveyList=[];
surveyList = user.surveys;
console.log(surveyList);
console.log("user: "+ user);
res.json(surveyList);
}
});
}};
This is the console output:
[ ]
user: { __v: 2,
_id: 52939b8c22a7efb720000003,
email: 'a#b.de',
password: '202cb962ac59075b964b07152d234b70',
surveys: []
}
These are the Mongoose models:
exports.SurveySchema = new Mongoose.Schema({
description : String,
questions : [question] });
exports.User = new Mongoose.Schema({
name : String,
email: { type: String, unique: true },
password: { type: String, required: true},
surveys : [{type: Schema.ObjectId, ref: 'SurveySchema'}] });
Btw:
I already tried User.findOne(...) and then a Survey.find() in the callback. It seemed that the second statement was not even executed. Apparently i am very new to mongoose..and i can't find a way around this problem
Do you have any ideas how to help me?
I couldn't really find any helpful solution here, but the problem shouldn't be a big one.
Thanks in advance, its really keeping me up for days now!!
Edit: So this is the index.js with the method:
var mongoose = require('mongoose');
var db = mongoose.createConnection('localhost', 'mytestapp');
var SurveySchema = require('../models/Survey.js').SurveySchema;
var Survey = mongoose.model('SurveySchema', SurveySchema, 'surveys');
var UserSchema = require('../models/Survey.js').User;
var User = mongoose.model('user', UserSchema, 'users');
//.. here are some more methods..
exports.getSurveysForUser = function(User) {
return function (req, res) {
User
.findOne({_id: req.params.userId})
.populate('surveys')
.exec(function (err, user){
if (err) return res.json({error: err})
else {
var surveyList=[];
surveyList = user.surveys;
console.log(surveyList);
console.log("user: "+ user);
res.json(surveyList);
}
});
}};
//this is the code, that saves a response to a survey
exports.addResponse = function(ResponseSet) {
return function (req, res) {
console.log("bin da: addResponse");
console.log("response zu: " + req.body.surveyId);
console.log("von user : " + req.body.userId);
//für user speichern
var pUser = User.findOne({_id:req.body.userId}, function (error, user) {
// Maybe populate doesnt work, because i only push the ID?
user.surveys.push(Mongoose.Types.ObjectId(req.body.surveyId));
user.save();
}
);
var pSurvey = Survey.findOne({_id:req.body.surveyId}, function (error, survey) {
survey.responses.push(Mongoose.Types.ObjectId(req.params.id));
survey.save();
}
);
//responseSet speichern
var responseSet = new ResponseSet(req.body);
responseSet.save(function(error, responseSet) {
if (error || !responseSet) {
res.json({ error : error });
} else {
res.json(responseSet);
}
});
};
};
And this is the app.js, which consumes the index.js:
var Mongoose = require('mongoose');
var db = Mongoose.createConnection('localhost', 'mytestapp');
var SurveySchema = require('./models/Survey.js').SurveySchema;
var Survey = db.model('surveys', SurveySchema);
var UserSchema = require('./models/Survey.js').User;
var User = db.model('user', UserSchema);
var ResponseSetSchema = require ('./models/Survey.js').responseSet;
var ResponseSet = db.model('responseSet', ResponseSetSchema);
var express = require('express')
, routes = require('./routes')
, http = require('http')
, path = require('path')
, passport = require('passport')
, pass = require('./config/pass')
, user_routes = require('./routes/user');
var app = express();
// all environments
app.set('port', process.env.PORT || 3000);
app.set('views', __dirname + '/views/app');
//app.engine('html', require('ejs').renderFile);
app.use(express.static(__dirname + '/views/app'));
app.use(express.cookieParser());
app.use(express.favicon());
app.use(express.logger('dev'));
app.use(express.bodyParser());
app.use(express.methodOverride());
app.use(express.session({ secret: 'securedsession' }));
app.use(passport.initialize()); // Add passport initialization
app.use(passport.session()); // Add passport initialization
app.use(app.router);
app.use(express.static(path.join(__dirname, 'public')));
// development only
if ('development' == app.get('env')) {
app.use(express.errorHandler());
}
app.all('/secure', pass.ensureAuthenticated);
app.get('/', function (req, res)
{
res.render('index.html');
} );
// some more code...
app.get('/api/secure/userSurveys/:userId', routes.getSurveysForUser(User));
http.createServer(app).listen(app.get('port'), function(){
console.log('Express server listening on port ' + app.get('port'));
});
Hope it helpsto fight the problem!
Many many thanks in advance!! :)
Try changing:
surveys: [{type: Schema.ObjectId, ref: 'SurveySchema'}] });
to
surveys: [{type: Schema.Types.ObjectId, ref: 'SurveySchema'}] });
Also, make sure you have surveys pushed as children to User.surveys.
http://mongoosejs.com/docs/populate.html, here this section talks about your requirement in detail:
Refs to children
We may find however, if we use the aaron object, we are unable to get a list of the stories. This is because no story objects were ever 'pushed' onto aaron.stories."
Ok, it took quite a lot to figure out the issue:
exports.getSurveysForUser = function(User) {
...
};
The var User - model is not being injected into the scope properly. Change to:
exports.getSurveysForUser = (function(User) {
...
})(User);
The returned function's signature for middleware do not require User model to be passed as they are re-initiated and passed within the middleware code.
In index.js, change this
app.get('/api/secure/userSurveys/:userId', routes.getSurveysForUser(User));
to
app.get('/api/secure/userSurveys/:userId', routes.getSurveysForUser());
I also request you to self test your code and read as many docs as possible. Also, there can be multiple ways of reaching your goals. With time and practice you will conquer them all. Good luck!
So i found a solution!
Firstly it seemed, that the mongoose Schemas were not correctly required.
So in the models, i did mongoose.model('modelname', schemaname); for every model and now i only use mongoose.model(...) for every model in the index.js.
Secondly i found out about an even more critical thing: There were suddenly no user.surveys for my testuser anymore! I am sure that it was filled with surveys a few days ago. Because i tested my code several times and some surveys were pushed to that collection. Maybe i dropped the collection it in some testing..i don't really remember. So i pushed a survey manually in the mongodb console and tested it again --> worked! the user.surveys were populated! maybe the function worked yesterday and didn't need any change. I am so sorry, if that was a waste of time.
Bad thing is, that right now the exports.addResponse(....) is only saving a response, but is not pushing the IDs to the arrays user.surveys and survey.responses. This seems to be a synchronizing Problem and i will figure that out somehow.
Anyways, thank you for your help and time!
I'm trying to follow this Express.js and MongoDB tutorial with the difference of making my index page have a form where you save an email address.
This is the main code so far:
emailprovider.js
var Db = require('mongodb').Db;
var Connection = require('mongodb').Connection;
var Server = require('mongodb').Server;
var BSON = require('mongodb').BSON;
var ObjectID = require('mongodb').ObjectID;
EmailProvider = function(host, port) {
this.db= new Db('email-test', new Server(host, port, {safe: false}, {auto_reconnect: true}, {}));
this.db.open(function(){});
};
EmailProvider.prototype.getCollection= function(callback) {
this.db.collection('emails', function(error, email_collection) {
if( error ) callback(error);
else callback(null, email_collection);
});
};
//save new email
EmailProvider.prototype.save = function(emails, callback) {
this.getCollection(function(error, email_collection) {
if( error ) callback(error)
else {
if( typeof(emails.address)=="undefined")
emails = [emails];
for( var i =0;i< emails.address;i++ ) {
email = emails[i];
email.created_at = new Date();
}
email_collection.insert(emails, function() {
callback(null, emails);
});
}
});
};
exports.EmailProvider = EmailProvider;
app.js
var express = require('express')
, routes = require('./routes')
, user = require('./routes/user')
, http = require('http')
, path = require('path')
, EmailProvider = require('./emailprovider').EmailProvider;
var app = express();
app.get('/', routes.index);
app.get('/users', user.list);
//Routes
//get new email form
app.get('/email/new', function(req, res) {
res.render('email_new', {
title: 'Email Address'
});
});
//save new email
app.post('/email/new', function(req, res){
emailProvider.save({
address: req.param('name')
}, function( error, docs) {
res.redirect('/')
});
});
index.js
.form-area
form(method='post')
input(type='text', name='address', placeholder='Your email address')
input(type='submit', value='Send')
I really don't have a clue what's going on in the emailprovider. I kinda got the routes down but right now when I try to save a new Email I get a Cannot POST / 404 error message. What's the correct way to do this?
Thanks.
EDIT: removed extra commas in jade syntax for input attributes.
Your EmailProvider variable is a reference to function and not an object instance. This may cause two problems:
The this operator on EmailProvider.js might not refer to EmailProvider as you wish.
Every call to EmailProvider.save() will run the db.collection again which can cause performance issues, memory leak as well other issues.
You should create an object instance from EmailProvider as follow:
var app = express();
var emailProvider = new EmailProvider(<some host>, <some port>);
...
If this operator causes problem (such as it does not recognize EmailProvider methods) call the call function after the save function as follow:
//save new email
app.post('/email/new', function(req, res){
emailProvider.save({
address: req.param('name')
}, function( error, docs) {
res.redirect('/')
}).call(emailProvider);
});
Hope it will help.