I am trying to build a data migration feature for my software. the client is providing data in csv format. I have built the field mapping based on my schema-model. The plan is to create an API which will return me an empty object for the model and I update the object with data collected from the CSV.
I understand that model.schema.obj will return me the schema, but that will have all the declarations for each field. I need my API to return an object based on the schema with all values set to undefined or empty string.
Can anyone help me ? Please
You can create an instance of your model with the default values to empty/undefined/null (you decide) and the return it.
Here's an example using Mongoose:
// Define your model
var Schema = mongoose.Schema;
// define the schema for our user model
var userSchema = Schema({
name : { type : String, default:'' },
lastname : { type : String, default: '' }
}, {collection: 'user', usePushEach: true});
Then on your API endpoint create an instance of the model and return it (example assuming you're using Express).
getSchemaObj: function(req, res, next){
var emptyUser = new User();
res.json(emptyUser);
}
I am trying to add a couple of attributes to the scaffolded MEAN.js User entity.
locationName: {
type: String,
trim: true
}
I also have created another entity Book connected with User. Unfortunately, I think I do not quite grasp the concept behind the populate method because I am not able to "populate" the User entity with the locationName attribute.
I tried the following:
/**
* List of Books
*/
exports.list = function(req, res) {
Book.find().sort('-created').populate('user', 'displayName', 'locationName').exec(function(err, books) {
if (err) {
return res.status(400).send({
message: errorHandler.getErrorMessage(err)
});
} else {
res.jsonp(books);
}
});
};
Unfortunately, I get the following error:
/home/maurizio/Workspace/sbr-v1/node_modules/mongoose/lib/connection.js:625
throw new MongooseError.MissingSchemaError(name);
^
MissingSchemaError: Schema hasn't been registered for model "locationName".
Any suggestion?
Thanks
Cheers
The error is clear, you should have a schema for the locationName.
If your location is just a string property in your user model and does not refer to separate model, you don't need and shouldn't use populate with it, it will simply be returned as a property of the returned user object from mongoose find() method.
If your want to make your location a stand alone entity (different mongodb document), you should have a mongoose model that defines your location object, aka have a file in your app\models name for example: location.server.model.js that contains something like:
var mongoose = require('mongoose'),
Schema = mongoose.Schema;
var LocationSchema = new Schema({
_id: String,
name: String
//, add any additional properties
});
mongoose.model('Location', LocationSchema);
Note that the _id here replaces the auto generated objectId, so this has to be unique, and this the property you should refer to in your User object, meaning if you have a location like this:
var mongoose = require('mongoose'),
Location = mongoose.model('Location');
var _location = new Location({_id:'de', name:'Deutschland'});
you should refer to it in your User object like this:
var _user=new User({location:'de'});
//or:
var _user=new User();
_user.location='de';
then you should be able to populate your location object with your user, like this:
User.find().populate('location').exec(function(err, _user) {
if (err) {
//handle error
} else {
//found user
console.log(_user);
//user is populated with location object, makes you able to do:
console.log(_user.location.name);
}
});
I suggest you to further read in mongodb data modeling and mongoose Schemas, Models, Population.
I want to give users the ability to create collections in my Node app. I have really only seen example of hard coding in collections with mongoose. Anyone know if its possible to create collections dynamically with mongoose? If so an example would be very helpful.
Basically I want to be able to store data for different 'events' in different collections.
I.E.
Events:
event1,
event2,
...
eventN
Users can create there own custom event and store data in that collection. In the end each event might have hundreds/thousands of rows. I would like to give users the ability to perform CRUD operations on their events. Rather than store in one big collection I would like to store each events data in a different collection.
I don't really have an example of what I have tried as I have only created 'hard coded' collections with mongoose. I am not even sure I can create a new collection in mongoose that is dynamic based on a user request.
var mongoose = require('mongoose');
mongoose.connect('localhost', 'events');
var schema = mongoose.Schema({ name: 'string' });
var Event1 = mongoose.model('Event1', schema);
var event1= new Event1({ name: 'something' });
event1.save(function (err) {
if (err) // ...
console.log('meow');
});
Above works great if I hard code 'Event1' as a collection. Not sure I create a dynamic collection.
var mongoose = require('mongoose');
mongoose.connect('localhost', 'events');
...
var userDefinedEvent = //get this from a client side request
...
var schema = mongoose.Schema({ name: 'string' });
var userDefinedEvent = mongoose.model(userDefinedEvent, schema);
Can you do that?
I believe that this is a terrible idea to implement, but a question deserves an answer. You need to define a schema with a dynamic name that allows information of 'Any' type in it. A function to do this may be a little similar to this function:
var establishedModels = {};
function createModelForName(name) {
if (!(name in establishedModels)) {
var Any = new Schema({ any: Schema.Types.Mixed });
establishedModels[name] = mongoose.model(name, Any);
}
return establishedModels[name];
}
Now you can create models that allow information without any kind of restriction, including the name. I'm going to assume an object defined like this, {name: 'hello', content: {x: 1}}, which is provided by the 'user'. To save this, I can run the following code:
var stuff = {name: 'hello', content: {x: 1}}; // Define info.
var Model = createModelForName(name); // Create the model.
var model = Model(stuff.content); // Create a model instance.
model.save(function (err) { // Save
if (err) {
console.log(err);
}
});
Queries are very similar, fetch the model and then do a query:
var stuff = {name: 'hello', query: {x: {'$gt': 0}}}; // Define info.
var Model = createModelForName(name); // Create the model.
model.find(stuff.query, function (err, entries) {
// Do something with the matched entries.
});
You will have to implement code to protect your queries. You don't want the user to blow up your db.
From mongo docs here: data modeling
In certain situations, you might choose to store information in
several collections rather than in a single collection.
Consider a sample collection logs that stores log documents for
various environment and applications. The logs collection contains
documents of the following form:
{ log: "dev", ts: ..., info: ... } { log: "debug", ts: ..., info: ...}
If the total number of documents is low you may group documents into
collection by type. For logs, consider maintaining distinct log
collections, such as logs.dev and logs.debug. The logs.dev collection
would contain only the documents related to the dev environment.
Generally, having large number of collections has no significant
performance penalty and results in very good performance. Distinct
collections are very important for high-throughput batch processing.
Say I have 20 different events. Each event has 1 million entries... As such if this is all in one collection I will have to filter the collection by event for every CRUD op.
I would suggest you keep all events in the same collection, especially if event names depend on client code and are thus subject to change. Instead, index the name and user reference.
mongoose.Schema({
name: { type: String, index: true },
user: { type: mongoose.Schema.Types.ObjectId, ref: 'User', index: true }
});
Furthermore I think you came at the problem a bit backwards (but I might be mistaken). Are you finding events within the context of a user, or finding users within the context of an event name? I have a feeling it's the former, and you should be partitioning on user reference, not the event name in the first place.
If you do not need to find all events for a user and just need to deal with user and event name together you could go with a compound index:
schema.index({ user: 1, name: 1 });
If you are dealing with millions of documents, make sure to turn off auto index:
schema.set('autoIndex', false);
This post has interesting stuff about naming collections and using a specific schema as well:
How to access a preexisting collection with Mongoose?
You could try the following:
var createDB = function(name) {
var connection = mongoose.createConnection(
'mongodb://localhost:27017/' + name);
connection.on('open', function() {
connection.db.collectionNames(function(error) {
if (error) {
return console.log("error", error)
}
});
});
connection.on('error', function(error) {
return console.log("error", error)
});
}
It is important that you get the collections names with connection.db.collectionNames, otherwise the Database won't be created.
This method works best for me , This example creates dynamic collection for each users , each collection will hold only corresponding users information (login details), first declare the function dynamicModel in separate file : example model.js
/* model.js */
'use strict';
var mongoose = require('mongoose'),
Schema = mongoose.Schema;
function dynamicModel(suffix) {
var addressSchema = new Schema(
{
"name" : {type: String, default: '',trim: true},
"login_time" : {type: Date},
"location" : {type: String, default: '',trim: true},
}
);
return mongoose.model('user_' + suffix, addressSchema);
}
module.exports = dynamicModel;
In controller File example user.js,first function to create dynamic collection and second function to save data to a particular collection
/* user.js */
var mongoose = require('mongoose'),
function CreateModel(user_name){//function to create collection , user_name argument contains collection name
var Model = require(path.resolve('./model.js'))(user_name);
}
function save_user_info(user_name,data){//function to save user info , data argument contains user info
var UserModel = mongoose.model(user_name) ;
var usermodel = UserModel(data);
usermodel.save(function (err) {
if (err) {
console.log(err);
} else {
console.log("\nSaved");
}
});
}
yes we can do that .I have tried it and its working.
REFERENCE CODE:
app.post("/",function(req,res){
var Cat=req.body.catg;
const link= req.body.link;
const rating=req.body.rating;
Cat=mongoose.model(Cat,schema);
const item=new Cat({
name:link,
age:rating
});
item.save();
res.render("\index");
});
I tried Magesh varan Reference Code ,
and this code works for me
router.post("/auto-create-collection", (req, res) => {
var reqData = req.body; // {"username":"123","password":"321","collectionName":"user_data"}
let userName = reqData.username;
let passWord = reqData.password;
let collectionName = reqData.collectionName;
// create schema
var mySchema = new mongoose.Schema({
userName: String,
passWord: String,
});
// create model
var myModel = mongoose.model(collectionName, mySchema);
const storeData = new myModel({
userName: userName,
passWord: passWord,
});
storeData.save();
res.json(storeData);
});
Create a dynamic.model.ts access from some where to achieve this feature.
import mongoose, { Schema } from "mongoose";
export default function dynamicModelName(collectionName: any) {
var dynamicSchema = new Schema({ any: Schema.Types.Mixed }, { strict: false });
return mongoose.model(collectionName, dynamicSchema);
}
Create dynamic model
import dynamicModelName from "../models/dynamic.model"
var stuff = { name: 'hello', content: { x: 1 } };
var Model = await dynamicModelName('test2')
let response = await new Model(stuff).save();
return res.send(response);
Get the value from the dynamic model
var Model = dynamicModelName('test2');
let response = await Model.find();
return res.send(response);
I'm writing an Express.js app that uses Mongoose ODM.
When a document is created/updated, I want it to automatically get some fields populated:
createdBy
createdOn
It seems to me that the right place to implement that would be in a Mongoose plugin that augments the document with those properties and provides defaults and/or mongoose middleware to populate the fields.
However, I have absolutely no idea how I could get the username from the session in my plugin.
Any suggestion?
/**
* A mongoose plugin to add mandatory 'createdBy' and 'createdOn' fields.
*/
module.exports = exports = function auditablePlugin (schema, options) {
schema.add({
createdBy: { type: String, required: true, 'default': user }
, createdOn: { type: Date, required: true, 'default': now }
});
};
var user = function(){
return 'user'; // HOW to get the user from the session ???
};
var now = function(){
return new Date;
};
You can't do this like this, bacause user function is added as a prototype method for this schema. In particular it is request/response independent. There is a hack however. If you have object obj of the type schema, then you can do
obj.session = req.session;
in request handler. Then you can access the session from the function. However this may lead to other issues ( for example running cron jobs on this collection ) and it looks like a very bad practice to me.
You can just do that manually, when a current user creates this schema object, can't you? Or create static method:
MySchema.statics.createObject = function ( user ) {
// create your object
var new_object = MySchema( ... );
// set necessary fields
new_object.createdBy = user.username;
}
How are you handling form validation with Express and Mongoose? Are you using custom methods, some plugin, or the default errors array?
While I could possibly see using the default errors array for some very simple validation, that approach seems to blow up in the scenario of having nested models.
I personally use node-validator for checking if all the input fields from the user is correct before even presenting it to Mongoose.
Node-validator is also nice for creating a list of all errors that then can be presented to the user.
Mongoose has validation middleware. You can define validation functions for schema items individually. Nested items can be validated too. Furthermore you can define asyn validations. For more information check out the mongoose page.
var mongoose = require('mongoose'),
schema = mongoose.Schema,
accountSchema = new schema({
accountID: { type: Number, validate: [
function(v){
return (v !== null);
}, 'accountID must be entered!'
]}
}),
personSchema = new schema({
name: { type: String, validate: [
function(v){
return v.length < 20;
}, 'name must be max 20 characters!']
},
age: Number,
account: [accountSchema]
}),
connection = mongoose.createConnection('mongodb://127.0.0.1/test');
personModel = connection.model('person', personSchema),
accountModel = connection.model('account', accountSchema);
...
var person = new personModel({
name: req.body.person.name,
age: req.body.person.age,
account: new accountModel({ accountID: req.body.person.account })
});
person.save(function(err){
if(err) {
console.log(err);
req.flash('error', err);
res.render('view');
}
...
});
I personaly use express-form middleware to do validation; it also has filter capabilities. It's based on node-validator but has additional bonuses for express. It adds a property to the request object indicating if it's valid and returns an array of errors.
I would use this if you're using express.