So am I learning nodejs and using mongodb/mongoose with it. I am making a todo list site and as part of it I am trying to add the option of hiding and then showing again the tasks that are completed. I have been doing a lot of research but cannot find a way to hide documents in mongoose model, so what I have been trying to do is delete the completed tasks but have the values stored first so that they can then be restored again but I have run into a bit of trouble with this.
This is my mongoose schema
const todoTaskSchema = new mongoose.Schema({
text: {
type: String,
required: true
},
isComplete: {
type: Boolean,
default: false
}
});
module.exports = mongoose.model('TodoTask',todoTaskSchema);
In my main index file I have imported this using
const TodoTask = require("./models/TodoTask");
The way it works is that the user will click on the button and then this code will run:
//Toggle Completed
app.route("/toggleCompleted/").get((req, res) => {
TodoTask.find({isComplete: true}, function(error, completed){
if (completed.length !== 0) {
return TodoTask.deleteMany({isComplete: true}, function(err) {
}
);
}
if (completed.length === 0)
console.log(completed)
TodoTask.insertMany([
{completed}
]).then(function(){
console.log("Data inserted") // Success
}).catch(function(error){
console.log(error) // Failure
});
})
res.redirect("/");
});
Obviously the issue with this is that once the completed tasks are deleted and the user clicks on the button again, it is going to overwrite the completed value and it will be blank. Not sure how to fix this though or if there is a method to just simply hide and then show the values again. Any help would be greatly appreciated
Logic like hiding and showing should generally be handled on the client side, this would be a lot faster, since you save yourself a trip to the server and the database and would also be easier to implement. If you still wanted to do it serverside, you could query for all tasks, where isComplete is either true or false instead of deleting and creating them again.
Related
I have an app that allows users to use their own custom data, so I can't know what the data is. However, I do want to allow them to sort the data.
This can be a significant amount of data, and mongodb ends up giving me memory errors (32MB limit)
What would be the best way to approach this? How can I allow the user to sort a large amount of data by an unknown field?
MongoDB allows you to design the schema in such a way that it can store Objects and Object relation in a schema, So you can allow the user to store any kind of information. As #kevinadi said, there is a limit of 32MB. As of sorting is concerned it can be done on your serverside.
This is an example I tried when storing objects in MongoDB and Mongoose ORM
var mongoose = require("mongoose");
var userSchema = new mongoose.Schema({
email: {
type: String,
unique: true,
required: true,
lowercase: true,
trim: true,
match: [/^\w+([\.-]?\w+)*#\w+([\.-]?\w+)*(\.\w{2,3})+$/, "Please fill a valid email address"]
},
custInfo:{
type:Object,
required: true
}
isConfirmed: {
type: Boolean,
required: true,
default: false
},
confirmedOn:{
type: Date,
required: true,
default: Date.now()
}
});
module.exports = mongoose.model("user",userSchema);
Since you have tagged this question Meteor I assume, you have the default Meteor environment, where you can use the client-side lightweight Mongo collections.
This gives you the opportunity to publish (Publication) / return (Method) your data mostly unsorted and let the client's handle this task.
Think this: just 100 clients asking for a publication that updates on every sort action (because the subscription parameters change, so the publication changes, too).
This causes already your server to consume a high amount of RAM to keep the observers (OPLOG etc.) running for 100 publications, each querying huge amounts of documents.
Possible performant solutions are described below. Please keep in mind, that they are not bound to any front-end and rather a conceptual description. You will have to include reactivity etc., based on your frontend environment.
Option A - Publish unsorted, let clients sort
server
Meteor.publish('hugeData', function () {
return MyCollection.find({ ...})
})
client
const handle = Meteor.subscribe('hugeData')
if (handle.ready()) {
const sortedData = MyCollection.find({ ... }, {sort: { someField: -1 } })
}
A big plus is here, that you can inform the clients about the completeness status, if using cursor.observeChanges.
Note, that if you want to scan backwards (return docs, with the newest) you can use the hint option on find:
Meteor.publish('hugeData', function () {
return MyCollection.find({ ...}, { hint: { $natural : -1 })
})
This is way more performant than { sort: { fieldName: -1} }.
Option B - return unsorted from Method, let clients sort
Now there may still be a problem with solution A, since it still has a lot of RAM to consume if there are lots of subscribers. An alternative (especially if live-data changes are not so relevant) is to use the Meteor Methods:
server
Meteor.method('hugeData', function () {
return MyCollection.find({ ...}).fetch()
})
Note that this requires to fetch the docs, otherwise and unhandledPromiseRejection is thrown.
client
This requires a LocalCollection on the client, that is not in sync with your server side collection, or you will get problems with document syncing:
const HugeData = new LocalCollection(null) // note the null as collection name!
const insertUpdate = document => {
if (LocalCollection.findOne(document._id)) {
delete document._id
return LocalCollection.update(document._id, document)
} else {
return LocalCollection.insert(document)
}
}
Meteor.call('hudeData', (err, data) => {
data.forEach(insertUpdate)
})
Then you can use the LocalCollection on the client for any projection of the received data.
All in all it is a good tradeoff to move the load to the clients. As long as you keep them informed when projections take a while it should be okay.
My current thought is an additional indexed collection holding 1.entity id, 2 fields name 3.field value.
Have that collection indexed, and then pull ordered entity ids from there, later on loading the full relevant documets by ID.
In Sequelize tutorials, it is said that a single model is generated in this way:
const User = sequelize.define('user', {
firstName: {
type: Sequelize.STRING
},
lastName: {
type: Sequelize.STRING
}
});
And than saved (i.e. create table) like this :
User.sync().then(() => {
// do whatever
});
But I expect to do that just once, I need to create tables just once. So the next time I run the script how to just retrieve models (i.e. tables) that were defined before with the above code.
in sync method you can pass an option to avoid sync of database tables every time. This option will make sure that your application checks for table in database, if it exist then it will not create otherwise it will create table.
User.sync(
{force: false}).then(() => {
// do whatever
});
Let me know if you still face issue. I am using sequalize and i am not getting this issue.
I'm trying to create a basic MEAN stack CRUD api to add shops into my database. I want every shop to have a unique name (to avoid adding duplicates). So far, everything gets saved into the database even if I post the same request 10 times. Went trough the code a couple of times and can't figure out what's wrong, if anyone could point me in the right direction I'd be very grateful.
shop model:
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
var bcrypt = require('bcrypt-nodejs');
//shop schema
var ShopSchema = new Schema({
name: { type: String, required: true, index: { unique: true }},
address: { type: String, required: true, index: { unique: true }}
});
module.exports = mongoose.model('Shop', ShopSchema);
post function:
apiRouter.route('/shops')
//create a shop
.post(function(req, res) {
//new instance of shop model
var shop = new Shop();
//set the shop information
shop.name = req.body.name;
shop.address = req.body.address;
//save shop and check for errors
shop.save(function(err) {
if(err) {
//duplicate entry
if(err.code == 11000) {
return res.json({ success: false, message: 'A shop with that name already exists.'});
}
else {
return res.send(err);
}
}
else {
res.json({ message:'Shop created! '});
}
});
})
I do not receive errors of any kind, like I said everything just gets written into the database.
Thanks for the help.
Basically your writes haven't finished before the new entries are saved. You can read more about creating unique keys Here, but the gist is below. The solution is to create an index over the unique fields ahead of time.
When we declare a property to be unique, we’re actually declaring that we want a database-level index on that property. Some database abstraction layers will issue a query to see if a there’s another record with the same value for the unique property, and if that query comes back empty, it allows the save or update to proceed. If you trust this method, you either have incredibly low traffic or you’re about to learn about race conditions, because 2 or more requests could have their checks to the database occur before any writes go out, and you end up with non-unique data in your DB.
In between the time that check query is issued, another insert could come along doing the exact same thing, and you still end up with duplication. Uniqueness can’t be correctly validated at the application level. So it’s good that Mongoose tries to create an index for us.
I am trying to create a many to many association between projects and tasks using node and sequelize. I cannot figure out how to create new 'Tasks' with variable names that can be accumulated as an array for the project.setTasks function as the documentation here gives as an example with ALOT of detail missing. http://sequelizejs.com/documentation#associations-many-to-many
here is the example they give:
Project.hasMany(Task)
Task.hasMany(Project)
Project.create()...
Task.create()...
Task.create()...
// save them... and then:
project.setTasks([task1, task2]).success(function() {
// saved!
})
I'm not sure what you mean by "variable names" exactly.
However, here's an example from the sequelize test suite, as copied below with my comments:
// Define the models
this.User = this.sequelize.define('User', { username: DataTypes.STRING });
this.Task = this.sequelize.define('Task', { title: DataTypes.STRING, active: DataTypes.BOOLEAN });
// Setup associations
this.User.hasMany(this.Task);
this.Task.hasMany(this.User);
// using promises, sync the database, then create a user and two tasks
return this.sequelize.sync({ force: true }).then(function() {
return Promise.all([
self.User.create({ username: 'John'}),
self.Task.create({ title: 'Get rich', active: true}),
self.Task.create({ title: 'Die trying', active: false})
]);
}).spread(function (john, task1, task2) {
// This line below is only relevant to the tests.
self.tasks = [task1, task2];
// Using the created (and saved) tasks and user, set the tasks and return the promise
return john.setTasks([task1, task2]);
});
Edit I agree that the sequelize documentation is still too light, but I've found it a good framework so far to use. I recommend when in doubt either reading the unit tests, or writing your own tests to confirm behaviours.
Edit Note you can also use .addTask and .removeTask instead of simply setting them all. See the docs
What is the best way to update a value within an array saved in a mongodb record? Currently, I'm trying it this way:
Record.find({ 'owner': owner}, {}, {sort: { date: -1 }}, function(err, record){
if(!err){
for (var i = 0; i < record[0].array.length; i++){
record[0].array[i].score = 0;
record[0].array[i].changed = true;
record[0].save();
}
}
});
And the schema looks like this:
var recordSchema = mongoose.Schema({
owner: {type: String},
date: {type: Date, default: Date.now},
array: mongoose.Schema.Types.Mixed
});
Right now, I can see that the array updates, I get no error in saving, but when I query the database again, the array hasn't been updated.
It would help if you explained your intent here as naming a property "array" conveys nothing about its purpose. I guess from your code you hope to go and set the score of each item there to zero. Note your save is currently being ignored because you can only save top-level mongoose documents, not nested documents.
Certain find-and-modify operations on arrays can be done with a single database command using the Array Update Operators like $push, $addToSet, etc. However I don't see any operators that can directly make your desired change in a single operation. Thus I think you need to find your record, alter the array date, and save it. (Note findOne is a convenience function you can use if you only care about the first match, which seems to be the case for you).
Record.findOne({ 'owner': owner}, {}, {sort: { date: -1 }}, function(err, record){
if (err) {
//don't just ignore this, log or bubble forward via callbacks
return;
}
if (!record) {
//Record not found, log or send 404 or whatever
return;
}
record.array.forEach(function (item) {
item.score = 0;
item.changed = true;
});
//Now, mongoose can't automatically detect that you've changed the contents of
//record.array, so tell it
//see http://mongoosejs.com/docs/api.html#document_Document-markModified
record.markModified('array');
record.save();
});
If you have a mongoose object of a document, you can of course update the array as in the question, with the following Caveat.
This is in fact a mongoose gotcha. Mongoose cannot track changes in the array of mixed, one has to use markModified:
doc.mixed.type = 'changed';
doc.markModified('mixed.type');
doc.save() // changes to mixed.type are now persisted