mongodb pull is not removing all items - node.js

I'm working on a project in nodejs using mongodb as my database. I'm trying to get rid of elements within my array that have dates before today. The problem that I'm having is that at most 5 elements are being deleted. I want all elements that meet this criteria to be deleted. Also, when I don't have user.possible.pull(items._id) const result = await user.save() all elements that meet this criteria are shown in my deletePossible array. However, when I do have user.possible.pull(items._id) const result = await user.save() at most 5 are being shown as well.
In my database, my User document looks like:
_id: '',
name: '',
possible: Array
0 Object
date: "Tues Jan 10 2023",
_id: "63c0b169b6fa12ac49874a13"
1 Object
date: "Wed Jan 11 2023",
_id: "63c0b172b6fa12ac49874a32"
...
My code:
const user = await User.findById(args.userId)
const deletePossible = [];
for (var items of user.possible) {
if (+new Date(items.date) < +new Date().setHours) {
deletePossible.push(items._id)
user.possible.pull(items._id)
const result = await user.save()
}
}
`
console.log(deletePossible)
I've tried a number of things such as:
for (var item of deletePossible) {
user.possible.pull(item)
const result = await user.save()
}
following deletePossible.push(items._id), and
const userInfo = await User.updateOne( { _id: args.userId}, {possible:{$pull:[...deletePossible] }} )
which removes all of the arrays from possible regardless of if it's contained within deletePossible and then adds a random _id. Nothing I have tried seems to work. Does anyone have any idea why this is happening and how to get this to work properly? I would really appreciate any help or advice. Thank you!

You can simply filter user.possible and save the updated User:
const user = await User.findById(args.userId);
if (!user) return;
// Change the condition based on your needs
user.possible = user.possible.filter(p => new Date(p.date) >= new Date());
await user.save();

The core of the issue appears to not be related to Mongo or Mongoose really, but is rather just a standard algorithmic logic problem.
Consider the following code, which iterates over an array, logs each element, and removes the third element when it arrives at it:
const array = [0, 1, 2, 3, 4];
for (const element of array) {
console.log(element);
if (element === 2) {
array.splice(2, 1); // remove the element at index 2 from the array
}
}
This code outputs:
0
1
2
4
Notice anything interesting? 3 has been skipped.
This happens because deleting an element from an array causes everything in front of it to move up a position. So if you're looking at 2 and you delete it, then 3 moves into 2's place, and 4 moves into 3's place. So then if you look at the next position, you're now looking at 4, not 3. The code never sees the 3.
This is why you should never change an array while iterating over it. A lot of languages won't even allow you to (if you're using iterators), they'll throw some sort of "underlying collection was modified during iteration" error. You can make it work if you know what you're doing (often just by iterating over the array backwards), but there are usually better solutions anyway, like using Array.prototype.filter().
One easy solution is to iterate over a copy of the array, so that when you do the delete, the array you're iterating over (the copy) isn't changed. That would look like this:
for (const item of [...user.possible]) {
if (/* some condition */) {
user.possible.pull(item._id);
}
}
Another problem with your code: +new Date().setHours will always evaluate to NaN since setHours is a function and converting a function to a number always results in NaN. I suspect this is just a typo you introduced while struggling with the original issue.
The suggestion to use filter() is even better.

Related

Create array and randomize node.js array

I'm trying to write a cisco webex bot which get all people in the space(room) and randomly write only one name.
I have this code
framework.hears("daily host", function (bot) {
console.log("Choosing a daily host");
responded = true;
// Use the webex SDK to get the list of users in this space
bot.webex.memberships.list({roomId: bot.room.id})
.then((memberships) => {
for (const member of memberships.items) {
if (member.personId === bot.person.id) {
// Skip myself!
continue;
}
let names = (member.personDisplayName) ? member.personDisplayName : member.personEmail;
let arrays = names.split('\n');
var array = arrays[Math.floor(Math.random()*items.length)];
console.log(array)
bot.say(`Hello ${array}`);
}
})
.catch((e) => {
console.error(`Call to sdk.memberships.get() failed: ${e.messages}`);
bot.say('Hello everybody!');
});
});
But this doesn't work.
Also name after i use let arrays = names.split('\n'); separated by space and don't have comma.
I think because of what code doesn't work
Output of console log:
[ 'George Washington' ]
[ 'John' ]
[ 'William Howard Taft' ]
Main question now how to turn output to array?
That's because arrays[Math.floor(Math.random()*items.length)] only assigns an array with length 3. You need to randomise the index and push to array or use a sort function on the original array
var array = arrays.sort((a,b)=>{
return Math.floor(Math.random()*arrays.length);
});
if you are looking to get the output as per you question you can use reduce instead of sort.
var arrays = [ 'George Washington', 'John', 'William Howard Taft'];
var array = arrays.reduce((a,i)=>{
if(!a) a = [];
a.splice(Math.floor(Math.random()*arrays.length), 0, [i]);
return a;
},[]);
Here is how to get a single name from your data, and ensuring it is a string. There are only four names in the array, so run the snippet several times if you keep getting the same name.
// A list of names. Notice that Arraymond is an array; the other names are strings.
const names = [ 'George Washington', 'John', 'William Howard Taft', ['Arraymond'] ];
// Randomize the names
const randomNames = names.sort(() => Math.random() - 0.5);
// Get the first name. Make sure the name is a string (not an array)
const name = randomNames[0].toString();
console.log(name)
A tip: don't name your array "array" or "arrays" - it is not meaningful. Use good naming conventions and meaningful variable names that help others understand what the code is doing.

Increment the Sort Key dynamoDB node.js

I am new to dynamodb.
I want to increment the Sort Key
If the id=0 the next id=1 and so on,
If the user(Partition key), id(Sort Key) add items the next add items the id increment 1.
The code use on PutItem with dynamodb.
Is possible to do that?
I did not want use the UUID( unique Key)
Most situations don't need an auto-incrementing attribute and DynamoDB doesn't provide this feature out of the box. This is considered to be an anti-pattern in distributed systems.
But, see How to autoincrement in DynamoDB if you really need to.
I understand that you may need this number because it is a legal obligation to have incremental invoice numbers for example.
One way would be to create a table to store your number sequences.
Add fields like:
{
name: "invoices",
prefix: "INV",
numberOfDigits: 5,
leasedValue: 1,
appliedValue: 1,
lastUpdatedTime: '2022-08-05'
},
{
name: "deliveryNotes",
prefix: "DN",
numberOfDigits: 5,
leasedValue: 1,
appliedValue: 1,
lastUpdatedTime: '2022-08-05'
}
You need 2 values (a lease and an applied value), to make sure you never skip a beat, even when things go wrong.
That check-lease-apply-release/rollback logic looks as follows:
async function useSequence(name: string, cb: async (uniqueNumber: string) => void) {
// 1. GET THE SEQUENCE FROM DATABASE
const sequence = await getSequence("invoices");
this.validateSequence(sequence);
// 2. INCREASE THE LEASED VALUE
const oldValue = sequence.appliedValue;
const leasedValue = oldValue + 1;
sequence.leasedValue = leasedValue;
await saveSequence(sequence);
try {
// 3. CREATE AND SAVE YOUR DOCUMENT
await cb(format(leasedValue));
// 4. INCREASE THE APPLIED VALUE
sequence.appliedValue++;
await saveSequence(sequence);
} catch(err) {
// 4B. ROLLBACK WHEN THINGS ARE BROKEN
console.err(err)
try {
const sequence = await getSequence(name);
sequence.leasedValue--;
this.validateSequence(sequence);
await saveSequence(sequence);
} catch (err2) {
console.error(err2);
}
throw err;
}
}
function validateSequence(sequence) {
// A CLEAN STATE, MEANS THAT THE NUMBERS ARE IN SYNC
if (sequence.leasedValue !== sequence.appliedValue) {
throw new Error("sequence is broken.");
}
}
Then, whenever you need a unique number you can use the above function to work in a protected scope, where the number will be rollbacked when something goes wrong.
const details = ...;
await useSequence("invoice", async (uniqueNumber) => {
const invoiceData = {...details, id: uniqueNumber};
const invoice = await this.createInvoice(invoiceData);
await this.saveInvoice(invoice);
})
Can it scale? Can it run on multiple instances? No, it can't. It never will be, because in most countries it's just not legal to do so. You're not allowed to send out invoice 6 before invoice 5 or to cancel invoice 5 after you've send invoice 6.
The only exception being, if you have multiple sequences. e.g. in some cases you're allowed to have a sequence per customer, or a sequence per payment system, ... Hence, you want them in your database.

How do I copy entries from one collection to another using mongoose?

I'm trying to create a little task management site for a work project. The overall goal is here is that the tasks stay the same each month (their status can be updated and whatnot), and they need to be duplicated at the start of each new month so they can be displayed and sorted by on a table.
I already figured out how to schedule the task, I have the table I need set up. A little explanation before the code - the way I'm planning on doing this is having two different task collections - one I've called "assignments", will have the tasks that need to be duplicated (with their description, status and other necessary data) and another collection, which I called "tasks", will have the exact same data but with an additional "date" field. This is where the table will get it's data from, the date is just for sorting purposes.
This is what I have so far -
Index.js: gets all the assignments from the database, and sends the object over to the duplicate function.
router.get('/test', async function(req, res, next) {
let allTasks = await dbModule.getAllAssignments();
let result = await dbModule.duplicateTasks(allTasks);
res.json(result);
});
dbmodule.js:
getAllAssignments: () => {
allAssignments = Assignment.find({});
return allAssignments;
},
duplicateTasks: (allTasksToAdd) => {
try {
for (let i = 0; i < allTasksToAdd.length; i++) {
let newTask = new Task({
customername: allTasksToAdd.customername,
provname: allTasksToAdd.provname,
description: allTasksToAdd.description,
status: allTasksToAdd.status,
date: "07-2020"
})
newTask.save();
}
return "Done"
} catch (error) {
return "Error"
}
}
The issue arises when I try and actually duplicate the tasks. For testing purposes I've entered the date manually this time, but that's all that ends up being inserted - just the date, the rest of the data is skipped. I've heard of db.collection.copyTo(), but I'm not sure if it'll allow me to insert the field I need or if it's supported in mongoose. I know there's absolutely an easier way to do this but I can't quite figure it out. I'd love some input and suggestions if anyone has any.
Thanks.
The problem is that allTasksToAdd.customername (and the other fields your trying to access) will be undefined. You need to access the fields under the current index:
let newTask = new Task({
customername: allTasksToAdd[i].customername,
provname: allTasksToAdd[i].provname,
description: allTasksToAdd[i].description,
status: allTasksToAdd[i].status,
date: "07-2020"
})
Note that you can simplify this by using a for .. of loop instead:
for (const task of allTasksToAdd) {
const newTask = new Task({
customername: task.customername,
provname: task.provname,
description: task.description,
status: task.status,
date: "07-2020"
});
newTask.save();
}

How to search data in mongodb with dynamic fields using mongoose?

I've a node.js api in which user sends the required fields as an array to be fetched from the mongodb database. I need to find the data of that fields using Find query. I've written forEach statement to loop through that array and got the array elements. But when I try to get the results by inserting the array elements in the query, it doesn't giving the required results. Could any one please help me in resolving the issue by seeing the code below?
templateLevelGraphData: async function(tid,payload){
let err, templateData, respData = [], test, currentValue;
[err,templateData] = await to(Template.findById(tid));
var templateId = templateData.templateId;
payload.variables.forEach(async data=>{
console.log(data); //data has the array elements like variables=["humidity"]
[err, currentValue] = await to(mongoose.connection.db.collection(templateId).find({},{data:1}).sort({"entryDayTime":-1}).limit(1).toArray());
console.log(currentValue);
});
return "success";
}
The expected output is,
[ { humidity: 36 } ]
But I'm getting only _id like,
[ { _id: 5dce3a2df89ab63ee4d95495 } ]
I think data is not applying in the query. But I'm printing the data in the console where it's giving the correct results by displaying the array elements like, humidity. What I need to do to make it work?
When you are passing {data: 1} you are passing an array where is expecting name of column.
You have to create an object where the keys are going to be the elements of the array and set them to 1.
const projection = data.reduce((a,b) => (a[b]=1, a), {});
[...] .find({}, projection) [...]
Actually I got the solution.
for(let i=0;i<payload.variables.length;i++){
var test = '{"'+ payload.variables[i] +'":1,"_id":0}';
var query = JSON.parse(test);
[err, currentValue] = await to(mongoose.connection.db.collection(templateId).find({"deviceId":deviceId},query).sort({"entryDayTime":-1}).limit(1).toArray());
console.log(currentValue); //It's giving the solution
}

MongoDB and Mongoose - Creating custom, autoindexed secondary IDs?

I initialize my DB in the usual way:
mongoose.connect(`mongodb://uname:pword#127.0.0.1:port/dbname?authSource=admin`, {useNewUrlParser: true, autoIndex: false});
And I have a Schema, something like:
var materialSchema = new Schema({
bookID: {type: String, required: true},
active: Boolean,
name: {type: String, required: true},
stockLength: {type: Number, required: true}
});
module.exports = mongoose.model('material', materialSchema);
When I create a new material and add it to the database, it is automatically assigned the usual _id - which is a behaviour I want to maintain. BUT, I'd also like for bookID to be a unique, auto-incrementing index. This is for physical shelf storage, and not for queries or anything like that.
I'd like for bookID to increment in the following way:
A-001
A-002
A-003
...
A-098
A-099
A-100
B-001
...
B-100
...
Z-001
...
Z-100
In case the pattern above isn't clear, the pattern starts at A-001 and ultimately ends at Z-100. Each letter goes from 001 through 100 before moving to the next letter. Each new collection entry is just the next ID in the pattern. It is unlikely that the end will ever be reached, but we'll cross that bridge when we get there.
I've only ever used the default _id for indexing, and can't figure out how to make this pattern.
Thanks for any insight!
Edit #1
The best solution I've come up with so far is to have a separate .txt file with all of the IDs listed in order. As each new object is created, pop (... shift) the next ID off the top of the file. This might also have the added benefit of easily adding additional IDs at a later date. This will probably be the approach I take, but I'm still interested in the mongoose solution requested above.
Edit #2
So I think the solution I'm going to use is a little different. Basically, findOne sorted by bookID descending. Then use the value returned to set the next.
Material.findOne()
.sort({bookID : -1})
.exec((err, mat) => {
if(err) {
// Send error
}else if(!mat) {
// First bookID
}else {
// Indexes exist...
let nextId = getNextID(mat.bookID);
// ...
}
});
Still easy to modify getNextID() to add new/different IDs in the future (if/when "Z100" is reached)
Thanks again!
Ok, so to expand a little bit on Edit #2, I've come up with the following solution.
Within the model (schema) file, we add a schema pre() middleware, that executes when .save() is called, before the save occurs:
// An arrow function will not work on this guy, if you want to use the "this" keyword
materialSchema.pre('save', function(next) {
this.model('material').findOne() // Don't forget the .model(...) bit!
.sort({bookID : -1}) // All I need is the highest (i.e. most recent) bookID
.select('bookID') // Ditto above (not really necessary)
.exec((err, result) => {
if(err) {
return next(err); // Oopsies, an error!
}else if(!result) {
this.bookID = 'A-001'; // The case when collection is empty
}else {
this.bookID = getNextID(result.bookID); // Otherwise, increment ID
}
next(); // Don't forget this sucker! This is how you save
});
});
And that's about it! It isn't an in-built solution direct from Mongoose, but it works a treat.
Just for completeness, the getNextID function looks like:
function getNextID(curID) {
let letter = curID.split('-')[0];
let number = parseInt(curID.split('-')[1]);
if(number >= 100) { // Increase the letter and reset the number
letter = String.fromCharCode(letter.charCodeAt(0) + 1)
number = '001';
}else { // Only increase the number
number = ('' + (number + 1)).padStart(3, '0'); // Makes sure the numbers are always 3 digits long
}
return `${letter}-${number}`;
}
This'll do just dandy for now. Until we get to Z100. But I'll cross that bridge if/when it comes. No big deal at all.
And you don't need to do anything special to use it. Just save a new doc as normal, and it automatically fires:
new Material({
// New material properties
}).save((err, mat) => {
// Handle errors and returns ...
});

Resources