I initialize my DB in the usual way:
mongoose.connect(`mongodb://uname:pword#127.0.0.1:port/dbname?authSource=admin`, {useNewUrlParser: true, autoIndex: false});
And I have a Schema, something like:
var materialSchema = new Schema({
bookID: {type: String, required: true},
active: Boolean,
name: {type: String, required: true},
stockLength: {type: Number, required: true}
});
module.exports = mongoose.model('material', materialSchema);
When I create a new material and add it to the database, it is automatically assigned the usual _id - which is a behaviour I want to maintain. BUT, I'd also like for bookID to be a unique, auto-incrementing index. This is for physical shelf storage, and not for queries or anything like that.
I'd like for bookID to increment in the following way:
A-001
A-002
A-003
...
A-098
A-099
A-100
B-001
...
B-100
...
Z-001
...
Z-100
In case the pattern above isn't clear, the pattern starts at A-001 and ultimately ends at Z-100. Each letter goes from 001 through 100 before moving to the next letter. Each new collection entry is just the next ID in the pattern. It is unlikely that the end will ever be reached, but we'll cross that bridge when we get there.
I've only ever used the default _id for indexing, and can't figure out how to make this pattern.
Thanks for any insight!
Edit #1
The best solution I've come up with so far is to have a separate .txt file with all of the IDs listed in order. As each new object is created, pop (... shift) the next ID off the top of the file. This might also have the added benefit of easily adding additional IDs at a later date. This will probably be the approach I take, but I'm still interested in the mongoose solution requested above.
Edit #2
So I think the solution I'm going to use is a little different. Basically, findOne sorted by bookID descending. Then use the value returned to set the next.
Material.findOne()
.sort({bookID : -1})
.exec((err, mat) => {
if(err) {
// Send error
}else if(!mat) {
// First bookID
}else {
// Indexes exist...
let nextId = getNextID(mat.bookID);
// ...
}
});
Still easy to modify getNextID() to add new/different IDs in the future (if/when "Z100" is reached)
Thanks again!
Ok, so to expand a little bit on Edit #2, I've come up with the following solution.
Within the model (schema) file, we add a schema pre() middleware, that executes when .save() is called, before the save occurs:
// An arrow function will not work on this guy, if you want to use the "this" keyword
materialSchema.pre('save', function(next) {
this.model('material').findOne() // Don't forget the .model(...) bit!
.sort({bookID : -1}) // All I need is the highest (i.e. most recent) bookID
.select('bookID') // Ditto above (not really necessary)
.exec((err, result) => {
if(err) {
return next(err); // Oopsies, an error!
}else if(!result) {
this.bookID = 'A-001'; // The case when collection is empty
}else {
this.bookID = getNextID(result.bookID); // Otherwise, increment ID
}
next(); // Don't forget this sucker! This is how you save
});
});
And that's about it! It isn't an in-built solution direct from Mongoose, but it works a treat.
Just for completeness, the getNextID function looks like:
function getNextID(curID) {
let letter = curID.split('-')[0];
let number = parseInt(curID.split('-')[1]);
if(number >= 100) { // Increase the letter and reset the number
letter = String.fromCharCode(letter.charCodeAt(0) + 1)
number = '001';
}else { // Only increase the number
number = ('' + (number + 1)).padStart(3, '0'); // Makes sure the numbers are always 3 digits long
}
return `${letter}-${number}`;
}
This'll do just dandy for now. Until we get to Z100. But I'll cross that bridge if/when it comes. No big deal at all.
And you don't need to do anything special to use it. Just save a new doc as normal, and it automatically fires:
new Material({
// New material properties
}).save((err, mat) => {
// Handle errors and returns ...
});
Related
I am new to dynamodb.
I want to increment the Sort Key
If the id=0 the next id=1 and so on,
If the user(Partition key), id(Sort Key) add items the next add items the id increment 1.
The code use on PutItem with dynamodb.
Is possible to do that?
I did not want use the UUID( unique Key)
Most situations don't need an auto-incrementing attribute and DynamoDB doesn't provide this feature out of the box. This is considered to be an anti-pattern in distributed systems.
But, see How to autoincrement in DynamoDB if you really need to.
I understand that you may need this number because it is a legal obligation to have incremental invoice numbers for example.
One way would be to create a table to store your number sequences.
Add fields like:
{
name: "invoices",
prefix: "INV",
numberOfDigits: 5,
leasedValue: 1,
appliedValue: 1,
lastUpdatedTime: '2022-08-05'
},
{
name: "deliveryNotes",
prefix: "DN",
numberOfDigits: 5,
leasedValue: 1,
appliedValue: 1,
lastUpdatedTime: '2022-08-05'
}
You need 2 values (a lease and an applied value), to make sure you never skip a beat, even when things go wrong.
That check-lease-apply-release/rollback logic looks as follows:
async function useSequence(name: string, cb: async (uniqueNumber: string) => void) {
// 1. GET THE SEQUENCE FROM DATABASE
const sequence = await getSequence("invoices");
this.validateSequence(sequence);
// 2. INCREASE THE LEASED VALUE
const oldValue = sequence.appliedValue;
const leasedValue = oldValue + 1;
sequence.leasedValue = leasedValue;
await saveSequence(sequence);
try {
// 3. CREATE AND SAVE YOUR DOCUMENT
await cb(format(leasedValue));
// 4. INCREASE THE APPLIED VALUE
sequence.appliedValue++;
await saveSequence(sequence);
} catch(err) {
// 4B. ROLLBACK WHEN THINGS ARE BROKEN
console.err(err)
try {
const sequence = await getSequence(name);
sequence.leasedValue--;
this.validateSequence(sequence);
await saveSequence(sequence);
} catch (err2) {
console.error(err2);
}
throw err;
}
}
function validateSequence(sequence) {
// A CLEAN STATE, MEANS THAT THE NUMBERS ARE IN SYNC
if (sequence.leasedValue !== sequence.appliedValue) {
throw new Error("sequence is broken.");
}
}
Then, whenever you need a unique number you can use the above function to work in a protected scope, where the number will be rollbacked when something goes wrong.
const details = ...;
await useSequence("invoice", async (uniqueNumber) => {
const invoiceData = {...details, id: uniqueNumber};
const invoice = await this.createInvoice(invoiceData);
await this.saveInvoice(invoice);
})
Can it scale? Can it run on multiple instances? No, it can't. It never will be, because in most countries it's just not legal to do so. You're not allowed to send out invoice 6 before invoice 5 or to cancel invoice 5 after you've send invoice 6.
The only exception being, if you have multiple sequences. e.g. in some cases you're allowed to have a sequence per customer, or a sequence per payment system, ... Hence, you want them in your database.
I'm trying to create a little task management site for a work project. The overall goal is here is that the tasks stay the same each month (their status can be updated and whatnot), and they need to be duplicated at the start of each new month so they can be displayed and sorted by on a table.
I already figured out how to schedule the task, I have the table I need set up. A little explanation before the code - the way I'm planning on doing this is having two different task collections - one I've called "assignments", will have the tasks that need to be duplicated (with their description, status and other necessary data) and another collection, which I called "tasks", will have the exact same data but with an additional "date" field. This is where the table will get it's data from, the date is just for sorting purposes.
This is what I have so far -
Index.js: gets all the assignments from the database, and sends the object over to the duplicate function.
router.get('/test', async function(req, res, next) {
let allTasks = await dbModule.getAllAssignments();
let result = await dbModule.duplicateTasks(allTasks);
res.json(result);
});
dbmodule.js:
getAllAssignments: () => {
allAssignments = Assignment.find({});
return allAssignments;
},
duplicateTasks: (allTasksToAdd) => {
try {
for (let i = 0; i < allTasksToAdd.length; i++) {
let newTask = new Task({
customername: allTasksToAdd.customername,
provname: allTasksToAdd.provname,
description: allTasksToAdd.description,
status: allTasksToAdd.status,
date: "07-2020"
})
newTask.save();
}
return "Done"
} catch (error) {
return "Error"
}
}
The issue arises when I try and actually duplicate the tasks. For testing purposes I've entered the date manually this time, but that's all that ends up being inserted - just the date, the rest of the data is skipped. I've heard of db.collection.copyTo(), but I'm not sure if it'll allow me to insert the field I need or if it's supported in mongoose. I know there's absolutely an easier way to do this but I can't quite figure it out. I'd love some input and suggestions if anyone has any.
Thanks.
The problem is that allTasksToAdd.customername (and the other fields your trying to access) will be undefined. You need to access the fields under the current index:
let newTask = new Task({
customername: allTasksToAdd[i].customername,
provname: allTasksToAdd[i].provname,
description: allTasksToAdd[i].description,
status: allTasksToAdd[i].status,
date: "07-2020"
})
Note that you can simplify this by using a for .. of loop instead:
for (const task of allTasksToAdd) {
const newTask = new Task({
customername: task.customername,
provname: task.provname,
description: task.description,
status: task.status,
date: "07-2020"
});
newTask.save();
}
I'm developing a small NodeJS web app using Mongoose to access my MongoDB database. A simplified schema of my collection is given below:
var MySchema = mongoose.Schema({
content: { type: String },
location: {
lat: { type: Number },
lng: { type: Number },
},
modifierValue: { type: Number }
});
Unfortunately, I'm not able to sort the retrieved data from the server the way it is more convenient for me. I wish to sort my results according to their distance from a given position (location) but taking into account a modifier function with a modifierValue that is also considered as an input.
What I intend to do is written below. However, this sort of sort functionality seems to not exist.
MySchema.find({})
.sort( modifierFunction(location,this.location,this.modifierValue) )
.limit(20) // I only want the 20 "closest" documents
.exec(callback)
The mondifierFunction returns a Double.
So far, I've studied the possibility of using mongoose's $near function, but this doesn't seem to sort, not allow for a modifier function.
Since I'm fairly new to node.js and mongoose, I may be taking a completely wrong approach to my problem, so I'm open to complete redesigns of my programming logic.
Thank you in advance,
You might have found an answer to this already given the question date, but I'll answer anyway.
For more advanced sorting algorithms you can do the sorting in the exec callback. For example
MySchema.find({})
.limit(20)
.exec(function(err, instances) {
let sorted = mySort(instances); // Sorting here
// Boilerplate output that has nothing to do with the sorting.
let response = { };
if (err) {
response = handleError(err);
} else {
response.status = HttpStatus.OK;
response.message = sorted;
}
res.status(response.status).json(response.message);
})
mySort() has the found array from the query execution as input and the sorted array as output. It could for instance be something like this
function mySort (array) {
array.sort(function (a, b) {
let distanceA = Math.sqrt(a.location.lat**2 + a.location.lng**2);
let distanceB = Math.sqrt(b.location.lat**2 + b.location.lng**2);
if (distanceA < distanceB) {
return -1;
} else if (distanceA > distanceB) {
return 1;
} else {
return 0;
}
})
return array;
}
This sorting algorithm is just an illustration of how sorting could be done. You would of course have to write the proper algorithm yourself. Remember that the result of the query is an array that you can manipulate as you want. array.sort() is your friend. You can information about it here.
I have a scenario in node/express/mongoose where I have users who accumulate points, and when that point total crosses a certain threshold, they "level up" (think games with point-based levels).
I have created a custom setter on the points field that checks if the value has changed, and if so tries to update the level field. Levels are defined in another collection, but are saved as simple JSON objects when associated with user docs (hence the .lean() in the query). I did it this way vs a virtual field or population for efficiency.
Problem: this doesn't actually seem to update the user 'level' field when it should. What am I doing wrong?
// Define our user schema
var UserSchema = new mongoose.Schema({
...
points: {type: Number, default: 0, set: pointsChangeHandler},
level: {name: String, minPoints: Number, maxPoints: Number},
...
});
And the setter looks like so:
function goodPointsChangeHandler(newValue) {
var self = this;
if (newValue != self.goodPoints) {
console.log('Point change detected.');
// Find level that corresponds with new point total
Level.findOne({
'minPoints': {$lte : self.goodPoints},
'maxPoints': {$gt : self.goodPoints}}, '-_id').lean().exec(function(err, level) {
if (self.goodLevel == undefined || self.goodLevel.rank != level.rank) {
console.log('Level changed.');
self.goodLevel = level;
}
return newValue;
});
}
return newValue;
}
Based on #laggingreflex's comment, I tried modifying this within the scope of the model method (i.e. not in the Level.findOne() callback, and changes made that way were persisted without an explicit save() call.
Also, I had a pretty silly error where I was returning newValue from thefindOne` callback.. not sure what I was thinking there...
Long story short, and this may be obvious to node/express/mongoose experts, but you can modify fields other than the one whose setter method you're currently in, but the moment you find yourself in the callback of another async method, you'll have to do an explicit save() or your modifications to this will not be persisted.
So:
function myFieldSetterMethod(newValue) {
this.myField = "a";
this.myOtherField = "b";
return newValue;
// no save() necessary, this will update both fields
}
function myFieldSetterMethod(newValue) {
this.myField = "a";
SomeModel.findOne(..., function(err, doc) {
this.myOtherField = doc.somethingIWantFromThisDoc;
// now you'll need an explicit save
// this.save(...)
});
return newValue;
}
I'd like to manage my own _id's through Mongoose/MongoDB - as the default ones are pretty long and consume bandwidth.
The difficulty is that I need to create them (say, by incrementing a counter), but I need to do this in a concurrent environment (Node.JS). Is there a simple example I could follow that creates a schema, with a custom _id and a static method (or anything better) that automatically generates the next unique _id, whenever a new document is created?
You could use Mongo's findAndModify() to generate sequential values. Below is an example of this:
// (assuming db is a reference to a MongoDB database)
var counters = db.collection('counters');
var query = {'name': 'counterName'};
var order = [['_id','asc']];
var inc = {$inc:{'next':1}};
var options = {new: true, upsert: true};
counters.findAndModify(query, order, inc, options, function(err, doc) {
if(err) {
callback(err);
return;
}
var id = doc.next;
callback(null, id);
});
Although generating sequential IDs looks pretty on applications keep in mind that there are some drawbacks to them (e.g. when you need to split your database geographically) which is why Mongo uses the long pseudo-random keys that it does.
As Chad briefly touched on, Mongo implements a uuid system for you, taking into account the timestamp, network address, and machine name, plus an autoincrementing 2 digit counter (in the event that multiple entries with the same timestamp occur). This schema is used to allow distributed databases (ie, running different database instances on different machines) while ensuring that each entry will still have a unique identifier (because the machine name section will be different).
Trying to role out your own schema would likely greatly limit the scalability that mongo provides.
This should work
import { randomString } from '#/helpers'
const taskSchema = new mongoose.Schema({
_id: {
type: String,
unique: true,
default: randomString
},
title: String,
...
})
Random string function
// helpers
export const randomString = (length?: number) => {
let result = ''
const characters = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789'
const charactersLength = characters.length
const n = length || 15
for (let i = 0; i < n; i++) {
result += characters.charAt(Math.floor(Math.random() * charactersLength))
}
return result
}
Tested result
{ "_id": "EIa9W2J5mY2lMDY", ... }