When using the library mongoose-uuid, I am able to setup UUID types for my schemas, so when I read the data it is in string (utf-8) format and when I save the data it is in UUID ObjectID BSON type 4 format. This works great with top level or flat direct values and ref definitions in my schema. However, when I have a UUID's in an array of ref's in a schema, the array saves to the database correctly, However when it is presented it is in its raw type. Based on the example below you can see scope_id is presented in the right format but the entitlements are not.
Here are the versions I am using:
mongoose-uuid - 2.3.0
mongoose - 5.5.11
I have tried modifying the library (mongoose-uuid) by changing the getter and converting the value, however, when I do so, it works when presenting but fails when it saves to the database. This is most likely due to the fact that the value is converted or casted before saving to the database.
Here is an example schema
{
"code": {
"type": String,
"required": true
},
"scope_id": {
"type": mongoose.Types.UUID,
"ref": "scopes"
},
"entitlements": [{
"type": mongoose.Types.UUID,
"ref": "entitlements"
}]
}
Example actual response
{
"entitlements": [
"zMihi1BKRomM1Q41p7hgLA==",
"ztOYL7n1RoGA6aoc0TcqoQ=="
],
"code": "APPUSR",
"scope_id": "b8f80c82-8325-4ffd-bfd7-e373a90e7c45",
"id": "32e79061-e531-45ad-b934-56811e2ad713"
}
Expected Response
{
"entitlements": [
"ccc8a18b-504a-4689-8cd5-0e35a7b8602c",
"ced3982f-b9f5-4681-80e9-aa1cd1372aa1"
],
"code": "APPUSR",
"scope_id": "b8f80c82-8325-4ffd-bfd7-e373a90e7c45",
"id": "32e79061-e531-45ad-b934-56811e2ad713"
}
As mentioned above, the code does work but breaks another part of the code. I found a solution that corrects this:
It is a slight amendment to the code above
SchemaUUID.prototype.cast = function (value, doc, init) {
console.log("cast", value, doc, init)
if (value instanceof mongoose.Types.Buffer.Binary) {
if (init && doc instanceof mongoose.Types.Embedded) {
return getter(value);
}
return value;
}
if (typeof value === 'string') {
var uuidBuffer = new mongoose.Types.Buffer(uuidParse.parse(value));
uuidBuffer.subtype(bson.Binary.SUBTYPE_UUID);
return uuidBuffer.toObject();
}
throw new Error('Could not cast ' + value + ' to UUID.');
};
This alternate version of the code allows for updates such as POST and PATCH to be applied.
As per my observation, if you change the below function in mongoose, it works fine
SchemaUUID.prototype.cast = function (value, doc, init) {
console.log("cast", value, doc, init)
if (value instanceof mongoose.Types.Buffer.Binary) {
if (init) {
return getter(value);
} else {
return value;
}
}
if (typeof value === 'string') {
var uuidBuffer = new mongoose.Types.Buffer(uuidParse.parse(value));
uuidBuffer.subtype(bson.Binary.SUBTYPE_UUID);
return uuidBuffer.toObject();
}
throw new Error('Could not cast ' + value + ' to UUID.');
};
Basically when you save objects init is false and when its initiated init is true
Related
I'm trying to follow this tutorial to filter the replication between a pouchdb and a couchdb databases
https://pouchdb.com/2015/04/05/filtered-replication.html
The problem is when I try to create the filtered function in the Fauxton webapp. In my created database, I click Design Document > New Docs and then paste the function:
{
"_id": "_design/app",
"filters": {
"by_agent": function(doc, req) {
return doc.agent === req.query.agent;
}.toString()
}
}
and when I click Create Document button, it crashes. The javascript console says
Uncaught SyntaxError: Unexpected token u in JSON at position 61
at JSON.parse ()
at t.checkDocIsValid (https://127.0.0.1:6984/_utils/dashboard.assets/js/bundle-b8e0ba71119195edb7ec64b98f53d9b9.js:529:19481)
at t.saveDoc (https://127.0.0.1:6984/_utils/dashboard.assets/js/bundle-b8e0ba71119195edb7ec64b98f53d9b9.js:529:19056)
...
how do I create the filtered function in couchDB? Maybe that isn't the procedure or I have to create it on another dababase. Thanks in advance
So what you're trying to do is use JavaScript code to create a view. Therefore, Fauxton takes only JSON as a document.
Here's how you can get the JSON from the JavaScript snippet :
//The snippet you had was a JavaScript object
//Even if it seems like a JSON object, there is a function() declaration followed by a .toString()
//By doing so, it easier to write functions instead of writing them in a raw string.
var javascriptObject = {
"_id": "_design/app",
"filters": {
"by_agent": function(doc, req) {
return doc.agent === req.query.agent;
}.toString()
}
}
console.info("You should use the following string in your Fauxton Editor:");
console.log(JSON.stringify(javascriptObject));
You should use the following string instead of the JavaScript snippet you tried:
{
"_id": "_design/app",
"filters": {
"by_agent": "function (doc, req) {\n return doc.agent === req.query.agent;\n }"
}
}
I need to create a view that lists the values for an attribute of a doc field.
Sample Doc:
{
"_id": "003e5a9742e04ce7a6791aa845405c17",
"title", "testdoc",
"samples": [
{
"confidence": "high",
"handle": "joetest"
}
]
}
Example using that doc, I want a view that will return the values for "handle"
I found this example with the heading - Get contents of an object with specific attributes e.g. doc.objects.[0].attribute. But when I fill in the attribute name, e.g. "handle" and replace doc.objects with doc.samples, I get no results:
Toggle line numbers
// map
function(doc) {
for (var idx in doc.objects) {
emit(doc.objects[idx], attribute)
}
}
That will create an array of key-value-pairs where the key is alway the value of handle. Replace null with a value you want e.g. doc.title. If you want to get the doc attached to every row use the query parameter ?include_docs=true while requesting the view.
// map
function (doc) {
var samples = doc.samples
for(var i = 0, sample; sample = samples[i++];) {
emit(sample.handle, null)
}
}
Like this ->
function(doc) {
for (var i in doc.samples) {
emit(doc._id, doc.samples[i].handle)
}
}
It will produce a result based on the doc._id field as the key. Or, if you want your key to be based on the .handle field you reverse the parameters in emit so you can search by startKey=, endKey=.
I have a mongodb document with an array that contains both normal numbers (32-bit) and
big ones (64-bits), and I query this document with the node javascript driver for mongodb.
When querying the document in a mongo shell, I can see 64-bit numbers printed as NumberLong. For example: NumberLong("1099511627776")
Basically I have two questions:
(1) Is there a clean way to check if a value is a 64-bit number? Today I use the following expression which is by elimination
if (typeof counters[slot] !== "number") {
// I assume counters[slot] is a 64-bit object
}
(2) How do I cast this value to an instance of mongo.Long? Today I cast to mongo.Long in a convoluted way. Is there a cleaner way to perform the cast?
var mongoLong = require('mongodb').Long;
if (typeof counters[slot] !== "number") {
var bigNumber = mongoLong(counters[slot].low_, counters[slot].high_);
// do things with bigNumber
}
Thanks!
Note:
One may wonder why I need to cast a Long to a Long.
The reason is because the "Long" value which was
returned by the MongoDB javascript driver query is not really an instance of the Long class. i.e.
it wasn't created with a Long() constructor, i.e. you cannot invoke the member functions on it, like getHighBits(), or isNegative().
Okay. There are a couple of ways you can get numbers stored as different BSON types in MongoDB. For starters let's consider this information in the shell first:
db.numbers.insert({ "n": NumberLong(1) }) // 64bit
db.numbers.insert({ "n": NumberInt(1) }) // 32bit
db.numbers.insert({ "n": 1 }) // Actually a Double!
All have difference BSON types as shown, so the best way to query for each is using the $type operator which tests for that field "type":
db.numbers.find({ "n": { "$type": 18 } }) // 64bit
db.numbers.find({ "n": { "$type": 16 } }) // 32bit
db.numbers.find({ "n": { "$type": 1 } }) // Double
If you tried to do this with JavaScript evaluation, then only the last two respond to your current test, and it's not the 64bit result since the "typeof" would just return object:
db.numbers.find(function() { return typeof(this.n) == "number" })
db.numbers.find(function() { return typeof(this.n) == "object" })
So only the last test matches the 64Bit integer since that is how this evaluates in JavaScript. But really the above $type test is better since it can be used directly in the query and use native code. Native code is better than JavaScript evaluation either inside or outside the query, so it is best to use it. And it's faster.
If you have a whole bunch to convert then you are going to need to loop the collection, covert to Long and write back. And best way to construct is using Long.fromInt, as well as of course handling the loop with Bulk Write Operations:
var async = require('async'),
MongoClient = require('mongodb').MongoClient,
Long = require('mongodb').Long;
MongoClient.connect('mongodb://localhost/test',function(err,db) {
db.collection('numbers',function(err,collection) {
var bulk = collection.initializeOrderedBulkOp();
var count = 0;
var stream = collection.find(
{
"$or": [
{ "n": { "$type": 16 } },
{ "n": { "$type": 1 } }
]
}
).stream();
stream.on('data',function(data) {
bulk.find({ "_id": data._id }).updateOne({
"$set": { "n": Long.fromInt( data.n ) }
});
if ( count % 1000 == 0 ) {
stream.pause();
bulk.execute(function(err,result) {
if (err) throw err;
bulk = collection.initializeOrderedBulkOp();
stream.resume();
});
}
});
stream.on('end',function() {
if ( count % 1000 != 0 )
bulk.execute(function(err,result) {
if (err) throw err;
bulk = collection.initializeOrderedBulkOp();
});
console.log("done");
process.exit();
});
});
});
That's just a basic premise working example, but it's the best way to go through your whole collection and covert the types as wanted.
Is it possible to $addToSet and determine which items were added to the set?
i.e. $addToSet tags to a post and return which ones were actually added
Not really, and not with a single statement. The closest you can get is with the findAndModify() method, and compare the orginal document form to the fields that you submitted in your $addToSet statement:
So considering an initial document:
{
"fields": [ "B", "C" ]
}
And then processing this code:
var setInfo = [ "A", "B" ];
var matched = [];
var doc = db.collection.findAndModify(
{ "_id": "myid" },
{
"$addToSet": { "fields": { "$each": setInfo } }
}
);
doc.fields.forEach(function(field) {
if ( setInfo.indexOf(field) != -1 ) {
matched.push(field);
}
});
return matched;
So that is a basic JavaScript abstraction of the methods and not actually nodejs general syntax for either the native node driver or the Mongoose syntax, but it does describe the basic premise.
So as long as you are using a "default" implementation method that returns the "original" state of the document before it was modified the you can play "spot the difference" as it were, and as is shown in the code example.
But doing this over general "update" operations is just not possible, as they are designed to possibly affect one or more objects and never return this detail.
I have a mongodb model called User, which has a mixed schema type variable called "inventory" (contains all the items a user contains). I would like to loop through all the users, and change the name of each item in their inventory. In particular, I would like to convert strings in the format of "10_alex_magician" or "3_maia_princess" to "alex_magician" and "maia_princess" respectively. The string conversion is relatively straightforward, and I'm using x.split('').slice(1).join('') to accomplish the conversion.
Where I'm having trouble is even when console.log shows that the conversion has been applied, it doesn't seem to be updating correctly to mongodb, yet no error message is being thrown. Does anyone know how to solve this?
Node.js function
//function to change old naming of items "10_alex_magician" to "alex_magician"
function modifyUser() {
User.find({}, function(err, results) {
_.map(results, function(result) {
var regex = /^\d+_[A-Za-z]+_[A-Za-z]+$/
for (var i = 0, len = result.inventory.length; i < len; i++) {
if(regex.test(result.inventory[i].itemName)) {
result.inventory[i].itemName = result.inventory[i].itemName.split('_').slice(1).join('_');
result.save(function(err, r) {
if(err) console.log(err);
//logging r shows that the text has been correctly updated
console.log(r)
});
}
}
})
})
}
Format of inventory variable
"inventory": [
{
"type": "sticker",
"numberOwned": 2,
"itemName": "1_alex_magician"
},
{
"type": "sticker",
"numberOwned": 1,
"itemName": "10_alex_scuba"
}
],
Mongoose only has automatic change detection for top-level properties and you are modifying a nested property, so mongoose doesn't know anything changed. Use markModified to tell mongoose you are mucking with inventory.
result.inventory[i].itemName = result.inventory[i].itemName.split('_').slice(1).join('_');
result.markModified('inventory');
result.save(....)
For efficiency, you may want to consider both .lean() and .stream() for this type of query and just do your updates with findByIdAndUpdate, passing just the updated inventory property.