Is the below query supported in Azure DocumentDB? It returns no documents.
Variables values at runtime:
1. collectionLink = "<link for my collection>"
2. feedOptions = new FeedOptions { MaxItemCount = 2 }
3. name = "chris"
client.CreateDocumentQuery<T>(collectionLink, feedOptions).Where(m => (m.Status == "Foo" && (m.Black.Name == null || m.Black.Name != name) && (m.White.Name == null || m.White.Name != name)));
I have tested with simpler queries, such as the below, which both return results I expect.
client.CreateDocumentQuery<T>(collectionLink, feedOptions).Where(m => m.Status == "Foo");
client.CreateDocumentQuery<T>(collectionLink, feedOptions).Where(m => m.Status == "Foo").Where(m => m.Size == 19);
Lastly, I've ensured there are documents which meet the problematic query's filter criteria:
{
"id": "1992db52-c9c6-4496-aaaa-f8cb83a8c6b0",
"status": "Foo",
"size": 19,
"black": {
"name": "charlie"
},
"white": {},
}
Thanks.
Turns out the "m.White.Name == null || m.White.Name != name" check is problematic because the Name field does not exist on the document in the DB.
When the document is edited to the following, the query returns it. Notice the explicit null value for Name field.
{
"id": "1992db52-c9c6-4496-aaaa-f8cb83a8c6b0",
"status": "Foo",
"size": 19,
"black": {
"name": "charlie"
},
"white": {
"name": null
},
}
The query can be written to handle missing properties using DocumentDB UDFs as follows. DocumentDB uses JavaScript's semantics, and an explicit null is different from a missing property ("undefined") in JavaScript. To check for explicit null is simple (== null like your query), but to query for a field that may or may not exist in DocumentDB, you must first create a UDF for ISDEFINED:
function ISDEFINED(doc, prop) {
return doc[prop] !== undefined;
}
And then use it in a DocumentDB query like:
client.CreateDocumentQuery<T>(
collectionLink,
"SELECT * FROM docs m WHERE m.Status == "Foo" AND (ISDEFINED(m.white, "name") OR m.white.name != name)");
Hope this helps. Note that since != and UDFs both require scans, it's a good idea for performance/scale to always use them only within queries that have other filters.
Related
When using the library mongoose-uuid, I am able to setup UUID types for my schemas, so when I read the data it is in string (utf-8) format and when I save the data it is in UUID ObjectID BSON type 4 format. This works great with top level or flat direct values and ref definitions in my schema. However, when I have a UUID's in an array of ref's in a schema, the array saves to the database correctly, However when it is presented it is in its raw type. Based on the example below you can see scope_id is presented in the right format but the entitlements are not.
Here are the versions I am using:
mongoose-uuid - 2.3.0
mongoose - 5.5.11
I have tried modifying the library (mongoose-uuid) by changing the getter and converting the value, however, when I do so, it works when presenting but fails when it saves to the database. This is most likely due to the fact that the value is converted or casted before saving to the database.
Here is an example schema
{
"code": {
"type": String,
"required": true
},
"scope_id": {
"type": mongoose.Types.UUID,
"ref": "scopes"
},
"entitlements": [{
"type": mongoose.Types.UUID,
"ref": "entitlements"
}]
}
Example actual response
{
"entitlements": [
"zMihi1BKRomM1Q41p7hgLA==",
"ztOYL7n1RoGA6aoc0TcqoQ=="
],
"code": "APPUSR",
"scope_id": "b8f80c82-8325-4ffd-bfd7-e373a90e7c45",
"id": "32e79061-e531-45ad-b934-56811e2ad713"
}
Expected Response
{
"entitlements": [
"ccc8a18b-504a-4689-8cd5-0e35a7b8602c",
"ced3982f-b9f5-4681-80e9-aa1cd1372aa1"
],
"code": "APPUSR",
"scope_id": "b8f80c82-8325-4ffd-bfd7-e373a90e7c45",
"id": "32e79061-e531-45ad-b934-56811e2ad713"
}
As mentioned above, the code does work but breaks another part of the code. I found a solution that corrects this:
It is a slight amendment to the code above
SchemaUUID.prototype.cast = function (value, doc, init) {
console.log("cast", value, doc, init)
if (value instanceof mongoose.Types.Buffer.Binary) {
if (init && doc instanceof mongoose.Types.Embedded) {
return getter(value);
}
return value;
}
if (typeof value === 'string') {
var uuidBuffer = new mongoose.Types.Buffer(uuidParse.parse(value));
uuidBuffer.subtype(bson.Binary.SUBTYPE_UUID);
return uuidBuffer.toObject();
}
throw new Error('Could not cast ' + value + ' to UUID.');
};
This alternate version of the code allows for updates such as POST and PATCH to be applied.
As per my observation, if you change the below function in mongoose, it works fine
SchemaUUID.prototype.cast = function (value, doc, init) {
console.log("cast", value, doc, init)
if (value instanceof mongoose.Types.Buffer.Binary) {
if (init) {
return getter(value);
} else {
return value;
}
}
if (typeof value === 'string') {
var uuidBuffer = new mongoose.Types.Buffer(uuidParse.parse(value));
uuidBuffer.subtype(bson.Binary.SUBTYPE_UUID);
return uuidBuffer.toObject();
}
throw new Error('Could not cast ' + value + ' to UUID.');
};
Basically when you save objects init is false and when its initiated init is true
I have a document structure in Cosmos that typically looks like this:
{
"Item No": "123456",
"Item Desc": "This is a description",
"images": [
"https://somedomain.com/image1.png",
"https://somedomain.com/image2.png"
]
}
Sometimes, there will be empty image values. I have written a UDF (user defined function) which will replace any empty values, with a default value:
function missingImage(doc, prop) {
if (typeof doc[prop] === "undefined" || doc[prop] === "" || doc[prop] === null) {
return "https://via.placeholder.com/150";
}
}
In the event an image url is blank, I get back this return (correct):
{
"id": "e3842b29-313c-4a84-bc94-bc43a9a55742",
"Item No": "123456",
"Item Desc": "This is a description.",
"image": "https://via.placeholder.com/150"
},
My SELECT query looks like this:
"c.id, c['Item No'], c['Item Desc'], udf.missingImage(c.images[0]) as image"
However, in situations where no image key exists at all, for example:
{
"Item No": "123456",
"Item Desc": "This is a description."
}
I don't get back my default.
My question: How can I modify my UDF or query, such that if the images key does not exist, I still return a default value?
Thanks #jay-gong for your time and response, however, this does not address the issue. I am looking for a way to return a default value when no images key exists in the document at all.
I feel the answer here, is not going to be through the UDF, rather it will need to be addressed at the query level. I am basing this off that fact that if I directly return a default, as you will see in the below UDF example, I don't get back images regardless.
The document in Cosmos:
{
"id": "8fdc9f47-6209-455d-9b9c-482341bb3170",
"Item No": "123456",
"Item Desc": "This is a description."
}
The UDF:
function missingImage(images) {
return "https://via.placeholder.com/150";
}
The query:
SELECT c.id, c['Item No'], c['Item Desc'], udf.missingImage(c.images) FROM c
The return:
[
{
"id": "8fdc9f47-6209-455d-9b9c-482341bb3170",
"Item No": "123456",
"Item Desc": "This is a description."
}
]
UPDATE:
I have come up with a solution, which is to use IS_DEFINED to check if the images key is defined. If its not, return false, which then gives me something to act on within the UDF.
The query:
SELECT c.id,
c['Item No'],
c['Item Desc'],
udf.missingImage((IS_DEFINED(c.images) = true ? c.images : false))
FROM c
The UDF:
function missingImage(images) {
if (images == false) {
return "https://via.placeholder.com/150";
}
return images;
}
Firstly, the sample document you provided is incorrect format of json.
I suppose that it should be like:
You could modify the udf function like:
function missingImage(images) {
for(var i =0;i<images.length;i++){
if (typeof images[i] === "undefined" || images[i] === "" || images[i] === null) {
images[i] = "https://via.placeholder.com/150";
}
}
return images;
}
Then use sql to make sure no "" value in the results:
SELECT udf.missingImage(c.images) FROM c
How can CosmosDB Query the values of the properties within a dynamic JSON?
The app allows storing a JSON as a set of custom properties for an object. They are serialized and stored in CosmosDb. For example, here are two entries:
{
"id": "ade9f2d6-fff6-4993-8473-a2af40f071f4",
...
"Properties": {
"fn": "Ernest",
"ln": "Hemingway",
"a_book": "The Old Man and the Sea"
},
...
}
and
{
"id": "23cb9d4c-da56-40ec-9fbe-7f5178a92a4f",
...
"Properties": {
"First Name": "Salvador",
"Last Name": "Dali",
"Period": "Surrealism"
},
...
}
How can the query be structured so that it searches in the values of Properties?
I’m looking for something that doesn’t involve the name of the
sub-propety, like SELECT * FROM c WHERE
some_function_here(c.Properties, ‘Ernest’)
Maybe I get your idea that you want to filter the documents by the value of the Properties, not the name. If so , you could use UDF in cosmos db.
sample udf:
function query(Properties,filedValue){
for(var k in Properties){
if(Properties[k] == filedValue)
return true;
}
return false;
}
sample query:
SELECT c.id FROM c where udf.query(c.Properties,'Ernest')
output:
Just summary here, Ovi's udf function like:
function QueryProperties (Properties, filedValue) {
for (var k in Properties) {
if (Properties[k] && Properties[k].toString().toUpperCase().includes(filedValue.toString().toUpperCase()))
return true;
return false;
}
Both of the following syntax's will work.
SELECT * FROM c where c.Properties["First Name"] = 'Salvador'
SELECT * FROM c where c.Properties.fn = 'Ernest'
I am using Cloudant which is just CouchDB with some more features and I'm looking for the syntax to count the number of elements in an array. Specifically, I'm creating a design document where I only index the document if its field 'color' has more than 0 elements. How do I check to see if there are more than 0 elements in 'color'?
{"_id": "_design/geodd",
"_rev": "5-2729b1453b11f81261ddb3cf3f3de72f",
"st_indexes": {
"geoidx": {
"index": "function(doc) {if (doc.geometry && doc.geometry.coordinates && doc.color != []) {st_index(doc.geometry);}}"
}
}
}
When writing your view, you can execute any Javascript code, so this is more a javascript question than a CouchDB question.
The mistake in your view is that you cannot compare arrays in javascript with the equality operator: doc.color != [] will always return false (because they are not the same objects).
Try replacing this by doc.color.length == 0 if you know in advance that doc.color is always an array.
If your not sure doc.color is an array, write isArray(doc.color) && doc.color.length == 0.
Your view code would become:
{"_id": "_design/geodd",
"_rev": "5-2729b1453b11f81261ddb3cf3f3de72f",
"st_indexes": {
"geoidx": {
"index": "function(doc) {if (doc.geometry && doc.geometry.coordinates && isArray(doc.color) && doc.color.length == 0) {st_index(doc.geometry);}}"
}
}
}
Is it possible to $addToSet and determine which items were added to the set?
i.e. $addToSet tags to a post and return which ones were actually added
Not really, and not with a single statement. The closest you can get is with the findAndModify() method, and compare the orginal document form to the fields that you submitted in your $addToSet statement:
So considering an initial document:
{
"fields": [ "B", "C" ]
}
And then processing this code:
var setInfo = [ "A", "B" ];
var matched = [];
var doc = db.collection.findAndModify(
{ "_id": "myid" },
{
"$addToSet": { "fields": { "$each": setInfo } }
}
);
doc.fields.forEach(function(field) {
if ( setInfo.indexOf(field) != -1 ) {
matched.push(field);
}
});
return matched;
So that is a basic JavaScript abstraction of the methods and not actually nodejs general syntax for either the native node driver or the Mongoose syntax, but it does describe the basic premise.
So as long as you are using a "default" implementation method that returns the "original" state of the document before it was modified the you can play "spot the difference" as it were, and as is shown in the code example.
But doing this over general "update" operations is just not possible, as they are designed to possibly affect one or more objects and never return this detail.