I have a json document in my DB that looks like this :
{
"uri" : "/me/myself/and/bd1e0f91656bfc713eb6560eeaad7ad1.json",
"category" : "content",
"format" : "json",
"versionId" : "14697362595356370",
"contentType" : "application/json",
"contentLength" : "1938",
"collections" : ["http://me.myself.com/collectionA"],
"properties" : {
"relatives" : ["/me/myself/and/B.json", "/me/myself/and/A.json"]
},
"content":{}
}
I'm trying to get all documents that have a specific relative in the properties:
qb.where(
qb.scope(
qb.property('relatives'),
qb.word("/me/myself/and/B.json"),
qb.fragmentScope('properties')
))
But i keep getting a large set of document that doesn't fit the query.
Any idea how to do this using the Marklogic NodeJS API?
I see two things that look like they might be problems. The first is qb.fragmentScope('properties'). This tells MarkLogic to look in the document's properties, rather than the document's content. That doesn't look like what you meant, given your sample JSON document.
The second problem is the word query -- "/me/myself/and/B.json" is likely being broken up into its constituent words (me, myself, and, B, json), which are then matching in other documents. You want to match exactly what's there, so try a value query:
qb.where(
qb.scope(
qb.properties('properties'),
qb.value('relatives', '/me/myself/and/B.json')
)
)
Note that the qb.scope and the qb.properties are to restrict the search to just match the value when it appears in relatives under a properties JSON property. This is different from the JSON property-versus-content point made above.
qb.where(
qb.propertiesFragment(
qb.term('/me/myself/and/B.json')
)
)
This worked for me.
Related
I have a collection of product in which I have document like this
"_id" : ObjectId("5acb1dad698eaa7a254c9017"),
"txtProductCode" : "1233A",
"txtModelCode" : "00M",
"txtPartNo" : "00P",
"txtSerialNo" : "00S",
"txtProductName" : "Watch",
"traderId" : ObjectId("5ac5fb29b0f9b3444e6c1ef2")
I want to search a product based on its name and traderId for which I used
db.getCollection('product').find( {$and:[{'txtProductName':"Watch"},{"traderId" : ObjectId("5ac5fb29b0f9b3444e6c1ef2")}]})
its working fine but now if a user have input model no then it shoud use model number also to search for a product if the user have not input the model no then it should without model number
So My question is do I have to use cases like this
if(req.body.modelNo)
db.getCollection('product').find( {$and:[{'txtProductName':"Watch"},{"traderId" : ObjectId("5ac5fb29b0f9b3444e6c1ef2")},{'txtModelCode':"00M"}]})
else
db.getCollection('product').find( {$and:[{'txtProductName':"Watch"},{"traderId" : ObjectId("5ac5fb29b0f9b3444e6c1ef2")}]})
or is there a way to do this without making cases I have to do this for multiple condtions so I am trying not to use cases
Create the query object first then add the extra key with a conditional check. No need to explicitly use the $and operator when specifying a comma separated list of expressions as it's implicitly provided:
let query = {
'txtProductName': 'Watch',
'traderId': ObjectId('5ac5fb29b0f9b3444e6c1ef2')
};
if (req.body.modelNo) query['txtModelCode'] = req.body.modelNo;
db.getCollection('product').find(query);
If using the $and operator, you can push the additional query into an array then use the list for the $and operator:
let andOperator = [
{ 'txtProductName': 'Watch' },
{ 'traderId': ObjectId('5ac5fb29b0f9b3444e6c1ef2') }
];
if (req.body.modelNo) andOperator.push({ 'txtModelCode': req.body.modelNo });
// if (req.body.modelNo) andOperator = [...andOperator, { 'txtModelCode': req.body.modelNo }];
db.getCollection('product').find({ '$and': andOperator });
Well, I would have done this in this way
First, you should send a json of specific from to backend. for example
[{'txtModelCode':"00M"},{'txtPartNo':"AC"},{'Yts':"xyz"}]
OR
[{'txtModelCode':"00M"},{'txtPartNo':"AC"}]
OR
[{'txtModelCode':"00M"}]
This is the payload that you should expect in req.body. And finally you can use it in your find() criteria. Something like
db.getCollection('product').find( {$and:[{'txtProductName':"Watch"},
{"traderId" : ObjectId("5ac5fb29b0f9b3444e6c1ef2")}, ...req.body]})
... is called spread operator. Spread syntax allows an iterable such as an array expression or string to be expanded. Read more about it here
This will make it totally dynamic. Any scaling in collection can directly be used in find criteria. you never have to add extra line of code
I want to run a query where an item at a specific position in an array needs to be compared.
For example, consider the GeoJSON format for storing location data.
//sample document from collection user
{
name: "Some name",
location : {
type: "Point",
coordinates : [<Longitude>, <Latitude>]
}
}
How would I query users located at a specific longitude?
I cant seem to find anything in the documentation which can help me do the same.
Queries I have tried:
db.users.find({"location.coordinates[0]" : -73.04303})
Change your query to the following
db.users.find({"location.coordinates.0" : -73.04303})
This is the case: A webshop in which I want to configure which items should be listed in the sjop based on a set of parameters.
I want this to be configurable, because that allows me to experiment with different parameters also change their values easily.
I have a Product collection that I want to query based on multiple parameters.
A couple of these are found here:
within product:
"delivery" : {
"maximum_delivery_days" : 30,
"average_delivery_days" : 10,
"source" : 1,
"filling_rate" : 85,
"stock" : 0
}
but also other parameters exist.
An example of such query to decide whether or not to include a product could be:
"$or" : [
{
"delivery.stock" : 1
},
{
"$or" : [
{
"$and" : [
{
"delivery.maximum_delivery_days" : {
"$lt" : 60
}
},
{
"delivery.filling_rate" : {
"$gt" : 90
}
}
]
},
{
"$and" : [
{
"delivery.maximum_delivery_days" : {
"$lt" : 40
}
},
{
"delivery.filling_rate" : {
"$gt" : 80
}
}
]
},
{
"$and" : [
{
"delivery.delivery_days" : {
"$lt" : 25
}
},
{
"delivery.filling_rate" : {
"$gt" : 70
}
}
]
}
]
}
]
Now to make this configurable, I need to be able to handle boolean logic, parameters and values.
So, I got the idea, since such query itself is JSON, to store it in Mongo and have my Java app retrieve it.
Next thing is using it in the filter (e.g. find, or whatever) and work on the corresponding selection of products.
The advantage of this approach is that I can actually analyse the data and the effectiveness of the query outside of my program.
I would store it by name in the database. E.g.
{
"name": "query1",
"query": { the thing printed above starting with "$or"... }
}
using:
db.queries.insert({
"name" : "query1",
"query": { the thing printed above starting with "$or"... }
})
Which results in:
2016-03-27T14:43:37.265+0200 E QUERY Error: field names cannot start with $ [$or]
at Error (<anonymous>)
at DBCollection._validateForStorage (src/mongo/shell/collection.js:161:19)
at DBCollection._validateForStorage (src/mongo/shell/collection.js:165:18)
at insert (src/mongo/shell/bulk_api.js:646:20)
at DBCollection.insert (src/mongo/shell/collection.js:243:18)
at (shell):1:12 at src/mongo/shell/collection.js:161
But I CAN STORE it using Robomongo, but not always. Obviously I am doing something wrong. But I have NO IDEA what it is.
If it fails, and I create a brand new collection and try again, it succeeds. Weird stuff that goes beyond what I can comprehend.
But when I try updating values in the "query", changes are not going through. Never. Not even sometimes.
I can however create a new object and discard the previous one. So, the workaround is there.
db.queries.update(
{"name": "query1"},
{"$set": {
... update goes here ...
}
}
)
doing this results in:
WriteResult({
"nMatched" : 0,
"nUpserted" : 0,
"nModified" : 0,
"writeError" : {
"code" : 52,
"errmsg" : "The dollar ($) prefixed field '$or' in 'action.$or' is not valid for storage."
}
})
seems pretty close to the other message above.
Needles to say, I am pretty clueless about what is going on here, so I hope some of the wizzards here are able to shed some light on the matter
I think the error message contains the important info you need to consider:
QUERY Error: field names cannot start with $
Since you are trying to store a query (or part of one) in a document, you'll end up with attribute names that contain mongo operator keywords (such as $or, $ne, $gt). The mongo documentation actually references this exact scenario - emphasis added
Field names cannot contain dots (i.e. .) or null characters, and they must not start with a dollar sign (i.e. $)...
I wouldn't trust 3rd party applications such as Robomongo in these instances. I suggest debugging/testing this issue directly in the mongo shell.
My suggestion would be to store an escaped version of the query in your document as to not interfere with reserved operator keywords. You can use the available JSON.stringify(my_obj); to encode your partial query into a string and then parse/decode it when you choose to retrieve it later on: JSON.parse(escaped_query_string_from_db)
Your approach of storing the query as a JSON object in MongoDB is not viable.
You could potentially store your query logic and fields in MongoDB, but you have to have an external app build the query with the proper MongoDB syntax.
MongoDB queries contain operators, and some of those have special characters in them.
There are rules for mongoDB filed names. These rules do not allow for special characters.
Look here: https://docs.mongodb.org/manual/reference/limits/#Restrictions-on-Field-Names
The probable reason you can sometimes successfully create the doc using Robomongo is because Robomongo is transforming your query into a string and properly escaping the special characters as it sends it to MongoDB.
This also explains why your attempt to update them never works. You tried to create a document, but instead created something that is a string object, so your update conditions are probably not retrieving any docs.
I see two problems with your approach.
In following query
db.queries.insert({
"name" : "query1",
"query": { the thing printed above starting with "$or"... }
})
a valid JSON expects key, value pair. here in "query" you are storing an object without a key. You have two options. either store query as text or create another key inside curly braces.
Second problem is, you are storing query values without wrapping in quotes. All string values must be wrapped in quotes.
so your final document should appear as
db.queries.insert({
"name" : "query1",
"query": 'the thing printed above starting with "$or"... '
})
Now try, it should work.
Obviously my attempt to store a query in mongo the way I did was foolish as became clear from the answers from both #bigdatakid and #lix. So what I finally did was this: I altered the naming of the fields to comply to the mongo requirements.
E.g. instead of $or I used _$or etc. and instead of using a . inside the name I used a #. Both of which I am replacing in my Java code.
This way I can still easily try and test the queries outside of my program. In my Java program I just change the names and use the query. Using just 2 lines of code. It simply works now. Thanks guys for the suggestions you made.
String documentAsString = query.toJson().replaceAll("_\\$", "\\$").replaceAll("#", ".");
Object q = JSON.parse(documentAsString);
Help me with understanding mongodb, please.
have three collections: threads, messages and users.
thread
{ "title" : "1212", "message" : "12121", "user_id" : "50ffdfa42437e00223000001", "date" : ISODate("2013-04-11T19:48:36.878Z"), "_id" : ObjectId("51671394e5b854b042000003") }
message
{ "message" : "text", "image" : null, "thread_id" : "51671394e5b854b042000003", "user_id" : "516d08a7772d141766000001", "date" : ISODate("2013-04-17T15:58:07.021Z"), "_id" : ObjectId("516ec68fb91b762476000001") }
user
{ "user" : "admin", "date" : ISODate("2013-04-16T08:15:35.497Z"), "status" : 1, "_id" : ObjectId("516d08a7772d141766000001") }
How can I display all messages for current thread and get user name (for comment) from users collection?
this code get only messages without user name
exports.getMessages = function(id, skip, callback) {
var skip = parseInt(skip);
messages.find({thread_id: id}).sort({date: 1}).skip(skip).limit(20).toArray(
function(e, res) {
if (e) {
callback(e)}
else callback(null, res)
});
};
Node.js and mongo native
Generally, Mongo uses embedded documents or references to maintain relationships. Here is a link from the mongo docs worth reading.
What you are currently doing is storing a manual reference to the user collection within your message collection. Mongo manual references require additional queries in order to get that referenced data. In this case, using a reference based relationship will work, but it would force the N+1 query problem. Meaning you will have to make an addition query for every message you wish to display plus the original query for messages. References are explained in further detail here. One solution would be to incorporate DBRefs, which would require language specific driver support.
Another alternative would be use embedded documents. In this case you would store the related user object embedded within the messages object. Here is another link to the mongo docs with a great example. In this case, you would make a single query, which will return all of the messages, with each related user object embedded inside. Although embedded documents encourage duplicate data, in many cases they provide performance benefits. All of this information is explained in the mongo docs and can be read in detail to further understand the data modeling of mongo.
Additionally, the mongoose library is pretty awesome and has a populate function which is helpful for references.
I'm new to search and am having trouble interpreting the documentation on boosting fields in the mapping.
I want to achieve a simple boosting where the title of some article is more important than the tags associated with the article.
Here's an attempt at the config, which I have put in config/[index_name]/[some_name].json:
{
"[type]": {
"properties": {
"_boost": {
"name": "title",
"null_value": 2.0
}
"title": {
"type": "string"
}
}
}
}
I can tell the file is being read because of error messages from previous attempts at this file. I have also been deleting the index and recreating it between attempts so that it will use this mapping.
Will this work? It doesn't give any error messages, but I can't tell if there is any boost in effect from the output of _search or get _mapping API calls.
Here is the result of the _mapping call:
{
"[type]" : {
"properties" : {
"title" : {
"type" : "string"
}
"tags": {
"type" : "string"
}
}
}
}
Have a look at the example in the boost field documentation.
The boost field mapping (applied on the root object) allows to define
a boost field mapping where its content will control the boost level
of the document
The following mapping defines a field named _boost. If the _boost field itself exists within the JSON document indexed, its value will control the boost level of the indexed document.
{
"tweet" : {
"_boost" : {"name" : "_boost", "null_value" : 1.0}
}
}
Nothing special, the example just tells elasticsearch to consider the _boost field as it is and give a default 1.0 value to it when not present. But you are defining a boost for a specific document: that means that when the document matches a query, its score will be boosted according to the _boost field mapping that you applied to the root object. This doesn't have anything to do with boosting at a field level.
With your mapping you're saying that the content of the title field should be used as _boost, and you're giving a default _boost value of 2.0.
"_boost": {
"name": "title",
"null_value": 2.0
}
This doesn't make sense since the title contains text, and it's not either what you want I guess.
There are different ways to give more importance to a match on the title field.
As far as I understood from the documentation you can do it in your mapping like this:
{
"[type]" : {
"properties" : {
"title" : {
"type" : "string",
"boost" : 2.0
}
"tags": {
"type" : "string"
}
}
}
}
Quite honestly I haven't tried it and never used it before, but lucene does allow you to specify a boost per field at index time. The boost becomes part of the norms for that field and taken into account when there's a match on that specific field. So, this would be what you were looking for.
Anyway, I would personally do boosting at query time instead of index time, so that you don't need to modify your mapping and you can change the weight without reindexing. You can for example use a query string and search on different fields giving them different weights like this:
{
"query_string" : {
"fields" : ["title^2", "content"],
"query" : "this AND that OR thus"
}
}
You need to take into account that the query string query gets parsed and allows you to use the lucene query syntax.
Furthermore, you can combine different queries together using the bool query. You can express a boost for a match on title with a should clause containing for example a term query and a specific boost for it like this:
"should" : [
{
"term" : { "title" : "your query", "boost" : 2.0 }
}
]
You can use whatever query you want as should clause. If you go for the term query you need to remember it's not analyzed.