Following the comments on Mongoose: how to define a combination of fields to be unique?
First let's get the array of data sorted by all values which supposed to be unique.
Assuming we're talking about strings (as in this question), we can combine them to create one long string that is supposed to be unique.
Being sorted, if there are duplicate values they'll show up right after the other, so let's look for results that repeat themselves:
var previousName;
Person.find().sort('firstName lastName').exec().each(function (person) {
var name = person.firstName + person.lastName;
if (name == previousName) {
console.log(name);
person.remove();
}
previousName = name;
})
Related
Hi I have two mongodb collections. The first one returns json data (array) and with the output of this, I want to return documents that match.
When I run Console.log (req.bidder.myBids) I get the following output:
[{"productId":"3798b537-9c7b-4395-9e41-fd0ba39aa984","price":3010},{"productId":"3798b537-9c7b-4395-9e41-fd0ba39aa984","price":3020},{"productId":"4c4bd71c-6664-4d56-b5d3-6428fe1bed19","price":1040},{"productId":"4c4bd71c-6664-4d56-b5d3-6428fe1bed19","price":1050},{"productId":"4c4bd71c-6664-4d56-b5d3-6428fe1bed19","price":1060},{"productId":"4c4bd71c-6664-4d56-b5d3-6428fe1bed19","price":1070},{"productId":"4c4bd71c-6664-4d56-b5d3-6428fe1bed19","price":1090},{"productId":"4c4bd71c-6664-4d56-b5d3-6428fe1bed19","price":1100}]
The productId has duplicates, I want to remove duplicates and then call a routine that finds all the products that match and output as json.
So far I have this code that only outputs one document, but cant figure out how to add the array of productId's and then fetch all corresponding products.
var agencyId = req.body.agencyId;
var productId = req.body.productId;
if (!validate.STRING(agencyId)) {
res.apiError(messages.server.invalid_request);
} else {
dbProduct.find({productId:{$in:['3798b537-9c7b-4395-9e41-fd0ba39aa984','4c4bd71c-6664-4d56-b5d3-6428fe1bed19']}
}).then(dbRes => {
console.log(dbRes);
Updated code and works with hard-wired productId and updated above code. Looking at how to get the array data and transpose replacing the hard-wired productId's
The $in operator is what you want. See the docs here: https://docs.mongodb.com/manual/reference/operator/query/in/
For example I have a thousands of documents with same structure, for example:
{
"key_1":"value_1",
"key_2":"value_2",
"key_3":"value_3",
...
...
}
And I need to get, let's say key_1, key_3 and key_23 from some set of documents with known IDs, for example, I need to process only 5 documents while my DB contains several thousands. Each time I have a different set of keys and document IDs. Is it possible to get that information for a one request?
You can use a list function (see: this, this, and this).
Since you know the ids, you can then query _all_docs with the list function:
POST /{db}/_design/{ddoc}/_list/{func}/_all_docs?include_docs=true&columns=["key_1","key_2","key_3"]
Accept: application/json
Content-Length: {whatever}
{
"keys": [
"docid002",
"docid005"
]
}
The list function needs to look at documents, and send the appropriate JSON for each one. Not tested:
(function (head, req) {
send('{"total_rows":' + head.total_rows + ',"offset":' + head.offset + ',"rows":[');
var columns = JSON.parse(req.query.columns);
var delim = '';
var row;
while (row = getRow()) {
var doc = {};
for (var k in columns) {
doc[k] = row.doc[k];
}
row.doc = doc;
send(delim + toJSON(row));
delim = ',';
}
send(']}');
})
Whether this is a good idea, I'm not sure. If your documents are big, and bandwidth savings important, it might.
Yes, that’s possible. Your question can be broken up into two distinct problems:
Getting only a part of the document (in your example: key_1, key_3 and key_23). This can be done using a view. A view is saved into a design document. See the wiki for more info on how to create views.
Retrieving only certain documents, which are defined by their ID. When querying views, you cannot only specify a single ID (or rather key), but also an array of keys, which is what you would need here. Again, see the section on querying views in the wiki for explanations and examples.
Even though you only need a subset of values from a document, you may find that the system as a whole performs better if you just ask for the entire document then select the values you need from that result.
To only get the specific key value pairs you need to create a view that has view entries with a multipart key consisting of the doc id and doc item name, with value of the corresponding doc item.
So your map function would look something like:
function(doc){
for(var i = 1; i < doc.keysInDoc; i++){
var k = "key_"+i;
emit([doc._id, k], doc.[k]);
}
}
You can then use multi key lookup with each key being of the form ["docid12345", "key_1"], ["docid56789", "key_23"], etc.
So a query like:
http://host:5984/db/_design/design/_view/view?&keys=[["docid002","key_8"],["docid005","key_7"]]
will return
{"total_rows":84,"offset":67,"rows":[
{"id":"docid002","key":["docid002","key_8"],"value":"value d2_k8"},
{"id":"docid005","key":["docid005","key_12"],"value":"value d5_k12"}
]}
I have documents like,
{_id:1,
name:"john"
}
{_id:2,
name:"john boss"
}
{_id:3,
name:"jim"
}
I have to search the data where ever john is stored in documents. Suppose, if i search "john" the documents should get _id:1 & _id:2 related data. Please guide me to get the result.
I appreciate if any one provide the solutions.
I suggest a CouchDB view to show you all "words" from the "name" field.
function(doc) {
// map function: _design/example/_view/names
if(!doc.name) // Optionally do more testing for doc type, etc. here.
return
// Emit one row per word in the name field (first name, last name, etc.).
var words = doc.name.split(/\s+/)
for(var i = 0; i < words.length; i++)
emit(words[i].toLowerCase(), doc._id)
}
Now if you query /db/_design/example/_view/names?key="john", you will get two rows: one for doc id 1, and another for id 2. I also added a conversion to lower case, so searching for "john" will match people named "John".
Duplicates are possible: the same doc ID listed multiple times, e.g. for {"name":"John John"}; however you are guaranteed that all duplicate rows will be adjacent.
You can also add ?include_docs=true to your request to get the full document for each row.
I have a list which contains more than 75 thousand object. To search item from list currently I am using following code.
from nd in this.m_ListNodes
where
nd.Label == SearchValue.ToString()
select
nd;
Is this code is efficient?
How often do you need to search the same list? If you're only searching once, you might as well do a straight linear search - although you can make your current code slightly more efficient by calling SearchValue.ToString() once before the query.
If you're going to perform this search on the same list multiple times, you should either build a Lookup or a Dictionary:
var lookup = m_ListNodes.ToLookup(nd => nd.Label);
or
var dictionary = m_ListNodes.ToDictionary(nd => nd.Label);
Use a dictionary if there's exactly one entry per label; use a lookup if there may be multiple matches.
To use these, for a lookup:
var results = lookup[SearchValue.ToString()];
// results will now contain all the matching results
or for a dictionary:
WhateverType result;
if (dictionary.TryGetValue(SearchValue.ToString(), out result))
{
// Result found, stored in the result variable
}
else
{
// No such item
}
No. It would be better if you used a Dictionary or a HashSet with the label as the key. In your case a Dictionary is the better choice:
var dictionary = new Dictionary<string, IList<Item>>();
// somehow fill dictionary
IList<Item> result;
if(!dictionary.TryGetValue(SearchValue.ToString(), out result)
{
// if you need an empty list
// instead of null, if the SearchValue isn't in the dictionary
result = new List<Item>();
}
// result contains all items that have the key SearchValue
I have a lookup field in sharepoint which just references another list. I wonder how do I programatically enumerate all possible values for this field?
For example, my lookup field "Actual City" refers list "Cities" and column "Title", I have 3 cities there. In code I would like to get list of all possible values for field "Actual City", smth like (metacode, sorry):
SPFieldLookup f = myList["Actual City"];
Collection availableValues = f.GetAllPossibleValues();
//this should return collection with all cities a user might select for the field
I wrote some code to handle this for my project just the other day. Perhaps it will help.
public static List<SPFieldLookupValue> GetLookupFieldValues(SPList list, string fieldName)
{
var results = new List<SPFieldLookupValue>();
var field = list.Fields.GetField(fieldName);
if (field.Type != SPFieldType.Lookup) throw new SPException(String.Format("The field {0} is not a lookup field.", fieldName));
var lookupField = field as SPFieldLookup;
var lookupList = list.ParentWeb.Lists[Guid.Parse(lookupField.LookupList)];
var query = new SPQuery();
query.Query = String.Format("<OrderBy><FieldRef Name='{0}'/></OrderBy>", lookupField.LookupField);
foreach (SPListItem item in lookupList.GetItems(query))
{
results.Add(new SPFieldLookupValue(item.ID, item[lookupField.LookupField].ToString()));
}
return results;
}
Then to use it, your code would look something like this:
var list = SPContext.Current.Web.Lists["My List"];
var results = GetLookupFieldValues(list, "Actual City");
foreach (SPFieldLookupValue result in results)
{
var value = result.LookupValue;
var id = result.LookupId;
}
I think there is no explicit method returning what you want. But the SPFieldLookup class stores all the info you need to request this information manually: LookupField and LookupList
So you could retrieve the information by getting it form the list you lookup field uses. To make it reusable you could implement it as a Extension Method. So the next time you could really call f.GetAllPossibleValues();.
As I understand you want to query all values that are in use?
If so, you would have to query items where Actual City is not null, query would look something like:
<Where><IsNotNull><FieldRef Name='Actual City'/></IsNotNull></Where>
Then, for each queried item you would
List<SPFieldLookupValue> result = new List<SPFieldLookupValue>(returnedItemCount * 5);
foreach (SPListItem item in queriedItems) {
object lookup = item["Actual City"];
SPFieldLookupValueCollection lookupValues = new SPFIeldLookupValueCollection(
(lookup != null) ? lookup.ToString() : ""
);
foreach (SPFieldLookupValue lookupValue in lookupValues) {
if (!result.Contains(lookupValue)) {
result.Add(lookupValue);
}
}
}
Or you could use HashTable where LookupId would be string and LookupValue would be int id and then check if HashTable.ContainsKey(lookupId)... must be faster to find an integer in hashtable rather than string in list, but the resource intensive part is to probably query all items where that field contains some value and then loop...
If you want to enumerate all possible values, that means you basically want to get all the Title field values from all the items in the Cities list. I don't think there is a method like GetAllPossibleValues() in SharePoint, but you can either just list all the items in Cities and get their titles, if there's just a few, or use a CAML query if there's plenty.