I have a collection that looks like:
[
{
'job': builder,
'name': bob
},
{
'job': doctor,
'name': bob
},
{
'job': builder,
'name': james
},
{
'job': lawyer,
'name': james
},
...
]
I also have an array where job is always the same, like:
[
{
'job': builder,
'name': jack
},
{
'job': builder,
'name': john
},
...
]
I want to replace all objects in my collection where job is builder. To do this I am currently using two separate queries.
mycollection.remove({'job': builder})
mycollection.insert(new_job_array);
Is there a way to combine this into one query?
You can use the update command.
mycollection.update({'job':'builder'}, {new document}, {multi:true})
More information here: MongoDB update
The end result is that, no, there is not one operation that does this. 2 steps are required and it should use really use a callback.
mycollection.remove({'job': builder}, function(err){
if ( err ) handle();
else mycollection.insert(new_job_array);
});
Related
I have this aggregation pipeline i wrote in NodeJS in my Atlas Trigger :
const pipeline = [
{$match: {"score": {$gt: 0}, "update": true}},
{$setWindowFields: {sortBy: {"score": -1}, output: {"rank": {$denseRank: {}}}}},
{$merge: {into: "ranking"}}
];
await ranking_col.aggregate(pipeline);
I have written this pipeline in python first for testing and it's working just fine :
self.db.ranking.aggregate([
{
"$match": {
"score": {"$gt": 0},
"update": True
}
},
{
'$setWindowFields': {
'sortBy': {'score': -1},
'output': {
'rank': {
'$denseRank': {
}
}
}
}
},
{
"$merge": {
"into": "ranking"
}
}
])
I have no errors from the Trigger logs but it seems that the pipeline is simply not executed as it should modify the ranking as it's done in python.
Can you please tell me what am i doing wrong here ?
EDIT : The database scheme (as simple as the query is)
See below one document of ranking_col :
{
"_id": "7dqe1kcA7R1YGjdwHsAkV83",
"score": 294,
"update": false,
"rank": 0,
}
The aggregation is simply here to calculate the rank attribute according to the score.
Ok so the issue relies in the Mongo driver the function uses on atlas and your understanding of it.
aggregate returns an AggregateCursor, which means until you trigger it no command is actually getting executed, this means your trigger is actually running fine, but because no one is using the cursor it just exits the function without doing anyways.
A super simple solution would be to just add .toArray(), this will convert the cursor to an array of documents. essentially triggering the functionality:
await ranking_col.aggregate(pipeline).toArray();
Imaging I have an array of objects, available before the aggregate query:
const groupBy = [
{
realm: 1,
latest_timestamp: 1318874398, //Date.now() values, usually different to each other
item_id: 1234, //always the same
},
{
realm: 2,
latest_timestamp: 1312467986, //actually it's $max timestamp field from the collection
item_id: 1234,
},
{
realm: ..., //there are many of them
latest_timestamp: ...,
item_id: 1234,
},
{
realm: 10,
latest_timestamp: 1318874398, //but sometimes then can be the same
item_id: 1234,
},
]
And collection (example set available on MongoPlayground) with the following schema:
{
realm: Number,
timestamp: Number,
item_id: Number,
field: Number, //any other useless fields in this case
}
My problem is, how to $group the values from the collection via the aggregation framework by using the already available set of data (from groupBy) ?
What have been tried already.
Okay, let skip crap ideas, like:
for (const element of groupBy) {
//array of `find` queries
}
My current working aggregation query is something like that:
//first stage
{
$match: {
"item": 1234
"realm" [1,2,3,4...,10]
}
},
{
$group: {
_id: {
realm: '$realm',
},
latest_timestamp: {
$max: '$timestamp',
},
data: {
$push: '$$ROOT',
},
},
},
{
$unwind: '$data',
},
{
$addFields: {
'data.latest_timestamp': {
$cond: {
if: {
$eq: ['$data.timestamp', '$latest_timestamp'],
},
then: '$latest_timestamp',
else: '$$REMOVE',
},
},
},
},
{
$replaceRoot: {
newRoot: '$data',
},
},
//At last, after this stages I can do useful job
but I found it a bit obsolete, and I already heard that using [.mapReduce][1] could solve my problem a bit faster, than this query. (But official docs doesn't sound promising about it) Does it true?
As for now, I am using 4 or 5 stages, before start working with useful (for me) documents.
Recent update:
I have checked the $facet stage and I found it curious for this certain case. Probably it will help me out.
For what it's worth:
After receiving documents after the necessary stages I am building a representative cluster chart, that you may also know as a heatmap
After that I was iterating each document (or array of objects) one-by-one to find their correct x and y coordinated in place which should be:
[
{
x: x (number, actual $price),
y: y (number, actual $realm),
value: price * quantity,
quantity: sum_of_quantity_on_price_level
}
]
As for now, it's old awful code with for...loop inside each other, but in the future, I will be using $facet => $bucket operators for that kind of job.
So, I have found an answer to my question in another, but relevant way.
I was thinking about using $facet operator and to be honest, it's still an option, but using it, as below is a bad practice.
//building $facet query before aggregation
const ObjectQuery = {}
for (const realm of realms) {
Object.assign(ObjectQuery, { `${realm.name}` : [ ... ] }
}
//mongoose query here
aggregation([{
$facet: ObjectQuery
},
...
])
So, I have chosen a $project stage and $switch operator to filter results, such as $groups do.
Also, using MapReduce could also solve this problem, but for some reason, the official Mongo docs recommends to avoid using it, and choose aggregation: $group and $merge operators instead.
EDIT 2: The following schema (provided by a colleague) works. I removed the quotation marks from the schema in the examples from Microsoft, but that still didn't work. I'm not sure what the issue is. I leave the question open in case someone else wants to provide an answer, but I've got it working.
const card = {
contentType: 'application/vnd.microsoft.card.adaptive',
content: {
$schema: 'http://adaptivecards.io/schemas/adaptive-card.json',
type: 'AdaptiveCard',
version: '1.0',
{
type: 'Input.Text',
placeholder: 'Name',
style: 'text',
maxLength: 50,
id: 'defaultInput'
},
actions: [
{
type: 'Action.Submit',
title: 'Siguiente',
data: {} // will be populated with form input values
}
]
}
};
I'm trying to make a form in my MS Bot using Adaptive Cards. I took the sample form from the MS site (https://blog.botframework.com/2019/07/02/using-adaptive-cards-with-the-microsoft-bot-framework/) but get the following error
The error seems to be thinking that my action type is Action.openUrl but I don't see that in my code, which is below. Any help much appreciated. Using Microsoft Bot Framework 3, Node 12.13.0.
function askPolicyNumber(session) {
const card = {
'$schema': 'https://adaptivecards.io/schemas/adaptive-card.json',
'type': 'AdaptiveCard',
'version': '1.1',
'body': [
{
'type': 'Input.Text',
'id': 'id_text'
},
{
'type': 'Input.Number',
'id': 'id_number'
}
],
'actions': [
{
'type': 'Action.messageBack',
'title': 'Submit',
'data': {
'prop1': true,
'prop2': []
}
}
]
};
const msg = new builder.Message(session).attachments([card]);
return session.send(msg);
}
EDIT:
It seems that no matter what I set the action to it keeps thinking it's an openUrl action. In fact, if I set it to openUrl and give it a url property, it works fine.
I looked at this page -- https://learn.microsoft.com/en-us/microsoftteams/platform/task-modules-and-cards/cards/cards-actions#adaptive-cards-actions -- and followed the instructions there for 'Adaptive Cards with messageBack action', but it didn't change anything
"actions": [
{
"type": "Action.Submit",
"title": "Click me for messageBack",
"data": {
"msteams": {
"type": "messageBack",
"displayText": "I clicked this button",
"text": "text to bots",
"value": "{\"bfKey\": \"bfVal\", \"conflictKey\": \"from value\"}"
}
}
}
]
}
There are a lot of problems with what you're doing. It is recommended that everyone use Bot Builder v4 instead of v3. The main problem that your colleague solved was that you were trying to use an Adaptive Card object as though it was an Attachment object.
The blog post you linked to explains that Adaptive Cards must follow the Adaptive Cards schema. There is no Action.messageBack in the Adaptive Cards schema. Please continue referring to the documentation for more information.
I'm rather new to elasticsearch, so i'm coming here in hope to find advices.
I have two indices in elastic from two different csv files.
The index_1 has this mapping:
{'settings': {
'number_of_shards' : 3
},
'mappings': {
'properties': {
'place': {'type': 'keyword' },
'address': {'type': 'keyword' },
}
}
}
The file is about 400 000 documents long.
The index_2 with a much smaller file(about 50 documents) has this mapping:
{'settings': {
"number_of_shards" : 1
},
'mappings': {
'properties': {
'place': {'type': 'text' },
'address': {'type': 'keyword' },
}
}
}
The field "place" in index_2 is all of the unique values from the field "place" in index_1.
In both indices the "address" fields are postcodes of datatype keyword with a structure: 0000AZ.
Based on the "place" field keyword in index_1 I want to assign the term of field "address" from index_2.
I have tried using the pandas library but the index_1 file is too large. I have also to tried creating modules based off pandas and elasticsearch, quite unsuccessfully. Although I believe this is a promising direction. A good solution would be to stay into the elasticsearch library as much as possible as these indices will be later be used for further analysis.
If i understand correctly it sounds like you want to use updateByQuery.
the request body should look a little like this:
{
'query': {'term': {'place': "placeToMatch"}},
'script': 'ctx._source.address = "updatedZipCode"'
}
This will update the address field of all documents with the matched place.
EDIT:
So what we want to do is use updateByQuery while iterating over all the documents in index2.
First step: get all the documents from index2, will just do this using the basic search feature
{
"index": 'index2',
"size": 100 // get all documents, once size is over 10,000 you'll have to padginate.
"body": {"query": {"match_all": {}}}
}
Now we iterate over all the results and use updateByQuery for each of the results:
// sudo
doc = response[i]
// update by query request.
{
index: 'index1',
body: {
'query': {'term': {'address': doc._source.address}},
'script': 'ctx._source.place = "`${doc._source.place}`"'
}
}
I have data in table in this format
emp_id,emp_name,title,supervisor_id,supervisor_name
11,Anant,Business Unit Executive,8,abc
15,Raina,Analysis Manager Senior,11,Anant
16,Kumar,Conversion Manager,11,Anant
18,amit,Analyst Specialist,11,Anant
25,anil,senior engineer,18,amit
35,Pang Pang,senior engineer,25,anil
38,Xiang Xiang,UE engineer,25,anil
I will enter supervisor_id and it will return all employee under that then after continue this until we achieve lower level, i want to do this in node and sql server with recursive function.
I want this data to be in hierarchical way like this .
var ds ={ 'emp_id':11,
'name': 'Anant',
'title': 'Business Unit Executive',
'children': [
{ 'name': 'Raina','emp_id':15, 'title': 'Analysis Manager Senior' },
{ 'name': 'Kumar','emp_id':16, 'title': 'Conversion Manager' },
{ 'name': 'amit', 'emp_id':18, 'title': 'Analyst Specialist',
'children': [
{ 'name': 'anil','emp_id':25, 'title': 'senior engineer' ,
'children': [
{ 'name': 'Pang Pang','emp_id':35, 'title': 'engineer' },
{ 'name': 'Xiang Xiang', 'emp_id':38,'title': 'UE engineer' }
]
}
]
}
]
};
I'm not familiar with the which library you are using to request form server so i will sudo code those portions
async getEmployeesBySupervisorId(supervidor_id){
const employees = await <get-employees-query> // you may also need to map the results to your {emp_id, name, title} depending on your query library default to [] if no employees are found
return Promise.all(...employees.map(employee=>{
employee.children = await getEmployeesBySupervisorId(employee.emp_id)
}))
}
That will get you an array of employees, with children until no more employees are found,
While this will work it fires many queries, it may be better for you to leverage sql and your ORM to make this more efficient in the future.