MongoError: The dollar ($) prefixed field '$push' in '$push' is not valid for storage - node.js

I am trying to upsert a dataset to a Mongo collection.
The intended document may or may not exist.
If it does exist, it will have at least one item in an embedded document (zips), and should append to that document rather than overwrite it.
If it does not exist, it should insert the new document to the collection.
When I run the below code, I am getting an error: MongoError: The dollar ($) prefixed field '$push' in '$push' is not valid for storage.
I put this together based on the docs: https://docs.mongodb.org/getting-started/node/update/#update-multiple-documents
Versions:
MongoDB (windows) = 3.2.0;
mongodb (npm package) = 2.1.4
var query = {
county: aCountyName,
state: aStateName
}
var params = {
'$set': {
county: 'Boone',
state: 'MO',
'$push': {
zips: {
'$each': [ '65203' ]
}
}
}
}
(could also be)
var params = {
'$set': {
county: 'Pierce',
state: 'WA',
'$push': {
zips: {
'$each': [ '98499', '98499' ]
}
}
}
}
db.collection(collectionName).updateMany(query, params, {'upsert': true},
function(err, results) {
callback();
}
);

I don't think $push is valid within a $set. Instead try adding it as another parameter, e.g.:
var params = {
'$set': {
county: 'Pierce',
state: 'WA'
},
'$push': {
zips: {
'$each': ['98499',
'98499']
}
}
}

The reason is because you didn't close the } so MongoDB think $push is a field's name and as mentioned in the documentation:
Field names cannot contain dots (i.e. .) or null characters, and they must not start with a dollar sign (i.e. $).
var query = {
county: aCountyName,
state: aStateName
};
var params = {};
params['$set'] = { county: 'Boone', state: 'MO' };
params['$push'] = { zips: { '$each': [ '65203' ] } };
Then:
db.collection(collectionName).updateMany(query, params, {'upsert': true},
function(err, results) {
callback();
}
);

Related

Update nested array objects in MongoDB

I have to deal with objects of the following type in a NodeJS app (using mongodb driver):
data_test = {
"id": "105-20090412",
"date": new Date('2020-09-04T14:00:00.000Z'),
"station": {
"name": "AQ105",
"loc": {
"type": "Point",
"coordinates": [14.324498, 40.821930]
},
"properties": {}
},
"samples": [{
"t": new Date('2020-09-04T14:14:00.000Z'),
"data": {
//"temp_celsius": 31.81,
//"humRelPercent": 39,
"press_mBar": 1021.12,
"PM10": 200
}
}]
}
I receive every 2 minutes data as above.
I want to:
If the data received has an id not yet present on MongoDB do an insert
If the data received has a sample object with a Date (t property) yet present then add properties to this one (for example readings of different sensors)
If the data received has a sample object with a Date (t property) not yet present in samples array, then add this new one
I would like to do what described above with the minor count possible of round-trips to the MongoDB server.
I hope to have been clear enough.
Any suggestion?
Thanks in advance.
Here's my suggestion, this is not the correct answer. You will need to fiddle with the query portion. The query below should work for 1 & 3, for 2 you will have to play around.
db.collection.updateOne(
{ "id" : "105-20090412", "samples.t": <Date> },
{ $push: { "samples" : <sample> } },
{ $setOnInsert: { station: <station> } },
{ upsert: true }
);
References:
https://docs.mongodb.com/manual/reference/method/db.collection.updateOne/
https://docs.mongodb.com/manual/reference/operator/update/setOnInsert/#up._S_setOnInsert
https://docs.mongodb.com/manual/reference/operator/update/push/
I finally came to the following solution, perhaps not the most efficient one:
try {
const db = client.db(dbName);
const collection = db.collection(collectionName);
// retrive id, station, date and samplesToAdd as separate objects
let {
id,
...dataToInsert
} = data
//id = new ObjectID(id)
const queryBy_id = {
_id: id
}
// first check if doc exists
let res_query = await collection.findOne(queryBy_id)
// if doc does not exists then insert a new one
if (!res_query) {
res_insert = await collection.insertOne({
_id: id,
...dataToInsert
})
return res_insert;
} else {
// retrive samples from initial query
let current_samples = res_query.samples
// check if sample in dataToInsert yet exists
// use getTime to correctly compare dates
let idx = current_samples.findIndex(x => x.t.getTime() == dataToInsert.samples[0].t.getTime())
if (idx >= 0) {
// find index of sample to update
let current_t = current_samples[idx].t
// merge data yet stored with new one
current_samples.data = {
...current_samples[idx].data,
...dataToInsert.samples[0].data
}
let resUpdateSample = await collection.updateOne({
_id: id,
'samples.t': current_t
}, {
$set: {
'samples.$.data': current_samples.data
}
})
return resUpdateSample
} else {
// add data to samples array
let resAddToSamples = await collection.updateOne({
_id: id
}, {
$push: {
samples: dataToInsert.samples[0]
}
})
return resAddToSamples
}
}
} catch (err) {
logger.error(err);
}
How can I improve it?
Thanks.

Mongoose: cannot infer query fields to set, path 'participants' is matched twice

I'm using mongoose with Node.js to create a document of chat with participants as one of the fields if the chat doesn't exist.
If it does exist then simply increment the status to 1 or any number.
My current Solution:
try {
let query = { participants: { $all: [CURRENT_USER_ID, TARGETED_ID] } };
let update = { $inc: { status: 1 }};
let options = { upsert: true, new: true };
let chat = await Chat.findOneAndUpdate(
query,
update,
options
).exec();
console.log(chat);
} catch (err) {
console.log(err.message);
}
I will receive an error
"cannot infer query fields to set, path 'participants' is matched
twice"
I even use this solution and it doesn't work, it created an empty list of participants instead.
let query = {
participants: {
$all: [
{ $elemMatch: { $eq: CURRENT_USER_ID } },
{ $elemMatch: { $eq: TARGETED_ID } }
]
}
};
Any help would be really helpful. Thanks

Select fields in mongoose query where field value not equal to something

I am basically trying to update a document and then select the fields from the result where the field value is not equal to something. Assume jwt_id to be b816cf00e9f649fbaf613e2ca2d523b5.
Query
const removeDevices = await Identity.findOneAndUpdate(
{
userID: user_id
},
{
$pull: {
activeTokens: {
jti: {
$ne: jwt_id
}
}
}
},
).select(["-_id", "activeTokens.jti"]);
Now, running this query gives the following output:
{ activeTokens:
[ { jti: '5d872359af2c47e5970c1fae531adf0e' },
{ jti: 'd3ac84f520614067b1caad504d7ab27f' },
{ jti: '25c6fa96705c4eec96e1427678c3ff50' },
{ jti: 'b816cf00e9f649fbaf613e2ca2d523b5' }
]
}
How can I get all the jti fields except { jti: b816cf00e9f649fbaf613e2ca2d523b5 } from the select command?
Desired Output
{ activeTokens:
[ { jti: '5d872359af2c47e5970c1fae531adf0e' },
{ jti: 'd3ac84f520614067b1caad504d7ab27f' },
{ jti: '25c6fa96705c4eec96e1427678c3ff50' },
]
}
It's hard to say for certain without testing, but i don't think mongoose returns the document after it was modified, but rather simply returns the matching document. So, i think in the case of findOneAndUpdate, you would have to have your query match to do the pull, and then manually filter the array again in application code to get the desired output.
This might work:
const removeDevices = await Identity.findOneAndUpdate(
{
userID: user_id
},
{
$pull: {
'activeTokens.jti': { $ne: jwt_id }
}
},
).select(["-_id", "activeTokens.jti"]).then(identity=>identity.activeTokens.filter(token=>token.jti!==jwt_id));
If the above doesn't work for some reason, then i would try something more simpler
simple:
const removeDevices = await Identity.findOne({userID: user_id}).select(["-_id", "activeTokens"]).then(identity=>{
const removedTokens = []
identity.activeTokens = identity.activeTokens.filter(token=>{
if(token.jti===jwt_id) {
return true;
}
removedTokens.push(token);
})
identity.save(err=>{
console.log('doc saved')
});
return removedTokens;
});
or (atomic):
const removeDevices = await Identity.findOne({userID: user_id}).select('activeTokens','jti _id').then(identity=>{
const removedTokens = identity.activeTokens.filter(token=>token.jti!==jwt_id);
const result = await Identity.update({userId:user_id},{$pull:{'activeTokens._id': { $in: removedTokens.map(t=>t._id) } }});
console.log(result.nModified);
return removedTokens;
});

mongoDB find, update and pull in One Query

I want to do all the find the data from the collection and then want to update some field as well as depending on want to empty the array.
const addCityFilter = (req, res) => {
if (req.body.aCities === "") {
res.status(409).jsonp({ message: adminMessages.err_fill_val_properly });
return false;
} else {
var Cities = req.body.aCities.split(","); // It will make array of Cities
const filterType = { "geoGraphicalFilter.filterType": "cities", "geoGraphicalFilter.countries": [], "geoGraphicalFilter.aCoordinates": [] };
/** While using $addToset it ensure that to not add Duplicate Value
* $each will add all values in array
*/
huntingModel
.update(
{
_id: req.body.id,
},
{
$addToSet: {
"geoGraphicalFilter.cities": { $each: Cities }
}
},
{$set:{filterType}},
).then(function(data) {
res.status(200).jsonp({
message: adminMessages.succ_cityFilter_added
});
});
}
};
Collection
geoGraphicalFilter: {
filterType: {
type:String,
enum: ["countries", "cities", "polygons"],
default: "countries"
},
countries: { type: Array },
cities: { type: Array },
aCoordinates: [
{
polygons: { type: Array }
}
]
}
But as result, the only city array is getting an update. No changes in filterType.
You appear to be passing the $set of filterType as the options argument, not the update argument.
huntingModel
.update(
{
_id: req.body.id,
},
{
$addToSet: {
"geoGraphicalFilter.cities": { $each: Cities }
},
$set: {
filterType
}
}
).then(function(data) {
res.status(200).jsonp({
message: adminMessages.succ_cityFilter_added
});
});

Node Mongo - find multiple parameters

I'm trying to find in a collection if there is already a session number, to avoid duplications. dadosORS.email and dadosORS.sessao (which is 3)come from a form. So when I do this:
mongoClient.collection('registosORS', function(err,collection){
collection.find({email:{$eq:dadosORS.email}},{sessao:{$eq:dadosORS.sessao}}).toArray(function(err,result){
try{
console.log(result);
}catch (err){
console.log(err);
}
if(result){
// callback(false)
return
} else {
I get result = undefined. If I change the query to
collection.find({email:dadosORS.email},{sessao:dadosORS.sessao}).toArray(function(err,result){
it lists my every occurence of the email:
[ { _id: 5a37b4c3da53ff1e825f94b4, sessao: '1' },
{ _id: 5a37b4e6da53ff1e825f94b6, sessao: '1' },
{ _id: 5a37b57ce500ca1ea5522e22, sessao: '2' } ]
So, how can I see if the dadosORS.sessao for that dadosORS.email already exists?
Just do an and query:
collection.find( { email : dadosORS.email, sessao : dadosORS.sessao } )
or can be expressed as
collection.find( { $and: [ { email : dadosORS.email }, { sessao : dadosORS.sessao } ] } )

Resources