I'm experiencing a strange problem in Mongoose related to find queries. When I run the query below, I get a variable number of results. I will get a consistent 210 results when querying in Mongo, but usually get between 198-210 results when doing the same thing through Mongoose. I've tried the query with and without indexes set.
Any suggestions on what might be causing this would be greatly appreciated.
Customer Model:
subscriptions: [
{
renewal: {
type: Boolean,
default: false
}
}
]
Query
Customer.find({ "subscriptions.renewal": true }, {}, { timeout: false })
The problem ultimately cleared up when I removed the Customer collection indexes from Mongo (not just the definitions in the schema). Anyone experiencing this issue might want to give that a try.
Related
the problem I am facing is as follows:
Search value: 'cooking'
JSON object::
data: {
skills: {
items: [ { name: 'cooking' }, ... ]
}
}
Expected result: Should find all the "skill items" that contain 'cooking' inside their name, using TypeORM and Nest.js.
The current code does not support search on the backend, and I should implement this. I want to use TypeORM features, rather than handling it with JavaScript.
Current code: (returns data based on the userId)
const allItems = this.dataRepository.find({ where: [{ user: { id: userId } }] })
I investigated the PostgreSQL documentation regarding the PostgreSQL functions and even though I understand how to create a raw SQL query, I am struggling to convert this to the TypeORM equivalent.
Note: I researched many StackOverflow issues before creating this question, but do inform me If I missed the right one. I will be glad to investigate.
Can you help me figure out the way to query this with TypeORM?
UPDATE
Let's consider the simple raw query:
SELECT *
FROM table1 t
WHERE t.data->'skills' #> '{"items":[{ "name": "cooking"}]}';
This query will provide the result for any item within the items array that will match exact name - in this case, "cooking".
That's totally fine, and it can be executed as a raw request but it is certainly not easy to maintain in the future, nor to use pattern matching and wildcards (I couldn't find a solution to do that, If you know how to do it please share!). But, this solution is good enough when you have to work on the exact matches. I'll keep this question updated with the new findings.
use Like in Where clause:
servicePoint = await this.servicePointAddressRepository.find({
where: [{ ...isActive, name: Like("%"+key+"%"), serviceExecutive:{id: userId} },
{ ...isActive, servicePointId: Like("%"+key+"%")},
{ ...isActive, branchCode: Like("%"+key+"%")},
],
skip: (page - 1) * limit,
take: limit,
order: { updatedAt: "DESC" },
relations:["serviceExecutive","address"]
});
This may help you! I'm matching with key here.
I am trying to update in a collection and there are 1400+ offices are there and after checking and running the query I am updating in a collection document and update in the subcollection with few details after querying but sometimes i am getting this error
10 ABORTED: Too much contention on these documents. Please try again.
and i am simply using batch for writing in the doc here is my code for updation in the collection.
batch.set(
rootCollections.x.doc(getISO8601Date())
.collection(subCollection.y)
.doc(change.after.get('Id')),
{
officeId: change.after.get('Id'),
office: change.after.get('office'),
status: change.after.get('status'),
creationTimestamp:
change.after.get('creationTimestamp') ||
change.after.get('createTimestamp') ||
change.after.get('timestamp'),
activeUsers: [...new Set(phoneNumbers)].length,
confirmedUers: activityCheckinSubscription.docs.length,
uniqueActivities: [...new Set(activities)].length,
payments: 0,
contact: [
`${change.after.data().creator.phoneNumber},${
change.after.data().attachment['First Contact'].value
}`,
],
},
{ merge: true },
);
batch.set(
rootCollections.x.doc(getISO8601Date()),
{
Added: admin.firestore.FieldValue.increment(1),
},
{ merge: true },
);
PromiseArray.push(batch.commit());
await Promise.all(PromiseArray);
It seems that you are facing the same issue from this similar case here, where there were thousands of records in the database being updated. As clarified there, you have limitation of how much writes you can perform in a document in one second - more details here - and even though Firestore sometimes might hang on with the faster writes, it will fail at some point.
As this is hard coded and a limit that it's imposed by Firestore, what you can try is the solution explained in this similar case here, that it's either change to Realtime Database, where the limit is not the number of writes, but the size of the data or in case of the usage of a counter or some other data aggregation in Firestore, to use a distributed counter solution, that you can get more details here.
To summarize, there isn't much you can do unless of workaround it with this solution, as this is a limitation documented of Firestore.
We are facing a timeout issue with our mongo updates. Our collection currently contains around 300 thousand documents. When we try to update a record via the UI, the server times out and the UI is stuck in limbo.
Lead.updateOne({
_id: body.CandidateID
}, {
$set: {
ingestionStatus: 'SUBMITTED',
program: body.program,
participant: body.participant,
promotion: body.promotion,
addressMeta: body.addressMeta,
CreatedByID: body.CreatedByID,
entryPerson: body.entryPerson,
lastEnteredOn: body.lastEnteredOn,
zipcode: body.zipcode,
state: body.state,
readableAddress: body.readableAddress,
promotionId: body.promotionId,
programId: body.programId,
phone1: body.phone1,
personId: body.personId,
lastName: body.lastName,
hasSignature: body.hasSignature,
firstName: body.firstName,
city: body.city,
email: body.email,
addressVerified: body.addressVerified,
address: body.address,
accountId: body.accountId
}
This is how we update a single record. We are using mlab and Heroku in our stack. Looking for advice on how to speed this up considerably.
Thank you.
If your indexes are fine then you could try rebuilding indexes on this collection.
collection indexes from the mango command line:
For example, rebuild the lead collection indexes from the mongo command line:
db.lead.reIndex();
Reference:
https://docs.mongodb.com/v3.2/tutorial/manage-indexes/
https://docs.mongodb.com/manual/reference/command/repairDatabase/
if you are not using this then try this one
Index builds can block write operations on your database, so you don’t want to build indexes in the foreground on large tables during peak usage. You can use the background creation of indexes by specifying background: true when creating.
db.collection.createIndex({ a:1 }, { background: true })
This will ultimately take longer to complete, but it will not block operations and will have less of an impact on performance.
1) Shard Lead collection by id as shard key.
2) Check if the memory taken by mongodb due to index is less than the memory of the mongoDb server.
Have you tried what this answer suggests? Namely, updating with no write-concern?
I have a strange problem. I want to update a document in my MongoDB with the mongoose.findByIdAndUpdate method, but it seems not to be working. The code is:
Device.findByIdAndUpdate(
req.params.id,
{ $set: { power: power } },
{ new: true },
(err, device) => { ... }
I get no error, but the device returned in the callback does not have the updated value. At first I thought maybe it was some sort of problem with the { new: true } option that tells mongoose to return the updated document, but then I checked the database, and the value there also has not been updated.
I also tried replacing findByIdAndUpdate with update function, but the results are the same - the db is not getting updated.
If it changes anything I use mongoose.update() function in other places and it works fine. I also tried the 'classical' way of updating the value here - meaning I used findOne function and then changed returned document's power field value and saved it and it also worked fine.
I will be really gratefull for any advice on fixing this. Thank you!
So I have an object using a dictionary to store products that a user has added to the cart in a shopping cart application. I am taking is object and attempting to insert into mongoDB with zero luck.
The piece of data I am attempting to insert looks like this:
products: '{"rJUg4uiGl":{"productPrice":"78.34","count":2},"BJ_7VOiGg":{"productPrice":"3","count":2}}' }
My process of attempting to insert it into mongoDB looks like this:
db.orders.insert("products":{"rJUg4uiGl":{"productPrice":"78.34","count":2},"BJ_7VOiGg":{"productPrice":"3","count":2}});
Currently with this approach I get the following error:
2016-12-15T18:11:43.862-0500 E QUERY [thread1] SyntaxError: missing ) after argument list #(shell):1:27
Which is implying there is some sort of a formatting issue with inserting it. I have moved quotation marks and parenthesis around plenty, simply to either get the above error, or a ... response from mongoDB implying that it is waiting for me to do something more to fix what exactly is causing an error.
Any chance anyone could help give some guidance in the best way to store this object in mongoDB?
My true question feels that it should have been in regards to the mongoose schema that would be used in order to store this data format. I hoped that getting how to initially insert it into mongodb was going to be enough but the way the data is being saved has me a bit confused. I know this is a bit of an awful question but could I get any assistance with setting up my schema for this as well?
"products" : {
"rJUg4uiGl" : {
"productPrice" : "78.34",
"count" : 2
},
"BJ_7VOiGg" : {
"productPrice" : "3",
"count" : 2
}
}
This is what the data looks like when it is stored in mongo. I think what is confusing me on how to set up is the "rJUg4uiGl" portion of the data. I am un-sure of how exactly that is suppose to look in mongoose schema. Here are a few of my rather poor attempts:
products: {
productId: {
productPrice: Number,
count: Number
}
}
Above simply doesn't store anything in the database
products: {
productId: [{
productPrice: Number,
count: Number
}]
}
Above gives:
"products" : {
"productId" : [ ]
}
Again, I know that this is quite specific but any help at all would be extremely appreciated.
Need to wrap your insert data in {}
db.orders.insert({"products":{"rJUg4uiGl":{"productPrice":"78.34","count":2},"BJ_7VOiGg":{"productPrice":"3","count":2}}});