I receive 3 post calls from client, let say in a second, and with nodejs-mongodb immediately(without any pause, sleep, etc) I try to insert the data that is posted in database using updateOne. All data is new, so in every call, insert would happen.
Here is the code (js):
const myCollection = mydb.collection("mydata")
myCollection.updateOne({name:req.data.name},{$set:{name:req.data.name, data:req.data.data}}, {upsert:true}, function(err, result) {console.log("UPDATEONE err: "+err)})
When I call just 1 time this updateOne, it works; 2 times successively, it works. But if I call 2+ times in succession, only the first two ones correctly inserted into database, and the rest, no.
The error that I get after updateOne is, MongoWriteConcernError: No write concern mode named 'majority;' found in replica set configuration. However, I always get this error, also even when the insertion is done correctly. So I don't think this is related to my problem.
Probably you will suggest to me to use updateMany, bulkWrite, etc. and you will be right, but I want to know the reason why after 2+ the insertion is not done.
Have in mind .updateOne() returns a Promise so it should be handled properly in order to avoid concurrency issues. More info about it here.
The error MongoWriteConcernError might be related to the connection string you are using. Check if there is any &w=majority and remove it as recommended here.
Related
I use jooq to generate objects against a local database, but when running "for real" later in production the actual databases will have different names. To remedy this I use the <outputSchemaToDefault>true</outputSchemaToDefault> config option (maven).
At the same time, we have multiple databases (schemas), and are using a connection pool to the server like "jdbc:mysql://localhost:3306/" (without specifying a database here).
How do I tell jooq which database to use when running queries?
I have tried all config I can think of:
new Settings()
.withRenderSchema(true) // true/false seems to make no difference.
.withRenderCatalog(true) // true/false seems to make no difference.
.withRenderMapping(new RenderMapping()
.withDefaultSchema("my_database") // Seems to have no effect.
// The above 3 configs always give me an error saying "no database selected".
// Adding this gives me 'my_database.my_table' does not exist - while it actually does.
.withSchemata(new MappedSchema()
.withInputExpression(Pattern.compile(".*"))
.withOutput("my_database")
));
I have also tried using a database/schema name, as in not configuring outputSchemaToDefault. But then, adding the MappedSchema code above, but that gives me errors with "'my_databasemy_database.my_table' does not exist", which is correct. I have no clue why that code gives me the database/schema name twice?
Edit:
When jooq tells me that the db.table does not exist, if I put a break point in a good place and get the sql from jooq and run exactly that against my database it does work. But jooq fails to run it.
Also, I'm using version 3.15.3 of jooq.
I solved it. Instead of using .withInputExpression(Pattern.compile(".*")), it seems to work with .withInput("").
I'm still not sure why it works, or if this is the "correct" way of solving it. But at least it is a way forward.
No clue why using the pattern, I got the name twice though. But that one I'll leave alone.
As you know, in mongoose, we can remove all users with age 30 like this:
User.find({age: 30}).remove(callback);
Now, replace find() with findOne(), and I think it should remove only 1 user:
User.findOne({age: 30}).remove(callback);
oh, not as I expected, the code above also remove ALL instead of ONE
So, why findOne().remove() remove ALL instead of ONE? Is that a bug or a feature and why?
Thanks in advance!
P/S: I know findOneAndRemove() would remove one user for me, but in this question I want to understand findOne().remove()
I have reported this question to mongoose team, and got a reply:
https://github.com/LearnBoost/mongoose/issues/1851#issuecomment-31355346
Here's the message from aheckmann
"that's a good catch. findOne just sets the command name to run, remove() changes it back to a rice command but no limit was ever set. We should probably change that in 3.9 so that findOne sets
the limit as well."
Both find and findOne returns mongoose Query objects which only contains information about the model and the specified query. It's not taking into account findOne which is applied first in the callback. What you expect to happen is to have options be set like this User.findOne({age: 30}, null, {limit: 1}).remove() as this would only remove one and you could argue that this is a bug, but that depends on the usage. Like you have already pointed out, the right way to go is to use findOneAndRemove().
I'm kind of a noob but wouldn't you need to put your remove in the callback because this is an asynchronous function? Try something like:
User.findOne({age: 30}, function(err, user){
user.remove()
})
Before saving a new document to a mongodb collection via nodejs in my MongoLab database, I'm using model.count to check certain fields to prevent a duplicate entry:
MyModel.count({field1: criteria1, field2: criteria2}, function (err, count) {
if (count == 0) {
// Create new document and call .save()
}
});
However, during testing I'm noticing many duplicates (inconsistent in number across test runs) in the collection after the process finishes, although not as many as if I did not do the .count() check.
Since the MyModel.count() statement is embedded in a callback being repeatedly called whenever the 'readable' event is emitted by one of several ReadStreams, I suspect there is an async issue caused by rapid writes to the collection. Specifically, two or more identical and nearly simultaneous calls to MyModel.count return a count of 0, and end up each creating and saving (identical) documents to the collection.
Does this sound probable? And if so how can I enforce uniqueness of document writes without setting timeouts or using a synchronous pattern?
As Peter commented, the right way to enforce uniqueness is to create a unique index on the collection over those fields and then handle the code: 11000 insert error to recover from attempts at creating duplicates.
You can add the index via your schema before you create the model from it:
mySchema.index({field1: 1, field2: 1}, {unique: true});
Why would a database named 'blog' not allow a record insert and also give no return error and why would a database named 'blogs' allow a record inserts and return errors?
I just spent several hours going through all my code thinking I did something wrong. I have written many mongoose connected apps but when using the following it would return success but not insert the record and return no error as to why:
mongooose.connect('mongodb://localhost:27017/blog');
After banging my head against a wall for a bit I decided to change the database name:
mongooose.connect('mongodb://localhost:27017/blogs');
It works! But why would this name convention matter? I can't find anything in the documentation for MongoDB or Mongoosejs.
So I'm fairly certain mongodb doesn't care about database name "blog" vs "blogs". However, do note that mongoose has the questionably-helpful feature of silently queueing up operations while the database connection is still not established and then firing them off if/when the database connection is ready. That could be causing your confusion. To test that theory, pass a callback to mongoose.connect and put a console.log in the callback so you know exactly when the connection is ready.
I want to create a "prepared statement" in postgres using the node-postgres module. I want to create it without binding it to parameters because the binding will take place in a loop.
In the documentation i read :
query(object config, optional function callback) : Query
If _text_ and _name_ are provided within the config, the query will result in the creation of a prepared statement.
I tried
client.query({"name":"mystatement", "text":"select id from mytable where id=$1"});
but when I try passing only the text & name keys in the config object, I get an exception :
(translated) message is binding 0 parameters but the prepared statement expects 1
Is there something I am missing ? How do you create/prepare a statement without binding it to specific value in order to avoid re-preparing the statement in every step of a loop ?
I just found an answer on this issue by the author of node-postgres.
With node-postgres the first time you issue a named query it is
parsed, bound, and executed all at once. Every subsequent query issued
on the same connection with the same name will automatically skip the
"parse" step and only rebind and execute the already planned query.
Currently node-postgres does not support a way to create a named,
prepared query and not execute the query. This feature is supported
within libpq and the client/server protocol (used by the pure
javascript bindings), but I've not directly exposed it in the API. I
thought it would add complexity to the API without any real benefit.
Since named statements are bound to the client in which they are
created, if the client is disconnected and reconnected or a different
client is returned from the client pool, the named statement will no
longer work (it requires a re-parsing).
You can use pg-prepared for that:
var prep = require('pg-prepared')
// First prepare statement without binding parameters
var item = prep('select id from mytable where id=${id}')
// Then execute the query and bind parameters in loop
for (i in [1,2,3]) {
client.query(item({id: i}), function(err, result) {...})
}
Update: Reading your question again, here's what I believe you need to do. You need to pass a "value" array as well.
Just to clarify; where you would normally "prepare" your query, just prepare the object you pass to it, without the value array. Then where you would normally "execute" your query, set the value array in the object and pass it to the query. If it's the first time, the driver will do the actual prepare for you the first time around, and simple do binding and execution for the rest of the iteration.