I have a pool of the threads and they do the bunch of the operations with the database. However the amount of the threads is huge so it's very possible that 2 threads would like to use the same record from the database at the same time.
In ado.net I'd use transactions to handle this situation.
What should I use in entity framework to avoid mentioned problem?
Related
I am creating a Node.js API consisting of multiple Microservices.
Each Microservice is responsible for one or more features of my application. However, my data is structured into multiple databases which each have multiple collections.
Now I need one sevice to perform atomic operations across multiple databases. If everything happened in the same database, I'd use a normal transaction. However, I don't know how to do this with multiple databases or if this is even possible?
Example:
One of the Microservices takes care of creating users. A user must be
created inside two databases. However, this should happen atomically,
i.e. if the user is created, it must be created in both databases.
UPDATE: MongoDB's official docs state the following:
With distributed transactions, transactions can be used across
multiple operations, collections, databases, documents, and shards.
I haven't found anything on how to perform distributed transactions with mongoose though.
I would be extremely glad if someone could give me some clarification on this topic.
You need to use the SAGA pattern of the microservice architecture.
The SAGA pattern is divided into two types:
Choreography-based saga
Orchestration-based saga
If you want to manage distributed transactions from a single service, then you can use Orchestration-based saga (2).
So with this pattern, you can implement a distributed transaction that either executes a chain of actions or rolls back along the chain, using compensating transactions.
I also recommend studying the patterns of microservice architecture on this site and recommend the book.
EDIT: Mongoose support Distributed Transactions, because it's a client to MongoDB Server. Form Mongoose point of view, a distributed transaction is just a transaction.
According to this video, on Distributed Transactions in MongoDB
the Distributed Transactions is defined above the level of mongoose, and can use it.
in the documentation of mongodb, they say:
Distributed Transactions and Multi-Document Transactions Starting in
MongoDB 4.2, the two terms are synonymous. Distributed transactions
refer to multi-document transactions on sharded clusters and replica
sets. Multi-document transactions (whether on sharded clusters or
replica sets) are also known as distributed transactions starting in
MongoDB 4.2.
Here is how I would try to solve this (Divide-and-conquer):
Try simple example of Distributed Transactions with MongoDB
Then try using simple mongoose with Transactions (it might be that there is be no different between , Distributed Transactions and non- Distributed Transactions as far as mongoose knows, because the Transactions is in higher level – see video).
Then try to combine the 2 solutions and see it this works,
If does not work with mongoose, I would try to implement Distributed Transactions with MongoDB, as the video implay that they spent a lot of effort in this, and since mongoose just let you do things that you can also do with MongoDB alone. Moving from mongoose to MongoDB maybe not so simple, but implementing Distributed Transactions is very hard.
I have a spring boot app that will be used on a fairly popular e-commerce platform. I need to create 3 threads to run Cassandra queries in parallel with some business logic to make the service performant. Is this unheard of? I have barely used threads in my young career.
That right. I usually create batch to query cassandra. Use ThreadPool query cassandra to make the service performant
Scenario: In any kind of online Reservation application there is possiblty that people may acesses the application at the same time.
Because of this there could be a concurrency issues in that perticular application.
Problem:
In this case, if one or more people accessing the application at a time then there is a problem of concurrency issue.
To solve concurrency the issue we have two solutions
1) Java Concurrency at Client level
2) JPA Optimistc or Pessimistic Locking at persistence level
If the database contains 10 millions or more records then which one is the optimal solution?
Performance issues and Concurrency issues are two different problem domains altogether. Its better not to mix both.
Regarding concurrency issues:
Java Concurrency or Multi threading may be of very limited use in a online reservation system. To avoid problem of lost updates, you must use either JPA Optimistic Locking (preferably) or Pessimistic Locking. That's because if multiple instances of your application are running in parallel (or multiple threads) then its impossible to avoid lost updates using JVM level thread-safety techniques.
You can refer to the below tutorials:
JPA Optimistic Locking - Oracle Blog
The Java EE 6 Tutorial
It completely depends on the application. If the Datamodel, not complex and not huge then the JPA Optimistic Locking is an optimal solution.
Both Java & Database Concurrency have there own advantages and disadvantages.
Database level:
Advantage:
1) its easier to implement and also for maintenance.
Disadvantage:
1)If the database contains huge and complex data then it may causes deadlocks and timeouts.
Java Level:
Advantages:
1) Better resource Utilization
Disadvantages:
1) Debugging & Testing very complex.
2) There is a possibility deadlock occurence.
3) Difficult to implement in Java Programming.
4) An Element of risk involved with a bad design.(Starvation)
5) it's not portable from one environment to another environment
ref: java Concurrency vs JPA concurrency
ref: database vs user level
I want to use my sqlite3 database with multiple threads in parallel. I read that using connection pools makes the access threadsafe but I still get errors while inserting data.
(make-thread
#'(lambda()
(dotimes (i 100)
(with-database (db ("/path/to/db")
:database-type :sqlite3 :pool T)
(do-stuff-with db)))))
When using multiple threads in this fashion in this error
While accessing database #
with expression "INSERT INTO ...":
Error 5 / database is locked
Is it even possible to do a multi threaded insert with an sqlite3 database? If yes how?
SQLite does not support concurrency of multiple write transactions. From the SQlite site:
SQLite supports an unlimited number of simultaneous readers, but it will only allow one writer at any instant in time. For many situations, this is not a problem. Writer queue up. Each application does its database work quickly and moves on, and no lock lasts for more than a few dozen milliseconds. But there are some applications that require more concurrency, and those applications may need to seek a different solution.
Cl-sql has been written to give a "unified" interface for the typical client-server relational DBMS, like other "standardized" libraries (e.g. JDBC or ODBC), but SQLite is an "untypical" database management system: in practice it is a library that offers SQL as language to access a simple "database-in-a-file", and a few other functionalities of DBMSs. For instance, it has no real concurrency control (it uses the Operating Systems functions to lock the db file), so it cannot be considered a "real" DBMS, and cl-sql cannot offer nothing more than the functionalities of the underlying system.
So, if you need concurrent insertions into a database, you should use something else, for instance PostgreSQL.
If there are multiple thread which access (read/write) to a same table into a DB, what considerations of thread-safety should I take?
Here are some good tips, for example if using MySQL
Use row-level locking.
Use the TRANSACTION_READ_COMMITTED isolation level.
Avoid queries that cannot use indexes; they require locking of all the rows in the table (if only very briefly) and might block an update.
Avoid sharing Statements among threads
Here is some more information and reference
check for mechanisms which implement transactions in different isolation levels. These mechanism are present in database system or your API.