I have found similar question on stackoverflow but it is solely focused on performance and answer is pretty obvious: creating new connection for each query = slower performance (how much slower? it depends)
I am more worried about transaction isolation aspect. In SQLite documentation I have found out that there is no isolation within a single connection. I am using sqlite3 library in my electron desktop app and I was planning on sharing a single connection throught the whole time that my app is running (to make it a little faster) but now I am wondering if it is safe. If there is no isolation within a single connection then is this scenario possible?:
Client triggers 2 unrelated processes
1.
db.serialize(()=>{
db.run("BEGIN");
try{
db.run("foo");
db.run("bar");
}catch(e){
db.run("ROLLBACK")
}
db.run("COMMIT")
});
2.
db.run("another foobar")
1. and 2. are ran parallel so it is possible that 2. finishes somewhere in between of "begin" and "commit"/"rollback" from 1.
Does that mean that it is possible for queries from 2. to be rolledback or commited by 1. even though they are entierly separate or is 2. using some implicit transaction to prevent this?
I think it is possible since there is no isolation within single connection but I might be missing something because I have never worked with SQLite and sqlite3 (or I might have missed something more basic) so I would like to confirm if this scenario is a potential danger of using single sqlite3 connection.
Initially I have created an sqlite database('temp.db') and shared its connection to multiple process. I started to get lots of database locked error.
I needed database for temporary storage only. The only operation performed are INSERT and SELECT on a single TABLE also no COMMIT is done on the database.
To overcome above lock issue I have created an in-memory(':memory:') sqlite database and shared its connection to multiple process. I have not run into any database lock error until now.
In both the cases I have not used any locking mechanisms. Using in first case might have resolved the issue, but dont want to increase execution time.
Is locking needed in second case? What could be other pitfalls to care for? Its impact on long running application?
I'm developing a Scala Play2 application that queries an OrientDB Graph in Scala Play2. Until today I didn't bother with indexes and all seemed to work fine but now that I have enabled a couple I get this error:
play.api.http.HttpErrorHandlerExceptions$$anon$1: Execution
exception[[ODatabaseException: Database instance is not set in current
thread. Assure to set it with:
ODatabaseRecordThreadLocal.INSTANCE.set(db);]]
From the documentation I understand that the database object is not thread safe but I'm uncertain how to proceed: my queries are picked up asynchronously by the Play2 executor pool and I'm not sure whether it would be a good idea to mess around with threadlocals. Will the driver block? Will the driver clobber its state if different threads from the pool handle the database connection? In any case I would like some advice from someone that knows Orient's driver architecture better than me :)
As suggested by the documentation and the error, the OrientDB driver uses thread locals to isolate thread-unsafe portions of itself.
The solution to this issue was to move away from the simplistic design I had at that point and run the queries from within an org.apache.tinkerpop.gremlin.orientdb.OrientGraph acquired through a org.apache.tinkerpop.gremlin.orientdb.OrientGraphFactory#getTx() called within the same block.
I didn't investigate whether a transaction is strictly required, although I don't think so.
We're experiencing a few problems using the Community.MvvmCross-SQLite project in an mvvmcross app on Windows Phone 8. We have a few background threads (using Task.Run()) that reads/writes to the database in response to updates from a server - since the update might take a few seconds we don't want the UI to freeze up.
When running the app we experience a few crashes where we see messages like "Database missing or corrupt", and when debugging the situation we see a crashes when accessing the database from the background threads.
We've tried to use one common SqliteConnection across the whole project, and everytime we try to write to the database we encapsulate the call in a lock() statement.
Is it ok to use the same connection, or should we create new ones for every operation?
What is the state of the multithreaded support in the underlying Community.CSharpLiteSqlite.WP7 library?
SQLite And Multiple Threads
SQLite support three different threading modes:
1.Single-thread. In this mode, all mutexes are disabled and SQLite is unsafe to use in more than a single thread at once.
2.Multi-thread. In this mode, SQLite can be safely used by multiple threads provided that no single database connection is used simultaneously in two or more threads.
3.Serialized. In serialized mode, SQLite can be safely used by multiple threads with no restriction.
SqliteTheadSafe Documentatioon
Why does being thread safe matter in a web app? Pylons (Python web framework) uses a global application variable which is not thread safe. Does this matter? Is it only a problem if I intend on using multi-threading? Or, does it mean that one user might not have updated state if another user... I'm just confusing myself. What's so important with this?
Threading errors can lead to serious and subtle problems.
Say your system has 10 members. One more user signs up to your system and the application adds him to the roster and increments the count of members; "simultaneously", another user quits and the application removes him from the roster and decrements the count of members.
If you don't handling threading properly, your member count (which should be 10) could easily be nine, 10, or 11, and you'll never be able to reproduce the bug.
So be careful.
You should care about thread safety. E.g in java you write a servlet that provides some functionality. The container will deploy an instance of your servlet, and as HTTP requests arrive from clients, over different TCP connections, each request is handled by a separate thread which in turn will call your servlet. As a result, you will have your servlet being call from multiple threads. So if it is not thread-safe, then erroneous result will be returned to the user, due to data corruption of access to shared data by threads.
It really depends on the application framework (which I know nothing about in this case) and how the web server handles it. Obviously, any good webserver is going to be responding to multiple requests simultaneously, so it will be operating with multiple threads. That web server may dispatch to a single instance of your application code for all of these requests, or it may spawn multiple instances of your web application and never use a given instance concurrently.
Even if the app server does use separate instances, your application will probably have some shared state--say, a database with a list of users. In that case, you need to make sure that state can be accessed safely from multiple threads/instances of your web app.
Then, of course, there is the case where you use threading explicitly in your application. In that case, the answer is obvious.
Your Web Application is almost always multithreading. Even though you might not use threads explicitly. So, to answer your questions: it's very important.
How can this happen? Usually, Apache (or IIS) will serve several request simultaneously, calling multiple times from multiple threads your python programs. So you need to consider that your programs run in multiple threads concurrently and act accordingly.
(This was too long to add a comment to the other fine answers.)
Concurrency problems (read: multiple access to shared state) is a super-set of threading problems. The (concurrency problems) can easily exist at an "above thread" level such as a process/server level (the global variable in the case you mention above is process-unique value, which in turn can lead to an inconsistent view/state if there are multiple processes).
Care must be taken to analyze the data consistency requirements and then implement the software to fulfill those requirements. I would always err on the side of safe, and only degrade in carefully analyzed areas where it is acceptable.
However, note that CPython runs only one thread context for Python code execution (to get true concurrent threads you need to write/use C extensions), so, while you can get a form of race condition upon expected data, you won't get (all) the same kind of partial-write scenarios and such that may plague C/C++ programs. But, once again. Err on the side of a consistent view.
There are a number of various existing methods of making access to a global atomic -- across threads or processes. Use them.