I'm currently trying to implement something similar to DataSourceUtils::doGetConnection in rust with actix web.
I've multiple repository traits with methods like get_user_for_update. Now currently the implementations of the trait receive an sqlx::PgPool and draw a connection from this pool. But in many cases multiple repository methods are called and should execute within one transaction. Is there any way to bind a connection to the current async context in actix web? So each repository method could use the logic "If there's a connection bound to this thread already, use it. If a transaction is requested, get a connection and bind it to this context, otherwise get a connection, execute, and close it".
I'd like to avoid passing a connection reference to every repository method, since I'd have to pass it between services too and mocking the connection is really difficult (associated generic types etc).
Related
Currently when using Rust Rocket framework it is necessary to get your database connection via the controller. Basically the connection is given to your handler from a pre-configured pool. Now we have to pass this connection down into any struct which needs a database connection.
If I would like to separate the concerns of reading from the data store, or potentially multiple data stores if caching is involved as well, then I will have to pass one, or potentially multiple, different connection structs from my handler into the lower layer.
While I am aware that I can encapsulate all of the connections into a single request guard, I am dissatisfied by lack of abstraction. I feel quite strongly that my handler should know nothing about the database in order to keep the concerns as separate as possible.
How would I proceed in Rust to obtain a connection from some shared pool of connections in an object, without usage of request guards and argument drilling?
Note: Terminology may be incorrect due to rather limited experience with Rocket
I have documents in different MongoDB databases referencing each other (mongoengine's LazyRefereneceField), so each time I need to get the field's value, I need to connect and disconnect from the field's relevant database, which I find very inefficient.
I've read about connection pooling, but I can't find a solution on how to implement it using MongoEngine. How can I create a connection pool and reuse connections from it every time I need to the value for a LazyReferenceField?
MongoEngine is managing the connection globally (i.e once connected, it auto-magically re-use that connection), usually you call connect just once, when the application/script starts and then you are good to go, and don't need to interfere with the connection.
LazyReferenceField is not different from any other field (ReferenceField, StringField, etc) in that context. The only difference is that it's not doing the de-referencing immediatly but only when you explicitly request it with the .fetch method
Continuing the conversation from
this question:
Two-part question here:
Can a TcpNetClientConnectionFactory have multiple connections to its upstream
server, if the host and port are the same?
If so, how can I programmatically build a new connection for that connection
factory? I see the buildNewConnection method, but it is protected.
The first connection is automatically built as soon as the first Message
passes through the factory. What we need to do is notice when following Messages
have a different ip_connectionId, stand up a new connection, and route those
Messages to that new connection. Obviously, Messages with the original
ip_connectionId would still be routed to the original connection.
Not sure whether it would be better to create multiple connections off of one
connection factory, or create a whole new connection factory, sending message
handler, and receiving channel adapter for each new connection.
If the inbound connection factory is a TcpNetServerConnectionFactory, you can simply use a ThreadAffinityClientConnectionFactory because each inbound connection gets its own thread.
You would call getConnection(). This will bind the connection to the thread (and you can obtain the connection id from it), but you don't really need to map the header in this direction because of the thread affinity, you would only have to map on the return path.
Bear in mind, though, if the ThreadAffinityClientConnectionFactory detects that a connection has been closed, it will create a new one. So, you might want to call getConnection() in your mapper on each call. However, there would still be a race condition, so you might also need to listen for TcpConnectionCloseEvents and TcpConnectionOpenEvents.
If you use NIO on the inbound, or otherwise hand off the work to other threads via an executor, this won't work.
In that case you would need your own wrapping connection factory - you could use the ThreadAffinityClientConnectionFactory as a model, but instead of storing the connections in a ThreadLocal, you'd store them in a map. But you'd still need a ThreadLocal (set upstream on each call) to tell the factory which connection to hand out when the adapter asks for one.
There's a trick you need to be aware of, however.
There is a property singleUse on the connection factory. This serves 2 purposes;
first, it tells the factory to create a new connection each time getConnection() is called instead of a single, shared, connection
second, it tells the inbound adapter to close the connection after the reply is received
So the trick is you need singleUse=true on the real factory (so it gives you a new connection each time getConnection() is called), but singleUse=false on the wrapping factory so the adapters don't close the connection.
I suggest you look at the ThreadAffinityClientConnectionFactory and CachingClientConnectionFactory connection factory to see how they work.
We should probably consider splitting this into two booleans; we could probably also make some improvements to avoid the need for a thread local by adding something like getConnection(String connectionId) to the client factory contract and have the factory look up the connection internally; but that will require work in the adapters.
I'll capture an issue for this and see if we can get something in 5.2.
Rather a long answer, but I hope it makes sense.
I'm using the Node native client 1.4 in my application and I found something in the document a little bit confusing:
A Connection Pool is a cache of database connections maintained by the driver so that connections can be re-used when new connections to the database are required. To reduce the number of connection pools created by your application, we recommend calling MongoClient.connect once and reusing the database variable returned by the callback:
Several questions come in mind when reading this:
Does it mean the db object also maintains the fail over feature provided by replica set? Which I thought should be the work of MongoClient (not sure about this but the C# driver document does say MongoClient maintains replica set stuff)
If I'm reusing the db object, when should I invoke the db.close() function? I saw the db.close() in every example. But shouldn't we keep it open if we want to reuse it?
EDIT:
As it's a topic about reusing, I'd also want to know how we can share the db in different functions/objects?
As the project grows bigger, I don't want to nest all the functions/objects in one big closure, but I also don't want to pass it to all the functions/objects.
What's a more elegant way to share it among the application?
The concept of "connection pooling" for database connections has been around for some time. It really is a common sense approach as when you consider it, establishing a connection to a database every time you wish to issue a query is very costly and you don't want to be doing that with the additional overhead involved.
So the general principle is there that you have an object handle ( db reference in this case ) that essentially goes and checks for which "pooled" connection it can use, and possibly if the current "pool" is fully utilized then and create another ( or a few others ) connection up to the pool limit in order to service the request.
The MongoClient class itself is just a constructor or "factory" type class whose purpose is to establish the connections and indeed the connection pool and return a handle to the database for later usage. So it is actually the connections created here that are managed for things such as replica set fail-over or possibly choosing another router instance from the available instances and generally handling the connections.
As such, the general practice in "long lived" applications is that "handle" is either globally available or able to be retrieved from an instance manager to give access to the available connections. This avoids the need to "establish" a new connection elsewhere in your code, which has already been stated as a costly operation.
You mention the "example" code which is often present through many such driver implementation manuals often or always calling db.close. But these are just examples and not intended as long running applications, and as such those examples tend to be "cycle complete" in that they show all of the "initialization", the "usage" of various methods, and finally the "cleanup" as the application exits.
Good application or ODM type implementations will typically have a way to setup connections, share the pool and then gracefully cleanup when the application finally exits. You might write your code just like "manual page" examples for small scripts, but for a larger long running application you are probably going to implement code to "clean up" your connections as your actual application exits.
I am working on an application in which I need a connection to a server. I also need to access this connection from different activities.
To achieve this I was going to override the Application class and create the connection there. This would allow for easy interaction from every Activity as I could simply call getApplicationContext().getConnection() to get access to my own connection class.
The problem with this approach is that the Application class does not have any onDestroy() method or similar in which I can release the connection and any related resources. I do not think that leaving it idle until onLowMemory() is called is the best approach here.
I cannot add a custom release() method, as I don't know when to call it (there are two Activities that can be the last one to be active, and depending on the users actions they do not know if the other is to be started when the active one is shut down).
Is there a good solution to this, should I just ignore releasing resources (before onLowMemory()) or is there a better way to achieve what I want (possibly a Service, but as there will be several calls to an underlying class it might get overly problematic with the Service?)
Just use Singleton Design Pattern. Making your Connection class Singleton gives you approach to access connection from different activities, and don`t forget to handle multithreading.