I have a requirement to connect to Redis and PostgreSQL databases. For performance reasons, I need to be able to connect to these databases with a connection pool. There are some packages for connection pool e.g. Redis redis-connection-pool, node-redis-connection-pool and Postgresnode-postgres. These work well individually.
How do I make them work across a Node-RED instance? Can I set-up a script that runs at Node-RED start-up which then 'exports' the pool object to any node that could 'require' it?
No, you would have to look at how the available node-red nodes for Redis and Postgres were written and then modify them if needed.
A lot of nodes already have the concept of shared config nodes which hold a connection so they may already have a point to anchor a pool.
Related
After month of research, we are here, hoping for someone to have a insight about these issue:
On a GKE cluster, our pods (node.JS) are having trouble connecting to our external oracle business database.
To be more precise, ~70% of our connection tentative are ending in error:
ORA-12545: Connect failed because target host or object does not exist
The 30% left are working well, and doesn't reset or end prematurely. Once it's connected, it's all good from here.
Our stack:
Our flux are handed by containers based on a node:12.15.0-slim image, at which we add LIBAIO1 and a instant oracle client (v12.2). We use oracleDB v5.0.0 as node module
We use cron job pod handling our node container, in a clusterIP service on a GKE cluster (1.16.15-gke.4300).
Our external oracle database in on a private network (which our cluster have access), in a Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bi version, behind a load balancer
I can give more detail if needed.
What we have already tried:
We have tried to pass directly on the database, cutting off the load balancer: no effect
We had cron job pod doing ping each min on the database server for a day: no error, although flux pod somehow encounter the ORA-12545 error
We redo all our code, connecting differently to the database and making update for our node module oracledb (v4 to v5): no effect
We tried to monitore the load up over the oracle database and take action spreading our flux over all night instead of a 1 hour window: no effect
We had our own kubernetes cluster before GKE, directly in our private network, causing the exactly same error.
We had a audit by some expert on kubernetes, without them finding the issue or seeing a critical issue over our cluster/k8s configuration
What works:
All our pods, some requesting into mySql database, micro service, web front, are all working fine.
All our business tool (dozen of, including Talend and some custom software) are using the oracle database without issue.
Our own flux handling node container are working fine with the oracle database as long they are into a docker env, and not a kube one.
To resume: We have a mysterious issue when trying to connect to an oracle database from a kubernetes env, where pods are randomly unable to reach the database
We are looking for any hint we can have
I have been given a task to configure Cassandra DB for the project. We are facing a problem - for all environments there is a dedicated server for Cassandra. But, for the DEV environment, the client does not want to provide a seperate server and current DEV servers are already fully packaged and we can't afford to install Cassandra on them.
My question is, is there any possibility to fake connection to Cassandra in an environment? I've created CassandraConfiguration.java class, configured session, cluster etc etc, it all works smoothly on other envs, but on DEV, well, it fails, as it cannot connect, because there's no Cassandra... Commiting the cassandraconfiguration file will kill the dev.
You can use scassandra (simulated cassandra), or Simulacron that are emulating Cassandra. Or you can use cassandra-unit that will run Cassandra in the same JVM as your test.
I've been using Couchbase for my database solution and so far it looks very good.
I'm confused however with connecting to a Cluster. A Cluster is just a group of nodes so when you use the API to connect to a Cluster what do you use as the IP? Do you just use one of the nodes in the Cluster? Does it matter which one?
I'm personally using the Node.js API.
Technically all you need is just one node in the list. As soon as it connects to that one, it will get the cluster map of the entire cluster and know all of the rest of the nodes. No it does not matter which node.
That being said, best practice is to have at least 3 nodes of the cluster listed in the connection string or better yet if the SDK you are using supports it, use a DNS SRV record with at least 3 nodes in there. With three nodes in the list if for some reason (e.g. server failure or maintenance) one of the nodes is unavailable, you can still bootstrap an application server to get that cluster map with one of the other nodes in the list.
I asked this question a few months ago on couchbase forums and the author of the node.js module answered that you should use "some" of them
like :
cluster.openBucket("couchbase://server1,server2,server3", function(err) {});
if you have server4 and 5 are added , they will be automatically added to the cluster as soon as they are available in the cluster.
Check here for details : https://forums.couchbase.com/t/couchnode-connection-to-cluster/6281
So I have an app I am working on and I am wondering if I am doing it correctly.
I am running cluster on my node.js app, here is a link to cluster. I couldn't find anywhere that states if I should only run cluster on a single server or if it is okay to run it on a cluster of servers. If I continue down the road I am going I will have a cluster inside a cluster.
So that it is not just opinions as answers, here is my question. Was cluster the package made to do what I am doing (cluster of workers on a single server inside a cluster of servers)?
Thanks in advance!
Cluster wasn't specifically designed for that, but there is nothing about it which would cause a problem. If you've designed your app to work with cluster, it's a good indication that your app will also scale across multiple servers. The main gotcha would be if you're doing anything stateful on the filesystem. For example, if a user uploads a photo and you store it on the server disk, that would be problematic when scaling out across multiple servers (that don't share the same disk).
How can i create a connection pool for cassandra using hector? can somebody give a simple example for this. i have searched the net but didn't get any solution.
Hector handles the connection pooling for you. From their docs:
The basic design of connection pooling logic in Hector is to create a master pool of individual pools of connections based on the number of Cassandra nodes explicitly identified to Hector. For example, if we had three Cassandra nodes configured, HConnectionManager would contain three individual pools of connections – one for each node.
...
The underlying HConnectionManager as well as the host-specific pool for localhost:9160 will both be created automatically if they do not already exist.
You can also read through all the possible configuration options of the connection and pooling settings.