We have hosted a Hazelcast cluster on a server and a different app on a different server in the same region uses Hazelcast Jet client instance to aggregate the data from Kafka source using pipeline.
In this setup, does Jet client instance send data which it receives from Kafka source Hazelcast cluster which will involve lot of IO or when we create pipeline, the Hazelcast cluster itself creates the connection to Kafka and this connection is from the Jet cluster rather than from the client app?
#Abhishek, you need to have a Jet cluster somewhere. The job itself run on the Jet cluster. If you have Kafka as a source, your app will submit the job to Jet cluster and Jet cluster will connect to data. If your sink is a Hazelcast map on the same Jet cluster, it'll be a local write, if it's a separate Hazelcast cluster, then you need to use Sinks.remoteMap configuring a client to connect to remote Hazelcast cluster.
Related
I have a Node JS application hosted on EC2 instances that I am trying to connect with an RDS Aurora Postgres cluster. The cluster consists of 1 writer and 2 readers. I have whitelisted the security group attached to the EC2 instances as an ingress rule on the security group associated with the database cluster which allows the EC2 instances to communicate with the database in the cluster.
I am having some issues connecting the application on the EC2 instance to the database. I have read this link, but that discusses using a JDBC driver in order to connect the database. I'm not sure if I can use a JDBC driver to allow a Node JS application to connect to this database cluster. I'm not finding any useful examples to connect a Node app to Aurora Postgres.
Any advice on connecting Node JS to Aurora Postgres DB cluster would be helpful.
I have two Presto clusters and I want to bring them under Presto gateway.
What are the challenges and what unexpected can happen if I have only one cluster in each group?
I want all of my users to use Gateway connection instead of coordinator.
I have a StatefulSet Cassandra deployment that works great for services deployed to Kubernetes with namespace access, but I also have an ETL job that runs in EMR and needs to load data into that Cassandra cluster.
What would be the main approach/Kubernetes way of doing this?
I can think of two options.
Simple one is you can create the server with Type: NodePort, with this you can connect server with Node IP Address:PortNumber.
Second option is you can create the Ingress Load Balancer and connect to Cassandra cluster.
Looking to interact with the traditional Spark Web GUI on default clusters in Dataproc.
This can be done by creating a SSH tunnel to the Dataproc master node. By using a SOCKS proxy you can then access all the applications running on YARN including your Spark sessions.
This guide will walk you through in detail:
Dataproc Cluster web interfaces
trying to figure out how to create Cassandra cluster in Azure across more than one datacentre.
I am not much interested in Cassandra topology settings yet, but more in how to set Azure endpoints or inter-datacentre communication to allow nodes to connect remotely. Do I need to set endpoint for node communication inside one datacentre?
What about security of azure endpoints?
Thank you.