I have used embedded elastic as part of a Spring application in Java like this:
Node node;
#SuppressWarnings("unused")
#Bean
public Client es() {
node = nodeBuilder().local(true).node();
Client client = node.client();
boolean indexExists = client.admin().indices().prepareExists(INDEX).execute().actionGet().isExists();
if (!indexExists) {
client.admin().indices().prepareCreate(INDEX).execute().actionGet();
}
return client;
}
I'm trying to do something similar with NodeJS so I don't have to create an elastic search instance separately(super low traffic). In the Spring case, I just set .local(true) and it's good to go. I can't find any option like that in Node.
This is what I'm doing now
var elasticsearch = require('elasticsearch');
var client = new elasticsearch.Client({
// log: 'trace',
host: 'localhost:9200'
});
and it works fine for an external server.
You can't have Elasticsearch Node Client in NodeJS. The second method is the way to go.
Related
I have installed 3 nodes of elastic search from Azure marketplace and any node can act as master node. Now how do I connect to the cluster ? If I had one node, I could simply use its IP with Port(9200) but here I have 3 nodes so how do I get the cluster endpoint ? Thanks
This how I did it and it worked well for me:
public class ElasticsearchConfig {
private Vector<String> hosts;
public void setHosts(String hostString) {
if(hostString == null || hostString.trim().isEmpty()) {
return;
}
String[] hostParts = hostString.split(",");
this.hosts = new Vector<>();
Collections.addAll(this.hosts, hostParts);
}
}
public class ElasticClient {
private final ElasticsearchConfig config;
private RestHighLevelClient client;
public ElasticClient(ElasticsearchConfig config) {
this.config = config;
}
public void start() throws Exception {
HttpHost[] httpHosts = new HttpHost[config.getHosts().size()];
config.getHosts()
.stream()
.map(host -> new HttpHost(host.split(":")[0], Integer.valueOf(host.split(":")[1])))
.collect(Collectors.toList())
.toArray(httpHosts);
client = new RestHighLevelClient(RestClient.builder(httpHosts));
System.out.println("Started ElasticSearch Client");
}
public void stop() throws Exception {
if (client != null) {
client.close();
}
client = null;
}
}
Set the ElasticsearchConfig as below:
ElasticsearchConfig config = new ElasticsearchConfig();
config.setHosts("ip1:port,ip2:port,ip3:port");
If all three nodes are part of same cluster than no need to specify all the nodes and even one single node ip is enough to connect to the cluster.
But above approach has some disadvantage and in small cluster with less workload its fine as in this case only one node which you configure in your elasticsearch client will act as co-ordinating node and can become a hot-spot in your cluster, its better to have all the nodes configured in your client so for every request any one node can act co-ordinating node and if you have huge workload you might also consider to have dedicated co-ordinating nodes for even better performance.
Hope this answer your question and I didn't provide the code snippet as don't know which language and client you are using and in your question I felt code is not issue but you want to understand the concept in detail.
Appreciate everyone's time and all those who replied; Turns out that by default, Azure marketplace copy of self managed ES, sets up only "Internal Load Balancer". I was able to get the cluster end point as soon as I configured "External Load Balancer". All set now.
I am new to Ignite and Kubernetes. I have a.Net Core 3.1 web application which is hosted Azure Linux App Service.
I followed the instructions (Apache Ignite Instructions Offical Site) and Apache Ignite could run on Azure Kubernetes. I could create a sample table and read-write actions worked successfully. Here is the screenshot of my success tests on PowerShell.
Please see the success test
Now, I try to connect Apache Ignite from my .net core web app but I couldn't make it.
My code is as below. I try to connect with IgniteConfiguration and SpringCfgXml, but both of them getting error.
private void Initialize()
{
var cfg = GetIgniteConfiguration();
_ignite = Ignition.Start(cfg);
InitializeCaches();
}
public IgniteConfiguration GetIgniteConfiguration()
{
var appSettingsJson = AppSettingsJson.GetAppSettings();
var igniteNodes = appSettingsJson["AppSettings:IgniteNodes"];
var nodeList = igniteNodes.Split(",");
var config = new IgniteConfiguration
{
Logger = new IgniteLogger(),
DiscoverySpi = new TcpDiscoverySpi
{
IpFinder = new TcpDiscoveryStaticIpFinder
{
Endpoints = nodeList
},
SocketTimeout = TimeSpan.FromSeconds(5)
},
IncludedEventTypes = EventType.CacheAll,
CacheConfiguration = GetCacheConfiguration()
};
return config;
}
The first error I get:
Apache.Ignite.Core.Common.IgniteException HResult=0x80131500
Message=Java class is not found (did you set IGNITE_HOME environment
variable?):
org/apache/ignite/internal/processors/platform/PlatformIgnition
Source=Apache.Ignite.Core
Also, I have no idea what I am gonna set for IGNITE_HOME, and username and secret to authentication.
Solution :
I finally connect the Ignite on Azure Kubernetes.
Here is my connection method.
public void TestConnection()
{
var cfg = new IgniteClientConfiguration
{
Host = "MyHost",
Port = 10800,
UserName = "user",
Password = "password"
};
using (IIgniteClient client = Ignition.StartClient(cfg))
{
var employeeCache1 = client.GetOrCreateCache<int, Employee>(
new CacheClientConfiguration(EmployeeCacheName, typeof(Employee)));
employeeCache1.Put(1, new Employee("Bilge Wilson", 12500, 1));
}
}
To find to host IP, user name and client secret please check the below images.
Client Id and Secret
IP Addresses
Note: I don't need to set any IGNITE_HOME ana JAVA_HOME variable.
The simplest way is to download Apache Ignite binary distribution (of the same version as one that you use), unzip it to a directory, and point IGNITE_HOME environment variable or IgniteConfiguration.IgniteHome configuration property to unzipped apache-ignite-n.n.n-bin/ directory absolute path.
We support doing that automatically for Windows-hosted apps but not for Linux-based deployments.
In my .NET Framework 4.6.1 application I am using StackExchange.Redis.StrongName 1.2.6 to connect to Azure Redis.
This is the code
public RedisContext(string connectionString = null)
{
if (connectionString == null) return;
Lazy<ConfigurationOptions> lazyConfiguration
= new Lazy<ConfigurationOptions>(() => ConfigurationOptions.Parse(connectionString));
var configuration = lazyConfiguration.Value;
configuration.SslProtocols = SslProtocols.Tls12;//just added
configuration.AbortOnConnectFail = false;
Lazy<ConnectionMultiplexer> lazyConnection =
new Lazy<ConnectionMultiplexer>(() => ConnectionMultiplexer.Connect(configuration));
_connectionMultiplexer = lazyConnection.Value;
LogProvider.IsDisabled = true;
var connectionEndpoints = _connectionMultiplexer.GetEndPoints();
_lockFactory = new RedisLockFactory(connectionEndpoints.Select(endpoint => new RedisLockEndPoint
{
EndPoint = endpoint,
Password = configuration.Password,
Ssl = configuration.Ssl
}));
}
In Azure, I have changed the Redis resource to use TLS1.2 and in code I have added this line:
configuration.SslProtocols = SslProtocols.Tls12;//just added
And now, nothing works anymore. This is the error I get in Application Insights:
Error connecting to Redis. It was not possible to connect to the redis server(s); ConnectTimeout
I have also tried to add ",ssl=True,sslprotocols=tls12" to the redis connection string, but with the same result.
Try referencing StackExchange.Redis instead of StackExchange.Redis.StrongName. I have done that in a few of my projects and now it works. However some 3rd party still use StrongName rather than the normal redis one. StackExchange.Redis.StrongName is now deprecated. https://github.com/Azure/aspnet-redis-providers/issues/107. I assume you are trying to connect to Azure Redis in relation to them stopping TLS 1.0 and 1.1 support?
I'm creating a cluster of applications that run on my server.
I use Hazelcast-cluster in combination with VertX in java.
now I would lite to extend the vertx eventbus into an NodeJs applicaion running on the same server.
Hazelcast is running in node and connecting correctly with the hazelcast members running on the JVM
var HazelcastClient = require('hazelcast-client').Client;
var Config = require('hazelcast-client').Config;
var config = new Config.ClientConfig();
config.networkConfig.addresses = [{host: '127.0.0.1', port: '5701'}];
var map = {};
HazelcastClient.newHazelcastClient(config)
.then(function (hazelcastClient) {
map = hazelcastClient.getMap("persons");
});
});
can someone help me with the eventbus part ?!?!
Thanks
After some long searching I found the answer to my problem: I had to let go of node and run my javascript application in a JVM provided by VertX itself.
Now I can cluster my JS application with JAVA application and use the native eventbus ( without bridge )
For anyone who comes along the same situation, here is my test code :
vertx-server.js:
var Vertx = require("vertx-js/vertx");
var options = {};
Vertx.clusteredVertx(options, function (res, res_err) {
if (res_err == null) {
var vertx = res;
var eventBus = vertx.eventBus();
console.log("We now have a clustered event bus: " + eventBus);
eventBus.consumer("system", function (message) {
console.log("I have received a system message: " + JSON.stringify(message.body()));
message.reply("ok from javascript");
});
eventBus.publish("system", "hoi van Javascript-node");
} else {
console.log("Failed: " + res_err);
}
});
to instal the packages needed :
npm install vertx3-full
to run the application :
./node_modules/.bin/vertx run vertx-server.js -cluster
You can't use the Hazelcast Node client to connect a Node application with a Vert.x cluster. You need to setup an event bus bridge use the bridge client in your Node app.
See the EventBus Bridge - Node.JS loader in the examples repo.
I assumed that we should use basicredisclientmanager or pooledredisclientmanager?
I tried this
private void dddddd()
{
for(int i=0;i<=1000;i++)
{
var client = new BasicRedisClientManager(new string[] { "host1", "host2", "host3" }).GetClient();
//do something with client
}
}
This loop runs fine for the first 100 plus but after that, I always got an error "Unknown Command Role"?? What is that and how to fix it? I need help!
I also tried to make a new class called MyRedisMgr and created a static property to create some sort of singleton but it didn't work either.
public BasicRedisClientManager MyMgr = new BasicRedisClientManager(new string[] { "host1", "host2", "host3" });
And then I use it like
for(int i=0;i<=1000;i++)
{
var client = MyRedisMgr.MyMgr.GetClient();
//do something with client
}
Please read the documentation on the proper usage of Redis Client Manager which should only be used as a singleton.
The BasicRedisClientManager doesn't have any connection pooling so every time you call GetClient() you're opening a new TCP connection with the redis-server. Unless you understand the implications you should be using one of the Pooled Redis Client Managers, e.g: RedisManagerPool.
You also need to always dispose the client after its used so that it can either be re-used or the TCP connection disposed of properly.
So your code sample should look like:
//Always use the same singleton instance of a Client Manager
var redisManager = new RedisManagerPool(masterHost);
for(int i=0;i<=1000;i++)
{
using (var redis = redisManager.GetClient())
{
//do something with client
}
}
The "Unknown Command Role" error is due to using an old version of Redis Server. The ROLE command was added in redis 2.8.12 but this API should only be used if your using redis-server v2.8.12+, so you shouldn't be getting this error by default. You can avoid this error by upgrading to either the stable v3.0 or old 2.8 versions of redis-server which has this command.
If you want to continue using an older version, use the INFO command to check what version you're running then tell ServiceStack.Redis what the version is with:
RedisConfig.AssumeServerVersion = 2600; //e.g. v2.6
RedisConfig.AssumeServerVersion = 2612; //e.g. v2.6.12