Does GridGain support SSL connection between each cluster member? If yes, can you show me how to do that?
Thanks,
Bill
GridGain supports SSL only for client connections (GridGain provides .NET and C++ thin clients), but not for communication between nodes.
To enable SSL for client connections, configure your server nodes like this:
<bean id="grid.cfg" class="org.gridgain.grid.GridConfiguration">
<!-- Enable REST. -->
<property name="restEnabled" value="true"/>
<!-- Client connection configuration. -->
<property name="clientConnectionConfiguration">
<bean class="org.gridgain.grid.GridClientConnectionConfiguration">
<!-- Enable SSL. -->
<property name="restTcpSslEnabled" value="true"/>
<!-- Provide SSL context factory (required). -->
<property name="restTcpSslContextFactory">
<bean class="org.gridgain.client.ssl.GridSslBasicContextFactory">
<property name="keyStoreFilePath" "keystore/server.jks"/>
<property name="keyStorePassword" value="123456"/>
<property name="trustStoreFilePath" "keystore/trust.jks"/>
<property name="trustStorePassword" value="123456"/>
</bean>
</property>
</bean>
</property>
</bean>
You will also need to provide SSL context factory on client configuration.
Ignite (and GridGain) allows you to use SSL for all possible connections.
To use SSL connection between nodes define property sslContextFactory in IgniteConfiguration.
<bean id="cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="sslContextFactory">
<bean class="org.apache.ignite.ssl.SslContextFactory">
<property name="keyStoreFilePath" value="keystore/server.jks"/>
<property name="keyStorePassword" value="123456"/>
<property name="trustStoreFilePath" value="keystore/trust.jks"/>
<property name="trustStorePassword" value="123456"/>
</bean>
</property>
</bean>
You can also check official security documentation.
https://apacheignite.readme.io/docs/ssltls
Related
I created an AKS and I deployed the Apache Ignite service on it.
When I check the pods I can see they are working.
Also, I can get the load balancer IP.
I follow the official instructions of Apache and try to connect Ignite with ThinClient.
I share my code and instructions code.
Here is my code:
public void ConnectIgnite()
{
var cfg = new IgniteClientConfiguration
{
Endpoints = new[] { "20.101.12.***:10800" }
};
var client = Ignition.StartClient(cfg);
}
But my code is getting below errors;
System.AggregateException: 'Failed to establish Ignite thin client
connection, examine inner exceptions for details.'
Inner Exception ExtendedSocketException: A connection attempt failed
because the connected party did not properly respond after a period of
time, or established connection failed because connected host has
failed to respond.
and here is the Apache's instruction code;
ClientConfiguration cfg = new ClientConfiguration().setAddresses("13.86.186.145:10800");
IgniteClient client = Ignition.startClient(cfg);
ClientCache<Integer, String> cache = client.getOrCreateCache("test_cache");
cache.put(1, "first test value");
System.out.println(cache.get(1));
client.close();
Also, here is the official instruction link
I didn't understand what is wrong? Also, The Instruction says I don't need clientid and clientsecret but I don't want to connect without any security but this is completely another issue.
I found what is wrong. Apache's official page says: use below XML for the configuration.
<bean class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder">
<constructor-arg>
<bean class="org.apache.ignite.kubernetes.configuration.KubernetesConnectionConfiguration">
<property name="namespace" value="default" />
<property name="serviceName" value="ignite" />
</bean>
</constructor-arg>
</bean>
</property>
</bean>
</property>
</bean>
but,
first, that XML needs a beans tag at the top of XML.
Also, namespace value and serviceName value are not compatible with apache's official instruction page. If you follow the apache's page for the setup;
you have to use the below values
<property name="namespace" value="ignite" />
<property name="serviceName" value="ignite-service" />
instead of
<property name="namespace" value="default" />
<property name="serviceName" value="ignite" />
end of the changes your XML will look like
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder">
<constructor-arg>
<bean class="org.apache.ignite.kubernetes.configuration.KubernetesConnectionConfiguration">
<property name="namespace" value="ignite" />
<property name="serviceName" value="ignite-service" />
</bean>
</constructor-arg>
</bean>
</property>
</bean>
</property>
</bean>
</beans>
I changed my configuration XML and restart the pods with below command and It worked.
kubectl -n service rollout restart deployment ignite-cluster
Try connecting using 20.101.12.*** address from the outside.
Btw, what is 13.86.186.145, why are you trying to connect to it?
An update:
Seems like you just copy-pasted it from the docs, you need to replace it with your own values.
I am using jms:outbound-channel-adapter.
We noticed that application server is starting way too many MQ connections. Application ultimately exhausts the maxChannel limit of MQ Server and starts getting :
Caused by: com.ibm.mq.MQException: JMSCMQ0001: WebSphere MQ call failed with compcode '2' ('MQCC_FAILED') reason '2537' ('MQRC_CHANNEL_NOT_AVAILABLE')
I am using MQTopicConnectionFactory, CachingConnectionFactory, outbound-channel-adapter.
Current configuration:
<bean id="mqQueueConnectionFactory" class="com.ibm.mq.jms.MQTopicConnectionFactory">
<property name="SSLCipherSuite" value="${mq.sslCipherSuite}"/>
<property name="hostName" value="${mq.hostName}"/>
<property name="port" value="${mq.port}"/>
<property name="channel" value="${mq.channel}"/>
<property name="transportType" value="1"/>
</bean>
<bean id="cachedConnectionFactory"
class="org.springframework.jms.connection.CachingConnectionFactory"
p:targetConnectionFactory-ref="jmsTConnectionFactory"
p:reconnectOnException="true" p:sessionCacheSize="10"
p:cacheProducers="true" />
<bean id="senderTopic" class="com.ibm.mq.jms.MQTopic" >
<constructor-arg value="${mq.topicName}"/>
</bean>
<bean id="jmsQueueTemplate" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory">
<ref bean="cachedConnectionFactory" />
</property>
<property name="defaultDestination">
<ref bean="senderTopic" />
</property>
</bean>
Outbound-channel-adapter:
<int-jms:outbound-channel-adapter
id="jmsOutToNE" channel="umpAlertNotificationJMSChannel"
destination="senderTopic" jms-template="jmsQueueTemplate">
<!-- <int:retry-advice/> -->
</int-jms:outbound-channel-adapter>
Spring integration flow having 250 concurrent threads, however, there are 2000+ MQ connections starting during peak load.
What is the significance of sessionCacheSize="10" in CachingConnectionFactory if the connections are exceeding 500?
How can I limit the MQ connections?
Can't JMS reuse the MQ connection from the a pool the way DB connections work?
I am trying to set Expiry Policy using xml config file:
<?xml version="1.0" encoding="UTF-8"?>
<!--
Ignite configuration with all defaults and enabled p2p deployment and enabled events.
-->
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:util="http://www.springframework.org/schema/util"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/util
http://www.springframework.org/schema/util/spring-util.xsd">
<bean abstract="true" id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
<!-- Set to true to enable distributed class loading for examples, default is false. -->
<property name="peerClassLoadingEnabled" value="true"/>
<!-- Enable task execution events for examples. -->
<property name="includeEventTypes">
<list>
<!--Task execution events-->
<util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_STARTED"/>
<util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_FINISHED"/>
<util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_FAILED"/>
<util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_TIMEDOUT"/>
<util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_SESSION_ATTR_SET"/>
<util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_REDUCED"/>
<!--Cache events-->
<util:constant static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_PUT"/>
<util:constant static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_READ"/>
<util:constant static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_REMOVED"/>
</list>
</property>
<!-- Explicitly configure TCP discovery SPI to provide list of initial nodes. -->
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<!--
Ignite provides several options for automatic discovery that can be used
instead os static IP based discovery. For information on all options refer
to our documentation: http://apacheignite.readme.io/docs/cluster-config
-->
<!-- Uncomment static IP finder to enable static-based discovery of initial nodes. -->
<!--<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">-->
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
<property name="addresses">
<list>
<!-- In distributed environment, replace with actual host IP address. -->
<value>127.0.0.1:47500..47509</value>
</list>
</property>
</bean>
</property>
</bean>
</property>
</bean>
<bean class="org.apache.ignite.configuration.CacheConfiguration">
<property name="setExpiryPolicyFactory" >
<bean class="javax.cache.expiry.CreatedExpiryPolicy">
<!---- Remove all data after 10 Seconds---->
<property name="expiryDuration" value="10000"/>
</bean>
</property>
<property name="eagerTtl" value="true"/>
</bean>
I am a bit new for Ignite. I walk through documentation but unable to find. Is any way to setExpiryPolicy into XMl?
Any help would be appreciated!
That requires a bit of Spring XML magic because there is no easy-to-use factory class for the CreatedExpiryPolicy (i.e. no CreatedExpiryPolicyFactory that you can just instantiate like a normal bean), but there is the CreatedExpiryPolicy::factoryOf that can be passed as factory-method:
<property name="expiryPolicyFactory">
<bean class="javax.cache.expiry.CreatedExpiryPolicy" factory-method="factoryOf">
<constructor-arg>
<bean class="javax.cache.expiry.Duration">
<constructor-arg value="SECONDS"/>
<constructor-arg value="10"/>
</bean>
</constructor-arg>
</bean>
</property>
Apache Ignite doesn`t support expiry policies and TTL for SQL tables.
An alternative would be to create cache with dynamically generated names based on desired time window.
For example you could append hour of the day if you need it hourly:
MY_CACHE_00
MY_CACHE_01
MY_CACHE_02
etc...
MY_CACHE_23
Then make your software write/read to the latest hour CACHE, and clear the oldest.
I'm trying to integrate Apache Ignite with Apache Cassandra(3.11.2) as I want to use Ignite to cache the data present in my already existing Cassandra database.
After going through the online resources, I've done the following till now:
Downloaded Apache Ignite.
Copied all the folders present in "libs/optional/" to "libs/"(I don't know which ones will be required for Cassandra).
Created 3 xmls in the config folder i.e. "cassandra-config.xml", "connection-settings.xml" and "persistance-settings.xml". Currently I'm using the same node(172.16.129.68) for both Cassandra and Ignite.
cassandra-config.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd">
<!-- Cassandra connection settings -->
<import resource="connection-settings.xml" />
<!-- Persistence settings for 'cache1' -->
<bean id="cache1_persistence_settings" class="org.apache.ignite.cache.store.cassandra.persistence.KeyValuePersistenceSettings">
<constructor-arg type="org.springframework.core.io.Resource" value="file:/home/cass/apache_ignite/apache-ignite-fabric-2.4.0-bin/config/persistance-settings.xml" />
</bean>
<!-- Ignite configuration -->
<bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="cacheConfiguration">
<list>
<!-- Configuring persistence for "cache1" cache -->
<bean class="org.apache.ignite.configuration.CacheConfiguration">
<property name="name" value="cache1"/>
<property name="readThrough" value="true"/>
<property name="writeThrough" value="true"/>
<property name="writeBehindEnabled" value="true"/>
<property name="cacheStoreFactory">
<bean class="org.apache.ignite.cache.store.cassandra.CassandraCacheStoreFactory">
<property name="dataSourceBean" value="cassandraAdminDataSource"/>
<property name="persistenceSettingsBean" value="cache1_persistence_settings"/>
</bean>
</property>
</bean>
</list>
</property>
<!-- Explicitly configure TCP discovery SPI to provide list of initial nodes. -->
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<!--
Ignite provides several options for automatic discovery that can be used
instead os static IP based discovery. For information on all options refer
to our documentation: http://apacheignite.readme.io/docs/cluster-config
-->
<!-- Uncomment static IP finder to enable static-based discovery of initial nodes. -->
<!--<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">-->
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
<property name="addresses">
<list>
<!-- In distributed environment, replace with actual host IP address. -->
<value>172.16.129.68:47500..47509</value>
</list>
</property>
</bean>
</property>
</bean>
</property>
</bean>
connection-settings.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean id="loadBalancingPolicy" class="com.datastax.driver.core.policies.TokenAwarePolicy">
<constructor-arg type="com.datastax.driver.core.policies.LoadBalancingPolicy">
<bean class="com.datastax.driver.core.policies.RoundRobinPolicy"/>
</constructor-arg>
</bean>
<bean id="cassandraAdminDataSource" class="org.apache.ignite.cache.store.cassandra.datasource.DataSource">
<property name="port" value="9042"/>
<property name="contactPoints" value="172.16.129.68"/>
<property name="readConsistency" value="ONE"/>
<property name="writeConsistency" value="ONE"/>
<property name="loadBalancingPolicy" ref="loadBalancingPolicy"/>
</bean>
persistance-settings.xml
<persistence keyspace="test" table="epc_table">
<keyPersistence class="java.lang.String" strategy="PRIMITIVE" column="imsi"/>
<valuePersistence strategy="BLOB"/>
</persistence>
I run the following command to start Ignite from bin folder.
ignite.sh ../config/cassandra-config.xml
Now, I want to take a look at the cassandra table via sqlline. I've tried the following:
./sqlline.sh -u jdbc:cassandra://172.16.129.68:9042/test //(test is the name of the keyspace)
I get the following output:
No known driver to handle "jdbc:cassandra://172.16.129.68:9042/test". Searching for known drivers...
java.lang.NullPointerException
sqlline version 1.3.0
0: jdbc:cassandra://172.16.129.68:9042/test>
I've also tried:
./sqlline.sh -u jdbc:ignite:thin://172.16.129.68
but when I use "!tables", I'm not able to see any table.
What exactly has been missing? How to access/modify the tables present in Cassandra using sqlline?
Operating System: RHEL 6.5
Apache Ignite is a key-value database and there are no tables created by default that you are able to view with JDBC connector. CacheStore is a way to integrate Ignite with external DB or any other storage, and it loads data as a key-value pair.
In your config you said Ignite that you want to store and load entries in/from Cassandra, but still Ignite doesn't know entries structure (BTW Ignite really doesn't care what objects were putted into it).
To be able to list tables and do queries on it, you need to create tables. For that you need to have ignite-indexing in /lib directory and set QueryEntity or indexed types if you have annotated POJOs. Here is example of such configuration:
<bean class="org.apache.ignite.configuration.CacheConfiguration">
<property name="name" value="mycache"/>
<!-- Configure query entities -->
<property name="queryEntities">
<list>
<bean class="org.apache.ignite.cache.QueryEntity">
<property name="keyType" value="java.lang.Long"/>
<property name="valueType" value="org.apache.ignite.examples.Person"/>
<property name="fields">
<map>
<entry key="id" value="java.lang.Long"/>
<entry key="orgId" value="java.lang.Long"/>
<entry key="firstName" value="java.lang.String"/>
<entry key="lastName" value="java.lang.String"/>
<entry key="resume" value="java.lang.String"/>
<entry key="salary" value="java.lang.Double"/>
</map>
</property>
<property name="indexes">
<list>
<bean class="org.apache.ignite.cache.QueryIndex">
<constructor-arg value="id"/>
</bean>
<bean class="org.apache.ignite.cache.QueryIndex">
<constructor-arg value="orgId"/>
</bean>
<bean class="org.apache.ignite.cache.QueryIndex">
<constructor-arg value="salary"/>
</bean>
</list>
</property>
</bean>
</list>
</property>
If you configure that, you'll get an ability to enlist and query that tables over SQLine. (Please note, that you cannot query data that are not loaded into Ignite. To load them, you may use IgniteCache.get() with enabled readThrough option or IgniteCache.loadCache() to load everything from Cassandra table).
To query Cassandra with JDBC, you need a JDBC driver for it, try, for example DBSchema.
I am novice in Ignite and trying to utilize Ignite to setup in-memory cache. I did some basic configuration and started the Ignite based pluggable persistence work on single node. Now, I am planning to test the performance on 2 node cluster and setting up the ignite configuration as per below:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean id="countryCacheStoreFactory" class="javax.cache.configuration.FactoryBuilder" factory-method="factoryOf">
<constructor-arg><value>com.xyz.exploreignite.cache.CustomCacheStore</value></constructor-arg>
</bean>
<bean id="stateCacheStoreFactory" class="javax.cache.configuration.FactoryBuilder" factory-method="factoryOf">
<constructor-arg><value>com.xyz.exploreignite.cache.CustomStateCacheStore</value></constructor-arg>
</bean>
<bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="peerClassLoadingEnabled" value="false"/>
<property name="clientMode" value="true"/>
<property name="gridName" value="clusterGrid"/>
<property name="cacheConfiguration">
<list>
<!-- Partitioned cache example configuration (Atomic mode). -->
<bean class="org.apache.ignite.configuration.CacheConfiguration">
<property name="atomicityMode" value="ATOMIC"/>
<property name="backups" value="1"/>
<property name="name" value="customCountryCache"/>
<property name="readThrough" value="true"/>
<property name="writeThrough" value="true"/>
<property name="cacheMode" value="PARTITIONED"/>
<property name="writeBehindEnabled" value="true"/>
<property name="copyOnRead" value="true"/>
<property name="memoryMode" value="OFFHEAP_TIERED"/>
<property name="atomicWriteOrderMode" value="PRIMARY"/>
<property name="indexedTypes" >
<list>
<value>java.lang.Integer</value>
<value>com.xyz.exploreignite.pojo.Country</value>
</list>
</property>
<!-- Cache store. -->
<property name="cacheStoreFactory" ref="countryCacheStoreFactory"/>
</bean>
<bean class="org.apache.ignite.configuration.CacheConfiguration">
<property name="atomicityMode" value="ATOMIC"/>
<property name="backups" value="1"/>
<property name="name" value="customStateCache"/>
<property name="readThrough" value="true"/>
<property name="writeThrough" value="true"/>
<property name="cacheMode" value="PARTITIONED"/>
<property name="writeBehindEnabled" value="true"/>
<property name="copyOnRead" value="true"/>
<property name="memoryMode" value="OFFHEAP_TIERED"/>
<property name="atomicWriteOrderMode" value="PRIMARY"/>
<property name="indexedTypes" >
<list>
<value>java.lang.Integer</value>
<value>com.xyz.exploreignite.pojo.State</value>
</list>
</property>
<!-- Cache store. -->
<property name="cacheStoreFactory" ref="stateCacheStoreFactory"/>
</bean>
</list>
</property>
<!-- Explicitly configure TCP discovery SPI to provide list of initial nodes. -->
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<!--
Ignite provides several options for automatic discovery that can be used
instead os static IP based discovery. For information on all options refer
to our documentation: http://apacheignite.readme.io/docs/cluster-config
-->
<!-- Uncomment static IP finder to enable static-based discovery of initial nodes. -->
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
<!--<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">-->
<property name="addresses">
<list>
<value>127.0.0.1:47500..47509</value>
<value>172.26.49.1:47500..47509</value>
<!-- In distributed environment, replace with actual host IP address. -->
<value>172.26.49.2:47500..47509</value>
</list>
</property>
</bean>
</property>
</bean>
</property>
</bean>
Now, while starting ignite with "bin/ignite.sh " on both the nodes it shows failed to connect to server. While running only "bin/ignite.sh" in parallel to above one, both the individual ignite config instance starts in standalone mode with only 1 client. I need to have both of them utilizing the shared instance. Please suggest possible issues in my deployment/execution.
Try to remove <value>127.0.0.1:47500..47509</value> line from the discovery configuration. It doesn't make much for a distributed cluster.
That configuration is in <property name="clientMode" value="true"/> You need to have at least one node set to <property name="clientMode" value="false"/> or else your clients won't have any server nodes to connect to.