I created an AKS and I deployed the Apache Ignite service on it.
When I check the pods I can see they are working.
Also, I can get the load balancer IP.
I follow the official instructions of Apache and try to connect Ignite with ThinClient.
I share my code and instructions code.
Here is my code:
public void ConnectIgnite()
{
var cfg = new IgniteClientConfiguration
{
Endpoints = new[] { "20.101.12.***:10800" }
};
var client = Ignition.StartClient(cfg);
}
But my code is getting below errors;
System.AggregateException: 'Failed to establish Ignite thin client
connection, examine inner exceptions for details.'
Inner Exception ExtendedSocketException: A connection attempt failed
because the connected party did not properly respond after a period of
time, or established connection failed because connected host has
failed to respond.
and here is the Apache's instruction code;
ClientConfiguration cfg = new ClientConfiguration().setAddresses("13.86.186.145:10800");
IgniteClient client = Ignition.startClient(cfg);
ClientCache<Integer, String> cache = client.getOrCreateCache("test_cache");
cache.put(1, "first test value");
System.out.println(cache.get(1));
client.close();
Also, here is the official instruction link
I didn't understand what is wrong? Also, The Instruction says I don't need clientid and clientsecret but I don't want to connect without any security but this is completely another issue.
I found what is wrong. Apache's official page says: use below XML for the configuration.
<bean class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder">
<constructor-arg>
<bean class="org.apache.ignite.kubernetes.configuration.KubernetesConnectionConfiguration">
<property name="namespace" value="default" />
<property name="serviceName" value="ignite" />
</bean>
</constructor-arg>
</bean>
</property>
</bean>
</property>
</bean>
but,
first, that XML needs a beans tag at the top of XML.
Also, namespace value and serviceName value are not compatible with apache's official instruction page. If you follow the apache's page for the setup;
you have to use the below values
<property name="namespace" value="ignite" />
<property name="serviceName" value="ignite-service" />
instead of
<property name="namespace" value="default" />
<property name="serviceName" value="ignite" />
end of the changes your XML will look like
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder">
<constructor-arg>
<bean class="org.apache.ignite.kubernetes.configuration.KubernetesConnectionConfiguration">
<property name="namespace" value="ignite" />
<property name="serviceName" value="ignite-service" />
</bean>
</constructor-arg>
</bean>
</property>
</bean>
</property>
</bean>
</beans>
I changed my configuration XML and restart the pods with below command and It worked.
kubectl -n service rollout restart deployment ignite-cluster
Try connecting using 20.101.12.*** address from the outside.
Btw, what is 13.86.186.145, why are you trying to connect to it?
An update:
Seems like you just copy-pasted it from the docs, you need to replace it with your own values.
Related
I try to build Apache Ignite on Azure Kubernetes Service. AKS version is 1.19.
I followed the below instructions from the official apache page.
Microsoft Azure Kubernetes Service Deployment
But when I check the status of my pods, they seem failed. The status of pods is CrashLoopBackOff.
When I check the logs, It says the problem is node-configuration.xml
Here is the XML of the node configuration.
<bean class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder">
<constructor-arg>
<bean class="org.apache.ignite.kubernetes.configuration.KubernetesConnectionConfiguration">
<property name="namespace" value="default" />
<property name="serviceName" value="ignite" />
</bean>
</constructor-arg>
</bean>
</property>
</bean>
</property>
</bean>
Also, here is the output of the log.
PS C:\Users\kaan.akyalcin\ignite> kubectl logs ignite-cluster-755f6665c8-djdcn -n ignite
class org.apache.ignite.IgniteException: **Failed to instantiate Spring XML application context [springUrl=file:/ignite/config/node-configuration.xml, err=Line 1 in XML document from URL** [file:/ignite/config/node-configuration.xml] is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 71; cvc-elt.1: Cannot find the declaration of element 'bean'.]
at org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:1098)
at org.apache.ignite.Ignition.start(Ignition.java:356)
at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:367)
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to instantiate Spring XML application context [springUrl=file:/ignite/config/node-configuration.xml, err=Line 1 in XML document from URL [file:/ignite/config/node-configuration.xml] is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 71; cvc-elt.1: Cannot find the declaration of element 'bean'.]
at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:392)
at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:104)
at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:98)
at org.apache.ignite.internal.IgnitionEx.loadConfigurations(IgnitionEx.java:736)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:937)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:846)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:716)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:685)
at org.apache.ignite.Ignition.start(Ignition.java:353)
I didn't find the solution. How can I fix this problem?
You have to pass a valid XML file, looks like the docs need to be adjusted accordingly:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder">
<property name="namespace" value="degault"/>
<property name="serviceName" value="ignite"/>
</bean>
</property>
</bean>
</property>
</bean>
</beans>
I am trying to set Expiry Policy using xml config file:
<?xml version="1.0" encoding="UTF-8"?>
<!--
Ignite configuration with all defaults and enabled p2p deployment and enabled events.
-->
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:util="http://www.springframework.org/schema/util"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/util
http://www.springframework.org/schema/util/spring-util.xsd">
<bean abstract="true" id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
<!-- Set to true to enable distributed class loading for examples, default is false. -->
<property name="peerClassLoadingEnabled" value="true"/>
<!-- Enable task execution events for examples. -->
<property name="includeEventTypes">
<list>
<!--Task execution events-->
<util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_STARTED"/>
<util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_FINISHED"/>
<util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_FAILED"/>
<util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_TIMEDOUT"/>
<util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_SESSION_ATTR_SET"/>
<util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_REDUCED"/>
<!--Cache events-->
<util:constant static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_PUT"/>
<util:constant static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_READ"/>
<util:constant static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_REMOVED"/>
</list>
</property>
<!-- Explicitly configure TCP discovery SPI to provide list of initial nodes. -->
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<!--
Ignite provides several options for automatic discovery that can be used
instead os static IP based discovery. For information on all options refer
to our documentation: http://apacheignite.readme.io/docs/cluster-config
-->
<!-- Uncomment static IP finder to enable static-based discovery of initial nodes. -->
<!--<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">-->
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
<property name="addresses">
<list>
<!-- In distributed environment, replace with actual host IP address. -->
<value>127.0.0.1:47500..47509</value>
</list>
</property>
</bean>
</property>
</bean>
</property>
</bean>
<bean class="org.apache.ignite.configuration.CacheConfiguration">
<property name="setExpiryPolicyFactory" >
<bean class="javax.cache.expiry.CreatedExpiryPolicy">
<!---- Remove all data after 10 Seconds---->
<property name="expiryDuration" value="10000"/>
</bean>
</property>
<property name="eagerTtl" value="true"/>
</bean>
I am a bit new for Ignite. I walk through documentation but unable to find. Is any way to setExpiryPolicy into XMl?
Any help would be appreciated!
That requires a bit of Spring XML magic because there is no easy-to-use factory class for the CreatedExpiryPolicy (i.e. no CreatedExpiryPolicyFactory that you can just instantiate like a normal bean), but there is the CreatedExpiryPolicy::factoryOf that can be passed as factory-method:
<property name="expiryPolicyFactory">
<bean class="javax.cache.expiry.CreatedExpiryPolicy" factory-method="factoryOf">
<constructor-arg>
<bean class="javax.cache.expiry.Duration">
<constructor-arg value="SECONDS"/>
<constructor-arg value="10"/>
</bean>
</constructor-arg>
</bean>
</property>
Apache Ignite doesn`t support expiry policies and TTL for SQL tables.
An alternative would be to create cache with dynamically generated names based on desired time window.
For example you could append hour of the day if you need it hourly:
MY_CACHE_00
MY_CACHE_01
MY_CACHE_02
etc...
MY_CACHE_23
Then make your software write/read to the latest hour CACHE, and clear the oldest.
I'm trying to integrate Apache Ignite with Apache Cassandra(3.11.2) as I want to use Ignite to cache the data present in my already existing Cassandra database.
After going through the online resources, I've done the following till now:
Downloaded Apache Ignite.
Copied all the folders present in "libs/optional/" to "libs/"(I don't know which ones will be required for Cassandra).
Created 3 xmls in the config folder i.e. "cassandra-config.xml", "connection-settings.xml" and "persistance-settings.xml". Currently I'm using the same node(172.16.129.68) for both Cassandra and Ignite.
cassandra-config.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd">
<!-- Cassandra connection settings -->
<import resource="connection-settings.xml" />
<!-- Persistence settings for 'cache1' -->
<bean id="cache1_persistence_settings" class="org.apache.ignite.cache.store.cassandra.persistence.KeyValuePersistenceSettings">
<constructor-arg type="org.springframework.core.io.Resource" value="file:/home/cass/apache_ignite/apache-ignite-fabric-2.4.0-bin/config/persistance-settings.xml" />
</bean>
<!-- Ignite configuration -->
<bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="cacheConfiguration">
<list>
<!-- Configuring persistence for "cache1" cache -->
<bean class="org.apache.ignite.configuration.CacheConfiguration">
<property name="name" value="cache1"/>
<property name="readThrough" value="true"/>
<property name="writeThrough" value="true"/>
<property name="writeBehindEnabled" value="true"/>
<property name="cacheStoreFactory">
<bean class="org.apache.ignite.cache.store.cassandra.CassandraCacheStoreFactory">
<property name="dataSourceBean" value="cassandraAdminDataSource"/>
<property name="persistenceSettingsBean" value="cache1_persistence_settings"/>
</bean>
</property>
</bean>
</list>
</property>
<!-- Explicitly configure TCP discovery SPI to provide list of initial nodes. -->
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<!--
Ignite provides several options for automatic discovery that can be used
instead os static IP based discovery. For information on all options refer
to our documentation: http://apacheignite.readme.io/docs/cluster-config
-->
<!-- Uncomment static IP finder to enable static-based discovery of initial nodes. -->
<!--<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">-->
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
<property name="addresses">
<list>
<!-- In distributed environment, replace with actual host IP address. -->
<value>172.16.129.68:47500..47509</value>
</list>
</property>
</bean>
</property>
</bean>
</property>
</bean>
connection-settings.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean id="loadBalancingPolicy" class="com.datastax.driver.core.policies.TokenAwarePolicy">
<constructor-arg type="com.datastax.driver.core.policies.LoadBalancingPolicy">
<bean class="com.datastax.driver.core.policies.RoundRobinPolicy"/>
</constructor-arg>
</bean>
<bean id="cassandraAdminDataSource" class="org.apache.ignite.cache.store.cassandra.datasource.DataSource">
<property name="port" value="9042"/>
<property name="contactPoints" value="172.16.129.68"/>
<property name="readConsistency" value="ONE"/>
<property name="writeConsistency" value="ONE"/>
<property name="loadBalancingPolicy" ref="loadBalancingPolicy"/>
</bean>
persistance-settings.xml
<persistence keyspace="test" table="epc_table">
<keyPersistence class="java.lang.String" strategy="PRIMITIVE" column="imsi"/>
<valuePersistence strategy="BLOB"/>
</persistence>
I run the following command to start Ignite from bin folder.
ignite.sh ../config/cassandra-config.xml
Now, I want to take a look at the cassandra table via sqlline. I've tried the following:
./sqlline.sh -u jdbc:cassandra://172.16.129.68:9042/test //(test is the name of the keyspace)
I get the following output:
No known driver to handle "jdbc:cassandra://172.16.129.68:9042/test". Searching for known drivers...
java.lang.NullPointerException
sqlline version 1.3.0
0: jdbc:cassandra://172.16.129.68:9042/test>
I've also tried:
./sqlline.sh -u jdbc:ignite:thin://172.16.129.68
but when I use "!tables", I'm not able to see any table.
What exactly has been missing? How to access/modify the tables present in Cassandra using sqlline?
Operating System: RHEL 6.5
Apache Ignite is a key-value database and there are no tables created by default that you are able to view with JDBC connector. CacheStore is a way to integrate Ignite with external DB or any other storage, and it loads data as a key-value pair.
In your config you said Ignite that you want to store and load entries in/from Cassandra, but still Ignite doesn't know entries structure (BTW Ignite really doesn't care what objects were putted into it).
To be able to list tables and do queries on it, you need to create tables. For that you need to have ignite-indexing in /lib directory and set QueryEntity or indexed types if you have annotated POJOs. Here is example of such configuration:
<bean class="org.apache.ignite.configuration.CacheConfiguration">
<property name="name" value="mycache"/>
<!-- Configure query entities -->
<property name="queryEntities">
<list>
<bean class="org.apache.ignite.cache.QueryEntity">
<property name="keyType" value="java.lang.Long"/>
<property name="valueType" value="org.apache.ignite.examples.Person"/>
<property name="fields">
<map>
<entry key="id" value="java.lang.Long"/>
<entry key="orgId" value="java.lang.Long"/>
<entry key="firstName" value="java.lang.String"/>
<entry key="lastName" value="java.lang.String"/>
<entry key="resume" value="java.lang.String"/>
<entry key="salary" value="java.lang.Double"/>
</map>
</property>
<property name="indexes">
<list>
<bean class="org.apache.ignite.cache.QueryIndex">
<constructor-arg value="id"/>
</bean>
<bean class="org.apache.ignite.cache.QueryIndex">
<constructor-arg value="orgId"/>
</bean>
<bean class="org.apache.ignite.cache.QueryIndex">
<constructor-arg value="salary"/>
</bean>
</list>
</property>
</bean>
</list>
</property>
If you configure that, you'll get an ability to enlist and query that tables over SQLine. (Please note, that you cannot query data that are not loaded into Ignite. To load them, you may use IgniteCache.get() with enabled readThrough option or IgniteCache.loadCache() to load everything from Cassandra table).
To query Cassandra with JDBC, you need a JDBC driver for it, try, for example DBSchema.
I am working with Ignite and i want to load data from cassandra table. Ignite is working on local system and Cassandra working on server. Here is the config file.
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:util="http://www.springframework.org/schema/util"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/util
http://www.springframework.org/schema/util/spring-util.xsd">
<bean id="loadBalancingPolicy" class="com.datastax.driver.core.policies.TokenAwarePolicy">
<constructor-arg type="com.datastax.driver.core.policies.LoadBalancingPolicy">
<bean class="com.datastax.driver.core.policies.RoundRobinPolicy"/>
</constructor-arg>
</bean>
<util:list id="contactPoints" value-type="java.lang.String">
<value>90.43.255.66:9160</value>
</util:list>
<bean id="cassandraAdminDataSource" class="org.apache.ignite.cache.store.cassandra.datasource.DataSource">
<property name="contactPoints" ref="contactPoints"/>
<!-- <property name="user" value="user"/>
<property name="password" value="p#ssw0rd"/> -->
<property name="readConsistency" value="ONE"/>
<property name="writeConsistency" value="ONE"/>
<property name="loadBalancingPolicy" ref="loadBalancingPolicy"/>
</bean>
When i try to connect to Cassandra server it shows an error All host(s) tried for query failed (tried: 90.43.255.66:9160 (com.datastax.driver.core.exceptions.TransportException: [null] Cannot connect)). When i connect to local system by changing ip to 127.0.0.1 its works fine.
This looks like a connectivity issue. Possible reasons are incorrect address and/or port provided, firewall block, etc.
Does GridGain support SSL connection between each cluster member? If yes, can you show me how to do that?
Thanks,
Bill
GridGain supports SSL only for client connections (GridGain provides .NET and C++ thin clients), but not for communication between nodes.
To enable SSL for client connections, configure your server nodes like this:
<bean id="grid.cfg" class="org.gridgain.grid.GridConfiguration">
<!-- Enable REST. -->
<property name="restEnabled" value="true"/>
<!-- Client connection configuration. -->
<property name="clientConnectionConfiguration">
<bean class="org.gridgain.grid.GridClientConnectionConfiguration">
<!-- Enable SSL. -->
<property name="restTcpSslEnabled" value="true"/>
<!-- Provide SSL context factory (required). -->
<property name="restTcpSslContextFactory">
<bean class="org.gridgain.client.ssl.GridSslBasicContextFactory">
<property name="keyStoreFilePath" "keystore/server.jks"/>
<property name="keyStorePassword" value="123456"/>
<property name="trustStoreFilePath" "keystore/trust.jks"/>
<property name="trustStorePassword" value="123456"/>
</bean>
</property>
</bean>
</property>
</bean>
You will also need to provide SSL context factory on client configuration.
Ignite (and GridGain) allows you to use SSL for all possible connections.
To use SSL connection between nodes define property sslContextFactory in IgniteConfiguration.
<bean id="cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="sslContextFactory">
<bean class="org.apache.ignite.ssl.SslContextFactory">
<property name="keyStoreFilePath" value="keystore/server.jks"/>
<property name="keyStorePassword" value="123456"/>
<property name="trustStoreFilePath" value="keystore/trust.jks"/>
<property name="trustStorePassword" value="123456"/>
</bean>
</property>
</bean>
You can also check official security documentation.
https://apacheignite.readme.io/docs/ssltls