I am newbie to Cassandra. I wanted to install cassandra-0.8.4 on 3 nodes and to run Map/Reduce job that uploads data from HDFS to Cassandra.
I have installed Cassnadra on 3 nodes lab02(199.168.0.2),lab03(199.168.0.3) & lab04(199.168.0.4) respectively and can create a keyspace & column family and they got distributed across the cluster.
When I run my map/reduce program it ended up with "UnknownHostException". the same map/reduce program works well on single node cluster.
Here are the steps which I have followed.
cassandra.yaml details
lab02(199.168.0.2): (seed node)
auto_bootstrap: false seeds: "199.168.0.2" listen_address: 199.168.0.2
rpc_address: 199.168.0.2
lab03(199.168.0.3): auto_bootstrap: true seeds: "199.168.0.2"
listen_address: 199.168.0.3 rpc_address: 199.168.0.3
lab04(199.168.0.4): auto_bootstrap: true seeds: "199.168.0.2"
listen_address: 199.168.0.4 rpc_address: 199.168.0.4
2.
When I run my map/reduce program it ended up with "UnknownHostException"
Error: java.net.UnknownHostException: /199.168.0.2 at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:849) at java.net.InetAddress.getAddressFromNameService(InetAddress.java:1200) at java.net.InetAddress.getAllByName0(InetAddress.java:1153) at java.net.InetAddress.getAllByName(InetAddress.java:1083) at java.net.InetAddress.getAllByName(InetAddress.java:1019) at java.net.InetAddress.getByName(InetAddress.java:969) at org.apache.cassandra.client.RingCache.refreshEndpointMap(RingCache.java:93) at org.apache.cassandra.client.RingCache.(RingCache.java:67) at org.apache.cassandra.hadoop.ColumnFamilyRecordWriter.(ColumnFamilyRecordWriter.java:98) at org.apache.cassandra.hadoop.ColumnFamilyRecordWriter.(ColumnFamilyRecordWriter.java:92) at org.apache.cassandra.hadoop.ColumnFamilyOutputFormat.getRecordWriter(ColumnFamilyOutputFormat.java:132) at org.apache.cassandra.hadoop.ColumnFamilyOutputFormat.getRecordWriter(ColumnFamilyOutputFormat.java:62) at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408) at org.apache.hadoop.mapred.Child.main(Child.java:170)
Here are the config line for map/reduce.
job4.setReducerClass(TblUploadReducer.class );
job4.setOutputKeyClass(ByteBuffer.class);
job4.setOutputValueClass(List.class);
job4.setOutputFormatClass(ColumnFamilyOutputFormat.class);
ConfigHelper.setOutputColumnFamily(job4.getConfiguration(), args[1],args[3] );
ConfigHelper.setRpcPort(job4.getConfiguration(), args[7]); // 9160
ConfigHelper.setInitialAddress(job4.getConfiguration(), args[9]); // 199.168.0.2
ConfigHelper.setPartitioner(job4.getConfiguration(), "org.apache.cassandra.dht.RandomPartitioner");
Steps which I have verified are
There is a passwordless ssh has been configured b/w lab02,lab03
&lab04. All the nodes can ping each other with out any issues.
When I ran "InetAddress.getLocalHost()" from java program on lab02 it
prints "lab02/199.168.0.2".
When I over looked "o/p" of bin/cassandra it prints couple of
messages and under InetAddress field "/199.168.0.3" etc. Here it does
not print "hostname/IP". Is that problem?
Kindly help me to resolve above issue.
Regards,
Thamizhannal
This has been resolved already in 0.8 tip: https://issues.apache.org/jira/browse/CASSANDRA-3044
It's trying to look up the literal string "/199.168.0.2" as an address, which is indeed invalid. If this is coming from your configuration, remove the slash. Otherwise, it might be coming from RingCache which would be a bug. If so, first upgrade to 0.7.8 to make sure it's not already fixed. If the problem persists, open a bug on https://issues.apache.org/jira/browse/CASSANDRA.
Edit: Nate is right, this is a known bug in 0.8.4 that is fixed for 0.8.5.
Related
I am using ubuntu 14.04 with apache cassandra 3.7. I am trying to start it but get the following error message:
ERROR [main] 2016-07-15 15:22:10,627 CassandraDaemon.java:731 - Cannot start node if snitch's data center (dc1) differs from previous data center (datacenter1). Please fix the snitch configuration, decommission and rebootstrap this node or use the flag -Dcassandra.ignore_dc=true.
I know I can set -Dcassandra.ignore_dc=true, BUT that is not a fix, its a band-aid and for development use only, this is suppose to be in production. I tried to clear out all the files and folders in /var/lib/cassandra, I MEAN every SINGLE FILE AND FOLDER, started apache cassandra again, AND STILL THE SAME ERROR MESSAGE... any other idea??
change in file:
/etc/cassandra/cassandra-rackdc.properties
entry from dc1 to datacenter1
on all nodes
and then do a rolling restart of nodes.
If have just switched to GossipingPropertyFileSnitch, start Cassandra with the option
-Dcassandra.ignore_dc=true
If it starts successfully, execute:
nodetool repair
nodetool cleanup
Afterwards, Cassandra should be able to start normally without the ignore option.
I faced the issue while upgrading my Apache cassandra from 3.11.1 to 3.11.4 .
cassandra.yaml
old_Config : endpoint_snitch: GossipingPropertyFileSnitch
New_Config: endpoint_snitch: SimpleSnitch
{changed it to GossipingPropertyFileSnitch}
cassandra-rackdc.properties
old_version_config: dc:Dc1 rack:Rack1
New_version_config: dc:dc rack:rack (changed this to Dc1 and Rack1)
this resolves my issue
I am trying to start cqlsh and this is what I get:
/bin$ ./cqlsh
Connection error: ('Unable to connect to any servers', {'127.0.0.1':
OperationTimedOut('errors=None, last_host=None',)})
I tried removing ~/.cassandra, did not work. I also compared cassandra.yaml with a version that worked.
Any ideas?
posting on this old thread as a memo for others, as i couln't managed to find any info to resolve these symptoms without debugging up to now:
got the same issue with a slow testing cluster, and i resolved it by setting a missing control_connection_timeout kwargs in Cluster() init, in cqlsh.py file.
issue opened and patch proposal provided at https://issues.apache.org/jira/browse/CASSANDRA-10959
Depending on your version and configuration, check the values specified for listen_address and/or rpc_address in your cassandra.yaml. If they are defined to anything other than localhost, you will need to provide that address when connecting with cqlsh.
$ grep listen_address: /etc/cassandra/cassandra.yaml
listen_address: 210.156.89.15
$ cqlsh 210.156.89.15 -u aploetz -p aploetz
Connected to PermanentWaves at 210.156.89.15.
[cqlsh 5.0.1 | Cassandra 2.1.4 | CQL spec 3.2.0 | Native protocol v3]
Use HELP for help.
aploetz#cqlsh>
I am trying to configure two windows servers in my network as Cassandra cluster.
I did some reading in various sites and changed the below in Cassandra.yalm
after changing the default value of 127.0.0.1 to actual IP the Cassandra service is not starting.
I also added the map to actual IP to localhost in (windows) hosts file.
After doing the above change, the service is coming up when I start the service. it is stopping immediately.
The reason I am changing this IP is to make this a cluster with two node setup,
Please let me know if I miss some thing.
Version: Datastax community version of Cassandra
Server : windows.
Thx
Muthu
Message from Cassandra.txt in logs dir:
ERROR [main] 2014-09-18 11:43:12,155 DatabaseDescriptor.java (line 116) Fatal configuration error
org.apache.cassandra.exceptions.ConfigurationException: Invalid yaml Caused by: Can't construct a java object for tag:yaml.org,2002:org.apache.cassandra.config.Config; exception=Cannot create property=seed_provider for JavaBean=org.apache.cassandra.config.Config#34e5190a; No suitable constructor with 2 arguments found for class org.apache.cassandra.config.SeedProviderDef in 'reader', line 8, column 1: cluster_name: 'Test Cluster'
If you want to create Cassandra cluster you must have at least two nodes and configure /etc/cassandra/cassandra.yaml
cassandra.yaml
cluster_name: 'Some Cluster Name'
listen_address: [Current IP]
rpc_address: [Current IP]
seed_provuder:
- seeds: "[Current IP], [Remote IP]"
Note: seeds must have at least two IPs which must be reachable for each other
Clean and start Cassandra instance
sudo rm -rf /var/lib/cassandra/* /var/log/cassandra/*
Note: Cassandra instance must be killed before cleaning those folders.
I have a problem with a single node Cassandra installation.
I can start it without any errors in the log.
I can create a keyspace, create tables, insert and delete data.
However truncate is not working
cqlsh> CREATE KEYSPACE mykeyspace WITH REPLICATION = {'class' : 'SimpleStrategy', 'replication_factor': 1};
cqlsh> use mykeyspace;
cqlsh:mykeyspace> create table test1 (num int, primary key (num));
cqlsh:mykeyspace> insert into test1 (num) values (12);
cqlsh:mykeyspace> select * from test1;
num
-----
12
(1 rows)
cqlsh:mykeyspace> truncate test1;
Unable to complete request: one or more nodes were unavailable.
Also if I try to run nodetool describecluster it doesn't return complete response
[XXXX#XXXX dsc-cassandra-2.0.6]$ ./bin/nodetool describecluster
Cluster Information:
Name: Test Cluster
Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
Schema versions:
UNREACHABLE: [127.0.0.1]
I'm using
Cassandra DSC 2.0.6.
Red Hat 5.8.
java version "1.7.0_51"
Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)
I get responses for ping 127.0.0.1 and ping localhost
I checked all the ports that I am aware of cassandra may need (7000, 9160, 7199, 9042) using telnet - for example
telnet 127.0.0.1 7199
telnet localhost 7199
I can connect to these ports.
I'm using the default cassandra.yaml. These are the lines where either IP or hostname shows up
listen_address: localhost
rpc_address: localhost
seed_provider:
- class_name: org.apache.cassandra.locator.SimpleSeedProvider
parameters:
- seeds: "127.0.0.1"
I also looked into the source code. I believe the problem can be close to the method org.apache.cassandra.service.StorageProxyMBean.describeSchemaVersions(). Most likely I get no response to the SCHEMA_CHECK message.
I tried to enable TRACE log in log4j for nodetool (conf/log4j-tools.properties) to get more information about the issue, but somehow log4j didn't start logging (it did create the file that I set in the appender, but the file was empty.)
There must be something specific to this environment because I can't repeat this problem in any other environments. So I can't figure out what's causing it.
The problem was that Cassandra couldn't load snappy.
org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] null
at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:239)
at org.xerial.snappy.Snappy.<clinit>(Snappy.java:48)
at org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
at org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
at org.apache.cassandra.net.OutboundTcpConnection.connect(OutboundTcpConnection.java:359)
at org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:150)
I turned off compression in cassanda.yaml
internode_compression: none
Now both nodetool describecluster and I truncate work.
I also found a similar post here Cassandra Startup Error 1.2.6 on Linux x86_64
Since I can't install another glibc on this machine for the sake of testing I downloaded snappy-java-1.0.4.1.jar and replaced libsnappyjava.so in my snappy-java-1.0.5.jar
With this jar I was able to run cassandra with
internode_compression: all
(I have glibc 2.5 installed)
I have a cluster running on datastax-cassandra 1.2.5, it works fine, because of vnodes adn leveled compaction strategy issue i tried promoting it to 1.2.6.
So upgrading involved -
1 - stopping all the nodes
2 - deleting 1.2.5 rpm
3 - installing 1.2.6 rpm
4 - fixing cassandra.yaml
5 - starting cassandra.
Problem Statement - The problem now is that all the nodes are up and running, but not in one cluster. They all are operating in their own cluster even though the seeds in yaml points to the original seed.
nodetool status also just shows the one node (the node on which we are on)
system log shows one error
ERROR [WRITE-/10.93.3.46] 2013-10-21 19:43:29,101 CassandraDaemon.java (line 192)
Exception in thread Thread[WRITE-/10.10.10.10,5,main]
java.lang.NoClassDefFoundError: Could not initialize class org.xerial.snappy.Snappy
at org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
at org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
at
org.apache.cassandra.net.OutboundTcpConnection.connect(OutboundTcpConnection.java:351)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:143)
**** 10.10.10.10 is the seed ip
Any help on how to pass through it
Try to set the internode_compression to none. It will disable compression between nodes, which is failing because snappy cannot initialize
internode_compression: none