Unable to Connect Sqoop to Oracle TimesTen through JDBC on Linux - linux

I have installed Timesten database (full version) on linux (Linux is guest OS installed through Oracle viritual box with cloudera VM)
I am trying to run following sqoop command on linux and getting below errors
command
sqoop list-tables --connect jdbc:timesten:direct:dsn=sampledb_1122 --driver com.timesten.jdbc.TimesTenDriver
**error**
ERROR manager.SqlManager: Error reading database metadata: java.sql.SQLException: Problems with loading native library/missing methods: no ttJdbc in java.library.path
java.sql.SQLException: Problems with loading native library/missing methods: no ttJdbc in java.library.path
at com.timesten.jdbc.JdbcOdbcConnection.connect(JdbcOdbcConnection.java:1809)
at com.timesten.jdbc.TimesTenDriver.connect(TimesTenDriver.java:305)
at com.timesten.jdbc.TimesTenDriver.connect(TimesTenDriver.java:161)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:233)
at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:878)
at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
at org.apache.sqoop.manager.SqlManager.listTables(SqlManager.java:520)
at org.apache.sqoop.tool.ListTablesTool.run(ListTablesTool.java:49)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
Could not retrieve tables list from server
18/02/18 18:56:04 ERROR tool.ListTablesTool: manager.listTables() returned null
TimesTen bin and lib folder location
/home/cloudra/timesten/TimesTen/tt1122_64/bin
/home/cloudera/timesten/TimesTen/tt1122_64/lib
Following values are setup in my environment and other parameters
USERNAME=cloudera
DESKTOP_SESSION=gnome
MAIL=/var/spool/mail/cloudera
PATH=/var/lib/sqoop:/home/cloudera/timesten/TimesTen/tt1122_64/bin:/home/cloudera/timesten/TimesTen/tt1122_64/lib:/home/cloudera/anaconda3/bin:/var/lib/sqoop:/home/cloudra/timesten/TimesTen/tt1122_64/bin:/home/cloudera/timesten/TimesTen/tt1122_64/lib:/home/cloudera/anaconda3/bin:/home/cloudera/anaconda3/bin:/usr/local/firefox:/sbin:/usr/java/jdk1.7.0_67-cloudera/bin:/usr/local/apache-ant/apache-ant-1.9.2/bin:/usr/local/apache-maven/apache-maven-3.0.4/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cloudera/bin
PWD=/home/cloudera
THREAD_FLAGS=native
HOME=/home/cloudera
SHLVL=2
M2_HOME=/usr/local/apache-maven/apache-maven-3.0.4
GNOME_DESKTOP_SESSION_ID=this-is-deprecated
LOGNAME=cloudera
CVS_RSH=ssh
CLASSPATH=/home/cloudera/timesten/TimesTen/tt1122_64/lib/ttjdbc6.jar
[cloudera#quickstart ~]$ echo $LD_LIBRARY_PATH
/home/cloudera/timesten/TimesTen/tt1122_64/lib:/home/cloudera/timesten/TimesTen/tt1122_64/lib:
[cloudera#quickstart ~]$ java -version
java version "1.7.0_67"
Java(TM) SE Runtime Environment (build 1.7.0_67-b01)
Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)
[cloudera#quickstart ~]$
cloudera#quickstart bin]$ ./ttversion
TimesTen Release 11.2.2.8.0 (64 bit Linux/x86_64) (tt1122_64:53396) 2015-01-20T08:36:31Z
Instance admin: cloudera
Instance home directory: /home/cloudera/timesten/TimesTen/tt1122_64
World accessible
Daemon home directory: /home/cloudera/timesten/TimesTen/tt1122_64/info
PL/SQL enabled.
In addition to above.. the ttjdbc6.jar file is located at following location
[cloudera#quickstart sqoop]$ pwd
/var/lib/sqoop
[cloudera#quickstart sqoop]$ ls -ltr
total 0
lrwxrwxrwx 1 root root 40 Jun 9 2015 mysql-connector-java.jar -> /usr/share/java/mysql-connector-java.jar
lrwxrwxrwx 1 root root 58 Feb 16 21:37 ttjdbc6.jar -> /home/cloudera/timesten/TimesTen/tt1122_64/lib/ttjdbc6.jar
[cloudera#quickstart timesten]$ pwd
/usr/lib/timesten
[cloudera#quickstart timesten]$ ls -ltr
total 276
-rwxrwxrwx 1 root root 279580 Feb 18 11:33 ttjdbc6.jar
Java_library_path output
[cloudera#quickstart timesten]$ java -XshowSettings:properties
Property settings:
awt.toolkit = sun.awt.X11.XToolkit
file.encoding = UTF-8
file.encoding.pkg = sun.io
file.separator = /
java.awt.graphicsenv = sun.awt.X11GraphicsEnvironment
java.awt.printerjob = sun.print.PSPrinterJob
java.class.path = /home/cloudera/timesten/TimesTen/tt1122_64/lib/ttjdbc6.jar
java.class.version = 51.0
java.endorsed.dirs = /usr/java/jdk1.7.0_67-cloudera/jre/lib/endorsed
java.ext.dirs = /usr/java/jdk1.7.0_67-cloudera/jre/lib/ext
/usr/java/packages/lib/ext
java.home = /usr/java/jdk1.7.0_67-cloudera/jre
java.io.tmpdir = /tmp
java.library.path = /home/cloudera/timesten/TimesTen/tt1122_64/lib
/home/cloudera/timesten/TimesTen/tt1122_64/lib
/usr/java/packages/lib/amd64
/usr/lib64
/lib64
/lib
/usr/lib
java.runtime.name = Java(TM) SE Runtime Environment
java.runtime.version = 1.7.0_67-b01
I execute ttenv.sh scripts but it is not setting up any parameter when I check env parameters, so I had to do it manually.
Gurus and experts.. please help me here.. not sure what is the issue and why I am getting the above error.
Thanks for your help..

The key line here is this:
java.sql.SQLException: Problems with loading native library/missing methods:
no ttJdbc in java.library.path
The TimesTen JDBC driver is a type 1 / 2 driver and it relies on the underlying TimesTen native libraries. Specifically it needs several shared libraries located in <TimesTen_install_dir>/lib such as libttJdbc.so (the one that the error is complaining about), libtten.so etc. Typically you need to make sure that the java.library.path includes this directory (which it appears is the case) and that the CLASSPATH includes the ttjdbc7.jar file in that directory. Another possibility is that your TimesTen installation is a 'client only' installation in which case you cannot use the 'direct' driver and if you try to do so then you would get this exact error. I suggest checking to see if you actually have the files libttJdbc.so and libtten.so in <TimesTen_install_dir>/lib and if not, then this means you have a client only install and need to configure / use client/server connectivity instead.

Related

Unable to locate JVM fatal error log file (hs_err_pid.log) after Dataproc Spark Job crash

After Apache Spark Executor JVM crash in C++ library I'm unable to locate hs_err_pid.log file, that is specified in the Executor JVM output log. Here's an example of Executor output log:
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007f6326dce8b0, pid=28580, tid=0x00007f630ea57700
#
# JRE version: OpenJDK Runtime Environment (8.0_212-b01) (build 1.8.0_212-8u212-b01-1~deb9u1-b01)
# Java VM: OpenJDK 64-Bit Server VM (25.212-b01 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C [libessence-jni.so+0x18b0] Java_com_evernote_service_nts_indexer_lib_Essence_EssProcess+0x0
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /hadoop/yarn/nm-local-dir/usercache/root/appcache/application_1559573462307_0002/container_1559573462307_0002_01_000005/hs_err_pid28580.log
10:50:00:[32m562[m [Executor task launch worker for task 41] [32mINFO[m .....NtsLibInternalIndexerProcessor(NtsLibInternalIndexerProcessor.java:50) [32mprocess [m Process for user: 18432
[thread 140063422109440 also had an error]
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
But when I'm SSH to target worker machine to locate hs_err_pid28580.log I can't find any traces of this file. I've tried:
vglazkov#reindex-cluster-vg-w-0:~$ sudo find / -name hs_err_pid28580.log
vglazkov#reindex-cluster-vg-w-0:~$
vglazkov#reindex-cluster-vg-w-0:~$ sudo ls -la /hadoop/yarn/nm-local-dir/usercache/root/appcache/
total 12
drwx--x--- 3 yarn yarn 4096 Jun 4 10:46 .
drwxr-x--- 4 yarn yarn 4096 May 15 15:47 ..
drwx--x--- 3 yarn yarn 4096 Jun 4 10:48 application_1557935076075_0097
But in the last case directory named application_1557935076075_0097 does not match my applicationId application_1559573462307_0002 and does not contain any hs_err_pid.log files

Spark master and worker seem to run on different JVM version

In standalone mode master process uses /usr/bin/java which resolves to JVM 1.8 and worker process /usr/lib/jvm/java/bin/java which resolves to 1.7. In my Spark application I'm using some APIs introduced in 1.8.
Looking at stack trace one line that comes up is: Caused by: java.lang.NoClassDefFoundError: Could not initialize class SomeClassDefinedByMe which internally creates instance from java.time which I believe is only in JDK 1.8.
How do I force worker to use JVM 1.8?
Update:
For now I renamed /usr/lib/jvm/java/bin/java and created a link that points to /usr/bin/java. This solved the problem but still would like to know why both processes use different binary location and where is this set.
On each Worker node, edit ${SPARK_HOME}/conf/spark-env.sh and define the appropriate $JAVA_HOME e.g.
export JAVA_HOME=/usr/bin/java
That file is sourced by ${SPARK_HOME}/bin/load-spark-env.sh which is invoked by each and every Spark command-line utility:
${SPARK_HOME}/bin/spark-shell via ${SPARK_HOME}/bin/spark-class
${SPARK_HOME}/bin/spark-submit via ${SPARK_HOME}/bin/spark-class
...
${SPARK_HOME}/sbin/start-slave.sh
...
Side note: the Linux alternatives are the standard way to define which JVM is on top of your PATH...
Typical setup with a "fixed" setting, not relying on the priority set by the OpenJDK RPM install:
$ ls -AFl $(which java)
lrwxrwxrwx. 1 root root 22 Feb 15 16:06 /usr/bin/java -> /etc/alternatives/java*
$ alternatives --display java | grep -v slave
java - status is manual.
link currently points to /usr/java/jdk1.8.0_92/jre/bin/java
/usr/lib/jvm/jre-1.8.0-openjdk.x86_64/bin/java - priority 18091
/usr/lib/jvm/jre-1.6.0-openjdk.x86_64/bin/java - priority 16000
/usr/java/jdk1.8.0_92/jre/bin/java - priority 18092
Current `best' version is /usr/java/jdk1.8.0_92/jre/bin/java.
...provided that $PATH is defined properly for the Linux account that launches the Spark slaves!

PHP exec(myexe) fails in PHP App, but not CLI. Fails Running Under User "apache"

I have a custom program (e.g. myexe) being executed by a web app using PHP's exec() function. It does not fail when run using the PHP CLI nor does myexe fail when run from the command line with me as a user. I have built myexe so that there are no memory issues when profiled using valgrind. myexe is about 26MB in size.
To simplify the situation, I have run myexe on the command line under the user 'apache' and reproduced the failure.
su -s /bin/sh apache -c "/usr/local/bin/myexe parm1 parm2..."
==> Segmentation fault (core dumped)
BUT when I change the user to myself and run the same command above, it works.
su -s /bin/sh mike -c "/usr/local/bin/myexe parm1 parm2..."
==> WORKS
Here's the error from the system log file:
Jul 9 18:26:15 DEVSTN-1 kernel: myexe[27352]: segfault at 7fffa2bf9ff8 ip 0000000000410324 sp 00007fffa2bfa000 error 6 in myexe[400000+5ae000]
Jul 9 18:26:16 DEVSTN-1 abrt[27353]: Saved core dump of pid 27352 (/usr/local/bin/myexe) to /var/spool/abrt/ccpp-2015-07-09-18:26:15-27352 (13631488 bytes)
Jul 9 18:26:16 DEVSTN-1 abrtd: Directory 'ccpp-2015-07-09-18:26:15-27352' creation detected
Jul 9 18:26:17 DEVSTN-1 abrtd: Executable '/usr/local/bin/myexe' doesn't belong to any package and ProcessUnpackaged is set to 'no'
Jul 9 18:26:17 DEVSTN-1 abrtd: 'post-create' on '/var/spool/abrt/ccpp-2015-07-09-18:26:15-27352' exited with 1
Jul 9 18:26:17 DEVSTN-1 abrtd: Deleting problem directory '/var/spool/abrt/ccpp-2015-07-09-18:26:15-27352'
My configuration:
CentOS6 2.6.32-504.23.4.el6.x86_64
Apache/2.2.15 (CentOS)
PHP Version 5.3.3
Am I correct with assuming that PHP has nothing to do with the error?
What should I do next?
Correct; PHP has nothing to do with the error. This is a segmentation fault caused by invalid memory access (either overflowing a buffer, or accessing already-freed memory) in myexe. It seems to have saved a core dump to /var/spool/abrt/ccpp-2015-07-09-18:26:15-27352, so, try debugging with GDB:
gdb /usr/local/bin/myexe -c /var/spool/abrt/ccpp-2015-07-09-18:26:15-27352
(gdb) bt
And try to see where the executable is failing. To get useful output, it will need to be compiled with debugging symbols. If it doesn't fail running as root or a different user, or running in an interactive terminal, I'd look for bugs that could be triggered by being unable to open a file, unable to read an expected environment variable, etc. to help isolate your problem.
Running the executable under strace might help figure out what's going on as well.
Found the problem by entering a bash shell user user apache and running the program using gdb.
Turns out myexe was trying to create a directory under the user's home dir (/home/apache) which doesn't exist.
What helped me was knowing how to start a shell under a different user and using gdb.
Here's the command to start a shell under another user (apache):
su -s /bin/bash apache

[01000][unixODBC][Driver Manager]Can't open lib '/usr/local/easysoft/oracle/InstantClient112/lib/libsqora.so' : file not found

I am trying to acces oracle from linux server. I am using unixODBC.
When i try to acces oracle using isql and i get error that driver manager can't open libsqora.so.12.1.
odbc.ini
[NAME]
Application Attributes = T
Attributes = W
BatchAutocommitMode = IfAllSuccessful
BindAsFLOAT = F
CloseCursor = F
DisableDPM = F
DisableMTS = T
Driver = Oracle 11g ODBC driver
DSN = DSN_NAME
EXECSchemaOpt =
EXECSyntax = T
Failover = T
FailoverDelay = 10
FailoverRetryCount = 10
FetchBufferSize = 64000
ForceWCHAR = F
Lobs = T
Longs = T
MaxLargeData = 0
MetadataIdDefault = F
QueryTimeout = T
ResultSets = T
ServerName = ServerName
SQLGetData extensions = F
Translation DLL =
Translation Option = 0
DisableRULEHint = T
UserID = xxxx
Password=<password>
StatementCache=F
CacheBufferSize=20
UseOCIDescribeAny=F
odbcinst.ini
[Oracle 11g ODBC driver]
Description=Oracle ODBC driver for Oracle 11g
Driver=/usr/local/easysoft/oracle/InstantClient112/lib/libsqora.so
FileUsage=1
Then, when i use isql to acces oracle i get the following error:
[root#xxxxx lib]# isql -v NAME
[01000][unixODBC][Driver Manager]Can't open lib '/usr/local/easysoft/oracle/InstantClient112/lib/libsqora.so' : file not found
[ISQL]ERROR: Could not SQLConnect
I had typo in odbcinst.ini. I have corrected but still same error.
[root#xxxxx tmp]# isql -v NAME
[01000][unixODBC][Driver Manager]Can't open lib '/usr/local/easysoft/oracle/InstantClient112/lib/libsqora.so.11.1' : file not found
[ISQL]ERROR: Could not SQLConnect
[root#xxxxx tmp]# ls -l /usr/local/easysoft/oracle/InstantClient112/lib/libsqora.so.11.1
-rw-r--r-- 1 bin bin 996363 Sep 5 2010 /usr/local/easysoft/oracle/InstantClient112/lib/libsqora.so.11.1
[Oracle 11g ODBC driver]
Description=Oracle ODBC driver for Oracle 11g
Driver=/usr/local/easysoft/oracle/InstantClient112/lib/libsqora.so.11.1
FileUsage=1
ENV
[root#xxxxx tmp]# env
HOSTNAME=xxxxx
SSH2_TTY=/dev/pts/0
SHELL=/bin/bash
TERM=xterm
HISTSIZE=1000
ODBC_DIR=/usr/local/easysoft/unixODBC
OLDPWD=/usr/local/easysoft
SSH_SESSION_ID=1424
SSH_TTY=/dev/pts/0
LD_LIBRARY_PATH=/usr/local/easysoft/lib:/usr/local/easysoft/unixODBC/lib
A__z="*SHLVL
TNS_ADMIN=/usr/local/easysoft/oracle/InstantClient112/network/
INPUTRC=/etc/inputrc
PWD=/tmp
LANG=en_US.UTF-8
ODBCSYSINI=/etc/
HOME=/root
SHLVL=3
ODBCINI=/etc
LESSOPEN=|/usr/bin/lesspipe.sh %s
ORACLE_HOME=/usr/local/easysoft/oracle/InstantClient112/
G_BROKEN_FILENAMES=1
_=/bin/env
PATH=/usr/kerberos/sbin:/usr/kerberos/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/X11R6/bin:/usr/local/easysoft/unixODBC/bin:/opt/oraClient/11.2.0.4/bin
Fixed original issue with LD_LIBRARY_PATH updates but now it is broken again since I am trying to use the 32bit Oracle client.
Installed 32bit oracle client in directory /opt/oraClient/11.2.0.4_32/.
Modified the odbcinst.ini:
[Oracle 11g ODBC driver]
Description=Oracle ODBC driver for Oracle 11g
#Driver=/usr/local/easysoft/oracle/InstantClient112/lib/libsqora.so.11.1
Driver=/opt/oraClient/11.2.0.4_32/lib/libsqora.so.11.1
FileUsage=1
Error:
[root#xxxxx lib]# /usr/local/bin/isql -v NAME
[01000][unixODBC] [Driver Manager]Can't open lib '/opt/oraClient/11.2.0.4_32/lib/libsqora.so.11.1' : file not found
[ISQL]ERROR: Could not SQLConnect
If I need to use the 32bit Oracle client, what am i doing wrong...I know it is something in environmental variables.
[root#xxxxx lib]# file /opt/oraClient/11.2.0.4_32/lib/libsqora.so.11.1
/opt/oraClient/11.2.0.4_32/lib/libsqora.so.11.1: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), not stripped
I enable trace but am not able to attach document here. I can email.
More debug info:
[root#xxxxx bin]# ldd /opt/oraClient/11.2.0.4_32/lib/libsqora.so.11.1
ldd: warning: you do not have execution permission for `/opt/oraClient/11.2.0.4_32/lib/libsqora.so.11.1'
linux-gate.so.1 => (0xffffe000)
libdl.so.2 => /lib/libdl.so.2 (0xf7f2b000)
libm.so.6 => /lib/libm.so.6 (0xf7f02000)
libpthread.so.0 => /lib/libpthread.so.0 (0xf7ee8000)
libnsl.so.1 => /lib/libnsl.so.1 (0xf7ecf000)
libclntsh.so.11.1 => not found
libodbcinst.so.1 => not found
libc.so.6 => /lib/libc.so.6 (0xf7d71000)
/lib/ld-linux.so.2 (0x00134000)
I don't get one the "not found" which may be causing some problems:
[root#xxxxx bin]# ls /opt/oraClient/11.2.0.4_32/lib/libclntsh.so.11.1
/opt/oraClient/11.2.0.4_32/lib/libclntsh.so.11.1
Below is the most recent env output:
[root#xxxxx]# env
HOSTNAME=xxxxx
SSH2_TTY=/dev/pts/0
TERM=xterm
SHELL=/bin/bash
HISTSIZE=1000
ODBC_DIR=/usr/local/easysoft/unixODBC
SSH_TTY=/dev/pts/0
LD_LIBRARY_PATH=/opt/oraClient/11.2.0.4_32/:/opt/oraClient/11.2.0.4_32/lib/:/usr/local/easysoft/oracle/InstantClient112:/usr/local/easysoft/oracle/InstantClient112/lib/
TNS_ADMIN=/opt/oraClient/11.2.0.4_32/network/
PATH=/usr/kerberos/sbin:/usr/kerberos/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/X11R6/bin:/usr/local/easysoft/unixODBC/bin:/opt/oraClient/11.2.0.4_32/bin
INPUTRC=/etc/inputrc
LANG=en_US.UTF-8
ODBCSYSINI=/etc/
SHLVL=4
HOME=/root
ODBCINI=/etc
ORACLE_HOME=/opt/oraClient/11.2.0.4_32/
G_BROKEN_FILENAMES=1
_=/bin/env
This error is misleading:
The file that the error claimed was not present did exist.
I was able to chase down the actual missing file by executing:
# ldd /path/to/your/socalled/missing/file.so
This returned a series of files and one said:
libodbcinst.so.2 => not found
That is was the culprit for me. I then created the link that I needed and voila, the error went away.
ref: http://mailman.unixodbc.org/pipermail/unixodbc-support/2011-November/003018.html
Instant client doesn't have a lib directory. If you've downloaded the instantclient-odbc-linux package and unzipped that in the same location as the basic package then the libsqora.so.11.1 file will be durectly under /usr/local/easysoft/oracle/InstantClient112.
So your .ini should point to:
Driver=/usr/local/easysoft/oracle/InstantClient112/libsqora.so.11.1
Note the extension though; you might want to soft-link that to libsqora.so. If you've moved the .so files into a subdirectory after unzipping (?) then you'd need:
Driver=/usr/local/easysoft/oracle/InstantClient112/lib/libsqora.so.11.1
It looks like you've reoriganised the instant client files after unzipping, creating a lib directory. I'm not sure if that will directly cause you problems. But you also haven't set LD_LIBRARY_PATH, as the installation instructions suggest:
Installation Steps:
Download the appropriate Instant Client packages for your platform. All installations REQUIRE the Basic package.
Unzip the packages into a single directory such as "instantclient".
Set the library loading path in your environment to the directory in Step 2 ("instantclient"). On many UNIX platforms, LD_LIBRARY_PATH
is the appropriate environment variable. On Windows, PATH should be
used.
Start your application and enjoy.
Clearly step (4) isn't working for you yet, but I think that's because you haven't done step (3); and I think as you've split the files - which might itself cause an issue - you may need to include both .../InstantClient112 and .../InstantClient112/lib in the LD_LIBRARY_PATH variable. And make sure the modified value is exported. I'm not sure why you'd want to rearrange the files though.
sudo apt-get install libaio1 libaio-dev

cassandra connection error : Could not connect to localhost:9160

my cassandra was working well but suddenly it stop working!
when i use cqlsh command i get this error:
connection error : Could not connect to localhost:9160
and in output.log file i seeing this :
Service exit with a return value of 1
OpenJDK Client VM warning: Insufficient space for shared memory file:
/tmp/hsperfdata_cassandra/10963
Try using the -Djava.io.tmpdir= option to select an alternate temp location.
INFO 12:23:31,307 Logging initialized
log4j:ERROR Failed to flush writer,
java.io.IOException: No space left on device
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:297)
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:220)
at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:290)
at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:294)
at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:140)
at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
at org.apache.log4j.helpers.QuietWriter.flush(QuietWriter.java:59)
at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:324)
at org.apache.log4j.RollingFileAppender.subAppend(RollingFileAppender.java:276)
at org.apache.log4j.WriterAppender.append(WriterAppender.java:162)
at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
at org.apache.log4j.Category.callAppenders(Category.java:206)
at org.apache.log4j.Category.forcedLog(Category.java:391)
at org.apache.log4j.Category.info(Category.java:666)
at org.apache.cassandra.service.CassandraDaemon.initLog4j(CassandraDaemon.java:118)
at org.apache.cassandra.service.CassandraDaemon.<clinit>(CassandraDaemon.java:65)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
at java.lang.Class.newInstance0(Class.java:374)
at java.lang.Class.newInstance(Class.java:327)
at org.apache.commons.daemon.support.DaemonLoader.load(DaemonLoader.java:190)
INFO 12:23:31,332 32bit JVM detected. It is recommended to run Cassandra on a 64bit JVM for better performance.
INFO 12:23:31,335 JVM vendor/version: OpenJDK Client VM/1.6.0_27
WARN 12:23:31,335 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release
INFO 12:23:31,335 Heap size: 252641280/253689856
INFO 12:23:31,335 Classpath: /usr/share/cassandra/lib/antlr-3.2.jar:/usr/share/cassandra/lib/avro-1.4.0-fixes.jar:/usr/share/cassandra/lib/avro-1.4.0-sources-fixes.jar$
INFO 12:23:31,691 JNA mlockall successful
INFO 12:23:31,715 Loading settings from filService exit with a return value of 1
OpenJDK Client VM warning: Insufficient space for shared memory file:
can somebody help me? :(
The /tmp directory on your host isn't large enough for the temp files that cassandra wishes to make. The temp files are related to the amount of data in the system. As your database is larger now than it was in the past, it started before but it does not start now.
Check the status of the /tmp directory with df. Here is my system
$ df /tmp
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda6 570881944 350121276 191761552 65% /
To alter the place that is used for these temp files like the error says set java.io.tmpdir
On my system (Ubuntu Linux) this can be done by editing the end of the file /etc/cassandra/cassandra-env.sh
JVM_OPTS="$JVM_OPTS $JVM_EXTRA_OPTS -Djava.io.tmpdir=/opt"
Ensure that the new temp directory has sufficient space and that the permissions are correct, probably allowing read/write for the cassandra user would be enough

Resources