[Question posted by a user on YugabyteDB Community Slack]
I found lots of "IO error" in tserver's ERROR log:
E0622 18:12:44.155575 460196 tablet_metadata.cc:400] Failed to destroy regular DB at: /home/yugabyte/data/yb-data/tserver/data/rocksdb/table-00004200000030008000000000004254/tablet-71c67832a05b4effa7462823983c7e6a: IO error (yb/rocksdb/util/env_posix.cc:317): /home/yugabyte/data/yb-data/tserver/data/rocksdb/table-00004200000030008000000000004254/tablet-71c67832a05b4effa7462823983c7e6a/LOCK: No such file or directory
E0622 18:12:44.158552 460196 tablet_metadata.cc:400] Failed to destroy regular DB at: /home/yugabyte/data/yb-data/tserver/data/rocksdb/table-000042000000300080000000000042e4/tablet-ac80c3319e784abaa9eb93d6caca6faf: IO error (yb/rocksdb/util/env_posix.cc:317): /home/yugabyte/data/yb-data/tserver/data/rocksdb/table-000042000000300080000000000042e4/tablet-ac80c3319e784abaa9eb93d6caca6faf/LOCK: No such file or directory
E0622 18:12:44.163367 460196 tablet_metadata.cc:400] Failed to destroy regular DB at: /home/yugabyte/data/yb-data/tserver/data/rocksdb/table-000042000000300080000000000042b7/tablet-c156f2fc85154a4c8d835660aa1c5244: IO error (yb/rocksdb/util/env_posix.cc:317): /home/yugabyte/data/yb-data/tserver/data/rocksdb/table-000042000000300080000000000042b7/tablet-c156f2fc85154a4c8d835660aa1c5244/LOCK: No such file or directory
E0622 18:12:44.166888 460196 tablet_metadata.cc:400] Failed to destroy regular DB at: /home/yugabyte/data/yb-data/tserver/data/rocksdb/table-0000420000003000800000000000420c/tablet-48c3441b5a6f469c99407b8cc04b7d26: IO error (yb/rocksdb/util/env_posix.cc:317): /home/yugabyte/data/yb-data/tserver/data/rocksdb/table-0000420000003000800000000000420c/tablet-48c3441b5a6f469c99407b8cc04b7d26/LOCK: No such file or directory
I have checked disk and network, both of them work fine.
I can create/drop/select/update/insert through psql too.
Can I ignore these errors, or anything I can do to follow this issue?
If these logs only generated after tserver started, without repeating and everything works fine you can ignore them.
Related
I recently switched to ubuntu and when I installed and tried to create a connection on Oracle SQL Developers 22.2.1 I get a error
Status : Failure -Test failed: IO Error: The Network Adapter could not establish the connection
Ive tried changing the ports and switching between JDK 19,17 and 11 non of them seem to fix the error, all the guides that i can find on this error seem to be specifically for windows.
any help is appreciated and apologies if I have couldnt explain my problem clearly.
Testing connection of server should produce successful but its giving Status : Failure -Test failed: IO Error: The Network Adapter could not establish the connection
I have a server running node and mongoDB. Mongo is suddenly throwing errors, I managed to get it started upon rebooting the server but the error reappears upon trying to handle a request.
The error when I try to run mongo in a shell is:
MongoDB shell version v4.4.6
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
Error: couldn't connect to server 127.0.0.1:27017, connection attempt failed: SocketException: Error connecting to 127.0.0.1:27017 :: caused by :: Connection refused :
connect#src/mongo/shell/mongo.js:374:17
#(connect):2:6
exception: connect failed
exiting with code 1
My mongod.conf has the correct data path and I also tried setting it manually using mongod --dbpath and can confirm that it is correct.
mongod --repair doesn't work either.
Any ideas?
Turns out the server was out of storage. Running df showed that there was no disk space left.
I ended up adding a volume and moving my database and files there.
I am at my wits end!
I have a minimal install of Ubuntu Server 18.04 and OpenJDK 11 (headless).
Downloaded, to a local folder are the java 9+ binaries for Derby (db-derby-10.15.2.0-bin)
Path and Environment settings are all correct!
When I start the server startNetworkServer -h 0.0.0.0, I get an error when doing a simple connect using the ij command line tool
ij> connect 'jdbc:derby://localhost:1527/dbname;create=true';
ERROR XJ041: DERBY SQL error: ERRORCODE: 40000, SQLSTATE: XJ041, SQLERRMC: Failed to create database 'dbname', see the next exception for details.::SQLSTATE: XBM01::SQLSTATE: XJ001
The derby.log file makes reference to:
java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "getenv.SOURCE_DATE_EPOCH")
Looking further into this error, I learned that I somehow need a security.profile. I found this website that seemed to be the answers to my problems. https://www.javacodegeeks.com/2020/04/apache-derby-database-jvm-security-policy.html
Following these pretty straight-forward instructions, I get:
java.security.AccessControlException: access denied
org.apache.derby.shared.common.security.SystemPermission( "engine", "usederbyinternals" )
For the next person who has this strange problem (it seems to happen with some regularity, here's a simple workaround, copied from this FAQ page at Chalmers Institute of Technology:
Q: When we try to create a database in Derby and the database explorer in NetBeans, we get one or more of the following error(s):
An error occurred while creating the database:
java.sql.NonTransientConnectionException: DERBY SQL error: ERRORCODE:
40000, SQLSTATE: XJ041, SQLERRMC: ...
Caused by: java.security.AccessControlException: access denied
("java.lang.RuntimePermission" "getenv.SOURCE_DATE_EPOCH")
A: This is some kind of missconfiguration in the JVM with a very aggressive security policy that doesn't allow applications to fetch the time on the system (since epoch). The solution is to edit ~/.java.policy or [java.home]/lib/security/java.policy and add the following:
grant {
permission java.lang.RuntimePermission "getenv.SOURCE_DATE_EPOCH", "read";
};
If you are on Windows you can read about where this policy file is supposed to be located here;
https://docs.oracle.com/javase/7/docs/technotes/guides/security/PolicyFiles.html
Apache-Derby is a database management system prepared for a multi-user environment, therefore, when you execute the startNetworkServer -h 0.0.0.0 instruction, you are telling it by default to take certain security into account, and that is why it does not let you do an insecure connection such as ij> connect 'jdbc:derby://172.16.17.31:1527/BBDD_server;create=true';
because you are connecting without specifying username and password, so you should either connect by specifying username + password, or start the server without any security:
startNetworkServer -h 0.0.0.0 -noSecurityManager
More help:
https://db.apache.org/derby/docs/10.4/adminguide/tadminnetservopen.html
https://db.apache.org/derby/docs/10.4/adminguide/tadminnetservbasic.html
MongoDB service crashes and below is the error log:
ERROR: mmap private failed with out of memory. (64 bit build)
Assertion: 13636:file /var/lib/mongodb/_tmp_repairDatabase_4/dbname.0 open/create failed in createPrivateMap (look in log for more information)
and I could not start mongodb until I clear the data from mongodb data directory
Which OS do you use ?
Did you look at:
MongoDB: out of memory
or
http://pranavl.wordpress.com/2012/09/12/mongodb-error-mmap-failed-with-out-of-memory-64-bit-build/
I am not expert in Mongo - but I run it without any problem..
We untar spark-0.9.0-incubating.tgz and trying to build it for use with Yarn.
SPARK_HADOOP_VERSION=2.0.0-cdh4.6.0 SPARK_YARN=true sbt/sbt assembly
...
[info] Resolving io.netty#netty-all;4.0.13.Final ...
[error] Server access Error: Connection timed out url=https://oss.sonatype.org/content/repositories/snapshots/io/netty/netty-all/4.0.13.Final/netty-all-4.0.13.Final.pom
[error] Server access Error: Connection timed out url=https://oss.sonatype.org/service/local/staging/deploy/maven2/io/netty/netty-all/4.0.13.Final/netty-all-4.0.13.Final.pom
...
If I just cut-paste the url into a browser, I get:
404 - ItemNotFoundException
Retrieval of /io/netty/netty-all/4.0.13.Final/netty-all-4.0.13.Final.pom from M2Repository(id=snapshots) is forbidden by repository policy SNAPSHOT.
org.sonatype.nexus.proxy.ItemNotFoundException: Retrieval of /io/netty/netty-all/4.0.13.Final/netty-all-4.0.13.Final.pom from M2Repository(id=snapshots) is forbidden by repository policy SNAPSHOT.
at org.sonatype.nexus.proxy.maven.AbstractMavenRepository.doRetrieveItem(AbstractMavenRepository.java:380)
at org.sonatype.nexus.proxy.maven.maven2.M2Repository.doRetrieveItem(M2Repository.java:396)
at org.sonatype.nexus.proxy.repository.AbstractRepository.retrieveItem(AbstractRepository.java:765)
at org.sonatype.nexus.proxy.repository.AbstractRepository.retrieveItem(AbstractRepository.java:608)
at org.sonatype.nexus.proxy.router.DefaultRepositoryRouter.retrieveItem(DefaultRepositoryRouter.java:155)
at org.sonatype.nexus.web.content.NexusContentServlet.doGet(NexusContentServlet.java:359)
at org.sonatype.nexus.web.content.NexusContentServlet.service(NexusContentServlet.java:331)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
I have seen this reported in a number of places but no solution. Is this error because we are behind a corporate firewall, or is this due to something else? Please advise.
I had proxy set as environment variables, but it appears they are not being picked up. Adding them in sbt directly worked for me.
Edit $SPARK_HOME/sbt/sbt
For example,
EXTRA_ARGS="-Dhttp.proxySet=true -Dhttp.proxyHost=myproxy.mycompany.com -Dhttp.proxyPort=80 -Dhttps.proxySet=true -Dhttps.proxyHost=myproxy.mycompany.com -Dhttps.proxyPort=80 -Dftp.proxySet=true -Dftp.proxyHost=myproxy.mycompany.com -Dftp.proxyPort=80 -Dhttp.nonProxyHosts=mydomain -Dhttps.nonProxyHosts=mydomain -Dftp.nonProxyHosts=mydomain"