I am using these steps to use apache pulsar on docker: https://github.com/streamnative/tgip/blob/master/episodes/001/demo.md
I was able to use these steps before to install and use pulsar but for some reason now when am creating a directory, it is going to write protected and pulsar zookeeper container is exiting with following logs as soon as it is created:
ERROR org.apache.zookeeper.server.ZooKeeperServerMain - Unable to access datadir, exiting abnormally
org.apache.zookeeper.server.persistence.FileTxnSnapLog$DatadirException: Unable to create data directory data/zookeeper/version-2
at org.apache.zookeeper.server.persistence.FileTxnSnapLog.<init>(FileTxnSnapLog.java:136) ~[org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3]
at org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:137) ~[org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3]
at org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:112) ~[org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3]
at org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:67) [org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3]
at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:140) [org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3]
at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:90) [org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3]
Unable to access datadir, exiting abnormally
23:36:15.223 [main] INFO org.apache.zookeeper.audit.ZKAuditProvider - ZooKeeper audit is disabled.
23:36:15.226 [main] ERROR org.apache.zookeeper.util.ServiceUtils - Exiting JVM with code 3
23:36:15.196 [PurgeTask] ERROR org.apache.zookeeper.server.DatadirCleanupManager - Error occurred while purging.
org.apache.zookeeper.server.persistence.FileTxnSnapLog$DatadirException: Unable to create data directory data/zookeeper/version-2
at org.apache.zookeeper.server.persistence.FileTxnSnapLog.<init>(FileTxnSnapLog.java:136) ~[org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3]
at org.apache.zookeeper.server.PurgeTxnLog.purge(PurgeTxnLog.java:80) ~[org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3]
at org.apache.zookeeper.server.DatadirCleanupManager$PurgeTask.run(DatadirCleanupManager.java:141) [org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3]
at java.util.TimerThread.mainLoop(Timer.java:556) [?:?]
at java.util.TimerThread.run(Timer.java:506) [?:?]
23:36:15.229 [PurgeTask] INFO org.apache.zookeeper.server.DatadirCleanupManager - Purge task completed
I have made sure that SELinux is disabled and tried changing permission using chmod 777 data/ and every other step available to resolve this but still unable to find any. Please help me with the possible resolution.
Related
Running an application in in client mode, the driver logs are printed with the below info messages, any idea on how to resolve this? Any spark configs to be updated? or missing?
[INFO ][dispatcher-event-loop-29][SparkRackResolver:54] Got an error when resolving hostNames. Falling back to /default-rack for all
The jobs runs fine, this msg is not in the executor logs.
Check this bug:
https://issues.apache.org/jira/browse/SPARK-28005
If you want to suppress this in the logs you can try to add this into your log4j.properties
log4j.logger.org.apache.spark.deploy.yarn.SparkRackResolver=ERROR
This can happen while using spart-submit with master yarn in a deploy mode local (not using --deploy-mode cluster) and the path to topology.py script is not correct into your core-site.xml.
Path to core-site.xml can be set via environment variable HADOOP_CONF_DIR (or YARN_CONF_DIR).
Check the path in the param net.topology.script.file.name value of core-site.xml.
If the path is incorrect, deploying driver in local mode will lead to error of executing with the following warning:
23/01/15 18:39:43 WARN ScriptBasedMapping: Exception running /home/alexander/xxx/.conf/topology.py 10.15.21.199
java.io.IOException: Cannot run program "/etc/hadoop/conf.cloudera.yarn/topology.py" (in directory "/home/john"): error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
...
23/01/15 18:39:43 INFO SparkRackResolver: Got an error when resolving hostNames. Falling back to /default-rack for all
I have installed a DSE 6.0 cassandra cluster using LCM| opscenter 6.5 and node is up is running . During LCM cluster install, it installed datastax agent as well .
But the agent is not connecting to DSE and opscenter is not showing any details about the node.Later I tried with tarball install of datastax agent but that too is showing same issue. Please see below agent.log and screenshots.
WARN [async-dispatch-2] 2018-07-24 09:23:19,915 JMX marked as down, restarting JMX components.
ERROR [async-dispatch-2] 2018-07-24 09:23:19,916 Error starting DynamicEnvrionmentComponent.
java.io.IOException: Process failed: bash -c /tmp/opsc_3882111672138551416/dense.sh
Exit val: 126
Output:
bash: /tmp/opsc_3882111672138551416/dense.sh: Permission denied
at opsagent.proc$handle_proc_results.invokeStatic(proc.clj:61)
at opsagent.proc$handle_proc_results.invoke(proc.clj:51)
at opsagent.proc$run_proc.invokeStatic(proc.clj:84)
at opsagent.proc$run_proc.doInvoke(proc.clj:65)
at clojure.lang.RestFn.invoke(RestFn.java:410)
at opsagent.environment.utils$package_config_paths.invokeStatic(utils.clj:161)
at opsagent.environment.utils$package_config_paths.invoke(utils.clj:141)
at opsagent.environment.utils$all_config_paths.invokeStatic(utils.clj:197)
at opsagent.environment.utils$all_config_paths.doInvoke(utils.clj:190)
at clojure.lang.RestFn.invoke(RestFn.java:805)
at opsagent.environment.dynamic$dynamic_env_state.invokeStatic(dynamic.clj:162)
at opsagent.environment.dynamic$dynamic_env_state.invoke(dynamic.clj:148)
at clojure.lang.AFn.applyToHelper(AFn.java:171)
at clojure.lang.AFn.applyTo(AFn.java:144)
at clojure.core$apply.invokeStatic(core.clj:652)
at clojure.core$partial$fn__4765.doInvoke(core.clj:2534)
at clojure.lang.RestFn.invoke(RestFn.java:397)
at opsagent.jmx$create_jmx_pool_with_config$wrapper__5941.doInvoke(jmx.clj:239)
at clojure.lang.RestFn.invoke(RestFn.java:410)
at opsagent.environment.dynamic$add_dynamic_state.invokeStatic(dynamic.clj:276)
at opsagent.environment.dynamic$add_dynamic_state.invoke(dynamic.clj:264)
at opsagent.environment.dynamic.DynamicEnvironmentComponent.start(dynamic.clj:299)
at com.stuartsierra.component$fn__2593$G__2587__2595.invoke(component.clj:4)
at com.stuartsierra.component$fn__2593$G__2586__2598.invoke(component.clj:4)
at clojure.lang.Var.invoke(Var.java:379)
at clojure.lang.AFn.applyToHelper(AFn.java:154)
at clojure.lang.Var.applyTo(Var.java:700)
at clojure.core$apply.invokeStatic(core.clj:648)
at clojure.core$apply.invoke(core.clj:641)
at com.stuartsierra.component$try_action.invokeStatic(component.clj:116)
at com.stuartsierra.component$try_action.invoke(component.clj:115)
at clojure.lang.Var.invoke(Var.java:401)
at opsagent.config_service$update_system$fn__22445.invoke(config_service.clj:223)
at clojure.lang.ArraySeq.reduce(ArraySeq.java:114)
at clojure.core$reduce.invokeStatic(core.clj:6544)
at clojure.core$reduce.invoke(core.clj:6527)
at opsagent.config_service$update_system.invokeStatic(config_service.clj:217)
at opsagent.config_service$update_system.doInvoke(config_service.clj:213)
at clojure.lang.RestFn.invoke(RestFn.java:425)
at opsagent.config_service$start_system_BANG_.invokeStatic(config_service.clj:243)
at opsagent.config_service$start_system_BANG_.invoke(config_service.clj:236)
at opsagent.config_service$fn__22551$fn__22552$state_machine__4942__auto____22553$fn__22555.invoke(config_service.clj:266)
at opsagent.config_service$fn__22551$fn__22552$state_machine__4942__auto____22553.invoke(config_service.clj:266)
at clojure.core.async.impl.ioc_macros$run_state_machine.invokeStatic(ioc_macros.clj:973)
at clojure.core.async.impl.ioc_macros$run_state_machine.invoke(ioc_macros.clj:972)
at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invokeStatic(ioc_macros.clj:977)
at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invoke(ioc_macros.clj:975)
at clojure.core.async.impl.ioc_macros$take_BANG_$fn__4958.invoke(ioc_macros.clj:986)
at clojure.core.async.impl.channels.ManyToManyChannel$fn__707$fn__708.invoke(channels.clj:95)
at clojure.lang.AFn.run(AFn.java:22)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
INFO [async-dispatch-2] 2018-07-24 09:23:19,917 Starting JMXComponent
please note "/tmp/opsc_3882111672138551416/dense.sh: Permission denied" in your logs.
You probably don't have permissions to create anything under /tmp/
You can try fix the permissions or to reconfigure your temporary directory with -Djava.io.tmpdir in datastax-agent-env.sh:
JVM_OPTS="$JVM_OPTS -Xmx128M -Djava.io.tmpdir=/other/temp/directory"
You can find it here: /usr/share/datastax-agent/bin/
In version 6, there is datastax-agent instead of datastax-agent-env.sh
Remember to add this line at the beggining of datastax-agent file
I have installed monitoring out of the box according to this link:
http://www.jhipster.tech/monitoring/
When I start with:
docker-compose up -d
Everything starts but not Elastalert:
First log:
ERROR: for monitoring_jhipster-alerter_1 Cannot start service jhipster-alerter: OCI runtime create failed: container_linux.go:296: starting container process caused "process_linux.go:398: container init caused \"rootfs_linux.go:58: mounting \\"/Users/john/source/intellij/company/app/myservice/alerts/config.yaml\\" to rootfs \\"/var/lib/docker/overlay2/5657c6e9e7bb2be5cf4fa9860c04269e34be15641f4e3f0c1449af7cbf82ced5/merged\\" at \\"/var/lib/docker/overlay2/5657c6e9e7bb2be5cf4fa9860c04269e34be15641f4e3f0c1449af7cbf82ced5/merged/opt/elastalert/config.yaml\\" caused \\"not a directory\\"\"": unknown: Are you trying to mount a direStarting monitoring_jhipster-import-dashboards_1
Second log:
ERROR: for jhipster-alerter Cannot start service jhipster-alerter: OCI runtime create failed: container_linux.go:296: starting container process caused "process_linux.go:398: container init caused \"rootfs_linux.go:58: mounting \\"/Users/john/source/intellij/company/app/myservice/alerts/config.yaml\\" to rootfs \\"/var/lib/docker/overlay2/5657c6e9e7bb2be5cf4fa9860c04269e34be15641f4e3f0c1449af7cbf82ced5/merged\\" at \\"/var/lib/docker/overlay2/5657c6e9e7bb2be5cf4fa9860c04269e34be15641f4e3f0c1449af7cbf82ced5/merged/opt/elastalert/config.yaml\\" caused \\"not a directory\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
ERROR: Encountered errors while bringing up the project.
Using the default docker-compose.yml file that I got with:
curl -O https://raw.githubusercontent.com/jhipster/jhipster-console/master/bootstrap/docker-compose.yml
Im not sure what this messages says?
This is because the volumes path for JHipster Alerter is incorrect. Change
jhipster-alerter:
image: jhipster/jhipster-alerter:latest
environment:
- ES_HOST=jhipster-elasticsearch
- ES_PORT=9200
volumes:
- ../jhipster-alerter/rules/:/opt/elastalert/rules/
- ../alerts/config.yaml:/opt/elastalert/config.yaml
To
- ../alerts/rules/:/opt/elastalert/rules/
- ../jhipster-alerter/config.yaml:/opt/elastalert/config.yaml
As shown in https://github.com/jhipster/jhipster-console/pull/102/commits/fa5bc75ec29ca357477ac1a22203ae6cbe2af2f7.
I'm trying to setup a HDInsight emulator on a Windows 8.1 PC following these instructions: https://azure.microsoft.com/en-us/documentation/articles/hdinsight-hadoop-emulator-get-started/
When trying to run a MapReduce job, I get a connection error.
How can I solve or further investigate this issue?
Details below.
Prerequisites:
Installed Azure Powershell and Azure SDK for VS 2015
Installed HDInsight Emulator for Azure incl. Hortonworks Data Platform
Started local hdp services (13 services running)
Connected Visual Studio to Emulator (had to follow troubleshooting point 2: replacing IP addresses in core-site.xml with '*' due to dynamic IP)
Created directories and copied text files as suggested
Problem:
When trying to run the first example, I get the following error:
16/01/11 10:36:39 INFO mapreduce.Job: Job job_1452503376359_0003 failed with state FAILED due to: Application application_1452503376359_0003 failed 2 times due to AM Container for appattempt_1452503376359_0003_000002 exited with exitCode: -1000 due to: Call From EH3HOST/192.168.56.1 to EH3HOST:8020 failed on connection exception: java.net.ConnectException: Connection refused: no further information; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
.Failing this attempt.. Failing the application.
16/01/11 10:36:39 INFO mapreduce.Job: Counters: 0
The following worked for me:
Search for XML files containing <your own host name>:8020 inside the c:\hdp\hdp-<Version Number>\etc\hadoop\ folder. (e.g. EH3HOST:8020)
You should find at least
mapred-site.xml
core-site.xml
yarn-site.xml
Replace all occurrences within these files with 127.0.0.1:8020.
I am running Linux in VirtualBox and am having an issue that I did not encounter on my machine with Linux as the primary OS.
When launching the neo4j service through sudo ./neo4j start in /opt/neo4j-community-2.3.1/bin I get a timeout with the message Failed to start within 120 seconds. Neo4j Server may have failed to start, please check the logs
my log from /opt/neo4j-community-2.3.1/data/graph.db/messages.log says:
http://pastebin.com/wUA715QQ
and data/log/console.log says:
2016-01-06 02:07:03.404+0100 INFO Successfully started database
2016-01-06 02:07:03.603+0100 INFO Successfully stopped database
2016-01-06 02:07:03.604+0100 INFO Successfully shutdown Neo4j Server
2016-01-06 02:07:03.608+0100 ERROR Failed to start Neo4j: Starting Neo4j failed: Component 'org.neo4j.server.security.auth.FileUserRepository#9ab182' was successfully initialized, but failed to start. Please see attached cause exception. Starting Neo4j failed: Component 'org.neo4j.server.security.auth.FileUserRepository#9ab182' was successfully initialized, but failed to start. Please see attached cause exception.
org.neo4j.server.ServerStartupException: Starting Neo4j failed: Component 'org.neo4j.server.security.auth.FileUserRepository#9ab182' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.server.exception.ServerStartupErrors.translateToServerStartupError(ServerStartupErrors.java:67)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:234)
at org.neo4j.server.Bootstrapper.start(Bootstrapper.java:97)
at org.neo4j.server.CommunityBootstrapper.start(CommunityBootstrapper.java:48)
at org.neo4j.server.CommunityBootstrapper.main(CommunityBootstrapper.java:35)
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.server.security.auth.FileUserRepository#9ab182' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:462)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:111)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:194)
... 3 more
Caused by: java.nio.file.AccessDeniedException: /opt/neo4j-community-2.3.1/data/dbms/auth
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
at java.nio.file.Files.newByteChannel(Files.java:361)
at java.nio.file.Files.newByteChannel(Files.java:407)
at java.nio.file.Files.readAllBytes(Files.java:3152)
at org.neo4j.server.security.auth.FileUserRepository.loadUsersFromFile(FileUserRepository.java:208)
at org.neo4j.server.security.auth.FileUserRepository.start(FileUserRepository.java:73)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:452)
... 5 more
Any idea why the server won't start?
Check the permissions on /opt/neo4j-community-2.3.1/data/dbms/auth
See the line that says:
Caused by: java.nio.file.AccessDeniedException: /opt/neo4j-community-2.3.1/data/dbms/auth