ArangoDB services restart reason diagnostic - arangodb

The arangodb services were restarted automatically (because of hang of some process probably) with the following messages in arangodb.log:
Shutting down services... component=arangodb
Wait for process termination without sending signal coordinator... component=arangodb pid=xxx type=coordinator
Terminated coordinator... component=arangodb pid=xxx type=coordinator
coordinator has terminated component=arangodb pid=xxx type=coordinator
Exited component=arangodb pid=xxx type=coordinator
Wait for process termination without sending signal dbserver... component=arangodb pid=yyy type=dbserver
Terminated agent component=arangodb pid=zzz type=agent
agent has terminated component=arangodb pid=zzz type=agent
Exited component=arangodb pid=zzz type=agent
Continue signal termination process dbserver... component=arangodb pid=yyy type=dbserver
Terminating dbserver... component=arangodb pid=yyy type=dbserver
Wait on yyy returned component=arangodb exit-status=1 trap-cause=-1
Terminated dbserver... component=arangodb pid=yyy type=dbserver
dbserver has terminated component=arangodb pid=yyy type=dbserver
Exited component=arangodb pid=yyy type=dbserver
Is it possible to configure the system to show the reason in some log / trace files?

Related

flutter: Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)

I'm working with flutter in ubuntu 18.04. I can't run my project although flutter doctor is no problem. I stuck in the problem of Gradle daemon. I have tried the method in gradle user guide and other questions in SO to add org.gradle.daemon=false in gradle.properties, but the same problem. What's wrong?
Launching lib/main.dart on M2006C3LC in debug mode...
The message received from the daemon indicates that the daemon has disappeared.
Build request sent: Build{id=0846277b-6ded-4171-9a9a-ce34e51b07be, currentDir=/home/byhuang/flutterOpenProject/flutter_init_demo/android}
Attempting to read last messages from the daemon log...
Daemon pid: 3422
log file: /home/byhuang/.gradle/daemon/5.6.2/daemon-3422.out.log
----- Last 20 lines from daemon log file - daemon-3422.out.log -----
11:38:04.753 [DEBUG] [org.gradle.launcher.daemon.server.DefaultIncomingConnectionHandler] Starting executing command: Build{id=0846277b-6ded-4171-9a9a-ce34e51b07be, currentDir=/home/byhuang/flutterOpenProject/flutter_init_demo/android} with connection: socket connection from /0:0:0:0:0:0:0:1:41393 to /0:0:0:0:0:0:0:1:46484.
11:38:04.755 [ERROR] [org.gradle.launcher.daemon.server.DaemonStateCoordinator] Command execution: started DaemonCommandExecution[command = Build{id=0846277b-6ded-4171-9a9a-ce34e51b07be, currentDir=/home/byhuang/flutterOpenProject/flutter_init_demo/android}, connection = DefaultDaemonConnection: socket connection from /0:0:0:0:0:0:0:1:41393 to /0:0:0:0:0:0:0:1:46484] after 0.0 minutes of idle
11:38:04.756 [INFO] [org.gradle.launcher.daemon.server.DaemonRegistryUpdater] Marking the daemon as busy, address: [f54344e9-fcb9-455f-a518-b1d7af504663 port:41393, addresses:[/0:0:0:0:0:0:0:1%lo, /127.0.0.1]]
11:38:04.763 [DEBUG] [org.gradle.launcher.daemon.registry.PersistentDaemonRegistry] Marking busy by address: [f54344e9-fcb9-455f-a518-b1d7af504663 port:41393, addresses:[/0:0:0:0:0:0:0:1%lo, /127.0.0.1]]
11:38:04.790 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire exclusive lock on daemon addresses registry.
11:38:04.791 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry.
11:38:04.804 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry.
11:38:04.805 [DEBUG] [org.gradle.launcher.daemon.server.DaemonStateCoordinator] resetting idle timer
11:38:04.805 [DEBUG] [org.gradle.launcher.daemon.server.DaemonStateCoordinator] daemon is running. Sleeping until state changes.
11:38:04.810 [INFO] [org.gradle.launcher.daemon.server.exec.StartBuildOrRespondWithBusy] Daemon is about to start building Build{id=0846277b-6ded-4171-9a9a-ce34e51b07be, currentDir=/home/byhuang/flutterOpenProject/flutter_init_demo/android}. Dispatching build started information...
11:38:04.811 [DEBUG] [org.gradle.launcher.daemon.server.SynchronizedDispatchConnection] thread 17: dispatching class org.gradle.launcher.daemon.protocol.BuildStarted
11:38:04.812 [DEBUG] [org.gradle.launcher.daemon.server.exec.EstablishBuildEnvironment] Configuring env variables: {PATH=/home/byhuang/src/FlutterSDK/flutter/bin/cache/dart-sdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/byhuang/.pub-cache/bin:~/src/FlutterSDK/flutter/bin, XAUTHORITY=/run/user/1000/gdm/Xauthority, INVOCATION_ID=bb1359976cdb403b93f2e6bd5fa0d6e7, XMODIFIERS=#im=ibus, GDMSESSION=ubuntu, XDG_DATA_DIRS=/usr/share/ubuntu:/usr/local/share:/usr/share:/var/lib/snapd/desktop, TEXTDOMAINDIR=/usr/share/locale/, GTK_IM_MODULE=ibus, DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus,guid=e1b93d29865be954f2c06fce5fe015cb, PUB_HOSTED_URL=https://pub.flutter-io.cn, XDG_CURRENT_DESKTOP=ubuntu:GNOME, JOURNAL_STREAM=9:49578, SSH_AGENT_PID=1797, COLORTERM=truecolor, QT4_IM_MODULE=xim, SESSION_MANAGER=local/byhuang-virtual-machine:#/tmp/.ICE-unix/1702,unix/byhuang-virtual-machine:/tmp/.ICE-unix/1702, USERNAME=byhuang, LOGNAME=byhuang, PWD=/home/byhuang/flutterOpenProject/flutter_init_demo/android, MANAGERPID=1667, IM_CONFIG_PHASE=2, LANGUAGE=zh_CN:zh, LESSOPEN=| /usr/bin/lesspipe %s, SHELL=/bin/bash, OLDPWD=/home/byhuang/flutterOpenProject/flutter_init_demo/android, GNOME_DESKTOP_SESSION_ID=this-is-deprecated, GTK_MODULES=gail:atk-bridge, GNOME_TERMINAL_SCREEN=/org/gnome/Terminal/screen/1c43d352_e494_406b_b4b1_8e1a6bf95400, CLUTTER_IM_MODULE=xim, TEXTDOMAIN=im-config, DBUS_STARTER_ADDRESS=unix:path=/run/user/1000/bus,guid=e1b93d29865be954f2c06fce5fe015cb, FLUTTER_STORAGE_BASE_URL=https://storage.flutter-io.cn, XDG_SESSION_DESKTOP=ubuntu, LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:, SHLVL=2, LESSCLOSE=/usr/bin/lesspipe %s %s, QT_IM_MODULE=xim, JAVA_HOME=/home/byhuang/software/android-studio/jre, TERM=xterm-256color, FLUTTER_ROOT=/home/byhuang/src/FlutterSDK/flutter, XDG_CONFIG_DIRS=/etc/xdg/xdg-ubuntu:/etc/xdg, GNOME_TERMINAL_SERVICE=:1.59, LANG=zh_CN.UTF-8, XDG_SESSION_TYPE=x11, XDG_SESSION_ID=4, DISPLAY=:1, FLUTTER_SUPPRESS_ANALYTICS=true, GPG_AGENT_INFO=/run/user/1000/gnupg/S.gpg-agent:0:1, DESKTOP_SESSION=ubuntu, USER=byhuang, XDG_MENU_PREFIX=gnome-, VTE_VERSION=5202, WINDOWPATH=1, QT_ACCESSIBILITY=1, XDG_SEAT=seat0, SSH_AUTH_SOCK=/run/user/1000/keyring/ssh, FLUTTER_ALREADY_LOCKED=true, GNOME_SHELL_SESSION_MODE=ubuntu, XDG_RUNTIME_DIR=/run/user/1000, XDG_VTNR=1, DBUS_STARTER_BUS_TYPE=session, HOME=/home/byhuang}
11:38:04.826 [DEBUG] [org.gradle.launcher.daemon.server.exec.LogToClient] About to start relaying all logs to the client via the connection.
11:38:04.826 [INFO] [org.gradle.launcher.daemon.server.exec.LogToClient] The client will now receive all logging from the daemon (pid: 3422). The daemon log file: /home/byhuang/.gradle/daemon/5.6.2/daemon-3422.out.log
11:38:04.832 [DEBUG] [org.gradle.launcher.daemon.server.exec.RequestStopIfSingleUsedDaemon] Requesting daemon stop after processing Build{id=0846277b-6ded-4171-9a9a-ce34e51b07be, currentDir=/home/byhuang/flutterOpenProject/flutter_init_demo/android}
11:38:04.832 [LIFECYCLE] [org.gradle.launcher.daemon.server.DaemonStateCoordinator] Daemon will be stopped at the end of the build stopping after processing
11:38:04.832 [DEBUG] [org.gradle.launcher.daemon.server.DaemonStateCoordinator] Stop as soon as idle requested. The daemon is busy: true
11:38:04.835 [DEBUG] [org.gradle.launcher.daemon.server.DaemonStateCoordinator] daemon stop has been requested. Sleeping until state changes.
11:38:04.835 [DEBUG] [org.gradle.launcher.daemon.server.exec.ExecuteBuild] The daemon has started executing the build.
11:38:04.839 [DEBUG] [org.gradle.launcher.daemon.server.exec.ExecuteBuild] Executing build with daemon context: DefaultDaemonContext[uid=cbbde454-54e4-47ec-b258-28179c1fddfe,javaHome=/home/byhuang/software/android-studio/jre,daemonRegistryDir=/home/byhuang/.gradle/daemon,pid=3422,idleTimeout=120000,priority=NORMAL,daemonOpts=-Xmx1536M,-Dfile.encoding=UTF-8,-Duser.country=CN,-Duser.language=zh,-Duser.variant]
----- End of the daemon log -----
FAILURE: Build failed with an exception.
* What went wrong:
Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Running Gradle task 'assembleDebug'...
Running Gradle task 'assembleDebug'... Done 29.2s
Exception: Gradle task assembleDebug failed with exit code 1
I faced the same error today in Ubuntu. To temporarily bypass the error, adding --no-daemon to whichever gradle command I was running worked. Deleting the daemon subfolder in /home/{username}/.gradle folder permanently fixed the error.
go to project file -> android-> delete .gradle file.
As the other answers suggest delete the .gradle folder. But that didn't work for me, I had to do that and then shut down and restart my ubuntu.

Gracefully killing container and pods in kubernetes with spark exception

What is the recommended way to gracefully kill a container and driver pods in kubernetes when an application fails or reaches an exception. Currently, when my application runs into an exception my pods and executors continue to run and I noticed that my container doesn't get killed unless an explicit exit 1 is used.For some reason my spark application doesn't cause an exit 1 status or sigterm signal to be sent to the container or pod.
Tried to add following to yaml specs based on recommendations but still not getting pod driver and executors to terminate:
spec:
terminationGracePeriodSeconds: 0
driver:
lifecycle:
preStop:
exec:
command:
- /bin/bash
- -c
- touch /var/run/killspark && sleep 65
The preStop LiveCycle hook you added won't have any effect, since it is only triggered
before a container is terminated due to an API request or management
event such as liveness probe failure, preemption, resource contention
and others
https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/
I suspect, what you really have to figure out is why your container main process keeps running despite the exception.

How to fix Heroku error: Process exited with status 129

2018-05-14T09:12:46.770011+00:00 app[api]: Starting process with command node musicology.js by user leshaber24#yandex.ru
2018-05-14T09:12:50.704836+00:00 heroku[run.5756]: Awaiting client
2018-05-14T09:12:50.760658+00:00 heroku[run.5756]: Starting process with command node musicology.js
2018-05-14T09:12:50.767575+00:00 heroku[run.5756]: State changed from starting to up
2018-05-14T09:12:56.951346+00:00 heroku[run.5756]: Client connection closed. Sending SIGHUP to all processes
2018-05-14T09:12:57.484760+00:00 heroku[run.5756]: Process exited with status 129
2018-05-14T09:12:57.499531+00:00 heroku[run.5756]: State changed from up to complete
These are the logs coming from Heroku. My app is here: https://github.com/Alesha24/mysicology_telegram_bot
I've already fixed an error with SIGKILL command, now (after doing node musicology.js) error is of status 129 and moreover this one:
Error R13 (Attach error) -> Failed to attach to process
This error also appears to be unclear for me. Any ideas of way, how can I fix it?

Why did Spark standalone Worker node-1 terminate after RECEIVED SIGNAL 15: SIGTERM?

Note: This error was thrown before the components were executed by spark.
Logs
Worker Node1:
17/05/18 23:12:52 INFO Worker: Successfully registered with master spark://spark-master-1.com:7077
17/05/18 23:58:41 ERROR Worker: RECEIVED SIGNAL 15: SIGTERM
Master Node:
17/05/18 23:12:52 INFO Master: Registering worker spark-worker-1com:56056 with 2 cores, 14.5 GB RAM
17/05/18 23:14:20 INFO Master: Registering worker spark-worker-2.com:53986 with 2 cores, 14.5 GB RAM
17/05/18 23:59:42 WARN Master: Removing spark-worker-1com-56056 because we got no heartbeat in 60 seconds
17/05/18 23:59:42 INFO Master: Removing spark-worker-2.com:56056
17/05/19 00:00:03 ERROR Master: RECEIVED SIGNAL 15: SIGTERM
Worker Node2:
17/05/18 23:14:20 INFO Worker: Successfully registered with master spark://spark-master-node-2.com:7077
17/05/18 23:59:40 ERROR Worker: RECEIVED SIGNAL 15: SIGTERM
TL;DR I think someone has explicitly called kill command or sbin/stop-worker.sh.
"RECEIVED SIGNAL 15: SIGTERM" is reported by a shutdown hook to log TERM, HUP, INT signals on UNIX-like systems:
/** Register a signal handler to log signals on UNIX-like systems. */
def registerLogger(log: Logger): Unit = synchronized {
if (!loggerRegistered) {
Seq("TERM", "HUP", "INT").foreach { sig =>
SignalUtils.register(sig) {
log.error("RECEIVED SIGNAL " + sig)
false
}
}
loggerRegistered = true
}
}
In your case it means that the process received SIGTERM to stop itself:
The SIGTERM signal is a generic signal used to cause program termination. Unlike SIGKILL, this signal can be blocked, handled, and ignored. It is the normal way to politely ask a program to terminate.
That's what is sent when you execute KILL or use ./sbin/stop-master.sh or ./sbin/stop-worker.sh shell scripts that in turn call sbin/spark-daemon.sh with stop command that kills a JVM process for a master or a worker:
kill "$TARGET_ID" && rm -f "$pid"

WinRT app crashes in release build but not debug

I have an app that runs fine in debug but crashes when trying to launch in release. This is the exception that is thrown.
Unhandled exception at 0x5D2F7717 (Windows.UI.Xaml.dll) in Napkin.exe:
0xC000027B: An application-internal exception has occurred
(parameters: 0x034F6C70, 0x00000001).
Any ideas what might be the source of this problem? Thanks.
Here is more of the output window
Unhandled exception at 0x55257717 (Windows.UI.Xaml.dll) in Napkin.exe:
0xC000027B: An application-internal exception has occurred
(parameters: 0x02D029E0, 0x00000001). Unhandled exception at
0x750D2722 (combase.dll) in Napkin.exe: 0xC0000602: A fail fast
exception occurred. Exception handlers will not be invoked and the
process will be terminated immediately.
STATUS_STACK_BUFFER_OVERRUN encountered Napkin.exe has triggered a
breakpoint. The thread 0x1b20 has exited with code -1073740791
(0xc0000409). The thread 0xb50 has exited with code -1073740791
(0xc0000409). The thread 0x108c has exited with code -1073740791
(0xc0000409). The thread 0x16cc has exited with code -1073740791
(0xc0000409). The thread 0x222c has exited with code -1073740791
(0xc0000409). The thread 0x78c has exited with code -1073740791
(0xc0000409). The thread 0x14a4 has exited with code -1073740791
(0xc0000409). The thread 0x2164 has exited with code -1073740791
(0xc0000409). The program '[6392] Napkin.exe' has exited with code
-1073740791 (0xc0000409). Activation of the Windows Store app 'cdbbf746-1a99-4261-8bfa-9cb2934d4ac0_qddg1e943mmj2!App' failed with
error 'Windows was unable to communicate with the target application.
This usually indicates that the target application's process aborted.
More information may be available in the Debug pane of the Output
window (Debug->Windows->Output)'.
Possibly an uninitialized variable. Debug builds will initialize them for you, but Release builds will not.
Could also be an invalid pointer dereference; Debug builds usually treat memory differently than Release builds do.

Resources