I'm trying to launch the command:
source /srv/dev-disk-by-uuid-e7874961-1866-43e8-9cdb-503276f500b6/Scripts/Scheduled\ Jobs/telegrambot.sh
but I keep getting this error:
Errore #0:
OMV\ExecException: Failed to execute command 'export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin; export LANG=C.UTF-8; export SHELL=/bin/sh; sudo --shell --non-interactive --user='root' -- /var/lib/openmediavault/cron.d/userdefined-6805cace-24e4-4582-9961-3347e2c3ed1b 2>&1' with exit code '127': /var/lib/openmediavault/cron.d/userdefined-6805cace-24e4-4582-9961-3347e2c3ed1b: 4: /var/lib/openmediavault/cron.d/userdefined-6805cace-24e4-4582-9961-3347e2c3ed1b:
source: not found
in /usr/share/openmediavault/engined/rpc/cron.inc:185
Stack trace:
#0 /usr/share/php/openmediavault/rpc/serviceabstract.inc(588): Engined\Rpc\Cron->Engined\Rpc\{closure}('/tmp/bgstatus70...', '/tmp/bgoutputfp...')
#1 /usr/share/openmediavault/engined/rpc/cron.inc(189): OMV\Rpc\ServiceAbstract->execBgProc(Object(Closure))
#2 [internal function]: Engined\Rpc\Cron->execute(Array, Array)
#3 /usr/share/php/openmediavault/rpc/serviceabstract.inc(123): call_user_func_array(Array, Array)
#4 /usr/share/php/openmediavault/rpc/rpc.inc(86): OMV\Rpc\ServiceAbstract->callMethod('execute', Array, Array)
#5 /usr/sbin/omv-engined(537): OMV\Rpc\Rpc::call('Cron', 'execute', Array, Array, 1)
#6 {main}
Can anyone help me please?
I faced a similar error, and what solved the issue for me was to remove the cron time indication in the command of the openmediavault scheduled job form.
For example:
instead of */5 * * * * ~/duckdns/duck.sh >/dev/null 2>&1
add ~/duckdns/duck.sh >/dev/null 2>&1
Related
When I launch vim it echoes a lot of variables (from my .vimrc I think) to the shell, before asking to continue. Why is this happening and what can I do to stop it?
Here's an example:
ycm_cache_omnifunc #0
oceanic_next_terminal_bold #1
ycm_seed_identifiers_with_syntax #1
vundle#lazy_load #0
vundle#updated_bundles []
terminal_color_background #1b2b34
terminal_color_12 #6699cc
vundle#bundle_dir /home/username/.vim/bundle
syntax_on #1
did_load_filetypes #1
ycm_confirm_extra_conf #0
...
v:errmsg E216: No such group or event: filetypedetect *
v:beval_lnum #0
v:mouse_lnum #0
v:completed_item {}
v:t_dict #4
v:false v:false
v:beval_bufnr #0
v:shell_error #0
v:testing #0
v:lc_time en_US.UTF-8
v:searchforward #1
v:event {}
v:mouse_col #0
Press ENTER or type command to continue
It then launches fine.
I try to update gitlab from version 9.5.10 to latest 10. I do 3 steps:
touch /etc/gitlab/skip-auto-migrations ( I don't need backup )
yum -y install gitlab-ce-10.8.7-ce.0.el7.x86_64
gitlab-ctl reconfigure
Gitlab crashes with an error after one hour:
...
Error executing action `run` on resource 'bash[migrate gitlab-rails database]'
================================================================================
Mixlib::ShellOut::CommandTimeout
--------------------------------
Command timed out after 3600s:
Command exceeded allowed execution time, process terminated
---- Begin output of "bash" "/tmp/chef-script20180907-58298-lhgdju" ----
STDOUT: == 20141126120926 AddMergeRequestRebaseEnabledToProjects: migrating ===========
-- transaction_open?()
-> 0.0000s
...
63: end
64: not_if "(test -f #{db_migrate_status_file}) && (cat #{db_migrate_status_file} | grep -Fx 0)"
65: only_if { node['gitlab']['gitlab-rails']['auto_migrate'] }
66: end
Compiled Resource:
------------------
# Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb:49:in `from_file'
bash("migrate gitlab-rails database") do
action [:run]
default_guard_interpreter :default
command nil
backup 5
returns 0
user nil
interpreter "bash"
declared_type :bash
cookbook_name "gitlab"
recipe_name "database_migrations"
code " set -e\n log_file=\"/var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log\"\n umask 077\n /opt/gitlab/bin/gitlab-rake gitlab:db:configure 2>& 1 | tee ${log_file}\n STATUS=${PIPESTATUS[0]}\n echo $STATUS > /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-141d998df14e11465b19f94d3c4ccfa1-eb600b0\n exit $STATUS\n"
domain nil
not_if "(test -f /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-141d998df14e11465b19f94d3c4ccfa1-eb600b0) && (cat /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-141d998df14e11465b19f94d3c4ccfa1-eb600b0 | grep -Fx 0)"
only_if { #code block }
end
System Info:
------------
chef_version=13.6.4
platform=centos
platform_version=7.4.1708
ruby=ruby 2.3.7p456 (2018-03-28 revision 63024) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/chef-client
executable=/opt/gitlab/embedded/bin/chef-client
Recipe: gitlab::unicorn
* service[unicorn] action restart
ESC[32m- restart service service[unicorn]
Recipe: gitlab::sidekiq
* service[sidekiq] action restart
ESC[32m- restart service service[sidekiq]
Recipe: gitlab::gitlab-rails
* execute[clear the gitlab-rails cache] action run
ESC[32m- execute /opt/gitlab/bin/gitlab-rake cache:clear
Running handlers:
Running handlers complete
Chef Client failed. 25 resources updated in 01 hours 01 minutes 28 seconds
I found workaround but I don't want to risk changing the update procedure. GitLab is used in the production and use of more than 1000 people (An update check is performed on the copy of the database).
Does anyone know how to increase the timeout?
Ok, turning this into an answer so other people might be able to use this:
Unless someone knows of a way to override the default timeout for Mixlib::ShellOut in this case, it is possible to override that default of 3600 in opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/chef-13.6.4/lib/chef/provider/execute.rb line 44-48, which currently read:
def timeout
# original implementation did not specify a timeout, but ShellOut
# *always* times out. So, set a very long default timeout
new_resource.timeout || 3600
end
Increasing the 3600 there should help.
My OS is RHEL 7, and I run a simple Go program:
package main
import (
"time"
)
func main() {
time.Sleep(1000 * time.Second)
}
During its running, I check the thread count of process:
# cat /proc/13858/status | grep Thread
Threads: 5
While using the pstack command shipped on RHEL, it only prints one thread's stack:
# pstack 13858
Thread 1 (process 13858):
#0 runtime.futex () at /usr/local/go/src/runtime/sys_linux_amd64.s:307
#1 0x0000000000422580 in runtime.futexsleep (addr=0x4c7af8 <runtime.timers+24>, val=0, ns=999999997446) at /usr/local/go/src/runtime/os1_linux.go:57
#2 0x000000000040b07b in runtime.notetsleep_internal (n=0x4c7af8 <runtime.timers+24>, ns=999999997446, ~r2=255) at /usr/local/go/src/runtime/lock_futex.go:174
#3 0x000000000040b1e6 in runtime.notetsleepg (n=0x4c7af8 <runtime.timers+24>, ns=999999997446, ~r2=false) at /usr/local/go/src/runtime/lock_futex.go:206
#4 0x000000000043e5de in runtime.timerproc () at /usr/local/go/src/runtime/time.go:209
#5 0x0000000000451001 in runtime.goexit () at /usr/local/go/src/runtime/asm_amd64.s:1998
#6 0x0000000000000000 in ?? ()
Why does pstack only print one thread's content?
P.S.: The pstack script is here:
#!/bin/sh
if test $# -ne 1; then
echo "Usage: `basename $0 .sh` <process-id>" 1>&2
exit 1
fi
if test ! -r /proc/$1; then
echo "Process $1 not found." 1>&2
exit 1
fi
# GDB doesn't allow "thread apply all bt" when the process isn't
# threaded; need to peek at the process to determine if that or the
# simpler "bt" should be used.
backtrace="bt"
if test -d /proc/$1/task ; then
# Newer kernel; has a task/ directory.
if test `/bin/ls /proc/$1/task | /usr/bin/wc -l` -gt 1 2>/dev/null ; then
backtrace="thread apply all bt"
fi
elif test -f /proc/$1/maps ; then
# Older kernel; go by it loading libpthread.
if /bin/grep -e libpthread /proc/$1/maps > /dev/null 2>&1 ; then
backtrace="thread apply all bt"
fi
fi
GDB=${GDB:-/usr/bin/gdb}
# Run GDB, strip out unwanted noise.
# --readnever is no longer used since .gdb_index is now in use.
$GDB --quiet -nx $GDBARGS /proc/$1/exe $1 <<EOF 2>&1 |
set width 0
set height 0
set pagination no
$backtrace
EOF
/bin/sed -n \
-e 's/^\((gdb) \)*//' \
-e '/^#/p' \
-e '/^Thread/p'
pstack uses gdb. This is a quote from golang doc (https://golang.org/doc/gdb):
GDB does not understand Go programs well. The stack management,
threading, and runtime contain aspects that differ enough from the
execution model GDB expects that they can confuse the debugger, even
when the program is compiled with gccgo. As a consequence, although
GDB can be useful in some situations, it is not a reliable debugger
for Go programs, particularly heavily concurrent ones.
4 out of 5 threads that you see in /proc are created even before you program enters main. I assume that golang runtime creates them.
Why does pstack only print one thread's content?
Judging from output of strace for gdb I see that gdb actually tries to attach to them but after something goes wrong and gdb does not try to inspect these threads. These are syscalls that gdb issued for these runtime threads but due to unknown reason decided stopped investigating them immediately:
5072 ptrace(PTRACE_ATTACH, 5023, 0, 0) = 0
5072 --- SIGCHLD (Child exited) # 0 (0) ---
5072 rt_sigreturn(0x11) = 0
5072 ptrace(PTRACE_ATTACH, 5024, 0, 0) = 0
5072 --- SIGCHLD (Child exited) # 0 (0) ---
5072 rt_sigreturn(0x11) = 0
5072 ptrace(PTRACE_ATTACH, 5025, 0, 0) = 0
5072 --- SIGCHLD (Child exited) # 0 (0) ---
5072 rt_sigreturn(0x11) = 0
However you can inspect them yourself. It seems that these threads belong to golang runtime
$ pstack 5094
Thread 1 (process 5094):
#0 0x0000000000459243 in runtime.futex ()
#1 0x00000000004271e0 in runtime.futexsleep ()
#2 0x000000000040d55b in runtime.notetsleep_internal ()
#3 0x000000000040d64b in runtime.notetsleep ()
#4 0x0000000000435677 in runtime.sysmon ()
#5 0x000000000042e6cc in runtime.mstart1 ()
#6 0x000000000042e5d2 in runtime.mstart ()
#7 0x00000000004592b7 in runtime.clone ()
#8 0x0000000000000000 in ?? ()
$ pstack 5095
Thread 1 (process 5095):
#0 0x0000000000459243 in runtime.futex ()
#1 0x0000000000427143 in runtime.futexsleep ()
#2 0x000000000040d3f4 in runtime.notesleep ()
#3 0x000000000042f6eb in runtime.stopm ()
#4 0x0000000000430a79 in runtime.findrunnable ()
#5 0x00000000004310ff in runtime.schedule ()
#6 0x000000000043139b in runtime.park_m ()
#7 0x0000000000455acb in runtime.mcall ()
#8 0x000000c820021500 in ?? ()
#9 0x0000000000000000 in ?? ()
$ pstack 5096
Thread 1 (process 5096):
#0 0x0000000000459243 in runtime.futex ()
#1 0x0000000000427143 in runtime.futexsleep ()
#2 0x000000000040d3f4 in runtime.notesleep ()
#3 0x000000000042f6eb in runtime.stopm ()
#4 0x000000000042fff7 in runtime.startlockedm ()
#5 0x0000000000431147 in runtime.schedule ()
#6 0x000000000043139b in runtime.park_m ()
#7 0x0000000000455acb in runtime.mcall ()
#8 0x000000c820020000 in ?? ()
Update for gdb 8.0
pstack that uses gdb 8.0 correctly prints backtraces for all threas. The command looks like:
$ GDB=$HOME/bin/gdb pstack $(pidof main)
And here is its output (shortened):
$ GDB=$HOME/bin/gdb pstack $(pidof main) | egrep "^Thread"
Thread 4 (LWP 18335):
Thread 3 (LWP 18334):
Thread 2 (LWP 18333):
Thread 1 (LWP 18332):
When you're passing LWP/thread id to pstack you get a stack of that thread only. Try to pass a PID of the process to pstack and you'll get stacks of all its threads. You may get a PID or Tgid (thread group id) of the process: cat /proc/13858/status | grep Tgid. To get all LWPs created by your process you may run ps -L <PID>
Although Linux doesn't provide pstack as Solaris does, RedHat provides a script can do the same thing:
#!/bin/bash
if test $# -ne 1; then
echo "Usage: `basename $0 .sh` <process-id>" 1>&2
exit 1
fi
if test ! -r /proc/$1; then
echo "Process $1 not found." 1>&2
exit 1
fi
# GDB doesn't allow "thread apply all bt" when the process isn't
# threaded; need to peek at the process to determine if that or the
# simpler "bt" should be used.
backtrace="bt"
if test -d /proc/$1/task ; then
# Newer kernel; has a task/ directory.
if test `/bin/ls /proc/$1/task | /usr/bin/wc -l` -gt 1 2>/dev/null ; then
backtrace="thread apply all bt"
fi
elif test -f /proc/$1/maps ; then
# Older kernel; go by it loading libpthread.
if /bin/grep -e libpthread /proc/$1/maps > /dev/null 2>&1 ; then
backtrace="thread apply all bt"
fi
fi
GDB=${GDB:-/usr/bin/gdb}
if $GDB -nx --quiet --batch --readnever > /dev/null 2>&1; then
readnever=--readnever
else
readnever=
fi
# Run GDB, strip out unwanted noise.
$GDB --quiet $readnever -nx /proc/$1/exe $1 <<EOF 2>&1 |
$backtrace
EOF
/bin/sed -n \
-e 's/^(gdb) //' \
-e '/^#/p' \
-e '/^Thread/p'
Executing the script in Suse:
linux:~ # pstack 7286
Thread 3 (Thread 0x7f7c074b5700 (LWP 7287)):
#0 0x00007f7c055f7a9d in read () from /lib64/libpthread.so.0
#1 0x00007f7c050a3b76 in ?? () from /usr/lib64/libxenstore.so.3.0
#2 0x00007f7c050a3c2f in ?? () from /usr/lib64/libxenstore.so.3.0
#3 0x00007f7c050a3f72 in ?? () from /usr/lib64/libxenstore.so.3.0
#4 0x00007f7c055f10a4 in start_thread () from /lib64/libpthread.so.0
#5 0x00007f7c03c9104d in clone () from /lib64/libc.so.6
Thread 2 (Thread 0x7f7bfe4b0700 (LWP 7288)):
#0 0x00007f7c055f505f in pthread_cond_wait##GLIBC_2.3.2 ()
#1 0x00007f7c07199d99 in ?? ()
#2 0x00007f7c0709a213 in ?? ()
#3 0x00007f7c0709a610 in ?? ()
#4 0x00007f7c055f10a4 in start_thread () from /lib64/libpthread.so.0
#5 0x00007f7c03c9104d in clone () from /lib64/libc.so.6
Thread 1 (Thread 0x7f7c07483980 (LWP 7286)):
#0 0x00007f7c03c88cdf in ppoll () from /lib64/libc.so.6
#1 0x00007f7c07060ec9 in ?? ()
#2 0x00007f7c07026654 in ?? ()
#3 0x00007f7c06edcb36 in ?? ()
#4 0x00007f7c03bcdb05 in __libc_start_main () from /lib64/libc.so.6
#5 0x00007f7c06ee0eec in ?? ()
My question is how to resolve the function name from the address? such as 0x00007fe4ab73eb36. I know maybe through installing debug-info packages, but how to know install which packages?
Update:
According to Mark Plotnick's comments, I use the following command to get which debuginfo packages are lacked:
linux:~ # gdb --quiet -nx --readnever /proc/7286/exe 7286
......
Missing separate debuginfos, use: zypper install glibc-debuginfo-2.19-31.5.x86_64 ......
After installing all need debuginfo packages, the symbols can be resolved:
linux:~ # pstack 7286
Thread 3 (Thread 0x7f7c074b5700 (LWP 7287)):
#0 0x00007f7c055f7a9d in read () from /lib64/libpthread.so.0
#1 0x00007f7c050a3b76 in read_all.part.1.constprop ()
#2 0x00007f7c050a3c2f in read_message.constprop ()
#3 0x00007f7c050a3f72 in read_thread () from /usr/lib64/libxenstore.so.3.0
#4 0x00007f7c055f10a4 in start_thread () from /lib64/libpthread.so.0
#5 0x00007f7c03c9104d in clone () from /lib64/libc.so.6
Thread 2 (Thread 0x7f7bfe4b0700 (LWP 7288)):
#0 0x00007f7c055f505f in pthread_cond_wait##GLIBC_2.3.2 ()
#1 0x00007f7c07199d99 in qemu_cond_wait ()
#2 0x00007f7c0709a213 in vnc_worker_thread_loop ()
#3 0x00007f7c0709a610 in vnc_worker_thread ()
#4 0x00007f7c055f10a4 in start_thread () from /lib64/libpthread.so.0
#5 0x00007f7c03c9104d in clone () from /lib64/libc.so.6
Thread 1 (Thread 0x7f7c07483980 (LWP 7286)):
#0 0x00007f7c03c88cdf in ppoll () from /lib64/libc.so.6
#1 0x00007f7c07060ec9 in qemu_poll_ns ()
#2 0x00007f7c07026654 in main_loop_wait ()
#3 0x00007f7c06edcb36 in main ()
But "objdump -t /proc/7286/exe | grep main" outputs nothing:
linux:~ # objdump -t /proc/7286/exe | grep main
linux:~ #
I have a problem with crontab generated by ISPConfig.
MAILTO=''
* * * * * web9 /usr/bin/wget -q -O /dev/null 'http://inz.isedo.pl/test/cron.php' >/dev/null 2>&1 #inz.isedo.pl
In log, I have a errors:
Feb 16 21:11:01 s /usr/sbin/cron[21697]: (*system*ispc_web9) RELOAD (/etc/cron.d/ispc_web9)
Feb 16 21:11:01 s /USR/SBIN/CRON[23817]: (web9) CMD (/usr/bin/wget -q -O /dev/null 'http://inz.isedo.pl/test/cron.php' >/dev/null 2>&1^I#inz.isedo.pl)
Feb 16 21:11:01 s /USR/SBIN/CRON[23816]: (CRON) error (grandchild #23817 failed with exit status 1)
Does it work from the command line ?
Can you redirect the stdout and stderr to a file (as opposed to null) and pass on the output ?