Cronjob doesn't execute "bundle exec rake -f ..." - linux

I am trying to automate redmine's issue creation via email within a docker container using cronjob.
I use a slightly modified version of the offical redmine docker image with cron installed and some additional environment variables.
Manually, when I run the following line in the terminal, everything works fine.
# cd /usr/src/redmine && bundle exec rake -f /usr/src/redmine/Rakefile redmine:email:receive_imap RAILS_ENV="production" host=$IMAP_HOST port=$IMAP_PORT ssl=1 username=$SMTP_USER password=$SMTP_PASSWORD -r "openssl" -E "OpenSSL::SSL::VERIFY_PEER = OpenSSL::SSL::VERIFY_NONE"
(eval):1: warning: already initialized constant OpenSSL::SSL::VERIFY_PEER
W, [2020-12-11T14:18:43.730617 #86] WARN -- : Creating scope :system. Overwriting existing method Enumeration.system.
W, [2020-12-11T14:18:45.264865 #86] WARN -- : Creating scope :sorted. Overwriting existing method Group.sorted.
W, [2020-12-11T14:18:45.273745 #86] WARN -- : Creating scope :sorted. Overwriting existing method User.sorted.
I, [2020-12-11T14:18:46.534306 #86] INFO -- : Performing ActionMailer::DeliveryJob (Job ID: eda80c3a-dc7d-4dcc-9fb9-ad25414cbad2) from Inline(mailers) with arguments: "Mailer", "issue_edit", "deliver_now", #<GlobalID:0x000055f759eb7818 #uri=#<URI::GID gid://redmine-app/User/1>>, #<GlobalID:0x000055f759eb6940 #uri=#<URI::GID gid://redmine-app/Journal/1>>
I, [2020-12-11T14:18:46.560561 #86] INFO -- : Rendering mailer/issue_edit.text.erb within layouts/mailer
I, [2020-12-11T14:18:46.597403 #86] INFO -- : Rendered mailer/_issue.text.erb (34.9ms)
I, [2020-12-11T14:18:46.597516 #86] INFO -- : Rendered mailer/issue_edit.text.erb within layouts/mailer (36.9ms)
I, [2020-12-11T14:18:46.600656 #86] INFO -- : Rendering mailer/issue_edit.html.erb within layouts/mailer
I, [2020-12-11T14:18:46.613771 #86] INFO -- : Rendered mailer/_issue.html.erb (0.9ms)
I, [2020-12-11T14:18:46.613876 #86] INFO -- : Rendered mailer/issue_edit.html.erb within layouts/mailer (13.1ms)
I, [2020-12-11T14:18:46.620620 #86] INFO -- : Performed ActionMailer::DeliveryJob (Job ID: eda80c3a-dc7d-4dcc-9fb9-ad25414cbad2) from Inline(mailers) in 86.17ms
I, [2020-12-11T14:18:46.621236 #86] INFO -- : Enqueued ActionMailer::DeliveryJob (Job ID: eda80c3a-dc7d-4dcc-9fb9-ad25414cbad2) to Inline(mailers) with arguments: "Mailer", "issue_edit", "deliver_now", #<GlobalID:0x000055f75a681d00 #uri=#<URI::GID gid://redmine-app/User/1>>, #<GlobalID:0x000055f75a681530 #uri=#<URI::GID gid://redmine-app/Journal/1>>
I, [2020-12-11T14:18:46.622072 #86] INFO -- : MailHandler: issue #1 updated by Redmine Admin
Now, I want to create a cronjob that executes this line, say every minute.
I added the cronjob below, but the desired outcome remains absent.
# crontab -l
* * * * * cd /usr/src/redmine && bundle exec rake -f /usr/src/redmine/Rakefile redmine:email:receive_imap RAILS_ENV="production" host=$IMAP_HOST port=$IMAP_PORT ssl=1 username=$SMTP_USER password=$SMTP_PASSWORD -r "openssl" -E "OpenSSL::SSL::VERIFY_PEER = OpenSSL::SSL::VERIFY_NONE"
From the /var/log/cron.log file I know, that the cronjob is executed.
Dec 11 14:43:01 26c0ec148cea CRON[155]: (root) CMD (cd /usr/src/redmine && bundle exec rake -f /usr/src/redmine/Rakefile redmine:email:receive_imap RAILS_ENV="production" host=$IMAP_HOST port=$IMAP_PORT ssl=1 username=$SMTP_USER password=$SMTP_PASSWORD -r "openssl" -E "OpenSSL::SSL::VERIFY_PEER = OpenSSL::SSL::VERIFY_NONE")
If I append &> /home/redmine/cron.log to the job, an empty file gets created.
I assume cron somehow breaks the command.
I already tried to use the full paths to bundle (/usr/local/bin/bundle) and rake (/usr/local/bundle/bin/rake) without success.
Can you help me to fix this cronjob or tell me how to debug this further?
Setup:
Dockerfile:
FROM redmine:latest
ENV SMTP_AUTHENTICATION="login" \
SMTP_DOMAIN="gmail.com" \
SMTP_HOST="smtp.gmail.com" \
SMTP_PASSWORD="<password>" \
SMTP_PORT="587" \
SMTP_TLS="true" \
SMTP_USER="<email>" \
IMAP_HOST="imap.gmail.com" \
IMAP_PORT="993"
RUN apt-get update > /dev/null && apt-get install -y cron vim rsyslog > /dev/null
RUN service rsyslog start
RUN crontab -l | { cat; echo '* * * * * cd /usr/src/redmine && bundle exec rake -f /usr/src/redmine/Rakefile redmine:email:receive_imap RAILS_ENV="production" host=$IMAP_HOST port=$IMAP_PORT ssl=1 username=$SMTP_USER password=$SMTP_PASSWORD -r "openssl" -E "OpenSSL::SSL::VERIFY_PEER = OpenSSL::SSL::VERIFY_NONE"'; } | crontab -
docker-compose.yml:
version: '3.1'
services:
redmine:
#image: redmine
build: '.'
restart: always
ports:
- 80:3000
volumes:
- 'redmine_data:/redmine'
environment:
REDMINE_DB_MYSQL: db
REDMINE_DB_PASSWORD: <password>
REDMINE_SECRET_KEY_BASE: <secretkey>
db:
image: mysql:5.7
restart: always
volumes:
- 'db_data:/db'
environment:
MYSQL_ROOT_PASSWORD: <password>
MYSQL_DATABASE: redmine
volumes:
db_data:
driver: local
redmine_data:
driver: local

Related

Preseed late_command

I am trying to add a late_command to my preseed.cfg file but I get following error.
command:
d-i preseed/late_command string \
no_proxy=0,1,2,3,4,5,6,7,8,9 preseed_fetch /scripts/preseed_late_commands.sh /tmp/preseed_late_commands.sh ; \
/target/usr/bin/dos2unix /tmp/preseed_late_commands.sh || true; \
log-output -t preseed_late_commands.sh sh /tmp/preseed_late_commands.sh
The error that I get:
Execution of preseeded command "d-i preseed/late_command string \
no_proxy=0,1,2,3,4,5,6,7,8,9 preseed_fetch /scripts/preseed_late_commands.sh /tmp/preseed_late_commands.sh ; \
/target/usr/bin/dos2unix /tmp/preseed_late_commands.sh || true; \
log-output -t preseed_late_commands.sh sh /tmp/preseed_late_commands.sh" failed with exit code 2.
Infos:
os: Debian 11 (Bullseye)
preseed.cfg directory: http/d-i/bullseye/preseed.cfg
preseed_late_commands.sh directory: http/d-i/bullseye/scripts/preseed_late_commands.sh
Error image:
Error image
What should I do to get this command to run?
How and where can I see a live log (console output) or find a log file?

How to upgrade rpm package using bash script?

I want to install the specific rpm packages using bash script. But I am getting error.
I am using the below code :
#!/bin/bash
systemctl stop edb-as-10.service
server_status=$(systemctl status edb-as-10.service|awk '{print $1,$2,$3}'|grep Active|cut -d ":" -f2|cut -d " " -f2|tr -d " ")
echo "$server_status"
if [ "$server_status" == inactive ]; then
echo "You can proceed with upgrading the Potgresql server"
rpm –Uvh --nodeps edb-as10-server-pltcl-10.9.17-1.rhel7.x86_64.rpm
systemctl start edb-as-10.service else
echo " Failed to do Upgrading"
fi
I'm getting this error :
RPM version 4.11.3 Copyright (C) 1998-2002 - Red Hat, Inc. This
program may be freely redistributed under the terms of the GNU GPL
Usage: rpm [-aKfgpqVcdLilsiv?] [-a|--all] [-f|--file] [-g|--group]
[-p|--package] [--pkgid] [--hdrid] [--triggeredby] [--whatrequires]
[--whatprovides]
[--nomanifest] [-c|--configfiles] [-d|--docfiles] [-L|--licensefiles] [--dump] [-l|--list] [--queryformat=QUERYFORMAT]
[-s|--state] [--nofiledigest]
[--nofiles] [--nodeps] [--noscript] [--allfiles] [--allmatches] [--badreloc] [-e|--erase +] [--excludedocs]
[--excludepath=] [--force]
[-F|--freshen +] [-h|--hash] [--ignorearch] [--ignoreos] [--ignoresize] [-i|--install] [--justdb] [--nodeps]
[--nofiledigest]
[--nocontexts] [--noorder] [--noscripts] [--notriggers] [--nocollections] [--oldpackage] [--percent] [--prefix=]
[--relocate==]
[--replacefiles] [--replacepkgs] [--test] [-U|--upgrade +] [--reinstall=+] [-D|--define 'MACRO
EXPR'] [--undefine=MACRO]
[-E|--eval 'EXPR'] [--macros=] [--noplugins] [--nodigest] [--nosignature] [--rcfile=] [-r|--root ROOT]
[--dbpath=DIRECTORY]
[--querytags] [--showrc] [--quiet] [-v|--verbose] [--version] [-?|--help] [--usage] [--scripts] [--setperms] [--setugids]
[--conflicts] [--obsoletes]
[--provides] [--requires] [--info] [--changelog] [--xml] [--triggers] [--last] [--dupes] [--filesbypkg] [--fileclass]
[--filecolor] [--fscontext]
[--fileprovide] [--filerequire] [--filecaps]
But when I run the command
rpm –Uvh --nodeps edb-as10-server-pltcl-10.9.17-1.rhel7.x86_64.rpm
on terminal its working but when it placed in script its not working.

Install docker and docker-compose on Azure using cloud-Init

I wanted to bootstrap my Azure VM with Docker and Docker-compose using cloud-init. So far I tried something like below.
#cloud-config
package_update: true
package_upgrade: true
groups:
- docker: [default]
runcmd:
- [ sh, -c, "curl -sSL https://get.docker.com/ | sh" ]
- [ sh, -c, "sudo curl -L "https://github.com/docker/compose/releases/download/$(git ls-remote https://github.com/docker/compose | grep refs/tags | grep -oP "[0-9]+\.[0-9][0-9]+\.[0-9]+$" | tail -n 1)/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose" ]
- [ sh, -c, "sudo chmod +x /usr/local/bin/docker-compose" ]
But it doesn't install the docker/ docker-compose and get below error
cloud-init[958]: 2019-02-28 00:51:02,447 - util.py[WARNING]: Failed loading yaml blob. Invalid format at line 11 column 32: "while scanning a plain scalar
[ 333.241244] cloud-init[958]: in "<unicode string>", line 11, column 32:
[ 333.245521] cloud-init[958]: - [ sh, -c, "sudo curl -L "https://github.com/docker/compos ...
2019/02/28 00:51:04.015216 INFO Daemon Wire protocol version:2012-11-30
[ 333.245619] cloud-init[958]: ^
[ 333.259509] cloud-init[958]: found unexpected ':'
[ 333.259584] 2019/02/28 00:51:04.019282 INFO Daemon Server preferred version:2015-04-05
cloud-init[958]: in "<unicode string>", line 11, column 37:
[ 333.474295] cloud-init[958]: - [ sh, -c, "sudo curl -L "https://github.com/docker/compose/rel ...
[ 333.514672] cloud-init[958]: ^
[ 333.554215] cloud-init[958]: Please check http://pyyaml.org/wiki/YAMLColonInFlowContext for details."
[ 333.593764] cloud-init[958]: 2019/02/28 00:51:04.361760 INFO Daemon Waiting for ssh host key be generated at /etc/ssh/ssh_host_rsa_key.pub [1800 attempts remaining, sleeping 1s]
2019-02-28 00:51:02,470 - util.py[WARNING]: Failed loading yaml blob. Invalid format at line 11 column 32: "while scanning a plain scalar
[ 333.705350] cloud-init[958]: in "<unicode string>", line 11, column 32:
[ 333.731074] cloud-init[958]: - [ sh, -c, "sudo curl -L "https://github.com/docker/compos ...
[ 333.769118] cloud-init[958]: ^
[ 333.789992] cloud-init[958]: found unexpected ':'
[ 333.808855] cloud-init[958]: in "<unicode string>", line 11, column 37:
[ 333.831502] cloud-init[958]: - [ sh, -c, "sudo curl -L "https://github.com/docker/compose/rel ...
[ 333.863028] cloud-init[958]: ^
[ 333.885648] cloud-init[958]: Please check http://pyyaml.org/wiki/YAMLColonInFlowContext for details."
[ 333.915174] cloud-init[958]: 2019-02-28 00:51:02,470 - util.py[WARNING]: Failed at merging in cloud config part from part-001
Apart from it I doubt how can I add Azure auto created default user (which was provided from UI) to docker group.
Just tested by myself on Azure using Ubuntu 18.04-LTS:
#cloud-config
package_upgrade: true
packages:
- docker.io
- docker-compose
# create the docker group
groups:
- docker
# assign a VM's default user, which is mydefaultuser, to the docker group
users:
- default
- name: mydefaultuser
groups: docker
runcmd can be left empty for this setup.
you have " inside of other " which doesnt work without escaping. as for the user. I think default user created has id of 1000, you can just use that (probably test this hypothesis before).
After spending some time I solved it myself.
#cloud-config
package_update: true
# Setup swap memory
disk_setup:
ephemeral0:
table_type: mbr
layout: [66, [33, 82]]
overwrite: True
fs_setup:
- device: ephemeral0.1
filesystem: ext4
- device: ephemeral0.2
filesystem: swap
mounts:
- ["ephemeral0.1", "/mnt"]
- ["ephemeral0.2", "none", "swap", "sw", "0", "0"]
# Enable Docker's swap limit support
bootcmd:
- [ sh, -c, 'sudo echo GRUB_CMDLINE_LINUX="cgroup_enable=memory swapaccount=1" >> /etc/default/grub' ]
- [ sh, -c, 'sudo update-grub' ]
- [ cloud-init-per, once, mymkfs, mkfs, /dev/vdb ]
# Install docker and docker-compose
runcmd:
- [ sh, -c, 'curl -sSL https://get.docker.com/ | sh' ]
- [ sh, -c, 'sudo curl -L https://github.com/docker/compose/releases/download/$(curl -s https://api.github.com/repos/docker/compose/releases/latest | grep "tag_name" | cut -d \" -f4)/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose' ]
- [ sh, -c, 'sudo chmod +x /usr/local/bin/docker-compose' ]
# Add default auto created user to docker group
system_info:
default_user:
groups: [docker]

Gitlab-CE reconfigure time out on step "migrate gitlab-rails database"

I try to update gitlab from version 9.5.10 to latest 10. I do 3 steps:
touch /etc/gitlab/skip-auto-migrations ( I don't need backup )
yum -y install gitlab-ce-10.8.7-ce.0.el7.x86_64
gitlab-ctl reconfigure
Gitlab crashes with an error after one hour:
...
Error executing action `run` on resource 'bash[migrate gitlab-rails database]'
================================================================================
Mixlib::ShellOut::CommandTimeout
--------------------------------
Command timed out after 3600s:
Command exceeded allowed execution time, process terminated
---- Begin output of "bash" "/tmp/chef-script20180907-58298-lhgdju" ----
STDOUT: == 20141126120926 AddMergeRequestRebaseEnabledToProjects: migrating ===========
-- transaction_open?()
-> 0.0000s
...
63: end
64: not_if "(test -f #{db_migrate_status_file}) && (cat #{db_migrate_status_file} | grep -Fx 0)"
65: only_if { node['gitlab']['gitlab-rails']['auto_migrate'] }
66: end
Compiled Resource:
------------------
# Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb:49:in `from_file'
bash("migrate gitlab-rails database") do
action [:run]
default_guard_interpreter :default
command nil
backup 5
returns 0
user nil
interpreter "bash"
declared_type :bash
cookbook_name "gitlab"
recipe_name "database_migrations"
code " set -e\n log_file=\"/var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log\"\n umask 077\n /opt/gitlab/bin/gitlab-rake gitlab:db:configure 2>& 1 | tee ${log_file}\n STATUS=${PIPESTATUS[0]}\n echo $STATUS > /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-141d998df14e11465b19f94d3c4ccfa1-eb600b0\n exit $STATUS\n"
domain nil
not_if "(test -f /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-141d998df14e11465b19f94d3c4ccfa1-eb600b0) && (cat /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-141d998df14e11465b19f94d3c4ccfa1-eb600b0 | grep -Fx 0)"
only_if { #code block }
end
System Info:
------------
chef_version=13.6.4
platform=centos
platform_version=7.4.1708
ruby=ruby 2.3.7p456 (2018-03-28 revision 63024) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/chef-client
executable=/opt/gitlab/embedded/bin/chef-client
Recipe: gitlab::unicorn
* service[unicorn] action restart
ESC[32m- restart service service[unicorn]
Recipe: gitlab::sidekiq
* service[sidekiq] action restart
ESC[32m- restart service service[sidekiq]
Recipe: gitlab::gitlab-rails
* execute[clear the gitlab-rails cache] action run
ESC[32m- execute /opt/gitlab/bin/gitlab-rake cache:clear
Running handlers:
Running handlers complete
Chef Client failed. 25 resources updated in 01 hours 01 minutes 28 seconds
I found workaround but I don't want to risk changing the update procedure. GitLab is used in the production and use of more than 1000 people (An update check is performed on the copy of the database).
Does anyone know how to increase the timeout?
Ok, turning this into an answer so other people might be able to use this:
Unless someone knows of a way to override the default timeout for Mixlib::ShellOut in this case, it is possible to override that default of 3600 in opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/chef-13.6.4/lib/chef/provider/execute.rb line 44-48, which currently read:
def timeout
# original implementation did not specify a timeout, but ShellOut
# *always* times out. So, set a very long default timeout
new_resource.timeout || 3600
end
Increasing the 3600 there should help.

Cron generated by ISPConfig

I have a problem with crontab generated by ISPConfig.
MAILTO=''
* * * * * web9 /usr/bin/wget -q -O /dev/null 'http://inz.isedo.pl/test/cron.php' >/dev/null 2>&1 #inz.isedo.pl
In log, I have a errors:
Feb 16 21:11:01 s /usr/sbin/cron[21697]: (*system*ispc_web9) RELOAD (/etc/cron.d/ispc_web9)
Feb 16 21:11:01 s /USR/SBIN/CRON[23817]: (web9) CMD (/usr/bin/wget -q -O /dev/null 'http://inz.isedo.pl/test/cron.php' >/dev/null 2>&1^I#inz.isedo.pl)
Feb 16 21:11:01 s /USR/SBIN/CRON[23816]: (CRON) error (grandchild #23817 failed with exit status 1)
Does it work from the command line ?
Can you redirect the stdout and stderr to a file (as opposed to null) and pass on the output ?

Resources