I wanted to run a particular function in .bashrc script file ( which actually does a job of removing a docker exited containers in the background)
I already looked into cron but it is not useful for me please suggest any other methods to do it.
I also tried writing a while loop along with sleep which is not the efficient method as we start it every time and stop it.
First choice is cron, but you can also use at.
Here is a little example. The script is started once per minute and loggt each run into logfile.dat
#!/bin/bash
echo "bash $0" | at now +1 minutes -M
date >> /tmp/logfile.dat
With atq you can see witch jobs waiting for next run an with atrm you can stop the cycle.
==> man at
I don't necessarily consider this a great idea either, but to answer the question you asked...
Here's a simple template you should be able to adapt.
chime() {
local chimeDelay=10 # seconds, adjust to your needs
echo "bong!"; date; # code that Does The Thing
sleep $chimeDelay && chime & # snooze and Do The Thing again
} >/tmp/chimelog 2>/tmp/chime.err # logs, not your console
Once you execute this it should keep spawning as long as you are logged in, but ought to collapse on a HUP, which I assume is what you wanted. If you just wanted a cron substitute, then write and run it as a simplistic daemon with a HUP trap, but you probably should add locks to keep multiple instances from running, etc.
Related
I want to prevent a bash command from executing that has been chained using ; from running while the previous command is still running.
e.g. I write and submit command a; command b, but while command a is running I change my mind and want to prevent command b from running.
I cannot use kill because the subsequent command is not actually executing. Does bash have a queue of commands that can be manipulated?
To clarify, I am sure it is possible to make a new script or something that would allow me to create a queue, but that is not what this question is about. I specifically want to know if bash can prevent commands after a semicolon from running after I've 'submitted' them.
Consider these two scripts:
runner.sh
#!/bin/bash
while true
do
next_command=$(head -1 next_commands.list)
$next_command
sleep 60 #added to simulate processing time
done
next_commands.list
id
ls
echo hello
You can modify the content of the next_commands.list file to create a type of queue of which commands should be executed next.
Good morning all,
I am trying to implement concurrency in a very specific environment, and keep getting stuck. Maybe you can help me.
this is the situation:
-I have N nodes that can read/write in a shared folder.
-I want to execute an application in one of them. this can be anything, like a shell script, an installed code, or whatever.
-To do so, I have to send the same command to all of them. The first one should start the execution, and the rest should see that somebody else is running the desired application and exit.
-The execution of the application can be killed at any time. This is important because does not allow relying on any cleaning step after the execution.
-if the application gets killed, the user may want to execute it again. He would then send the very same command.
My current approach is to create a shell script that wraps the command to be executed. This could also be implemented in C. Not python or other languages, to avoid library dependencies.
#!/bin/sh
# (folder structure simplified for legibility)
mutex(){
lockdir=".lock"
firstTask=1 #false
if mkdir "$lockdir" &> /dev/null
then
controlFile="controlFile"
#if this is the first node, start coordinator
if [ ! -f $controlFile ]; then
firstTask=0 #true
#tell the rest of nodes that I am in control
echo "some info" > $controlFile
fi
# remove control File when script finishes
trap 'rm $controlFile' EXIT
fi
return $firstTask
}
#The basic idea is that a task executes the desire command, stated as arguments to this script. The rest do nothing
if ! mutex ;
then
exit 0
fi
#I am the first node and the only one reaching this, so I execute whatever
$#
If there are no failures, this wrapper works great. The problem is that, if the script is killed before the execution, the trap is not executed and the control file is not removed. Then, when we execute the wrapper again to restart the task, it won't work as every node will think that somebody else is running the application.
A possible solution would be to remove the control script just before the "$#" call, but that it would lead to some race condition.
Any suggestion or idea?
Thanks for your help.
edit: edited with correct solution as future reference
Your trap syntax looks wrong: According to POSIX, it should be:
trap [action condition ...]
e.g.:
trap 'rm $controlFile' HUP INT TERM
trap 'rm $controlFile' 1 2 15
Note that $controlFile will not be expanded until the trap is executed if you use single quotes.
I think it's related to the parent process creating new subprocess and does not have tty. Can anyone explain the detail under the hood? i.e. the related working model of bash, process creation, etc?
It may be a very broad topic so pointers to posts are also very appreciated. I've Googled for a while, all the results are about very specific case and none is about the story behind the scene. To provide more context, below is the shell script resulting the 'bash: no job control in this shell'.
#! /bin/bash
while [ 1 ]; do
st=$(netstat -an |grep 7070 |grep LISTEN -o | uniq)
if [ -z $st ]; then
echo "need to start proxy #$(date)"
bash -i -c "ssh -D 7070 -N user#my-ssh.example.com > /dev/null"
else
echo "proxy OK #$(date)"
fi
sleep 3
done
This line:
bash -i -c "ssh -D 7070 -N user#my-ssh.example.com > /dev/null"
is where "bash:no job control in this shell” come from.
You may need to enable job control:
#! /bin/bash
set -m
Job control is a collection of features in the shell and the tty driver which allow the user to manage multiple jobs from a single interactive shell.
A job is a single command or a pipeline. If you run ls, that's a job. If you run ls|more, that's still just one job. If the command you run starts subprocesses of its own, then they will also belong to the same job unless they are intentionally detached.
Without job control, you have the ability to put a job in the background by adding & to the command line. And that's about all the control you have.
With job control, you can additionally:
Suspend a running foreground job with CtrlZ
Resume a suspended job in the foreground with fg
Resume a suspend job in the background with bg
Bring a running background job into the foreground with fg
The shell maintains a list of jobs which you can see by running the jobs command. Each one is assigned a job number (distinct from the PIDs of the process(es) that make up the job). You can use the job number, prefixed with %, as an argument to fg or bg to select a job to foreground or background. The %jobnumber notation is also acceptable to the shell's builtin kill command. This can be convenient because the job numbers are assigned starting from 1, so they're shorter than PIDs.
There are also shortcuts %+ for the most recently foregrounded job and %- for the previously foregrounded job, so you can switch back and forth rapidly between two jobs with CtrlZ followed by fg %- (suspend the current one, resume the other one) without having to remember the numbers. Or you can use the beginning of the command itself. If you have suspended an ffmpeg command, resuming it is as easy as fg %ff (assuming no other active jobs start with "ff"). And as one last shortcut, you don't have to type the fg. Just entering %- as a command foregrounds the previous job.
"But why do we need this?" I can hear you asking. "I can just start another shell if I want to run another command." True, there are many ways of multitasking. On a normal day I have login shells running on tty1 through tty10 (yes there are more than 6, you just have to activate them), one of which will be running a screen session with 4 screens in it, another might have an ssh running on it in which there is another screen session running on the remote machine, plus my X session with 3 or 4 xterms. And I still use job control.
If I'm in the middle of vi or less or aptitude or any other interactive thing, and I need to run a couple of other quick commands to decide how to proceed, CtrlZ, run the commands, and fg is natural and quick. (In lots of cases an interactive program has a ! keybinding to run an external command for you; I don't think that's as good because you don't get the benefit of your shell's history, command line editor, and completion system.) I find it sad whenever I see someone launch a secondary xterm/screen/whatever to run one command, look at it for two seconds, and then exit.
Now about this script of yours. In general it does not appear to be competently written. The line in question:
bash -i -c "ssh -D 7070 -N user#my-ssh.example.com > /dev/null"
is confusing. I can't figure out why the ssh command is being passed down to a separate shell instead of just being executed straight from the main script, let alone why someone added -i to it. The -i option tells the shell to run interactively, which activates job control (among other things). But it isn't actually being used interactively. Whatever the purpose was behind the separate shell and the -i, the warning about job control was a side effect. I'm guessing it was a hack to get around some undesirable feature of ssh. That's the kind of thing that when you do it, you should comment it.
One of the possible options would be not having access to the tty.
Under the hood:
bash checks whether the session is interactive, if not - no job
control.
if forced_interactive is set, then check that stderr is
attached to a tty is skipped and bash checks again, whether in can
open /dev/tty for read-write access.
then it checks whether new line discipline is used, if not, then job control is disabled too.
If (and only if) we just set our process group to our pid, thereby becoming a process group leader, and the terminal is not in the same process group as our (new) process group, then set the terminal's process group to our (new) process group. If that fails, set our process group back to what it was originally (so we can still read from the terminal) and turn off job control.
if all of the above has failed, you see the message.
I partially quoted the comments from bash source code.
As per additional request of the question author:
http://tiswww.case.edu/php/chet/bash/bashtop.html Here you can find bash itself.
If you can read the C code, get the source tarball, inside it you will find job.c - that one will explain you more "under the hood" stuff.
I ran into a problem on my own embedded system and I got rid of the "no job control" error by running the getty process with "setsid", which according to its manpage starts a process with a new session id.
Faced this problem only because I've miscopied a previously executed command together with % prefix into zsh, like % echo this instead of echo this. The error was very unclear to such a stupid typo
First the background to this intriguing challenge. The continuous integration build can often have failures during development and testing of deadlocks, loops, or other issues that result in a never ending test. So all the mechanisms for notifying that a build has failed become useless.
The solution will be to have the build script timeout if there's zero output to the build log file for more than 5 minutes since the build routinely writes out the names of unit tests as it proceeds. So that's the best way to identify it's "frozen".
Okay. Now the nitty gritty...
The build server uses Hudson to run a simple bash script that invokes the more complex build script based on Nant and MSBuild (all on Windows).
So far all solutions around the net involve a timeout on the total run time of the command. But that solution fails in this case because the tests might hang or freeze in the first 5 minutes.
What we've thought of so far:
First, here's the high level bash command run the full test suite in Hudson.
build.sh clean free test
That command simply sends all the Nant and MSBuild build logging to stdout.
It's obvious that we need to tee that output to a file:
build.sh clean free test 2>&1 | tee build.out
Then in parallel a command needs to sleep, check the modify time of the file and if more than 5 minutes kill the main process. A kill -9 will be fine at that point--nothing graceful needed once it has frozen.
That's the part you can help with.
In fact, I made a script like this over 15 years ago to kill the connection with a data phone line to japan after periods of inactivity but can't remember how I did it.
Sincerely,
Wayne
build.sh clean free test 2>&1 | tee build.out &
sleep 300
kill -KILL %1
You may be able to use timeout:
timeout 300 command
Solved this myself by writing a bash script.
It's called iotimeout with one parameter which is the number of seconds.
You use it like this:
build.sh clean dev test | iotimeout 120
iotimeout has 2 loops.
One is a simple while read line loop that echos echo line but
it also uses the touch command to update the modified time of a
tmp file every time it writes a line. Unfortunately, it wasn't
possible to monitor a build.out file because Windoze doesn't
update the file modified time until you close the file. Oh well.
Another loop runs in the background, that's a forever loop
which sleeps 10 seconds and then checks the modified time
of the temp file. If that ever exceeds 120 seconds old then
that loop forces the entire process group to exit.
The only tricky stuff was returning the exit code of the original
program. Bash gives you a PIPESTATUS array to solve that.
Also, figuring out how to kill the entire program group was
some research but turns out to be easy just--kill 0
I need to stop the overlapping of cron jobs for example:If a cron job is scheduled at morning 2 o clock for DB backup and other cron job is scheduled at morning 7 o clock for DB backup again.So i need to stop the 7 o clock scheduled cron job if the DB backup for 2 o clock is not completed.
This is what flock is for. From man flock:
...
The third form is convenient inside shell scripts, and is usually used
the following manner:
(
flock -n 9 || exit 1
# ... commands executed under lock ...
) 9>/var/lock/mylockfile
...
In this case, flock will try to get a lock on fd 9 for /var/lock/myfile. If it can't (because the earlier job is still running), it will exit.
You need to put it in a script.
The most straightforward way to do this would be to have your 2am and 7am tasks coordinate. So for instance, the DB backup script could create a lock file when it executes, and check for the existence of this file when it starts up. If the file exists it could simply exit without doing any work (or sleep for a bit, or whatever you decide).
Doing this in a completely task-agnostic way might be possible, but it's going to be trickier and unnecessary in this particular case.
Assuming you are on a Unix system: The easiest way to do this is
create a file and write the cron job's PID to it. If the file can't be
created using open(2) with O_CREAT|O_EXCL flags, it already
exists, thus the previous job is still running or has crashed without
cleaning up. The PID can be used to determine whether the job is still
running.
In your case, the safest approach is probably for the cron job to die
if the PID file is found on startup.
Since you put the perl tag to the question, I suggest to have a look
at the File::Pid and Proc::PID::File modules.
You just need something that you can check to see if the first task has finished running, and this check should be done right before the job runs again.
I'd implement this with a database (if available) or a registery key or even a text file. So your task would look like this:
read job_flag
if job_flag == 0
step job_flag = 1
run job
step job_flag = 0
else
stop job
end
Why not just do this in a single cron job?
/path/to/job1 && /path/to/job2
Here is my solution:
$ crontab -l
* * * * * cd /patch/to/script; . /patch/to/.profile; overlap.pl process.sh -t Device || process.sh -t Device
$
$
$ cat overlap.pl
#!/usr/bin/env perl
use warnings;
use strict;
my $cmd = join(' ',#ARGV);
my #result = `pgrep -f \"$cmd\"`;
scalar #result > 2 ? exit 0 : exit 1;
EXPLANATION:
this job runs script every minute to check if it works (* * * * *)
cron job is complex, fisrt change directory, load profile so I need to launch script directly from this job not from script that is checking if overlapped
and then check if this particular process with the same parameters is already running (overlap.pl with exact command that I want to check)
if "no", return 0, if "yes", return >0
then there is logical "or" checking result and launch desired script or not
Pay attention to the array length in overlap.pl, you can check it by yourself by doing `ps -elf | grep \"$cmd\" >> /tmp/croncheck`;
You can do that just using cron. You are going to have to add some logic to your script or code to check if the other one is running or not. A total hack would be to have your script check for the exists of a particular file. If it does not exist then you should create this file, perform the backup, then remove it. If the file does exist, do nothing.
That is a hack and only really works because your jobs are spaced so far apart.