A cygwin 32-bit version installed on windows 7 64-bit has worked well for a few years. Lately it started to have a lot more errors with fork problem. An example is like:
+ cnt=44
+ '[' 44 -le 50 ']'
40 [main] bash 3348 fork: child -1 - forked process 4248 died unexpectedly, retry 0, exit code -1073741819, errno 11
./testsh: fork: retry: Resource temporarily unavailable
++ printf %03d 44
The example is by running this script:
$ cat testsh
#!/bin/bash -x
cnt=0
while [ $cnt -le 50 ]; do
echo $( printf "%03d" $cnt )
cnt=$(($cnt + 1))
done
Any idea about the specific cause?
Since cygwin has been installed long ago and working well, it cannot be updated. I know it might be better to re-install a late version, but the question is what caused the problem?
Moving my comment to an answer.
As I said above, this error usually indicates that you need to do a rebaseall.
See Here: https://stackoverflow.com/a/14509551/1549185
Related
These are my test files:
std-test.sh:
#!/bin/bash
for i in {0..20}
do
number=$RANDOM
let "number %= 10"
if [ $number -le 6 ]
then
echo $i;
else
echo $i 1>&2;
fi
done
process.sh:
#!/bin/bash
while read line; do
[[ $1 = "err" ]] && echo "$(date +%s%3N) $line" >> stderr;
[[ $1 = "out" ]] && echo "$(date +%s%3N) $line" >> stdout;
done
std-test.sh creates twenty lines containing their line number, and process.sh is able to read another commands output when piped to it, while it saves to stderr when err is passed to it as argument, and to stdout when out is passed. The command I use to test the whole thing is:
./std-test.sh 2> >(./process.sh err) > >(./process.sh out) (taken from here). My test outputs are as follows:
stdout:
1486297129986 0
1486297129987 1
1486297129988 2
1486297129988 3
1486297129989 4
1486297129990 6
1486297129991 7
1486297129991 8
1486297129992 9
1486297129993 10
1486297129993 11
1486297129994 12
1486297129995 14
1486297129995 17
1486297129996 18
stderr:
1486297129986 5
1486297129987 13
1486297129987 15
1486297129988 16
1486297129989 19
1486297129990 20
When I tried parsing the output, I realized, that the order of stderr and stdout is completely messed up. Why does 5 in stderr come before 1 in stdout, for instance? Or 13 right after 1? What's the mistake I made here, and how could I resolve it?
Edit #1: I know I could pipe stderr and stdout directly to files, but process.sh will curl the results to a remote server. Saving it to files is just kind of a POC.
Edit #2: I am actually trying to timestamp and process docker logs programmatically.
Update:
From the comments it appears OP was really trying to timestamp and log the output from an application running in a docker container.
docker logs -t already does this and is the best tool for job.
Original answer:
Probably because you're timestamping each line with the time it is processed and not the time they're actually generated?
You launch 3 processes when you run that command, not all of them run in parallel.
The processes aren't running in a truly parallel fashion, they can get preempted whenever the kernel feels like. Because the stdout is longer/processing more lines, it is more likely to get preempted and then resumed at a later point. Which would explain the timestamping.
What do you really need the timestamps on the lines for? Do you simply want ordering between the lines, or do you want real true timestamps?
Edit: as #Sven Festersen pointed out in the comments, it also might be because stdout is buffered when piped by default.
When I set up a Jenkins job and found a problem about timeout for shell script.
It works like this:
Start Jenkins → control.sh is launched → test1.sh is launched in control.sh
Part code of control.sh is like:
#!/bin/sh
source func.sh
export TIMEOUT=30
# set timeout as 30s for test1.sh
( ( sleep $TIMEOUT && function_Timeout ) & ./test1.sh )
# this line of code is in a = loop actually
# it will launch test2.sh, test3.sh... one by one
# later, I want to set 30s time out for each of them.
function_Timeout() {
if [ ! -f test1_result_file]: then
killall test1.sh
# the test1_result_file will not
# be created if test1.sh is not finished executing.
fi
}
part of func.sh is as below
#!/bin/sh
function trap_fun() {
TRAP_CODE=$?
{ if [ $TRAP_CODE -ne 0 ]; then
echo "test aborted"
else
echo "test completed"
} 2>/dev/null
trap "trap_fun" EXIT
After control.sh is launched by Jenkins job, the whole control.sh will be terminated when time is over, and the line of killall test1.sh is reached, and the Jenkins job stop and fail.
I guess it's because test1.sh is killed and exit code is not 0, so it cause this problem.
So my question is, is there someway to terminate or end the sub-script (launched by the main one, like control.sh in my case) exit with code 0?
Updated on July 1:
Thanks for the answers so far, I tried #Leon's suggestion, but I found the code 124 sent by timeout's kill action, is still caught by the trap code - trap "trap_fun" EXIT, which is in func.sh.
I added more details. I did a lot google job but still not found a proper way to resolve this problem:(
Thanks for your kind help!
Use the timeout utility from coreutils:
#!/bin/sh
timeout 30 ./test1.sh
status=$?
if [ $status -eq 124 ] #timed out
then
exit 0
fi
exit $status
Note that this is slightly different from your version of timeout handling, where all running instances of test1.sh are being terminated if any one of them times out.
I resolved this problem finally, I added the code below in each testX.sh.
trap 'exit 0' SIGTERM SIGHUP
It is to make test1.sh exit normally after it receives killall signal.
Thanks to all the help!
I'd like to provide an optional logging parameter in a bash script, and would like to use exec to tee a pipe from the beginning. However, opening the tee process is causing the script to hang, I believe because stdout is not closed:
# Output to a log file, if set
if [[ $OPT_LOG ]]; then
exec > >(tee -a $OPT_LOG)
fi
I've attempted to close with:
exec >&-
But it still hangs - is there another way to properly close tee so the script will exit properly at the end of execution?
It seems like, for some reason, using tee stops the prompt ($PS1) from appearing because the shell script has not exited. As a workaround, I generally use a short sleep after calling tee.
#!/bin/bash
exec > >(tee -a mylog)
sleep .1
# my code
Converting comments into an answer, with minor edits.
I noted:
The following code exits OK for me on Mac OS X 10.10.3.
OPT_LOG=file.name
if [[ $OPT_LOG ]]
then exec > >(tee -a $OPT_LOG)
fi
for ((i = 0; i < 10; i++))
do
echo "Logging message $i at $(date)"
sleep 1
done
Your problem is probably in the code you've not shown. What commands are you running? What do you get from bash -x yourscript.sh?
And chatraed observed
If you remove the sleep and date calls from your example, the script does not exit properly any more, as told by Andrew:
OPT_LOG=file.name;
if [[ $OPT_LOG ]]; then exec > >(tee -a $OPT_LOG); fi;
for ((i = 0; i < 10; i++)); do echo "Logging message $i"; done
And I responded:
Now that's an interesting observation! I can reproduce your result.
I experimented a bit:
I added a pwd before the for loop with 'just the echo', and that didn't affect things (but pwd is probably a Bash built-in).
I added bash -c 'exit 0' (which is not a built-in) and the code terminated OK.
I tried using > >(tee -a $OPT_LOG &) and the output didn't appear — on screen or in the file. (I find this surprising, but it was an attempted workaround, not a major part of the work.)
My impression is that Andrew has found a bug in Bash that could viably be reported. (See the Bash manual on Reporting Bugs for how to do that.) I don't think that not exiting simply because no external commands have been executed since the I/O redirection is OK. I can confirm that Apple's Bash 3.2.57 has the problem; so does Bash 4.3.27 (which I built for myself so it is patched with the fixes to the ShellShock bug).
It is relatively unusual that a shell script does not invoke any external command, and invoking any external command after the exec redirection seems to suppress the bug.
It also confirms that chatraed's workaround works, albeit more slowly than minimally necessary. For production use, I'd choose sleep 0, which is most of a tenth of a second faster than sleep 0.1. And sleeping for a fraction of a second only works on some systems; classically (and according to POSIX), sleep does not sleep for fractional seconds. OTOH, if you write sleep 0.1, on systems without support for fractional seconds, including the leading 0 probably makes it sleep for zero seconds anyway; writing .1 might or might not have the same effect.
I am running a script to install Oracle DB 11g with silent option and response file.
I noticed the shell after executing the command
$ /directory_path/runInstaller -silent -responseFile responsefilename
The install session just closes just giving me a log file location.
The installation process is active in the background. But for me no way to no about the progress and what is going on... till the prompt to run root scripts come. What if I close the putty windows etc?
Any way to keep the installer session active till finished ? and show some sort of progress on-screen ?
Any way to keep the installer session active till finished ?
Yes, you can wait for oracle silent install to finish on linux eg in a shell script as follows.
(Below was with oracle 11g release 2 for Redhat Enterprise Linux.)
You can wait for it to finish by doing:
$ /directory_path/runInstaller -silent -responseFile responsefilename |
while read l ;
do
echo "$l" ;
done
(this relies on the fact that even though the java universal installer runs in the background it keeps using stdout, so "read l" keeps succeeding until the background universal installer process exits)
and show some sort of progress on-screen ?
A bit more tricky but we can do it by figuring out the name of the logfile from the output of runInstaller before it exits. The output contains a line like:
Preparing to launch Oracle Universal Installer from /tmp/xxxxOraInstallTTT. ...
... where the TTT is a timestamp which leads us to the correct log file, /opt/oraInventory/logs/installActionsTTT.log.
Something like (I have not tested this because in my install I do not need progress output):
$ /directory_path/runInstaller -silent -responseFile responsefilename |
(
while read l ;
do
echo "$l" &&
if expr "$l" : "Preparing to launch Oracle Universal Installer from " >/dev/null
then
t=$(expr "$1" : ".*OraInstall\([^.]*\)") &&
log="/opt/oraInventory/logs/installActions${t}.log" &&
tail -f "$log" &
tpid=$!
fi
done
if [ -n "$tpid" ]
then
kill $tpid
fi
#[1]
)
... we can also tell if the install succeeded, because the universal installer always puts its exit status into the log via the two lines:
INFO: Exit Status is 0
INFO: Shutdown Oracle Database 11g Release 2 Installer
... so by adding to the above at #[1] ...
exitStatus=$(expr $(grep -B1 "$log" | head -1) : "INFO: Exit Status is\(.*\)") &&
exit $exitStatus
... the above "script" will exit with 0 status only if the oracle instal completes successfully.
(Note that including the space in the status captured by expr just above is deliberate, because bizarrely expr exits with status 1 if the matched substring is literally "0")
It is astounding that oracle would go to so much trouble to "background" the universal installer on linux/unix because:
it is trivial for the customer to generically run a script in the background:
runInstaller x y z &
... or...
setsid runInstaller x y z
it is very difficult (as we can see above) to wait for a "buried" background process to finish, and it cannot be done generically
Oracle would have saved themselves and everyone else lots of effort by just running the universal installer syncronously from runInstaller/.oui.
One useful option I found is -waitforcompletion and -nowait for windows. setup.exe will wait for completion instead of
spawning the java engine and exiting.
But still trying to figure out a way to tail that log file automatically when running the runInstaller.
UPDATE: Finally this is the only solution that worked for me.
LOGFILE=$(echo /path/oraInventory/logs/$(ls -t /path/oraInventory/logs | head -n 1))
./runInstaller -silent -responseFile /path/db.rsp -ignorePrereq
grep -q 'INFO: Shutdown Oracle Database' $LOGFILE
while [[ $? -ne 0 ]] ; do
tail -1 $LOGFILE
grep -q 'INFO: Shutdown Oracle Database' $LOGFILE
done
Instead of tailing the latest log in oraInventory, kept the process active till the last message is written to the log file. Old school but works.
It's been a while since I did an installation, but I don't think you can make it stay in the foreground; it just launches the JVM and that doesn't (AFAIK) produce any console output until the end, and you see that anyway, so it wouldn't help you much.
You should have plenty to look at though, in the logs directory under your oraInventory directory, which is identified in the oraInst.loc file you either created for the installation or already had, and might have specified on the command line. You mentioned it told you where the log file is when it started, so that would seem a good place to start. The installActions log in that directory will give you more detail than you probably want.
If you do close the terminal, the silentInstall and oraInstall logs tell you when the root scripts need to be run, and the latter even gives you the progress as a percentage - though I'm not sure how reliable that is.
Tailing the oraInstall or installActions files is probably the only way you can show some sort of progress on screen. 'Silent' works both ways, I suppose.
You probably want to "tail" the most up to date logs(ls -t would give that) generated under /tmp/OraInstallXXX directory.
I want to limit the execution time of a program I am running under Linux. I put in my scons script a line like:
Command("com","","ulimit -t 1; myprogram")
and tested it with an infinite loop program: it did not work and the program ran forever.
Am I missing something?
-- tsf
ulimit -t 1 means that the limit is set to 1 second of CPU time. If your infinite loop program uses any sort of sleep in its inner loop then it will use practically no CPU time. This means it will not get killed in 1 second of real, on the clock time. In fact it may take minutes or hours to use up its 1 second allocation.
What happens if you run the command outside of SCons? Perhaps you don't have permission to change the limit at all...
ulimit -t 1; ./myprogram
For example, it may say the following if the limit is already set to 0:
bash: ulimit: cpu time: cannot modify limit: Operation not permitted
Edit: it seems that the -t option is broken on Ubuntu 9.04. A fix has been committed 05 June 2009, but it may take a while to trickle into the updates - it may not be fixed until 9.10.
As an historical note, this problem no longer exists in Ubuntu 10.04.
You can also use this script:
(taken from http://newsgroups.derkeiler.com/Archive/Comp/comp.sys.mac.system/2005-12/msg00247.html)
#!/bin/sh
# timeout script
#
usage()
{
echo "usage: timeout seconds command args ..."
exit 1
}
[[ $# -lt 2 ]] && usage
seconds=$1; shift
timeout()
{
sleep $seconds
kill -9 $pid >/dev/null 2>/dev/null
}
eval "$#" &
pid=$!
timeout &
wait $pid
.