I am trying to take output of an command into array and for this tried following:
#!/bin/sh
cmd=($(date +%s;sleep 5; date +%s))
start_time=$cmd[0]
end_time=$cmd[1]
echo $start_time
#EOF
I was expection echo $start_time to give me start time but it print the following:
1572443382 1572443386[0]
Can't switch to bash shell and have only access to sh
Plain sh has no arrays. You have to cope without arrays. In your case that's easy:
start=$(date +%s)
sleep 5
end=$(date +%s)
echo "start=$start end=$end"
If you really, really want to have everything in one subshell then you have to store the output as a plain string and parse that string to retrieve the individual values. You can think of that one string as an "array" where each line is an array entry. Individual lines can be retrieved using sed (which uses indices starting from 1 instead of 0).
times=$(date +%s; sleep 5; date +%s)
echo "start=$(echo "$times" | sed -n 1p) end=$(echo "$times" | sed -n 2p)"
To store individual lines in variables use subshells:
times=$(date +%s; sleep 5; date +%s)
start=$(echo "$times" | sed -n 1p)
end=$(echo "$times" | sed -n 2p)
echo "start=$start end=$end"
However, if you just want to compute how long sleep 5 took you might as well use time sleep 5 which already does that for you.
The positional parameters are the closest thing sh has to an array:
sh-3.2$ set -- "$(date +%s)"; sleep 5; set -- "$#" "$(date +%s)"
sh-3.2$ start=$1 end=$2; echo "$start -> $end"
1572448562 -> 1572448567
You need to use ${array[index]} syntax to access array elements. So, change your start_time and end_time to as follows.
start_time=${cmd[0]}
end_time=${cmd[1]}
Also change
#!/bin/sh
to
#!/bin/bash
Related
I wrote small script under Debian Linux 11 that should check how many instances of application is currently running and what is power usage of GPU cards.
I save it under name test , and she is started every time I access instance over SSH
#!/bin/sh
clear
a=$(nvidia-smi -q -i 0 | grep "Power Draw" | cut -c45-50)
b=$(nvidia-smi -q -i 1 | grep "Power Draw" | cut -c45-50)
c=$(nvidia-smi -q -i 2 | grep "Power Draw" | cut -c45-50)
d=$(nvidia-smi -q -i 3 | grep "Power Draw" | cut -c45-50)
zet=$( echo "$a" + "$b" + "$c" + "$d" | bc -l )
echo "SYSTEM DRAW:" "$zet"
if [ "${zet}" -gt 150 ]; then
echo WARRNING - SYSTEM DRAW LOW
else
echo OK
fi
sleep 8
exit
All I need to add is this line
x=${x%.*}
That convert decimal number in number without decimals and script works perfect.
You could add set -x right before the part you want to debug, which will show you a debug of what is happening in bash. and stop it by inserting after that set +x
like:
set -x
power=$150
echo "SYSTEM DRAW :" $total
if [ $total \> $power ] ; then # escape > otherwise it redirects output
I do not think you are setting the value of $150
The script might be failing if one of the compared values is not set.. so you should initialize your variables to be let' say equal to 0 as a default at the beginning of your script, or via bash
like:
power=${150:-10} # if $150 does not have a value or empty, the default value of $power will be set to `10`
So many possible issues but you can compare possibly decimal values using bc:
if [ "$(echo "$total > $power" | bc)" = 1 ]; then
One problem is that [ (and [[) do string comparisons, and you want a numeric comparison. For that you want to use ((, so something like
if (( $total > 150 )); then
echo "WARNING..."
else
echo "OK"
fi
will work better. Otherwise a total of 1000 would print ok, and 90 would give a warning.
Other problems:
$150 gives you the value of a variable called 150 -- you probably want to remove the $
Outside of special forms like [[ and ((, a > will do an output redirection, rather than being a 'normal' argument to a command such as [.
As the comments recommend, use shellcheck, however, I think your intention is not what you wrote.
Try this, create a script (i.e. myscripy)
#! /bin/bash
power=$150
echo "power=$power"
then run it
./myscript abc
and prints
power=abc50
which is probably very different than what you expect.
That is because power will take the first argument's value ($1) and append 50.
If you wanted argument number 150 (also very unlikely), you should write
power=${150}
but if you want just the number
power=150
edit
based on the comment, use
zet=$(bc <<<"$a+$b+$c+$d")
to calculate zet if the values are floating points.
For the comparison use
if [ "$(bc <<<"$zet>150")" == 1 ]; then
...
fi
I have below script and for a requirement I have to place some function for each of these script to get time information for each script and at last show total time.
My main scripts looks like below:
/u01/scripts/stop.sh ${1} | tee ${stop_log}
/u01/scripts/kill_proc.sh ${1} | tee ${kill_log}
/u01/scripts/detach.sh ${1}| tee ${detach_log}
/u01/scripts/copy.sh ${1} | tee ${copy_log}
I want to use something like below function to get every script execution time and at last with a global variable I can show Total time taken by all the scripts.
I created below but unfortunately I could not use properly , if you have something kindly help here.
time_check()
{
export time_log=${log}/time_log_${dts}.log
echo 'StartingTime:'date +%s > ${time_log}
echo 'EndingTime:'date +%s >> ${time_log}
}
I want to use something like above function to get every script execution time and at last with a global variable I can show total time taken by all the scripts . Could anyone please guide how to get the desired result.
If you are OK with the time granularity of seconds, you could simply do this:
start=$SECONDS
/u01/scripts/stop.sh ${1} | tee ${stop_log}
stop=$SECONDS
/u01/scripts/kill_proc.sh ${1} | tee ${kill_log}
kill_proc=$SECONDS
/u01/scripts/detach.sh ${1}| tee ${detach_log}
detach=$SECONDS
/u01/scripts/copy.sh ${1} | tee ${copy_log}
end=$SECONDS
printf "%s\n" "stop=$((stop-start)), kill_proc=$((kill_proc-stop)), detach=$((detach-kill_proc)), copy=$((end-detach)), total=$((end-start))"
You can write a function to do this as well:
time_it() {
local start=$SECONDS rc
echo "$(date): Starting $*"
"$#"; rc=$?
echo "$(date): Finished $*; elapsed = $((SECONDS-start)) seconds"
return $rc
}
With Bash version >= 4.2 you can use printf to print date rather than invoking an external command:
time_it() {
local start=$SECONDS ts rc
printf -v ts '%(%Y-%m-%d_%H:%M:%S)T' -1
printf '%s\n' "$ts Starting $*"
"$#"; rc=$?
printf -v ts '%(%Y-%m-%d_%H:%M:%S)T' -1
printf '%s\n' "$ts Finished $*; elapsed = $((SECONDS-start)) seconds"
return $rc
}
And invoke it as:
start=$SECONDS
time_it /u01/scripts/stop.sh ${1} | tee ${stop_log}
time_it /u01/scripts/kill_proc.sh ${1} | tee ${kill_log}
time_it /u01/scripts/detach.sh ${1}| tee ${detach_log}
time_it /u01/scripts/copy.sh ${1} | tee ${copy_log}
echo "Total time = $((SECONDS-start)) seconds"
Related:
How do I measure duration in seconds in a shell script?
I've implemented a way to have concurrent jobs in bash, as seen here.
I'm looping through a file with around 13000 lines. I'm just testing and printing each line, as such:
#!/bin/bash
max_bg_procs(){
if [[ $# -eq 0 ]] ; then
echo "Usage: max_bg_procs NUM_PROCS. Will wait until the number of background (&)"
echo " bash processes (as determined by 'jobs -pr') falls below NUM_PROCS"
return
fi
local max_number=$((0 + ${1:-0}))
while true; do
local current_number=$(jobs -pr | wc -l)
if [[ $current_number -lt $max_number ]]; then
echo "success in if"
break
fi
echo "has to wait"
sleep 4
done
}
download_data(){
echo "link #" $2 "["$1"]"
}
mapfile -t myArray < $1
i=1
for url in "${myArray[#]}"
do
max_bg_procs 6
download_data $url $i &
((i++))
done
echo "finito!"
I've also tried other solutions such as this and this, but my issue is persistent:
At a "random" given step, usually between the 2000th and the 5000th iteration, it simply gets stuck. I've put those various echo in the middle of the code to see where it would get stuck but it the last thing it prints is the $url $i.
I've done the simple test to remove any parallelism and just loop the file contents: all went fine and it looped till the end.
So it makes me think I'm missing some limitation on the parallelism, and I wonder if anyone could help me out figuring it out.
Many thanks!
Here, we have up to 6 parallel bash processes calling download_data, each of which is passed up to 16 URLs per invocation. Adjust per your own tuning.
Note that this expects both bash (for exported function support) and GNU xargs.
#!/usr/bin/env bash
# ^^^^- not /bin/sh
download_data() {
echo "link #$2 [$1]" # TODO: replace this with a job that actually takes some time
}
export -f download_data
<input.txt xargs -d $'\n' -P 6 -n 16 -- bash -c 'for arg; do download_data "$arg"; done' _
Using GNU Parallel it looks like this
cat input.txt | parallel echo link '\#{#} [{}]'
{#} = the job number
{} = the argument
It will spawn one process per CPU. If you instead want 6 in parallel use -j:
cat input.txt | parallel -j6 echo link '\#{#} [{}]'
If you prefer running a function:
download_data(){
echo "link #" $2 "["$1"]"
}
export -f download_data
cat input.txt | parallel -j6 download_data {} {#}
Hi Im making a script to do some rsync process, for the rsync process, Sys admin has created the script, when it run it is asking select options, so i want to create a script to pass that argument from script and run it from cron.
list of directories to rsync take from file.
filelist=$(cat filelist.txt)
for i in filelist;do
echo -e "3\nY" | ./rsync.sh $i
#This will create a rsync log file
so i check the some value of log file and if it is empty i moving to the second file. if the file is not empty, i have to start rsync process as below that will take more that 2 hours.
if [ a != 0 ];then
echo -e "3\nN" | ./rsync.sh $i
above rsync process need to send to the background and take next file to loop. i check with the screen command, but screen is not working with server. also i need to get the duration that take to run process and passing to the log, when i use the time command i am unable to pass the echo variable. Also need to send this to background and take next file. appreciate any suggestions to success this task.
Question
1. How to send argument with Time command
echo -e "3\nY" | time ./rsync.sh $i
above one not working
how to send this to background and take next file to rsync while running previous rsync process.
Full Code
#!/bin/bash
filelist=$(cat filelist.txt)
Lpath=/opt/sas/sas_control/scripts/Logs/rsync_logs
date=$(date +"%m-%d-%Y")
timelog="time_result/rsync_time.log-$date"
for i in $filelist;do
#echo $i
b_i=$(basename $i)
echo $b_i
echo -e "3\nY" | ./rsync.sh $i
f=$(cat $Lpath/$(ls -tr $Lpath| grep rsync-dry-run-$b_i | tail -1) | grep 'transferred:' | cut -d':' -f2)
echo $f
if [ $f != 0 ]; then
#date=$(date +"%D : %r")
start_time=`date +%s`
echo "$b_i-start:$start_time" >> $timelog
#time ./rsync.sh $i < echo -e "3\nY" 2> "./time_result/$b_i-$date" &
time { echo -e "3\nY" | ./rsync.sh $i; } 2> "./time_result/$b_i-$date"
end_time=`date +%s`
s_time=$(cat $timelog|grep "$b_i-start" |cut -d ':' -f2)
duration=$(($end_time-$s_time))
echo "$b_i duration:$duration" >> $timelog
fi
done
Your question is not very clear, but I'll try:
(1) If I understand you correctly, you want to time the rsync.
My first attempt would be to use echo xxxx | time rsycnc. On my bash, this was however broken (or not supposed to work?). I'm normally using Zsh instead of bash, and on zsht, this indeed runs fine.
If it is important for you to use bash, an alternative (since the time for the echo can likely be neglected) would be to time the whole pipe, i.e. time (echo xxxx | time rsync), or even simpler time rsync <(echo xxxx)
(2) To send a process to the background, add an & to the line. However, the time command produces of course output (that's it purpose), and you don't want to receive output from a program in background. The solution is to redirect the output:
(time rsync <(echo xxxx) >output.txt 2>error.txt) &
If you want to time something, you can use:
time sleep 3
If you want to time two things, you can do a compound statement like this (note semicolon after second sleep):
time { sleep 3; sleep 4; }
So, you can do this to time your echo (which will take no time at all) and your rsync:
time { echo "something" | rsync something ; }
If you want to do that in the background:
time { echo "something" | rsync something ; } &
Full Code
#!/bin/bash
filelist=$(cat filelist.txt)
Lpath=/opt/sas/sas_control/scripts/Logs/rsync_logs
date=$(date +"%m-%d-%Y")
timelog="time_result/rsync_time.log-$date"
for i in $filelist;do
#echo $i
b_i=$(basename $i)
echo $b_i
echo -e "3\nY" | ./rsync.sh $i
f=$(cat $Lpath/$(ls -tr $Lpath| grep rsync-dry-run-$b_i | tail -1) | grep 'transferred:' | cut -d':' -f2)
echo $f
if [ $f != 0 ]; then
#date=$(date +"%D : %r")
start_time=`date +%s`
echo "$b_i-start:$start_time" >> $timelog
#time ./rsync.sh $i < echo -e "3\nY" 2> "./time_result/$b_i-$date" &
time { echo -e "3\nY" | ./rsync.sh $i; } 2> "./time_result/$b_i-$date"
end_time=`date +%s`
s_time=$(cat $timelog|grep "$b_i-start" |cut -d ':' -f2)
duration=$(($end_time-$s_time))
echo "$b_i duration:$duration" >> $timelog
fi
done
I am looking for a command (or way of doing) the following:
echo -n 6 | doif -criteria "isgreaterthan 4" -command 'do some stuff'
The echo part would obviously come from a more complicated string of bash commands. Essentially I am taking a piece of text from each line of a file and if it appears in another set of files more than x (say 100) then it will be appended to another file.
Is there a way to perform such trickery with awk somehow? Or is there another command.. I'm hoping that there is some sort of xargs style command to do this in the sense that the -I% portion would be the value with which to check the criteria and whatever follows would be the command to execute.
Thanks for thy insight.
It's possible, though I don't see the reason why you would do that...
function doif
{
read val1
op=$1
val2="$2"
shift 2
if [ $val1 $op "$val2" ]; then
"$#"
fi
}
echo -n 6 | doif -gt 3 ls /
if test 6 -gt 4; then
# do some stuff
fi
or
if test $( echo 6 ) -gt 4; then : ;fi
or
output=$( some cmds that generate text)
# this will be an error if $output is ill-formed
if test "$output" -gt 4; then : ; fi