This question already has answers here:
Brace expansion with variable? [duplicate]
(6 answers)
Closed 1 year ago.
I'm trying to create sequences as follows:
startDay=1
endDay=2
dayRange="{$startDay..$endDay}"
echo \[\"$dayRange\",\"{00..02}\"\]
The output is:
["{1..2}","00"] ["{1..2}","01"] ["{1..2}","02"]
When specifying the sequence directly {00..02}, it auto creates "00", "01", "02", but it does not understand the dayRange variable.
What I expect it to return is:
["1","00"] ["1","01"] ["1","02"] ["2","00"] ["2","01"] ["2","02"]
Not sure what I missed.
Please advise.
First idea would be a simple nested for loop:
startDay=1
endDay=2
pfx=
out=
for ((i=startDay; i<=endDay; i++))
do
for j in {00..02}
do
out+="${pfx}[\"${i}\",\"${j}\"]"
pfx=" "
done
done
echo "${out}"
This generates:
["1","00"] ["1","01"] ["1","02"] ["2","00"] ["2","01"] ["2","02"]
A bit less coding, and a bit faster, which uses OP's echo ... {00..02} to eliminate one of the for loops:
NOTE: this eliminates the subprocess $(echo ...) call I had in a previous edit.
startDay=1
endDay=2
for ((i=startDay; i<=endDay; i++))
do
echo -n "[\""${i}"\",\""{00..02}"\"]"
echo -n " "
done
echo ""
This also generates:
["1","00"] ["1","01"] ["1","02"] ["2","00"] ["2","01"] ["2","02"]
Here's one awk idea:
awk -v start=$"${startDay}" -v end="${endDay}" '
BEGIN {
pfx=""
out=""
for (i=start; i<=end; i++)
for (j=0; j<=2; j++) {
out=out pfx "[\"" i "\",\"" sprintf("%02d", j) "\"]"
pfx=" "
}
print out
}'
This also generates:
["1","00"] ["1","01"] ["1","02"] ["2","00"] ["2","01"] ["2","02"]
With the elimination of the earlier subprocess $(echo ...) the first 2 solutions come in with single-digit millisecond timings while the awk solution comes in with low double-digit millisecond timings.
As the number of days (and/or sequence size) increases the first 2 solutions start taking longer (the nested for loop falling farther behind) while the awk solution tends to maintain the same speed.
For really large increases (number of days and/or sequence size) I would expect awk to close in on, and eventually take, the lead.
Related
Case scenario:
$ cat Status.txt
1,connected
2,connected
3,connected
4,connected
5,connected
6,connected
7,disconnected
8,disconnected
9,disconnected
10,disconnected
11,disconnected
12,disconnected
13,disconnected
14,connected
15,connected
16,connected
17,disconnected
18,connected
19,connected
20,connected
21,disconnected
22,disconnected
23,disconnected
24,disconnected
25,disconnected
26,disconnected
27,disconnected
28,disconnected
29,disconnected
30,connected
As can be seen, there are "hollows", understanding them as lines with the "disconnected" value inside the sequence file.
I want, in fact, to detect these "holes", but it would be useful if I could set a minimum n of missing numbers in the sequence.
I.e: for ' n=5' a detectable hole would be the 7... 13 part, as there are at least 5 "disconnected" in a row on the sequence. However, the missing 17 should not be considered as detectable in this case. Again, at line 21 whe get a valid disconnection.
Something like:
$ detector Status.txt -n 5 --pattern connected
7
21
... that could be interpreted like:
- Missing more than 5 "connected" starting at 7.
- Missing more than 5 "connected" starting at 21.
I need to script this on Linux shell, so I was thinking about programing some loop, parsing strings and so on, but I feel like if this could be done by using linux shell tools and maybe some simpler programming. Is there a way?
Even when small programs like csvtool are a valid solution, some more common Linux commands (like grep, cut, awk, sed, wc... etc) could be worth for me when working with embedded devices.
#!/usr/bin/env bash
last_connected=0
min_hole_size=${1:-5} # default to 5, or take an argument from the command line
while IFS=, read -r num state; do
if [[ $state = connected ]]; then
if (( (num-last_connected) > (min_hole_size+1) )); then
echo "Found a hole running from $((last_connected + 1)) to $((num - 1))"
fi
last_connected=$num
fi
done
# Special case: Need to also handle a hole that's still open at EOF.
if [[ $state != connected ]] && (( num - last_connected > min_hole_size )); then
echo "Found a hole running from $((last_connected + 1)) to $num"
fi
...emits, given your file on stdin (./detect-holes <in.txt):
Found a hole running from 7 to 13
Found a hole running from 21 to 29
See:
BashFAQ #1 - How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?
The conditional expression -- the [[ ]] syntax used to make it safe to do string comparisons without quoting expansions.
Arithmetic comparison syntax -- valid in $(( )) in all POSIX-compliant shells; also available without the expansion side effects as (( )) as a bash extension.
This is the perfect use case for awk, since the machinery of line reading, column splitting, and matching is all built in. The only tricky bit is getting the command line argument to your script, but it's not too bad:
#!/usr/bin/env bash
awk -v window="$1" -F, '
BEGIN { if (window=="") {window = 1} }
$2=="disconnected"{if (consecutive==0){start=NR}; consecutive++}
$2!="disconnected"{if (consecutive>window){print start}; consecutive=0}
END {if (consecutive>window){print start}}'
The window value is supplied as the first command line argument; left out, it defaults to 1, which means "display the start of gaps with at least two consecutive disconnections". Probably could have a better name. You can give it 0 to include single disconnections. Sample output below. (Note that I added series of 2 disconnections at the end to test the failure that Charles metions).
njv#organon:~/tmp$ ./tst.sh 0 < status.txt # any number of disconnections
7
17
21
31
njv#organon:~/tmp$ ./tst.sh < status.txt # at least 2 disconnections
7
21
31
njv#organon:~/tmp$ ./tst.sh 8 < status.txt # at least 9 disconnections
21
Awk solution:
detector.awk script:
#!/bin/awk -f
BEGIN { FS="," }
$2 == "disconnected"{
if (f && NR-c==nr) c++;
else { f=1; c++; nr=NR }
}
$2 == "connected"{
if (f) {
if (c > n) {
printf "- Missing more than 5 \042connected\042 starting at %d.\n", nr
}
f=c=0
}
}
Usage:
awk -f detector.awk -v n=5 status.txt
The output:
- Missing more than 5 "connected" starting at 7.
- Missing more than 5 "connected" starting at 21.
This question already has answers here:
Variables in bash seq replacement ({1..10}) [duplicate]
(7 answers)
Brace expansion with a Bash variable - {0..$foo}
(5 answers)
Closed 8 years ago.
I'm making a program in bash that creates a histoplot, using numbers I have created. The numbers are stored as such (where the 1st number is how many words are on a line of a file, and the 2nd number is how many times this amount of words on a line comes up, in a given file.)
1 1
2 4
3 1
4 2
this should produce:
1 #
2 ####
3 #
4 ##
BUT the output I'm getting is:
1 #
2 #
3 #
4 #
however the for loop is not recognising that my variable "hashNo" is a number.
#!/bin/bash
if [ -e $f ] ; then
while read line
do
lineAmnt=${line% *}
hashNo=${line##* }
#VVVV Problem is this line here VVVV
for i in {1..$hashNo}
#This line ^^^^^^^ the {1..$hashNo}
do
hashes+="#"
done
printf "%4s" $lineAmnt
printf " $hashes\n"
hashes=""
done < $1
fi
the code works if I replace hashNo with a number (eg 4 makes 4 hashes in my output) but it needs to be able to change with each line (no all lines on a file will have the same amount of chars in them.
thanks for any help :D
A sequence expression in bash must be formed from either integers or characters, no parameter substitutions take place before hand. That's because, as per the bash doco:
The order of expansions is: brace expansion, tilde expansion, parameter, variable and arithmetic expansion and command substitution (done in a left-to-right fashion), word splitting, and pathname expansion.
In other words, brace expansion (which includes the sequence expression form) happens first.
In any case, this cries out to be done as a function so that it can be done easily from anywhere, and also made more efficient:
#!/bin/bash
hashes() {
sz=$1
while [[ $sz -ge 10 ]]; do
printf "##########"
((sz -= 10))
done
while [[ $sz -gt 0 ]]; do
printf "#"
((sz--))
done
}
echo 1 "$(hashes 1)"
echo 2 "$(hashes 4)"
echo 3 "$(hashes 1)"
echo 4 "$(hashes 2)"
which outputs, as desired:
1 #
2 ####
3 #
4 ##
The use of the first loop (doing ten hashes at a time) will almost certainly be more efficient than adding one character at a time and you can, if you wish, do a size-50 loop before that for even more efficiencies if your values can be larger.
I tried this for (( i=1; i<=$hashNo; i++ )) for the for loop, it seems to be working
Your loop should be
for ((i=0; i<hashNo; i++))
do
hashes+="#"
done
Also you can stick with your loop by the use of eval and command substitution $()
for i in $(eval echo {1..$hashNo})
do
hashes+="#"
done
In bash I'm trying to create an array and then run through a loop a number of times (determined by the user of the file) and then add a choice to that array the predetermined number of times. This is trade data, so for example, I choose 2 for factors. Then the program asks me to input the factor I want, and I put in open (open price of the day), then bid is added to the array arr and the question is asked again. Then I put in close (close price of the day) then close is added to the array, and in the end the
arr = open close like that. But I run the code and the question: "How many factors would you like to check total: " simply runs over and over again and I never leave the loop and it never appears that the inputs are being put into the array. Any help as to my mistake here is greatly appreciated. Thanks.
factor=""
total=0
declare -a arr
read -p "How many factors would you like to check total: " -e -i "$total" total
for (( x=1; x=total; x++ ))
do
read -p "Enter factor from list: " -e -i "$factor" factor
arr+=(${arr[#]} "$factor")
done
echo ${arr[#]}
You almost got it correct on array append. Just remember that += operator doesn't need full reference to array again on RHS. e.g. just
arr+=($factor)
Would be suffice to append $factor at the end of array variable arr.
Modify your script a little bit like this:
factor=""
total=0
declare -a arr
read -p "How many factors would you like to check total: " -e -i "$total" total
for (( x=1; x<=total; x++ ))
do
read -p "Enter factor from list: " -e -i "$factor" factor
arr+=($factor)
done
echo ${arr[#]}
You have a typo
for (( x=1; x=total; x++ ))
should be
for (( x=1; x==total; x++ ))
In the first one you are assigning total to x which is always true. In the second one you are checking for equality.
I'm trying to do something like these:
while[$read!="0"];
In this program
#!/bin/sh
i=0
cont=0
while[$read!="0"]; do
read number
cont=`expr $cont + $number`
i++
done
cont=`expr $cont / $i -1`
echo
I want to stop suming the entries when I give it a 0
tnx
The variable you're reading into is $number, so reference that rather than $read in your loop.
Whitespace is significant, so make sure to include spaces before, after, and between all of the items in your loop. (Confusingly, you must not include spaces in an assignment statement like i=0. i = 0 is wrong.)
For good measure, use double quotes around the variable. That's a good practice so that if the user hits enter without typing a number your script doesn't barf on the empty string.
while [ "$number" != "0" ]; do
Also, your i++ isn't right. There are various ways to write that, the simplest being:
let i++
In this, an infinite loop would be appropriate, since you know the condition on which you want to (hint) break out of the loop. The way to get an infinite loop in sh is: while true; do ...; done
Also, read has a -p option that lets you have a prompt (so you know what you're being asked to enter): read -p "Enter a number: " number
I have created a little password generation script. I'm curious to what improvements can be made for it except input error handling, usage information etc. It's the core functionality I'm interested in seeing improvements upon.
This is what it does (and what I like it to do):
Keep it easy to change which Lowercase characters (L), Uppercase characters (U), Numbers (N) and Symbols (S) that are used in passwords.
I'd like it to find a new password of legnth 10 for me in max two seconds.
It should take a variable length of the password string as an argument.
Only a password containing at least one L, U, N and S should be accepted.
Here is the code:
#!/bin/bash
PASSWORDLENGTH=$1
RNDSOURCE=/dev/urandom
L="acdefghjkmnpqrtuvwxy"
U="ABDEFGHJLQRTY"
N="012345679"
S="\-/\\)?=+.%#"
until [ $(echo $password | grep [$L] | grep [$U] | grep [$N] | grep -c [$S] ) == 1 ]; do
password=$(cat $RNDSOURCE | tr -cd "$L$U$N$S" | head -c $PASSWORDLENGTH)
echo In progress: $password # It's simply for debug purposes, ignore it
done
echo Final password: $password
My questions are:
Is there a nicer way of checking if the password is acceptable than the way I'm doing it?
What about the actual password generation?
Any coding style improvements? (The short variable names are temporary. Though I'm using uppercase names for "constants" [I know there formally are none] and lowercase for variables. Do you like it?)
Let's vote on the most improved version. :-)
For me it was just an exercise mostly for fun and as a learning experience, albeit I will start using it instead of the generation from KeepassX which I'm using now. It will be interesting to see which improvements and suggestions will come from more experienced Bashistas (I made that word up).
I created a little basic script to measure performance: (In case someone thinks it's fun)
#!/bin/bash
SAMPLES=100
SCALE=3
echo -e "PL\tMax\tMin\tAvg"
for p in $(seq 4 50); do
bcstr=""; max=-98765; min=98765
for s in $(seq 1 $SAMPLES); do
gt=$(\time -f %e ./genpassw.sh $p 2>&1 1>/dev/null)
bcstr="$gt + $bcstr"
max=$(echo "if($max < $gt ) $gt else $max" | bc)
min=$(echo "if($min > $gt ) $gt else $min" | bc)
done
bcstr="scale=$SCALE;($bcstr 0)/$SAMPLES"
avg=$(echo $bcstr | bc)
echo -e "$p\t$max\t$min\t$avg"
done
You're throwing away a bunch of randomness in your input stream. Keep those bytes around and translate them into your character set. Replace the password=... statement in your loop with the following:
ALL="$L$U$N$S"
password=$(tr "\000-\377" "$ALL$ALL$ALL$ALL$ALL" < $RNDSOURCE | head -c $PASSWORDLENGTH)
The repetition of $ALL is to ensure that there are >=255 characters in the "map to" set.
I also removed the gratuitous use of cat.
(Edited to clarify that what appears above is not intended to replace the full script, just the inner loop.)
Edit: Here's a much faster strategy that doesn't call out to external programs:
#!/bin/bash
PASSWORDLENGTH=$1
RNDSOURCE=/dev/urandom
L="acdefghjkmnpqrtuvwxy"
U="ABDEFGHJLQRTY"
N="012345679"
# (Use this with tr.)
#S='\-/\\)?=+.%#'
# (Use this for bash.)
S='-/\)?=+.%#'
ALL="$L$U$N$S"
# This function echoes a random index into it's argument.
function rndindex() { echo $(($RANDOM % ${#1})); }
# Make sure the password contains at least one of each class.
password="${L:$(rndindex $L):1}${U:$(rndindex $U):1}${N:$(rndindex $N):1}${S:$(rndindex $S):1}"
# Add random other characters to the password until it is the desired length.
while [[ ${#password} -lt $PASSWORDLENGTH ]]
do
password=$password${ALL:$(rndindex $ALL):1}
done
# Now shuffle it.
chars=$password
password=""
while [[ ${#password} -lt $PASSWORDLENGTH ]]
do
n=$(rndindex $chars)
ch=${chars:$n:1}
password="$password$ch"
if [[ $n == $(( ${#chars} - 1 )) ]]; then
chars="${chars:0:$n}"
elif [[ $n == 0 ]]; then
chars="${chars:1}"
else
chars="${chars:0:$n}${chars:$((n+1))}"
fi
done
echo $password
Timing tests show this runs 5-20x faster than the original script, and the time is more predictable from one run to the next.
you could just use uuidgen or pwgen to generate your random passwords, maybe later shuffling some letters around or something of the sort
secpwgen is very good (it can also generate easier to remember diceware passwords) - but has almost disappeared from the net. I managed to track down a copy of the 1.3 source & put it on github.
It is also now part of Alpine Linux.