I have a=10 var that I want to pad from left with integer 1 with 5 padding.
Output should be below:
11110
when Im doing:
printf "%05d\n" "${a}"
it gives me 00010.
As per bash printf man page, it supports only zero padding, so you can't directly use integer padding other than zero.
You can do by string printing or pad by manipulating
a=10
printf '%s\n' 111${a}
or
printf "% 5d\n" "10" | tr ' ' '1'
I think I would just:
printf "% 5d\n" "10" | tr ' ' '1'
fill with spaces and then substitute spaces for ones.
You may use this instruction:
printf "%5d" "$a" | sed 's/ /1/g'
It outputs:
11110
Related
I have values such as
B146XYZ,
G638XYZ,
G488xBC
I have to write a bash script where when it sees comma it has to remove the comma and add 7 spaces to it and also if it sees comma and a space or just space(no punctuations) it has to add 7 spaces to make all of them fixed length.
if [[ $row = *’,’* ]]
then
first= “${ row%%,*}”
echo “${first } “
I tried but can’t understand how to add conditions for the remaining criteria specially struggling with single value conditions such as G488xBC
What about just:
sed -E 's/[, ]+/ /g' file
Or something like this will print a padded table, so long as no field is longer than 13 characters:
awk -F '[,[:space:]]+' \
'{
for (i=1; i<NF; i++) {
printf("%-14s", $i)
}
print $NF
}'
Or the same thing in pure bash:
while IFS=$', \t' read -ra vals; do
last=$((${#vals[#]} - 1))
for ((i=0; i<last; i++)); do
printf "%-14s" "${vals[i]}"
done
printf '%s\n' "${vals[last]}"
done
newrow="${row//,/ }"
VALUES=`echo $VALUES | sed 's/,/ /g' | xargs`
The sed command will replace the comma with a single space.
The xargs will consolidate any number of whitespaces into a single space.
With that you now have your values in space separated string instead of comma, separated by unknown number of whitespaces.
From there you can use for i in $VALUES; do printf "$i\t"; done
Using the tab character like above will give you aligned output in case your values may be different in length.
But if your values are always same length then you can make it a bit more simple by doing
VALUES=`echo $VALUES | sed 's/,/ /g' | xargs | sed 's/1 space/7 spaces/g'`
echo $VALUES
My aim is to echo a character, for example #, based on a value such as num=6 and it must print # 6 times on the screen.
Not sure how to get this.
You could do something like
printf '#%.0s' {1..6}
or, in the more general case,
printf '#%.0s' $(seq 1 $num)
printf "%*s" "$num" " " | tr " " "#"
or
yes '#' | head -"$num" | tr -d "\n"
I have string contains a path
string="toto.titi.12.tata.2.abc.def"
I want to extract only the numbers from this string.
To extract the first number:
tmp="${string#toto.titi.*.}"
num1="${tmp%.tata*}"
To extract the second number:
tmp="${string#toto.titi.*.tata.*.}"
num2="${tmp%.abc.def}"
So to extract a parameter I have to do it in 2 steps. How to extract a number with one step?
You can use tr to delete all of the non-digit characters, like so:
echo toto.titi.12.tata.2.abc.def | tr -d -c 0-9
To extract all the individual numbers and print one number word per line pipe through -
tr '\n' ' ' | sed -e 's/[^0-9]/ /g' -e 's/^ *//g' -e 's/ *$//g' | tr -s ' ' | sed 's/ /\n/g'
Breakdown:
Replaces all line breaks with spaces: tr '\n' ' '
Replaces all non numbers with spaces: sed -e 's/[^0-9]/ /g'
Remove leading white space: -e 's/^ *//g'
Remove trailing white space: -e 's/ *$//g'
Squeeze spaces in sequence to 1 space: tr -s ' '
Replace remaining space separators with line break: sed 's/ /\n/g'
Example:
echo -e " this 20 is 2sen\nten324ce 2 sort of" | tr '\n' ' ' | sed -e 's/[^0-9]/ /g' -e 's/^ *//g' -e 's/ *$//g' | tr -s ' ' | sed 's/ /\n/g'
Will print out
20
2
324
2
Here is a short one:
string="toto.titi.12.tata.2.abc.def"
id=$(echo "$string" | grep -o -E '[0-9]+')
echo $id // => output: 12 2
with space between the numbers.
Hope it helps...
Parameter expansion would seem to be the order of the day.
$ string="toto.titi.12.tata.2.abc.def"
$ read num1 num2 <<<${string//[^0-9]/ }
$ echo "$num1 / $num2"
12 / 2
This of course depends on the format of $string. But at least for the example you've provided, it seems to work.
This may be superior to anubhava's awk solution which requires a subshell. I also like chepner's solution, but regular expressions are "heavier" than parameter expansion (though obviously way more precise). (Note that in the expression above, [^0-9] may look like a regex atom, but it is not.)
You can read about this form or Parameter Expansion in the bash man page. Note that ${string//this/that} (as well as the <<<) is a bashism, and is not compatible with traditional Bourne or posix shells.
This would be easier to answer if you provided exactly the output you're looking to get. If you mean you want to get just the digits out of the string, and remove everything else, you can do this:
d#AirBox:~$ string="toto.titi.12.tata.2.abc.def"
d#AirBox:~$ echo "${string//[a-z,.]/}"
122
If you clarify a bit I may be able to help more.
You can also use sed:
echo "toto.titi.12.tata.2.abc.def" | sed 's/[0-9]*//g'
Here, sed replaces
any digits (class [0-9])
repeated any number of times (*)
with nothing (nothing between the second and third /),
and g stands for globally.
Output will be:
toto.titi..tata..abc.def
Convert your string to an array like this:
$ str="toto.titi.12.tata.2.abc.def"
$ arr=( ${str//[!0-9]/ } )
$ echo "${arr[#]}"
12 2
Use regular expression matching:
string="toto.titi.12.tata.2.abc.def"
[[ $string =~ toto\.titi\.([0-9]+)\.tata\.([0-9]+)\. ]]
# BASH_REMATCH[0] would be "toto.titi.12.tata.2.", the entire match
# Successive elements of the array correspond to the parenthesized
# subexpressions, in left-to-right order. (If there are nested parentheses,
# they are numbered in depth-first order.)
first_number=${BASH_REMATCH[1]}
second_number=${BASH_REMATCH[2]}
Using awk:
arr=( $(echo $string | awk -F "." '{print $3, $5}') )
num1=${arr[0]}
num2=${arr[1]}
Hi adding yet another way to do this using 'cut',
echo $string | cut -d'.' -f3,5 | tr '.' ' '
This gives you the following output:
12 2
Fixing newline issue (for mac terminal):
cat temp.txt | tr '\n' ' ' | sed -e 's/[^0-9]/ /g' -e 's/^ *//g' -e 's/ *$//g' | tr -s ' ' | sed $'s/ /\\\n/g'
Assumptions:
there is no embedded white space
the string of text always has 7 period-delimited strings
the string always contains numbers in the 3rd and 5th period-delimited positions
One bash idea that does not require spawning any subprocesses:
$ string="toto.titi.12.tata.2.abc.def"
$ IFS=. read -r x1 x2 num1 x3 num2 rest <<< "${string}"
$ typeset -p num1 num2
declare -- num1="12"
declare -- num2="2"
In a comment OP has stated they wish to extract only one number at a time; the same approach can still be used, eg:
$ string="toto.titi.12.tata.2.abc.def"
$ IFS=. read -r x1 x2 num1 rest <<< "${string}"
$ typeset -p num1
declare -- num1="12"
$ IFS=. read -r x1 x2 x3 x4 num2 rest <<< "${string}"
$ typeset -p num2
declare -- num2="2"
A variation on anubhava's answer that uses parameter expansion instead of a subprocess call to awk, and still working with the same set of initial assumptions:
$ arr=( ${string//./ } )
$ num1=${arr[2]}
$ num2=${arr[4]}
$ typeset -p num1 num2
declare -- num1="12"
declare -- num2="2"
I've got almost the same question as here.
I have an array which contains aa ab aa ac aa ad, etc.
Now I want to select all unique elements from this array.
Thought, this would be simple with sort | uniq or with sort -u as they mentioned in that other question, but nothing changed in the array...
The code is:
echo `echo "${ids[#]}" | sort | uniq`
What am I doing wrong?
A bit hacky, but this should do it:
echo "${ids[#]}" | tr ' ' '\n' | sort -u | tr '\n' ' '
To save the sorted unique results back into an array, do Array assignment:
sorted_unique_ids=($(echo "${ids[#]}" | tr ' ' '\n' | sort -u | tr '\n' ' '))
If your shell supports herestrings (bash should), you can spare an echo process by altering it to:
tr ' ' '\n' <<< "${ids[#]}" | sort -u | tr '\n' ' '
A note as of Aug 28 2021:
According to ShellCheck wiki 2207 a read -a pipe should be used to avoid splitting.
Thus, in bash the command would be:
IFS=" " read -r -a ids <<< "$(echo "${ids[#]}" | tr ' ' '\n' | sort -u | tr '\n' ' ')"
or
IFS=" " read -r -a ids <<< "$(tr ' ' '\n' <<< "${ids[#]}" | sort -u | tr '\n' ' ')"
Input:
ids=(aa ab aa ac aa ad)
Output:
aa ab ac ad
Explanation:
"${ids[#]}" - Syntax for working with shell arrays, whether used as part of echo or a herestring. The # part means "all elements in the array"
tr ' ' '\n' - Convert all spaces to newlines. Because your array is seen by shell as elements on a single line, separated by spaces; and because sort expects input to be on separate lines.
sort -u - sort and retain only unique elements
tr '\n' ' ' - convert the newlines we added in earlier back to spaces.
$(...) - Command Substitution
Aside: tr ' ' '\n' <<< "${ids[#]}" is a more efficient way of doing: echo "${ids[#]}" | tr ' ' '\n'
If you're running Bash version 4 or above (which should be the case in any modern version of Linux), you can get unique array values in bash by creating a new associative array that contains each of the values of the original array. Something like this:
$ a=(aa ac aa ad "ac ad")
$ declare -A b
$ for i in "${a[#]}"; do b["$i"]=1; done
$ printf '%s\n' "${!b[#]}"
ac ad
ac
aa
ad
This works because in any array (associative or traditional, in any language), each key can only appear once. When the for loop arrives at the second value of aa in a[2], it overwrites b[aa] which was set originally for a[0].
Doing things in native bash can be faster than using pipes and external tools like sort and uniq, though for larger datasets you'll likely see better performance if you use a more powerful language like awk, python, etc.
If you're feeling confident, you can avoid the for loop by using printf's ability to recycle its format for multiple arguments, though this seems to require eval. (Stop reading now if you're fine with that.)
$ eval b=( $(printf ' ["%s"]=1' "${a[#]}") )
$ declare -p b
declare -A b=(["ac ad"]="1" [ac]="1" [aa]="1" [ad]="1" )
The reason this solution requires eval is that array values are determined before word splitting. That means that the output of the command substitution is considered a single word rather than a set of key=value pairs.
While this uses a subshell, it uses only bash builtins to process the array values. Be sure to evaluate your use of eval with a critical eye. If you're not 100% confident that chepner or glenn jackman or greycat would find no fault with your code, use the for loop instead.
I realize this was already answered, but it showed up pretty high in search results, and it might help someone.
printf "%s\n" "${IDS[#]}" | sort -u
Example:
~> IDS=( "aa" "ab" "aa" "ac" "aa" "ad" )
~> echo "${IDS[#]}"
aa ab aa ac aa ad
~>
~> printf "%s\n" "${IDS[#]}" | sort -u
aa
ab
ac
ad
~> UNIQ_IDS=($(printf "%s\n" "${IDS[#]}" | sort -u))
~> echo "${UNIQ_IDS[#]}"
aa ab ac ad
~>
If your array elements have white space or any other shell special character (and can you be sure they don't?) then to capture those first of all (and you should just always do this) express your array in double quotes! e.g. "${a[#]}". Bash will literally interpret this as "each array element in a separate argument". Within bash this simply always works, always.
Then, to get a sorted (and unique) array, we have to convert it to a format sort understands and be able to convert it back into bash array elements. This is the best I've come up with:
eval a=($(printf "%q\n" "${a[#]}" | sort -u))
Unfortunately, this fails in the special case of the empty array, turning the empty array into an array of 1 empty element (because printf had 0 arguments but still prints as though it had one empty argument - see explanation). So you have to catch that in an if or something.
Explanation:
The %q format for printf "shell escapes" the printed argument, in just such a way as bash can recover in something like eval!
Because each element is printed shell escaped on it's own line, the only separator between elements is the newline, and the array assignment takes each line as an element, parsing the escaped values into literal text.
e.g.
> a=("foo bar" baz)
> printf "%q\n" "${a[#]}"
'foo bar'
baz
> printf "%q\n"
''
The eval is necessary to strip the escaping off each value going back into the array.
'sort' can be used to order the output of a for-loop:
for i in ${ids[#]}; do echo $i; done | sort
and eliminate duplicates with "-u":
for i in ${ids[#]}; do echo $i; done | sort -u
Finally you can just overwrite your array with the unique elements:
ids=( `for i in ${ids[#]}; do echo $i; done | sort -u` )
this one will also preserve order:
echo ${ARRAY[#]} | tr [:space:] '\n' | awk '!a[$0]++'
and to modify the original array with the unique values:
ARRAY=($(echo ${ARRAY[#]} | tr [:space:] '\n' | awk '!a[$0]++'))
To create a new array consisting of unique values, ensure your array is not empty then do one of the following:
Remove duplicate entries (with sorting)
readarray -t NewArray < <(printf '%s\n' "${OriginalArray[#]}" | sort -u)
Remove duplicate entries (without sorting)
readarray -t NewArray < <(printf '%s\n' "${OriginalArray[#]}" | awk '!x[$0]++')
Warning: Do not try to do something like NewArray=( $(printf '%s\n' "${OriginalArray[#]}" | sort -u) ). It will break on spaces.
Without loosing the original ordering:
uniques=($(tr ' ' '\n' <<<"${original[#]}" | awk '!u[$0]++' | tr '\n' ' '))
If you want a solution that only uses bash internals, you can set the values as keys in an associative array, and then extract the keys:
declare -A uniqs
list=(foo bar bar "bar none")
for f in "${list[#]}"; do
uniqs["${f}"]=""
done
for thing in "${!uniqs[#]}"; do
echo "${thing}"
done
This will output
bar
foo
bar none
cat number.txt
1 2 3 4 4 3 2 5 6
print line into column: cat number.txt | awk '{for(i=1;i<=NF;i++) print $i}'
1
2
3
4
4
3
2
5
6
find the duplicate records: cat number.txt | awk '{for(i=1;i<=NF;i++) print $i}' |awk 'x[$0]++'
4
3
2
Replace duplicate records: cat number.txt | awk '{for(i=1;i<=NF;i++) print $i}' |awk '!x[$0]++'
1
2
3
4
5
6
Find only Uniq records: cat number.txt | awk '{for(i=1;i<=NF;i++) print $i|"sort|uniq -u"}
1
5
6
How about this variation?
printf '%s\n' "${ids[#]}" | sort -u
Another option for dealing with embedded whitespace, is to null-delimit with printf, make distinct with sort, then use a loop to pack it back into an array:
input=(a b c "$(printf "d\ne")" b c "$(printf "d\ne")")
output=()
while read -rd $'' element
do
output+=("$element")
done < <(printf "%s\0" "${input[#]}" | sort -uz)
At the end of this, input and output contain the desired values (provided order isn't important):
$ printf "%q\n" "${input[#]}"
a
b
c
$'d\ne'
b
c
$'d\ne'
$ printf "%q\n" "${output[#]}"
a
b
c
$'d\ne'
All the following work in bash and sh and are without error in shellcheck but you need to suppress SC2207
arrOrig=("192.168.3.4" "192.168.3.4" "192.168.3.3")
# NO SORTING
# shellcheck disable=SC2207
arr1=($(tr ' ' '\n' <<<"${arrOrig[#]}" | awk '!u[$0]++' | tr '\n' ' ')) # #estani
len1=${#arr1[#]}
echo "${len1}"
echo "${arr1[*]}"
# SORTING
# shellcheck disable=SC2207
arr2=($(printf '%s\n' "${arrOrig[#]}" | sort -u)) # #das.cyklone
len2=${#arr2[#]}
echo "${len2}"
echo "${arr2[*]}"
# SORTING
# shellcheck disable=SC2207
arr3=($(echo "${arrOrig[#]}" | tr ' ' '\n' | sort -u | tr '\n' ' ')) # #sampson-chen
len3=${#arr3[#]}
echo "${len3}"
echo "${arr3[*]}"
# SORTING
# shellcheck disable=SC2207
arr4=($(for i in "${arrOrig[#]}"; do echo "${i}"; done | sort -u)) # #corbyn42
len4=${#arr4[#]}
echo "${len4}"
echo "${arr4[*]}"
# NO SORTING
# shellcheck disable=SC2207
arr5=($(echo "${arrOrig[#]}" | tr "[:space:]" '\n' | awk '!a[$0]++')) # #faustus
len5=${#arr5[#]}
echo "${len5}"
echo "${arr5[*]}"
# OUTPUTS
# arr1
2 # length
192.168.3.4 192.168.3.3 # items
# arr2
2 # length
192.168.3.3 192.168.3.4 # items
# arr3
2 # length
192.168.3.3 192.168.3.4 # items
# arr4
2 # length
192.168.3.3 192.168.3.4 # items
# arr5
2 # length
192.168.3.4 192.168.3.3 # items
Output for all of these is 2 and correct. This answer basically summarises and tidies up the other answers in this post and is a useful quick reference. Attribution to original answer is given.
In zsh you can use (u) flag:
$ ids=(aa ab aa ac aa ad)
$ print ${(u)ids}
aa ab ac ad
Try this to get uniq values for first column in file
awk -F, '{a[$1];}END{for (i in a)print i;}'
# Read a file into variable
lines=$(cat /path/to/my/file)
# Go through each line the file put in the variable, and assign it a variable called $line
for line in $lines; do
# Print the line
echo $line
# End the loop, then sort it (add -u to have unique lines)
done | sort -u
I know how to iterate through alphabets:
for c in {a..z}; do ...; done
But I can't figure out how to iterate through all ASCII characters. Does anyone know how?
What you can do is to iterate from 0 to 127 and then convert the decimal value to its ASCII value(or back).
You can use these functions to do it:
# POSIX
# chr() - converts decimal value to its ASCII character representation
# ord() - converts ASCII character to its decimal value
chr() {
[ ${1} -lt 256 ] || return 1
printf \\$(printf '%03o' $1)
}
# Another version doing the octal conversion with arithmetic
# faster as it avoids a subshell
chr () {
[ ${1} -lt 256 ] || return 1
printf \\$(($1/64*100+$1%64/8*10+$1%8))
}
# Another version using a temporary variable to avoid subshell.
# This one requires bash 3.1.
chr() {
local tmp
[ ${1} -lt 256 ] || return 1
printf -v tmp '%03o' "$1"
printf \\"$tmp"
}
ord() {
LC_CTYPE=C printf '%d' "'$1"
}
# hex() - converts ASCII character to a hexadecimal value
# unhex() - converts a hexadecimal value to an ASCII character
hex() {
LC_CTYPE=C printf '%x' "'$1"
}
unhex() {
printf \\x"$1"
}
# examples:
chr $(ord A) # -> A
ord $(chr 65) # -> 65
A possibility using only echos octal escape sequences:
for n in {0..7}{0..7}{0..7}; do echo -ne "\\0$n"; done
Here is what I came up with for a one-liner taking some pieces from sampson-chen's and mata's answers:
for n in {0..127}; do awk '{ printf("%c", $0); }' <<< $n; done
Or alternatively:
for n in {0..127}; do echo $n; done | awk '{ printf("%c", $0); }'
Here's how you print an integer as its corresponding ascii character with awk:
echo "65" | awk '{ printf("%c", $0); }'
which will print:
A
And here's how you might iterate through the uppercase alphabet this way:
# ascii for A starts at 65:
ascii=65
index=1
total=26
while [[ $total -ge $index ]]
do
letter=$(echo "$ascii" | awk '{ printf("%c", $0); }')
echo "The $index'th letter is $letter"
# Increment the index counter as well as the ascii counter
index=$((index+1))
ascii=$((ascii+1))
done
Well... If you really want them all, and you want it to be something script-like, you could do this, I guess:
awk 'function utf32(i) {printf("%c%c%c%c",i%0x100,i/0x100%0x100,i/0x10000%0x100,i/0x1000000) } BEGIN{for(i=0;i<0x110000;i++){utf32(i);utf32(0xa)}}' | iconv --from-code=utf32 --to-code=utf8 | grep -a '[[:print:]]'
But the list is pretty huge, and not very useful. Awk may not be the most elegant way of generating the binary integers from 0 to 0x110000 - substitute something more elegant if you find it.
Edit: Oh, I see you only wanted ascii. Well, I will let this answer stay here in case somebody else actually wants all the UTF printable characters.
It depends what you mean by iterate. Be aware that NUL can't be assigned or passed to commands.
This generates all ascii characters
seq 0 127 |\
xargs printf '\\x%x ' |\
xargs printf '%b '
seq 0 127 generate all integers between 0 and 127
xargs printf '\\x%x ' convert it to hex, separated by space
xargs printf '%b ' convert hex to byte, separated by space