I'm having trouble understanding why there is a difference in the outputs of these 3 commands when executed on different distributions
Could you guys help me understand why ?
printf "%s" `echo ../.. | sed 's/[.]/\\&/g'`
output on kali: &&/&&~ output on ubuntu: &&/&&
printf "%s" $(echo ../.. | sed 's/[.]/\\&/g')
output on kali: \.\./\.\. output on ubuntu: ../..
printf "%s" "$(echo ../.. | sed 's/[.]/\\&/g')"
output on kali: \.\./\.\. output on ubuntu: \.\./\.\.
Kali uses Zsh shell. Ubuntu uses Bash shell.
output on ubuntu: &&/&&
Do not use backticks - prefer $(...). Backticks remove \ before execution.
printf "%s" `echo ../.. | sed 's/[.]/\\&/g'`
becomes:
printf "%s" $(echo ../.. | sed 's/[.]/\&/g')
So replaces . by & character.
output on kali: &&/&&~
The ~ on the end in Zsh in your configuration represents output from a command without a newline.
output on ...: \.\./\.\.
That should be understandable.
output on ubuntu: ../..
Aaaand this is the hard one around Bash 5.0. Unquoted result of command substitution undergo filename expansion (also called pathname expansion). The \.\./\.\. matches the directory path ../... Bash 5.0 introduced a change that would make the result of expansion undergo pathname expansion even if it doesn't have any globbing special characters.
The change was reversed in Bash 5.1. Bash-5.1-alpha available for download https://lists.gnu.org/archive/html/bug-bash/2019-01/msg00063.html.
Related
I'm developping a script to search for patterns within scripts executed from CRON on a bunch of remote servers through SSH.
Script on client machine -- SSH --> Remote Servers CRON/Scripts
For now I can't get the correct output.
Script on client machine
#!/bin/bash
server_list=( '172.x.x.x' '172.x.x.y' '172.x.x.z' )
for s in ${server_list[#]}; do
ssh -i /home/user/.ssh/my_key.rsa user#${s} crontab -l | grep -v '^#\|^[[:space:]]*$' | cut -d ' ' -f 6- | awk '{print $1}' | grep -v '^$\|^echo\|^find\|^PATH\|^/usr/bin\|^/bin/' | xargs -0 grep -in 'server.tld\|10.x.x.x'
done
This only gives me the paths of scripts from crontab, not the matched lines and line number plus the first line is prefixed with "grep:" keyword (example below):
grep: /opt/directory/script1.sh
/opt/directory/script2.sh
/opt/directory/script3.sh
/opt/directory/script4.sh
How to get proper output, meaning the script path plus line number plus line of matching pattern?
Remote CRON examples
OO 6 * * * /opt/directory/script1.sh foo
30 6 * * * /opt/directory/script2.sh bar
Remote script content examples
1 ) This will match grep pattern
#!/bin/bash
ping -c 4 server.tld && echo "server.tld ($1)"
2 ) This won't match grep pattern
#!/bin/bash
ping -c 4 8.x.x.x && echo "8.x.x.x ($1)"
Without example input, it's really hard to see what your script is attempting to do. But the cron parsing could almost certainly be simplified tremendously by refactoring all of it into a single Awk script. Here is a quick stab, with obviously no way to test.
#!/bin/sh
# No longer using an array for no good reason, so /bin/sh will work
for s in 172.x.x.x 172.x.x.y 172.x.x.z; do
ssh -i /home/user/.ssh/my_key.rsa "user#${s}" crontab -l |
awk '! /^#|^[[:space:]]*$/ && $6 !~ /^$|^(echo|find|PATH|\/usr\/bin|\/bin\/)/ { print $6 }' |
# no -0; use grep -E and properly quote literal dot
xargs grep -Ein 'server\.tld|10.x.x.x'
done
Your command would not output null-delimited data to xargs so probably the immediate problem was that xargs -0 would receive all the file names as a single file name which obviously does not exist, and you forgot to include the ": file not found" from the end of the error message.
The use of grep -E is a minor hack to enable a more modern regex syntax which is more similar to that in Awk, where you don't have to backslash the "or" pipe etc.
This script, like your original, runs grep on the local system where you run the SSH script. If you want to run the commands on the remote server, you will need to refactor to put the entire pipeline in single quotes or a here document:
for s in 172.x.x.x 172.x.x.y 172.x.x.z; do
ssh -i /home/user/.ssh/my_key.rsa "user#${s}" <<\________HERE
crontab -l |
awk '! /^#|^[[:space:]]*$/ && $6 !~ /^$|^(echo|find|PATH|\/usr\/bin|\/bin\/)/ { print $6 }' |
xargs grep -Ein 'server\.tld|10.x.x.x'
________HERE
done
The refactored script contains enough complexities in the quoting that you probably don't want to pass it as an argument to ssh, which requires you to figure out how to quote strings both locally and remotely. It's easier then to pass it as standard input, which obviously just gets transmitted verbatim.
If you get "Pseudo-terminal will not be allocated because stdin is not a terminal.", try using ssh -t. Sometimes you need to add multiple -t options to completely get rid of this message.
Is there an easy way to concatenate the result of two linux commands, in one line? (i.e without using variables)
I pull the local outdoor temperature from a nearby weather station. The result today is:
5.2
I simply want to add the units, so result should look like this:
5.2°C
An example command, that almost gives me what I want is:
wget -q -O- "http://meteocentre.com/montreal/home_e.html" | grep -oP '(?<=Tn= ).*(?=°)' ; printf "°C\n"
You are already concatenating the results.
wget .. | grep .. outputs: 5.2\n
printf outputs °C\n
The concatenated result is therefore 5.2\n°C\n , exactly what you're getting.
What you want to do is strip the linefeed after 5.2. You can take advantage of the fact that $(command substitution) strips trailing linefeeds and pass it to printf:
printf '%s°C\n' "$(wget -q -O- "http://meteocentre.com/montreal/home_e.html" | grep -oP '(?<=Tn= ).*(?=°)')"
in bash, add pipe at the end:
echo $(cat) [more text]
or
echo `cat` [more text]
example:
wget -q -O- "http://meteocentre.com/montreal/home_e.html" | grep -oP '(?<=Tn= ).*(?=°)' | echo $(cat) °C
-> 5.2 °C
I am new to bash and writing a script that needs to compare the minor version of the kernel to see if it is greater than or equal to 10, and exit if it is not. Currently I have something like this:
KERNEL=$(uname -r)
declare -i MINOR_VERSION=$(echo $KERNEL | cut -c 3-4)
if [ "10" -gt "$MINOR_VERSION"]; then exit 0; fi
This is bad code, and doesn't work if the minor version is < 10 since I am using cut and depending on it being two digits. What I probably need is something that parses the minor version by using the dots.
Example:
$ uname -r
3.4.0-60-generic
$ MNR_VAR=<awesome bash code, with cut or sed or something>
$ echo $MNR_VAR
4
I have been reading cut and sed documentation but have just been slow picking it up. I would appreciate the help!
TL;DR - looking for a bash command that will extract an int surrounded by the first two dots in a variable. "3.13.0.x" returns '13', "3.2.0.x" returns '2', etc.
EDIT:
Some answers as one liners below for those curious.
uname -r | cut -d '.' -f2
uname -r | awk -F . '{print $2}'
kernel="$(uname -r)" | tmp="${kernel#*.}" | minor="${tmp%%.*}" | echo "$minor"
In pure bash:
#!/bin/bash
ker="$(uname -r)"
minker="${ker#*.}"
minker="${minker%%.*}"
echo "$minker"
"${ker#*.}" is the string after the first match of a . in $ker. Thus
$minker becomes 13.0-generic... from 3.13.0-generic...
"${minker%%.*}" is the string left by cutting all matches (from right) of . and whatever after it, in $minker. Thus $minker becomes 13 from 13.0-generic...
See the Bash Parameter Expansion Manual for more info
Using Bash Regex:
#!/bin/bash
regex='([0-9]+)\.([0-9]+)'
[[ $(uname -r) =~ $regex ]]
echo ${BASH_REMATCH[2]}
The problem is you are using -c to cut. Don't do that.
Use the -f and -d flags instead to control the delimiter and fields to output.
Or use awk -F . '{print $2}' <<< "$(uname -r)".
Or use IFS=. read -r _ minor _rest <<< "$(uname -r)"; echo "$minor" (which has the benefit of not using any external utilities).
The usage of <<< "$(uname -r)" is bash-specific (I believe) but avoids the need for a pipe (|) and the sub-shell that it involves.
Extracting just minor version & comparing it with something is risky, because major number can change too...
I normally prefer padding the numbers with zeros, so that they can be easily compared using simple string compare.
kernel_version=$(uname -r | sed -r 's/([0-9]+)/0000\1/g; s/0*([0-9]{4})/\1/g') # gives 0003.0004.0000-0060-generic
if [[ "$kernel_version" < "0003.0010" ]]; then exit 0; fi
I am beginner in unix programming and a way to automate my work
I want to run a list a grep commands and get the output of all the grep command in a in a single delimited file .
i am using the following bash script. But it's not working .
Mockup sh file:
!/bin/sh
grep -l abcd123
grep -l abcd124
grep -l abcd125
and while running i used the following command
$ ./Mockup.sh > output.txt
Is it the right command?
How can I get both the grep command and output in the output file?
how can i delimit the output after each command and result?
How can I get both the grep command and output in the output file
You can use bash -v (verbose) to print each command before execution on stderr and it's output will be as usual be available on stdout:
bash -v ./Mockup.sh > output.txt 2>&1
cat output.txt
Working Demo
A suitable shell script could be
#!/bin/sh
grep -l 'abcd123\|abcd124\|abcd125' "$#"
provided that the filenames you pass on the invocation of the script are "well behaved", that is no whitespace in them. (Edit Using the "$#" expansion takes care of generic whitespace in the filenames, tx to triplee for his/her comment)
This kind of invocation (with alternative matching strings, as per the \| syntax) has the added advantage that you have exactly one occurrence of a filename in your final list, because grep -l prints once the filename as soon as it finds the first occurrence of one of the three strings in a file.
Addendum about "$#"
% ff () { for i in "$#" ; do printf "[%s]\n" "$i" ; done ; }
% # NB "a s d" below is indeed "a SPACE s TAB d"
% ff "a s d" " ert " '345
345'
[a s d]
[ ert ]
[345
345]
%
cat myscript.sh
########################
#!/bin/bash
echo "Trying to find the file contenting the below string, relace your string with below string"
grep "string" /path/to/folder/* -R -l
########################
save above file and run it as below
sh myscript.sh > output.txt
once the command prmpt get return you can check the output.txt for require output.
Another approach, less efficient, that tries to address the OP question
How can I get both the grep command and output in the output file?
% cat Mockup
#!/bin/sh
grep -o -e string1 -e string2 -e string3 "$#" 2> /dev/null | sort -t: -k2 | uniq
Output: (mocked up as well)
% sh Mockup file{01..99}
file01:string1
file17:string1
file44:string1
file33:string2
file44:string2
file48:string2
%
looking at the output from POV of a consumer, one foresees problems with search strings and/or file names containing colons... oh well, that's another Q maybe
I need to get only first two numbers of "uname -r" command in bash
example of regular out put:
uname -r
3.5.0-18-generic
what I expect using magic bash options:
3.5
assuming you want everything before the second dot, this will do what you want:
uname -r | cut -d. -f1-2
uname itself does not support cutting the output, afaik. The pipe through cut will show you fields 1 and 2 (-f1-2), delimited by dots (-d.)
uname -r | sed 's/\([0-9]\+\.[0-9]\+\)\..*/\1/'
You could also accomplish this with parameter expansion:
$ r="$(uname -r)"
$ echo ${r%.*}
3.5
${VAR%pat} non-greedily removes pat from the end of VAR. Note that pat is a glob pattern i.e. dot just means "dot" and star means "any-number-of-chars".