I want to parse each line from a text with this structure:
ipv4address: 1.2.3.4/29
ipv4gateway: 1.2.3.1
ipv4mtu: 1500
ipv4dnsserver: 8.8.8.8
ipv4dnsserver: 8.8.4.4
Newlines are seperated by \n.
To generate this file I use a program which will output some information:
CONFIG=$(umbim $DBG -d $device -n -t $tid config) || {
echo "mbim[$$]" "config failed"
return 1
}
then I write out the $CONFIG variable to a file, just to reread it again, which seems wrong to me.
echo "$CONFIG" > /tmp/ip
Then after that I use grep to get the information:
IP=$(grep "ipv4address" /tmp/ip |grep -E -o "(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)")
NM=$(grep "ipv4address" /tmp/ip |grep -o '.\{2\}$')
GW=$(grep "ipv4gateway" /tmp/ip |grep -E -o "(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)")
I want to avoid writing to a file. It would be better, or at least it seems better if I could grep on the $CONFIG variable. But using echo $CONFIG will not yield the results as newlines are ommitted with this. The same with printf.
I am using busybox if that helps.
BusyBox v1.25.1 () built-in shell (ash)
Edit: This is what happens when I want to print out the variable with echo:
$ CONFIG=$(cat /tmp/ip)
$ echo -e $CONFIG
ipv4address: 1.2.3.4/29 ipv4gateway: 1.2.3.1 ipv4mtu: 1500 ipv4dnsserver: 8.8.8.8 ipv4dnsserver: 8.8.4.4
Shell variable should almost always be quoted. If instead of echo $CONFIG | grep ... you use echo "$CONFIG" | grep ..., the newlines will be preserved and you'll get the expected result.
Why won't you just use something like this?
eval $(umbim $DBG -d $device -n -t $tid config | tr -d ' ' | grep ^ipv4 | tr a-z: A-Z=)
IP=${IPV4ADDRESS%/*}
NM=${IPV4ADDRESS##*/}
GW=$IPV4GATEWAY
Related
I have a parameter file (param.env) having the following content.
MY_PARAM=com:27}WMV\)pviZN
also, a bash file where I am fetching the value of MY_PARAM and writing into a random file.
#!/bin/bash
value=$(grep "^MY_PARAM=" param.env | cut -d '=' -f2-)
value1=$(cat param.env | grep "^MY_PARAM" | sed 's/=/ /' | awk '{print $2}')
echo $value
echo $value1
printf '%s\n' "$value"
Output:
com:27}WMV\)pviZN
com:27}WMV\)pviZN
com:27}WMV\)pviZN
However, I am expecting \ to be escaped and should not be part of the output.
I am also not allowed to edit the param.env.
Expected output:
com:27}WMV)pviZN
You could source the file, then the string will behave as if you'd assigned it like that in an interactive shell:
$ (. param.env; echo "$MY_PARAM")
com:27}WMV)pviZN
I've put the commands in a subshell so they don't pollute the environment.
I'm trying to grep multiple arguments in shell.
I put orders like ./script arg1 arg2.. argN
And I want them to act
egrep -i "arg1" mydata | egrep -i "arg2" | ... egrep -i "argN" | awk -f display.awk
in order to match patterns in AND format.
What's wrong in my process?
Is it even right to code like
egrep -i "arg1" mydata | egrep -i "arg2" | ... egrep -i "argN" | awk -f display.awk
to get multiple patterns in AND format??
if [ $# -eq 0 ]
then
echo "Usage:phone searchfor [...searchfor]"
echo "(You didn't tell me what you want to search for.)"
exit 0
else
for arg in $*
do
if [ $arg -eq $1 ]
then
egrep -i "arg" mydata |
else
egrep -i "arg" |
fi
done
awk -f display.awk
fi
If my data has
'happy sunny bunny',
'sleepy bunny',
and 'happy sunny'
I want them to perform if I tried ./script happy sunny bunny
then only
'happy sunny bunny'
comes out.
and if i tried ./script bunny then
'happy sunny bunny'
'sleepy bunny'
both coming out.
The immediate fix is to move the pipe character to after the done.
Also, you should loop over "$#" to preserve the quoting of your arguments, and generally quote your variables.
if [ $# -eq 0 ]
then
# print diagnostics to stderr
echo "Usage: phone searchfor [...searchfor]" >&2
echo "(You didn't tell me what you want to search for.)" >&2
exit 0
fi
for arg in "$#"
do
# Add missing dash before eq
if [ "$arg " -eq "$1" ]
then
# Surely you want "$arg" here, not the static string "arg"?
grep -E -i "$arg" mydata
else
grep -E -i "$arg"
fi
done |
awk -f display.awk
The overall logic still seems flawed; you will be grepping standard input for the first argument if there are more than two arguments. Perhaps you want to add an option to allow the user to specify an input file name, with - to specify standard input? And then all the regular arguments will be search strings, like the usage message suggests.
If indeed the intent is to loop over all the arguments to produce a logical AND, try this:
also () {
local what
what=$1
shift
if [ $# -gt 0 ]; then
grep -E -i "$what" | also "$#"
else
grep -E -i "$what"
fi
}
also "$#" <mydata | awk -f display.awk
... though a better implementation might be to build a simple Awk or sed script from the arguments:
script='1'
for arg in "$#"; do
script="$script && tolower(\$0) ~ tolower(\"$arg\")"
done
awk "$script" | awk -f display.awk
This breaks down if the search phrases could contain regex specials, though (which of course is true for the grep -E version as well; but then you could easily switch to grep -F).
Merging the two Awk scripts into one should probably not be hard either, though without seeing display.awk, this is speculative.
You can solve it recursively:
#! /bin/bash
if (( $# == 0)); then
exec cat
else
arg=$1; shift
egrep "$arg" | "$0" "$#"
fi
The recursion ends, if the script is called with no arguments. In this case it behaves like cat. In your example you can put your awk there. If the script is called with one or more arguemnts, it calles egrep with the first argument ($1) and passes the remaining arguments ($# after shift) to itself ($0).
Example:
$ ./recursive-egrep sys < /etc/passwd
sys:x:3:3:sys:/dev:/usr/sbin/nologin
systemd-timesync:x:100:102:systemd Time Synchronization,,,:/run/systemd:/bin/false
systemd-network:x:101:103:systemd Network Management,,,:/run/systemd/netif:/bin/false
systemd-resolve:x:102:104:systemd Resolver,,,:/run/systemd/resolve:/bin/false
systemd-bus-proxy:x:103:105:systemd Bus Proxy,,,:/run/systemd:/bin/false
$ ./recursive-egrep sys no < /etc/passwd
sys:x:3:3:sys:/dev:/usr/sbin/nologin
Use G from https://gitlab.com/ole.tange/tangetools/tree/master/G which does this (except for the awk part).
SYNOPSIS
G [[grep options] string] [[grep options] string] ...
DESCRIPTION
G is a shorthand of writing (search for single lines matching expressions):
grep --option string | grep --option2 string2
or with -g (search full files matching expressions):
find . -type f | xargs grep -l string1 | xargs grep -l string1
When I try to run this script this error appears : operating extra /home/ubuntu/Desktop/Destino/, and I do not know why , someone help me please.
#!/bin/bash
input="/home/ubuntu/Desktop/Output/SAIDA.txt"
dt=`date +"%Y%m%d%H%M%S"`
layout='C'
if [ -e "$input" ] ; then
header=$(head -n 1 $input)
export header
tail -n +2 $input | split -l 99 -d --additional-suffix=.txt \ --filter='{ printf %s\\n "$header"; cat; }' >/home/ubuntu/Desktop/Destino/$FILE - NOMENCLATURA_${dt}_
for arquivo in ´Is/home/ubuntu/Desktop/*.txt´
do
NOME= ´cat $arquivo | cut -d "." -f1´
touch/home/ubuntu/Desktop/Destino/$NOME.cfg
echo $dt > $NOME.cfg
echo $layout > $NOME.cfg
done
else
echo "The input file does not exist."
fi
You have some strange quote characters in your script. To substitute the output of a command, wrap it with $() or backticks, not ´ characters.
for arquivo in ´Is/home/ubuntu/Desktop/*.txt´
I guess Is was meant to be ls, but you left out the space after it. But there's no need to parse the output of ls, just use the wildcard directly.
for arquivo in /home/ubuntu/Desktop/*.txt
On this line:
tail -n +2 $input | split -l 99 -d --additional-suffix=.txt \ --filter='{ printf %s\\n "$header"; cat; }' >/home/ubuntu/Desktop/Destino/$FILE - NOMENCLATURA_${dt}_
you need to put the output filename in quotes because of the spaces.
tail -n +2 $input | split -l 99 -d --additional-suffix=.txt \ --filter='{ printf %s\\n "$header"; cat; }' >"/home/ubuntu/Desktop/Destino/$FILE - NOMENCLATURA_${dt}_"
Also, the FILE variable is not set, you need to assign that earlier.
On this line:
NOME= ´cat $arquivo | cut -d "." -f1´
you're again using the wrong type of quotes to capture the output of the command. Also, you must not have a space between = and the value you want to assign. It should be:
NOME=$(cat $arquivo | cut -d "." -f1)
There's no need to do export header. The variable is only being used in this script, not in any child processes.
I'm using awk to parse /etc/hosts and produce a command which will format MapR for me. It's being done in a bash utility in Chef:
egrep '^[0-9]' /etc/hosts | grep -v 127.0.0.1 \
| awk 'NR==1{ips=$1}
NR>1{ips=ips ", " $1}
$2=="namenode"{nn=$1}
END{ printf "/opt/mapr/server/configure.sh -C %s -Z %s -N mycluster --create-user -D /dev/xvdb\n", ips, nn}' \
| bash
sleep 60
The command above should execute the following command:
/opt/mapr/server/configure.sh -C 10.32.237.251 -Z 10.32.237.251 -N mycluster --create-user -D /dev/xvdb
However, looking into my chef output I see:
==> namenode: Executing awk utility
==> namenode: awk: line 1: runaway string constant "/opt/mapr/ ...
The command never got executed in the MapR node... However when i execute it directly on the terminal it works nicely in the way it's supposed to be. What am I doing wrong?
I'm updating the question to show the complete bash script that executes that utility:
DISK_CONFIG=/home/ubuntu/disk_config
if [ -f $DISK_CONFIG ];
then
echo "File already exists"
else
echo "Executing awk utility\n"
touch $DISK_CONFIG
egrep '^[0-9]' /etc/hosts | grep -v 127.0.0.1 \
| awk 'NR==1{ips=$1}
NR>1{ips=ips ", " $1}
$2=="namenode"{nn=$1}
END{ printf "/opt/mapr/server/configure.sh -C %s -Z %s -N mycluster --create-user -D /dev/xvdb\n", ips, nn}' \
| bash
sleep 60
fi
Assuming you're using HEREDOC syntax in your bash resource:
bash "whatever" do
code <<-EOH
DISK_CONFIG=/tmp/disk_config
if [ -f $DISK_CONFIG ];
then
echo "File already exists"
else
echo "Executing awk utility\n"
touch $DISK_CONFIG
egrep '^[0-9]' /etc/hosts | grep -v 127.0.0.1 \
| awk 'NR==1{ips=$1}
NR>1{ips=ips ", " $1}
$2=="namenode"{nn=$1}
END{ printf "/opt/mapr/server/configure.sh -C %s -Z %s -N mycluster --create-user -D /dev/xvdb\n", ips, nn}' \
| bash
fi
EOH
end
this one leads to your error:
Executing awk utility
awk: line 4: runaway string constant "/opt/mapr/ ...
This is due to the \n in your comand (the one into the awk command is likely to be problematic too)
This resource should do (warning I did replace the DISK_CONFIG path for my tests):
bash "whatever" do
code <<-EOH
DISK_CONFIG=/tmp/disk_config
if [ -f $DISK_CONFIG ];
then
echo "File already exists"
else
echo "Executing awk utility"
touch $DISK_CONFIG
egrep '^[0-9]' /etc/hosts | grep -v 127.0.0.1 \
| awk 'NR==1{ips=$1}
NR>1{ips=ips ", " $1}
$2=="namenode"{nn=$1}
END{ printf "/opt/mapr/server/configure.sh -C %s -Z %s -N mycluster --create-user -D /dev/xvdb", ips, nn}' \
| bash
sleep 60
fi
EOH
end
The reason is that Chef already interpret the \n in the code and so awk see a string never ending (runaway).
As you pipe to bash you can omit the \n as the pipe will end the line.
I get the following error:
> echo "${$(qstat -a | grep kig):0:7}"
-bash: ${$(qstat -a | grep kig):0:7}: bad substitution
I'm trying to take the number before. of
> qstat -a | grep kig
1192530.perceus- kigumen lr_regul pbs.sh 27198 2 16 -- 24:00:00 R 00:32:23
and use it as an argument to qdel in openPBS so that I can delete all process that I started with my login kigumen
so ideally, this should work:
qdel ${$(qstat -a | grep kig):0:7}
so far, only this works:
str=$(qstat -a | grep kig); qdel "${str:0:7}"
but I want a clean one-liner without a temporary variable.
The shell substring construct you're using (:0:7) only works on variables, not command substitution. If you want to do this in a single operation, you'll need to trim the string as part of the pipeline, something like one of these:
echo "$(qstat -a | grep kig | sed 's/[.].*//')"
echo "$(qstat -a | awk -F. '/kig/ {print $1}')"
echo "$(qstat -a | awk '/kig/ {print substr($0, 1, 7)}')"
(Note that the first two print everything before the first ".", while the last prints the first 7 characters.) I don't know that any of them are particularly cleaner, but they do it without a temp variable...
qstat -u palle | cut -f 1 -d "." | xargs qdel
Kills all my jobs... normally I grep out the jobname(s) before cut'ing...
So I use a small script "idlist":
qstat -u palle | grep -E "*.in" | grep -E "$1" | cut -f 1 -d "." | xargs
To see all my "map_..." jobs:
idlist "map_*"
For killing all my "map_...." jobs:
idlist "map_*" | xargs qdel
yet another ways :
foreach m1 in $(qstat -a );do
if [[ $m1 =~ kig ]];then
m2=${m1%.kig}
echo "kig found $m2 "
break
fi
done