I have this content in file.csv
cat file.csv
QUOTA,landscape=test,region=europe,limit=N2_CPUS quota=24.0,quota_used=0.0,quota_used_percent=0
QUOTA,landscape=test,region=europe,limit=COMMITTED_N2_CPUS quota=0.0,quota_used=0.0,quota_used_percent=0
QUOTA,landscape=test,region=europe,limit=COMMITTED_C2_CPUS quota=0.0,quota_used=0.0,quota_used_percent=0
QUOTA,landscape=test,region=europe,limit=RESERVATIONS quota=100.0,quota_used=0.0,quota_used_percent=0
I need to remove values which contain strings "RESERVATIONS" and "N2_CPUS" and the variables can be random
variable=("RESERVATIONS","N2_CPUS")
I am able to do when i use one value as variable using
cat file.csv | grep -v $variable
When there are more values in a variable, even loops are not working as expected. Could you please suggest?
I would use egrep (or grep -E, depending on your flavor of linux)
variable="RESERVATIONS|N2_CPUS"
cat file.csv | egrep -v $variable
or
cat file.csv | grep -Ev $variable
Note, though, in your example, the cat is not required:
grep -Ev "${variable}" file.csv
Notice the quotes around the variable, you may need those as well, depending on your shell & Linux version.
egrep (or grep -E) is an grep with Extended Regular Expression. The vertical bar, or pipe | separates the values. Effectively it is saying OR. Thus,
egrep -Ev "A|B" means look for 'A' or 'B' and remove them.
Use grep -E so you can use an extended regular expression, and then use | in the regexp to match multiple strings.
variable=RESERVATIONS|N2_CPUS
grep -v -E "$variable" file.csv
Related
All the users in the system that dosen't have as an ending character on their names a, s, t, r, m, z must be shown in Bash.
The users names can be obtained from the /etc/passwd file, in the first column. But I can not perceive the correct approach to exclude those characters from the search.
Should I use grep? Or just a cut?
Something like
grep -o '^[^:]*[^astrmz:]:' /etc/passwd | tr -d :
or
cut -d: -f1 /etc/passwd | grep '[^astrmz]$'
[^blah] matches any character but the ones listed, the opposite of [blah].
GNU grep using a lookahead:
grep -Po '^[^:]*[^astrmz:](?=:)' /etc/passwd
Or using awk instead:
awk -F: '$1 ~ /[^astrmz]$/ { print $1 }' /etc/passwd
Or in pure bash without external commands:
while IFS=: read -r name rest; do
if [[ $name =~ [^astrmz]$ ]]; then
echo "$name"
fi
done < /etc/passwd
As you can see, there's lots of potential approaches.
Simple one liner when using bash:
compgen -u | grep -v '[astrmz]$'
The compgen -u command will produce a list of users (without all of the extra fields present in /etc/passwd); compgen is a builtin in bash where it's normally used for username completion.
This should do the trick:
cut -d: -f1 /etc/passwd | grep -vE 's$|t$|r$|m$|z$'
The cut command strips out the username from the password file.
Then grep -v (does the UNMATCHING)
grep -E does multiple matching (OR OR OR)
the $ sign indicates the last character to match
For example , on my mac, I get:
_gamecontrollerd
_ondemand
_wwwproxy
_findmydevice
_ctkd
_applepay
_hidd
_analyticsd
_fpsd
_timed
_reportmemoryexception
(You see no names end with those 5 letters).
Good Luck.
I have a text file with layout as:
tableName1|counterVariable1
tableName2|counterVariable2
I want to replace the counterVariable1 with some other variable say counterVariableNew.
How can I accomplish this?
I have tried various SED/AWK approaches, closest one is mentioned below:
cat $fileName | grep -w $tableName | sed -i 's/$tableName\|counterVariable/$tableName\|counterVariableNew'
But all the 3 commands are not merging properly, please help!
Your script is an example of [ useless use of cat ]. But the key point here is to escape the pipe delimiter which has a special meaning(it stands for OR) when used with awk FS. So below script should do
# cat 42000479
tableName1|counterVariable1
tableName2|counterVariable2
tableName3|counterVariable2
# awk -F\| '$1=="tableName2"{$2="counterVariableNew"}1' 42000479
tableName1|counterVariable1
tableName2 counterVariableNew
tableName3|counterVariable2
An alternate way of doing the same stuff is below
# awk -v FS='|' '$1=="tableName2"{$2="counterVariableNew"}1' 42000479
Stuff inside the single quote will not be expanded.
awk -F'|' -v OFS='|' '/tableName1/ {$2="counterVariableNew"}1' file
tableName1|counterVariableNew
tableName2|counterVariable2
This will search for A (tableName1) and replace B (counterVariable1) to counterVariableNew.
Or by using sed :
sed -r '/tableName1/ s/(^.*\|)(.*)/\1counterVariableNew/g' file
tableName1|counterVariableNew
tableName2|counterVariable2
For word bounded search: Enclose the pattern inside \< and \> .
sed -r '/\<tableName1\>/ s/(^.*\|)(.*)/\1counterVariableNew/g' file
awk -F'|' -v OFS='|' '/\<tableName1\>/ {$2="counterVariableNew"}1' file
I am trying to read/grep a particular word or content that is before a period (.).
e.g. file1 has abinaya.ashok and I want to grep whatever is before the period (.) without hardcoding anything.
if I try
grep \.\ file1
it gives abinaya.ashok.
I've tried: grep\*\.\ file1
it doesn't give anything.Can we find it using grep commands or should we do it only using awk command? Any thoughts?
Using GNU grep for PCRE regex (for non-greedy and positive look-ahead), you can do:
echo 'abinaya.ashok' | grep -oP '.*?(?=\.)'
abinaya
Using awk:
echo 'abinaya.ashok' | awk -F\. '{print $1}'
abinaya
Check the following simple examples.
Including the dot:
$ echo abinaya.ashok | grep -o '.*[.]'
abinaya.
Without the dot:
$ echo abinaya.ashok | grep -o '^[^.]\+'
abinaya
Hope I understand you correctly:
sed -n 's/\..*//p' file1 | grep whatever
sed expression will print only part before dot (lines without dot are not printed).
Now use grep to search what you need.
I have a file values.properties which contain data, like:
$ABC=10
$XYZ=20
I want to create a shell script that will take each element one by one from above file.
Say $ABC, then go to file ABC.txt & replace the value of $ABC with 10.
Similarly, then go to file XYZ.txt and replace $XYZ with 20.
I think maybe this should be in the Unix and Linux section, the solution I've hacked together is as follows:
cat values.properties | grep "=" | cut -d "$" -f2 | awk -F "=" '{print "s/$"$1"/"$2"/g "$1".txt"}' | xargs -n2 sed -i
The flow is like so:
Filter out all the value assignments via: grep "="
Remove the '$' via: cut -d "$" -f2
Use awk to split the variable name and value and construct sed replacement command
Use xargs to pull in the replacement parameter and target file via: xargs -n2
Finally pass sed to as the command to xargs: xargs -n2 sed
I have a small bash script that greps/awk paragraph by using a keyword.
But after adding in the extra codes : set var = "(......)" it only prints a blank line and not the paragraph.
So I would like to ask if anyone knows how to properly pass the awk output into a variable for outputting?
My codes:
#!/bin/sh
set var = "(awk 'BEGIN{RS=ORS="\n\n";FS=OFS="\n"}/FileHeader/' /root/Desktop
/logs/Default.log)"
echo $var;
Thanks!
Use command substitution to capture the output of a process.
#!/bin/sh
VAR="$(awk 'BEGIN{RS=ORS="\n\n";FS=OFS="\n"}/FileHeader/' /root/Desktop/logs/Default.log)"
echo "$VAR"
some general advice with regards to shell scripting:
(almost) always quote every variable reference.
never put spaces around the equals sign in variable assignment.
You need to use "command substitution". Place the command inside either backticks, `COMMAND` or, in a pair of parentheses preceded by a dollar sign, $(COMMAND).
To set a variable you don't use set and you can't have spaces before and after the =.
Try this:
var=$(awk 'BEGIN{RS=ORS="\n\n";FS=OFS="\n"}/FileHeader/' /root/Desktop/logs/Default.log)
echo $var
You gave me the idea of this for killing a process :). Just chromium to whatever process you wanna kill.
Try this:
VAR=$(ps -ef | grep -i chromium | awk '{print $2}'); kill -9 $VAR 2>/dev/null; unset VAR;
anytime you see grep piped to awk, you can drop the grep. for the above,
awk '/^password/ {print $2}'
awk can easily replace any text command like cut, tail, wc, tr etc. and especally multiple greps piped next to each other. i.e
grep some_co.mand | a | grep b ... to | awk '/a|b|and so on/ {some action}.
Try to create a variable coming from vault/Hashicorp, when using packer template variables, like so:
BUILD_PASSWORD=$(vault read secret/buildAccount| grep ^password | awk '{print $2}')
echo $BUILD_PASSWORD
You can to the same with grep ^user