I have a small bash script that greps/awk paragraph by using a keyword.
But after adding in the extra codes : set var = "(......)" it only prints a blank line and not the paragraph.
So I would like to ask if anyone knows how to properly pass the awk output into a variable for outputting?
My codes:
#!/bin/sh
set var = "(awk 'BEGIN{RS=ORS="\n\n";FS=OFS="\n"}/FileHeader/' /root/Desktop
/logs/Default.log)"
echo $var;
Thanks!
Use command substitution to capture the output of a process.
#!/bin/sh
VAR="$(awk 'BEGIN{RS=ORS="\n\n";FS=OFS="\n"}/FileHeader/' /root/Desktop/logs/Default.log)"
echo "$VAR"
some general advice with regards to shell scripting:
(almost) always quote every variable reference.
never put spaces around the equals sign in variable assignment.
You need to use "command substitution". Place the command inside either backticks, `COMMAND` or, in a pair of parentheses preceded by a dollar sign, $(COMMAND).
To set a variable you don't use set and you can't have spaces before and after the =.
Try this:
var=$(awk 'BEGIN{RS=ORS="\n\n";FS=OFS="\n"}/FileHeader/' /root/Desktop/logs/Default.log)
echo $var
You gave me the idea of this for killing a process :). Just chromium to whatever process you wanna kill.
Try this:
VAR=$(ps -ef | grep -i chromium | awk '{print $2}'); kill -9 $VAR 2>/dev/null; unset VAR;
anytime you see grep piped to awk, you can drop the grep. for the above,
awk '/^password/ {print $2}'
awk can easily replace any text command like cut, tail, wc, tr etc. and especally multiple greps piped next to each other. i.e
grep some_co.mand | a | grep b ... to | awk '/a|b|and so on/ {some action}.
Try to create a variable coming from vault/Hashicorp, when using packer template variables, like so:
BUILD_PASSWORD=$(vault read secret/buildAccount| grep ^password | awk '{print $2}')
echo $BUILD_PASSWORD
You can to the same with grep ^user
Related
I have this content in file.csv
cat file.csv
QUOTA,landscape=test,region=europe,limit=N2_CPUS quota=24.0,quota_used=0.0,quota_used_percent=0
QUOTA,landscape=test,region=europe,limit=COMMITTED_N2_CPUS quota=0.0,quota_used=0.0,quota_used_percent=0
QUOTA,landscape=test,region=europe,limit=COMMITTED_C2_CPUS quota=0.0,quota_used=0.0,quota_used_percent=0
QUOTA,landscape=test,region=europe,limit=RESERVATIONS quota=100.0,quota_used=0.0,quota_used_percent=0
I need to remove values which contain strings "RESERVATIONS" and "N2_CPUS" and the variables can be random
variable=("RESERVATIONS","N2_CPUS")
I am able to do when i use one value as variable using
cat file.csv | grep -v $variable
When there are more values in a variable, even loops are not working as expected. Could you please suggest?
I would use egrep (or grep -E, depending on your flavor of linux)
variable="RESERVATIONS|N2_CPUS"
cat file.csv | egrep -v $variable
or
cat file.csv | grep -Ev $variable
Note, though, in your example, the cat is not required:
grep -Ev "${variable}" file.csv
Notice the quotes around the variable, you may need those as well, depending on your shell & Linux version.
egrep (or grep -E) is an grep with Extended Regular Expression. The vertical bar, or pipe | separates the values. Effectively it is saying OR. Thus,
egrep -Ev "A|B" means look for 'A' or 'B' and remove them.
Use grep -E so you can use an extended regular expression, and then use | in the regexp to match multiple strings.
variable=RESERVATIONS|N2_CPUS
grep -v -E "$variable" file.csv
This question already has answers here:
Command not found error in Bash variable assignment
(5 answers)
Closed 4 years ago.
Im having trouble calling a variable that should bring out the output of a command.
#!/bin/bash
ipAddresses = 'ifconfig | awk -v OFS=": " -v RS= '$1!="lo" && split($0, a, /inet addr:/) > 1{sub(/ .*/, "", a[2]); print $1, a[2]}''
echo -e "Sus direcciones IP son: \n " $(ipAddresses)
Appreciating any advice
Variable assignments cannot have space around the = in the shell. Also, you don't want single quotes there, you want either backticks or $(). The single quotes should only be for your awk command. Your awk is needlessly complicated as well, and you are using command substitution ($()) when printing, but ipAdresses is a variable, not a command.
Try something like this:
#!/bin/bash
ipAddresses=$(ifconfig | sed 's/^ *//' | awk -F'[: ]' '/^ *inet addr:/{print $3}')
printf 'Sus direcciones IP son:\n%s\n' "$ipAddresses"
But that is really not portable. You didn't mention your OS, but I am assuming it's a Linux and the output suggests Ubuntu (I don't have addr after inet in the output of ifconfig on my Arch, for example).
If you are running Linux, you could use grep instead:
ipAddresses=$(ifconfig | grep -oP 'inet addr:\K\S+')
ip is generally replacing ifconfig, so try this instead:
ipAddresses=$(ip addr | awk '/inet /{print $2}')
or
ipAddresses=$(ip addr | grep -oP 'inet \K\S+')
Or, to remove the trailing /N:
ipAddresses=$(ip addr | grep -oP 'inet \K[\d.]+')
And you don't need the variable anyway, you can just:
printf 'Sus direcciones IP son:\n%s\n' "$(ip addr | awk '/inet /{print $2}')"
I am not sure about your intention, since they are not stated, so I am trying to guess them from the script.
Option 1: you are trying to get IP address to into the variable ipAddresses and that is not happenning.
Start by changing single quotes around the long command and debug the command.
Option 2: you are storing a command in variable ipAddresses that you want to execute on the second line.
For both of the options you need to use the the value of the variable through $ipAdresses on the second line.
Also fix the assignment to following formart:
varName="value" # Note no spaces around = sign
Replace the final $(ipAddresses) with ${ipAddresses} or just "$ipAddresses", but also save the output of your command using $().
Check Difference between ${} and $() in Bash.
A basic example:
#!/bin/sh
OUTPUT=$(uname -a)
echo "The output: $OUTPUT"
I am writing a small script to map all the current memory being used by services running in a server. However, I am facing a problem doing that. My script is quite simple. I'm using pmap to find out memory being used and trying add up all the pid of a service running.
#!/bin/bash
result=`$pgrep java`
wc=`$pmap -x $result | wc -l`
gawk=`$pmap -x $result | gawk 'NR==$wc{print $3}'`
echo "$gawk"
Now, my problem is that gawk uses single quote when searching for a specific pattern (gawk 'NR==$wc{print $3}') but shell script gives me error because then meaning of single quote is different in shell from gawk.
Based on your comment, it looks like you're trying to do this:
pmap -x "$(pgrep java)" | awk '{s=$3}END{print s}'
This prints the third column of the last line of the output of pmap -x, with the PID of the running java process. In some versions of awk, you can simply do 'END{print $3}' but this isn't guaranteed to work.
pmap -x $result | gawk 'NR==$wc{print $3}' is not doing what you think it is. (I have replaced your $pmap with pmap, but my analysis is only of the gawk command so if that is incorrect it should be irrelevant.) The shell is going to pass the literal string NR==$wc{print $3} to awk, but it appears that you want awk to see the value of the shell variable $wc rather than the literal string $wc. When awk sees $wc, it treats wc an an uninitialized value, so $wc become equivalent to $0, and awk will print any line whose content matches the line number. The standard way to pass the shell variable into awk is:
pmap -x $result | gawk 'NR==w{print $3}' w=$wc
This assignes the shell variable wc to the awk variable w, and will print the third column of that line.
Note that there are a number of issues with this shell script, but this seems to be the core confusion.
So I have this string:
$var=server#10.200.200.20:/home/some/directory/file
I just want to extract the directory address meaning I only want the bit after the ":" character and get:
/home/some/directory/file
thanks.
I need a generic command so the cut command wont work as the $var variable doesn't have a fixed length.
Using sed:
$ var=server#10.200.200.20:/home/some/directory/file
$ echo $var | sed 's/.*://'
/home/some/directory/file
This might work for you:
echo ${var#*:}
See Example 10-10. Pattern matching in parameter substitution
This will also do.
echo $var | cut -f2 -d":"
For completeness, using cut
cut -d : -f 2 <<< $var
And using only bash:
IFS=: read a b <<< $var ; echo $b
You don't say which shell you're using. If it's a POSIX-compatible one such as Bash, then parameter expansion can do what you want:
Parameter Expansion
...
${parameter#word}
Remove Smallest Prefix Pattern.
The word is expanded to produce a pattern. The parameter expansion then results in parameter, with the smallest portion of the prefix matched by the pattern deleted.
In other words, you can write
$var="${var#*:}"
which will remove anything matching *: from $var (i.e. everything up to and including the first :). If you want to match up to the last :, then you could use ## in place of #.
This is all assuming that the part to remove does not contain : (true for IPv4 addresses, but not for IPv6 addresses)
This should do the trick:
$ echo "$var" | awk -F':' '{print $NF}'
/home/some/directory/file
awk -F: '{print $2}' <<< $var
I want to do this:
run a command
capture the output
select a line
select a column of that line
Just as an example, let's say I want to get the command name from a $PID (please note this is just an example, I'm not suggesting this is the easiest way to get a command name from a process id - my real problem is with another command whose output format I can't control).
If I run ps I get:
PID TTY TIME CMD
11383 pts/1 00:00:00 bash
11771 pts/1 00:00:00 ps
Now I do ps | egrep 11383 and get
11383 pts/1 00:00:00 bash
Next step: ps | egrep 11383 | cut -d" " -f 4. Output is:
<absolutely nothing/>
The problem is that cut cuts the output by single spaces, and as ps adds some spaces between the 2nd and 3rd columns to keep some resemblance of a table, cut picks an empty string. Of course, I could use cut to select the 7th and not the 4th field, but how can I know, specially when the output is variable and unknown on beforehand.
One easy way is to add a pass of tr to squeeze any repeated field separators out:
$ ps | egrep 11383 | tr -s ' ' | cut -d ' ' -f 4
I think the simplest way is to use awk. Example:
$ echo "11383 pts/1 00:00:00 bash" | awk '{ print $4; }'
bash
Please note that the tr -s ' ' option will not remove any single leading spaces. If your column is right-aligned (as with ps pid)...
$ ps h -o pid,user -C ssh,sshd | tr -s " "
1543 root
19645 root
19731 root
Then cutting will result in a blank line for some of those fields if it is the first column:
$ <previous command> | cut -d ' ' -f1
19645
19731
Unless you precede it with a space, obviously
$ <command> | sed -e "s/.*/ &/" | tr -s " "
Now, for this particular case of pid numbers (not names), there is a function called pgrep:
$ pgrep ssh
Shell functions
However, in general it is actually still possible to use shell functions in a concise manner, because there is a neat thing about the read command:
$ <command> | while read a b; do echo $a; done
The first parameter to read, a, selects the first column, and if there is more, everything else will be put in b. As a result, you never need more variables than the number of your column +1.
So,
while read a b c d; do echo $c; done
will then output the 3rd column. As indicated in my comment...
A piped read will be executed in an environment that does not pass variables to the calling script.
out=$(ps whatever | { read a b c d; echo $c; })
arr=($(ps whatever | { read a b c d; echo $c $b; }))
echo ${arr[1]} # will output 'b'`
The Array Solution
So we then end up with the answer by #frayser which is to use the shell variable IFS which defaults to a space, to split the string into an array. It only works in Bash though. Dash and Ash do not support it. I have had a really hard time splitting a string into components in a Busybox thing. It is easy enough to get a single component (e.g. using awk) and then to repeat that for every parameter you need. But then you end up repeatedly calling awk on the same line, or repeatedly using a read block with echo on the same line. Which is not efficient or pretty. So you end up splitting using ${name%% *} and so on. Makes you yearn for some Python skills because in fact shell scripting is not a lot of fun anymore if half or more of the features you are accustomed to, are gone. But you can assume that even python would not be installed on such a system, and it wasn't ;-).
try
ps |&
while read -p first second third fourth etc ; do
if [[ $first == '11383' ]]
then
echo got: $fourth
fi
done
Your command
ps | egrep 11383 | cut -d" " -f 4
misses a tr -s to squeeze spaces, as unwind explains in his answer.
However, you maybe want to use awk, since it handles all of these actions in a single command:
ps | awk '/11383/ {print $4}'
This prints the 4th column in those lines containing 11383. If you want this to match 11383 if it appears in the beginning of the line, then you can say ps | awk '/^11383/ {print $4}'.
Using array variables
set $(ps | egrep "^11383 "); echo $4
or
A=( $(ps | egrep "^11383 ") ) ; echo ${A[3]}
Similar to brianegge's awk solution, here is the Perl equivalent:
ps | egrep 11383 | perl -lane 'print $F[3]'
-a enables autosplit mode, which populates the #F array with the column data.
Use -F, if your data is comma-delimited, rather than space-delimited.
Field 3 is printed since Perl starts counting from 0 rather than 1
Getting the correct line (example for line no. 6) is done with head and tail and the correct word (word no. 4) can be captured with awk:
command|head -n 6|tail -n 1|awk '{print $4}'
Instead of doing all these greps and stuff, I'd advise you to use ps capabilities of changing output format.
ps -o cmd= -p 12345
You get the cmmand line of a process with the pid specified and nothing else.
This is POSIX-conformant and may be thus considered portable.
Bash's set will parse all output into position parameters.
For instance, with set $(free -h) command, echo $7 will show "Mem:"