I am writing a small script to map all the current memory being used by services running in a server. However, I am facing a problem doing that. My script is quite simple. I'm using pmap to find out memory being used and trying add up all the pid of a service running.
#!/bin/bash
result=`$pgrep java`
wc=`$pmap -x $result | wc -l`
gawk=`$pmap -x $result | gawk 'NR==$wc{print $3}'`
echo "$gawk"
Now, my problem is that gawk uses single quote when searching for a specific pattern (gawk 'NR==$wc{print $3}') but shell script gives me error because then meaning of single quote is different in shell from gawk.
Based on your comment, it looks like you're trying to do this:
pmap -x "$(pgrep java)" | awk '{s=$3}END{print s}'
This prints the third column of the last line of the output of pmap -x, with the PID of the running java process. In some versions of awk, you can simply do 'END{print $3}' but this isn't guaranteed to work.
pmap -x $result | gawk 'NR==$wc{print $3}' is not doing what you think it is. (I have replaced your $pmap with pmap, but my analysis is only of the gawk command so if that is incorrect it should be irrelevant.) The shell is going to pass the literal string NR==$wc{print $3} to awk, but it appears that you want awk to see the value of the shell variable $wc rather than the literal string $wc. When awk sees $wc, it treats wc an an uninitialized value, so $wc become equivalent to $0, and awk will print any line whose content matches the line number. The standard way to pass the shell variable into awk is:
pmap -x $result | gawk 'NR==w{print $3}' w=$wc
This assignes the shell variable wc to the awk variable w, and will print the third column of that line.
Note that there are a number of issues with this shell script, but this seems to be the core confusion.
Related
I'm trying to build a script in Linux (Debian 10) that shows the net usage (%) of a process passed as an argument.
This is the code, but there isn't any output:
ProcessName=$1
(nethogs -t ens33 | awk '/$ProcessName/{print $3}') &> output.txt
While using tracemode nethogs -t, first field of output is program and it can consists of irregular number of arguments.
In case of brave:
/usr/lib/brave-bin/brave --type=utility --utility-sub-type=network.mojom.NetworkService --field-trial-handle=18208005703828410459,4915436466583499460,131072 --enable-features=AutoupgradeMixedContent,DnsOverHttps,LegacyTLSEnforced,PasswordImport,PrefetchPrivacyChanges,ReducedReferrerGranularity,SafetyTip,WebUIDarkMode --disable-features=AutofillEnableAccountWalletStorage,AutofillServerCommunication,DirectSockets,EnableProfilePickerOnStartup,IdleDetection,LangClientHintHeader,NetworkTimeServiceQuerying,NotificationTriggers,SafeBrowsingEnhancedProtection,SafeBrowsingEnhancedProtectionMessageInInterstitials,SharingQRCodeGenerator,SignedExchangePrefetchCacheForNavigations,SignedExchangeSubresourcePrefetch,SubresourceWebBundles,TabHoverCards,TextFragmentAnchor,WebOTP --lang=en-US --service-sandbox-type=none --shared-files=v8_context_snapshot_data:100/930/1000 0.0554687 0.0554687
so $3 will no longer be as expected, you need to get last column of output using $(NF) as follow:
... | awk /$ProcessName/'{print $(NF)}'
for second last column:
... | awk /$ProcessName/'{print $(NF - 1)}'
What I'm doing wrong?
What you're doing wrong is single-quoting $ProcessName while wanting this to be parameter-expanded. To get the expansion, just don't quote there, e. g.:
… awk /$ProcessName/'{print $3}' …
I was unsure on how to correctly script a particular awk command which uses a shell variable, when I read the answers to How do I use shell variables in an awk script?.
The accepted answer demonstrates how interpolating a shell variable in an awkcommand would be prone to malicious code injection, and while I was able to reproduce the demo, I could not find the same problem with either of the following two commands:
#HWLINK=enp10s0
ip -o route | awk '/'$HWLINK'/ && ! /default/ {print $1}'
ip -o route | awk "/$HWLINK/"' && ! /default/ {print $1}'
So, the main question is if any of these (or both) is vulnerable.
A secondary question would be which form is preferred. I tried ip -o route | awk -v hwlink="$HWLINK" '/hwlink/ && ! /default/ {print $1}' but that doesn't work.
p.s. this is a refactoring; the original command was ip -o route | grep $HWLINK | grep -v default | awk '{print $1}'.
Sure, both are vulnerable, the first a bit less so.
This breaks your second line:
HWLINK="/{}BEGIN{print \"Your mother was a hamster and your father smelt of elderberries\"}/"
The only reason it doesn't break your first line is, in order to be able to be injected into the first line it must not contain spaces.
HWLINK="/{}BEGIN{print\"Your_mother_was_a_hamster_and_your_father_smelt_of_elderberries\"}/"
I see you already got the correct syntax to use :)
Your idea was right about letting the shell variables getting interpolated inside awk could let malicious code injection. As rightly pointed use the -v syntax, but your attempt fails because the pattern match with variable doesn't work in the form /../, use the direct ~ match
ip -o route | awk -v hwlink="$HWLINK" '$0 ~ hwlink && ! /default/ {print $1}'
Recommended way to sanitize your variables passed to awk would be to use the ARGV array or ENVIRON variable. Variables passed this way don't undergo expansion done by the shell
value='foo\n\n'
awk 'BEGIN {var=ARGV[1]; delete ARGV[1]}' "$value"
If you printed the value of var inside the awk it would be a literal foo\n\n and not the multi-line string which usually happens when the shell expands it.
This question already has answers here:
How do I set a variable to the output of a command in Bash?
(15 answers)
Closed 6 years ago.
I am not sure how to save the output of a command via bash into a variable:
PID = 'ps -ef | grep -v color=auto | grep raspivid | awk '{print $2}''
Do I have to use a special character for the apostrophe or for the pipes?
Thanks!
To capture the output of a command in shell, use command substitution: $(...). Thus:
pid=$(ps -ef | grep -v color=auto | grep raspivid | awk '{print $2}')
Notes
When making an assignment in shell, there must be no spaces around the equal sign.
When defining shell variables for local use, it is best practice to use lower case or mixed case. Variables that are important to the system are defined in upper case and you don't want to accidentally overwrite one of them.
Simplification
If the goal is to get the PID of the raspivid process, then the grep and awk can be combined into a single process:
pid=$(ps -ef | awk '/[r]aspivid/{print $2}')
Note the simple trick that excludes the current process from the output: instead of searching for raspivid we search for [r]aspivid. The string [r]aspivid does not match the regular expression [r]aspivid. Hence the current process is removed from the output.
The Flexibility of awk
For the purpose of showing how awk can replace multiple calls to grep, consider this scenario: suppose that we want to find lines that contain raspivid but that do not contain color=auto. With awk, both conditions can be combined logically:
pid=$(ps -ef | awk '/raspivid/ && !/color=auto/{print $2}')
Here, /raspivid/ requires a match with raspivid. The && symbol means logical "and". The ! before the regex /color=auto/ means logical "not". Thus, /raspivid/ && !/color=auto/ matches only on lines that contain raspivid but not color=auto.
A more straightforward approach:
pid=$(pgrep raspivid)
... or a little different
echo pgrep [t]eleport
I have a small bash script that greps/awk paragraph by using a keyword.
But after adding in the extra codes : set var = "(......)" it only prints a blank line and not the paragraph.
So I would like to ask if anyone knows how to properly pass the awk output into a variable for outputting?
My codes:
#!/bin/sh
set var = "(awk 'BEGIN{RS=ORS="\n\n";FS=OFS="\n"}/FileHeader/' /root/Desktop
/logs/Default.log)"
echo $var;
Thanks!
Use command substitution to capture the output of a process.
#!/bin/sh
VAR="$(awk 'BEGIN{RS=ORS="\n\n";FS=OFS="\n"}/FileHeader/' /root/Desktop/logs/Default.log)"
echo "$VAR"
some general advice with regards to shell scripting:
(almost) always quote every variable reference.
never put spaces around the equals sign in variable assignment.
You need to use "command substitution". Place the command inside either backticks, `COMMAND` or, in a pair of parentheses preceded by a dollar sign, $(COMMAND).
To set a variable you don't use set and you can't have spaces before and after the =.
Try this:
var=$(awk 'BEGIN{RS=ORS="\n\n";FS=OFS="\n"}/FileHeader/' /root/Desktop/logs/Default.log)
echo $var
You gave me the idea of this for killing a process :). Just chromium to whatever process you wanna kill.
Try this:
VAR=$(ps -ef | grep -i chromium | awk '{print $2}'); kill -9 $VAR 2>/dev/null; unset VAR;
anytime you see grep piped to awk, you can drop the grep. for the above,
awk '/^password/ {print $2}'
awk can easily replace any text command like cut, tail, wc, tr etc. and especally multiple greps piped next to each other. i.e
grep some_co.mand | a | grep b ... to | awk '/a|b|and so on/ {some action}.
Try to create a variable coming from vault/Hashicorp, when using packer template variables, like so:
BUILD_PASSWORD=$(vault read secret/buildAccount| grep ^password | awk '{print $2}')
echo $BUILD_PASSWORD
You can to the same with grep ^user
I want to do this:
run a command
capture the output
select a line
select a column of that line
Just as an example, let's say I want to get the command name from a $PID (please note this is just an example, I'm not suggesting this is the easiest way to get a command name from a process id - my real problem is with another command whose output format I can't control).
If I run ps I get:
PID TTY TIME CMD
11383 pts/1 00:00:00 bash
11771 pts/1 00:00:00 ps
Now I do ps | egrep 11383 and get
11383 pts/1 00:00:00 bash
Next step: ps | egrep 11383 | cut -d" " -f 4. Output is:
<absolutely nothing/>
The problem is that cut cuts the output by single spaces, and as ps adds some spaces between the 2nd and 3rd columns to keep some resemblance of a table, cut picks an empty string. Of course, I could use cut to select the 7th and not the 4th field, but how can I know, specially when the output is variable and unknown on beforehand.
One easy way is to add a pass of tr to squeeze any repeated field separators out:
$ ps | egrep 11383 | tr -s ' ' | cut -d ' ' -f 4
I think the simplest way is to use awk. Example:
$ echo "11383 pts/1 00:00:00 bash" | awk '{ print $4; }'
bash
Please note that the tr -s ' ' option will not remove any single leading spaces. If your column is right-aligned (as with ps pid)...
$ ps h -o pid,user -C ssh,sshd | tr -s " "
1543 root
19645 root
19731 root
Then cutting will result in a blank line for some of those fields if it is the first column:
$ <previous command> | cut -d ' ' -f1
19645
19731
Unless you precede it with a space, obviously
$ <command> | sed -e "s/.*/ &/" | tr -s " "
Now, for this particular case of pid numbers (not names), there is a function called pgrep:
$ pgrep ssh
Shell functions
However, in general it is actually still possible to use shell functions in a concise manner, because there is a neat thing about the read command:
$ <command> | while read a b; do echo $a; done
The first parameter to read, a, selects the first column, and if there is more, everything else will be put in b. As a result, you never need more variables than the number of your column +1.
So,
while read a b c d; do echo $c; done
will then output the 3rd column. As indicated in my comment...
A piped read will be executed in an environment that does not pass variables to the calling script.
out=$(ps whatever | { read a b c d; echo $c; })
arr=($(ps whatever | { read a b c d; echo $c $b; }))
echo ${arr[1]} # will output 'b'`
The Array Solution
So we then end up with the answer by #frayser which is to use the shell variable IFS which defaults to a space, to split the string into an array. It only works in Bash though. Dash and Ash do not support it. I have had a really hard time splitting a string into components in a Busybox thing. It is easy enough to get a single component (e.g. using awk) and then to repeat that for every parameter you need. But then you end up repeatedly calling awk on the same line, or repeatedly using a read block with echo on the same line. Which is not efficient or pretty. So you end up splitting using ${name%% *} and so on. Makes you yearn for some Python skills because in fact shell scripting is not a lot of fun anymore if half or more of the features you are accustomed to, are gone. But you can assume that even python would not be installed on such a system, and it wasn't ;-).
try
ps |&
while read -p first second third fourth etc ; do
if [[ $first == '11383' ]]
then
echo got: $fourth
fi
done
Your command
ps | egrep 11383 | cut -d" " -f 4
misses a tr -s to squeeze spaces, as unwind explains in his answer.
However, you maybe want to use awk, since it handles all of these actions in a single command:
ps | awk '/11383/ {print $4}'
This prints the 4th column in those lines containing 11383. If you want this to match 11383 if it appears in the beginning of the line, then you can say ps | awk '/^11383/ {print $4}'.
Using array variables
set $(ps | egrep "^11383 "); echo $4
or
A=( $(ps | egrep "^11383 ") ) ; echo ${A[3]}
Similar to brianegge's awk solution, here is the Perl equivalent:
ps | egrep 11383 | perl -lane 'print $F[3]'
-a enables autosplit mode, which populates the #F array with the column data.
Use -F, if your data is comma-delimited, rather than space-delimited.
Field 3 is printed since Perl starts counting from 0 rather than 1
Getting the correct line (example for line no. 6) is done with head and tail and the correct word (word no. 4) can be captured with awk:
command|head -n 6|tail -n 1|awk '{print $4}'
Instead of doing all these greps and stuff, I'd advise you to use ps capabilities of changing output format.
ps -o cmd= -p 12345
You get the cmmand line of a process with the pid specified and nothing else.
This is POSIX-conformant and may be thus considered portable.
Bash's set will parse all output into position parameters.
For instance, with set $(free -h) command, echo $7 will show "Mem:"