Need less to say after reading this, I am a newie. The code:
#!/bin/bash
IBL=`blkid |grep "deviceLabel" | awk '{print $1}'`
BSHOW="echo ${IBL::-1}"
seems to work; however, when I do this:
SP_CHECK=`df -h | grep ${BSHOW}`
echo "this is SP_CHECK= ${SP_CHECK}
sp shows nothing. Any idea why? any pointers to what topic I can look at to learn better?
Related
I'm trying to create an alias for getting memory on my machine, currently I have alias mem="lshw | grep size | awk -F: '{print $2}'", and when I run it as a non-super user, I get the following warning message:
WARNING: you should run this program as super-user.
WARNING: output may be incomplete or inaccurate, you should run this program as super-user.
size: 23GiB
I'm not worried about the results being potentially incomplete, in fact when I diff the output when running as root vs a standard user, it's exactly the same. Does anybody know how to get rid of these warnings? I tried piping stderr to /dev/null, but that didn't work. Does anyone else know how to get rid of these warnings?
Can I interest you in
alias mem='free -g | grep Mem | awk '\''{print $2 " GiB"}'\'
free -m will give MiB; you can change the " GiB" part to whatever you want (or remove it).
I don't have lshw installed on my machine, so I can't help you debug your version, unfortunately.
alias mem="lshw 2> /dev/null| grep size | awk -F: '{print $2}'"
Alternatively you can use free or read from /proc/meminfo
cat /proc/meminfo |grep MemTotal
I'm not sure how you piped to dev/null, but this works for me:
lshw 2> /dev/null | grep size | awk -F: '{print $2}'
Ignoring that there are other tools more suited to getting the memory, if there is something you need and lshw is your only option, you would be better suited to use -json or -xml output and use a tool to parse it like jq or xmllint. The version of lshw on my distro outputs invalid json that can't be parsed, but does have valid xml output.
This would accomplish your goal, although the path may very well be different for you:
lshw -xml 2> /dev/null | xmllint --xpath '/list/node/node/node[#id="memory"]/size/text()' -
Or add a one grep:
... | grep "size:"
I have a problem with one script in bash with CAT command.
This works:
#!/bin/bash
fil="| grep LSmonitor";
log="/var/log/sys.log ";
lines=`cat $log | grep LSmonitor | wc -l`;
echo $lines;
Output: 139
This does not:
#!/bin/bash
fil="| grep LSmonitor";
log="/var/log/sys.log ";
string="cat $log $fil | wc -l";
echo $string;
`$string`;
Output:
cat /var/log/sys.log | grep LSmonitor | wc -l
cat: opcion invalida -- 'l'
Pruebe 'cat --help' para mas informacion.
$fil is a parameter in this example static, but in real script, parameter is get from html form POST, and if I print I can see that the content of $fil is correct.
In this case, since you're building a pipeline as a string, you would need:
eval "$string"
But DON'T DO THIS!!!! -- someone can easily enter the filter
; rm -rf *
and then you're hosed.
If you want a regex-based filter, get the user to just enter the regex, and then you'll do:
grep "$fil" "$log" | wc -l
Firstly, allow me to say that this sounds like a really bad idea:
[…] in real script, parameter is get from html form POST, […]
You should not be allowing the content of POST requests to be run by your shell. This is a massive attack vector, and whatever mechanisms you have in place to try to protect it are probably not as effective as you think.
Secondly, | inside variables are not treated as special. This isn't specific to backticks. Parameter expansion (e.g., replacing $fil with | grep LSmonitor) happens after the command is parsed and mostly processed. There's a little bit of post-processing that's done on the results of parameter expansion (including "word splitting", which is why $fil is equivalent to the three arguments '|' grep LSmonitor rather than to the single argument '| grep LSmonitor'), but nothing as dramatic as you describe. So, for example, this:
pipe='|'
echo $pipe cat
prints this:
| cat
Since your use-case is so frightening, I'm half-tempted to not explain how you can do what you want — I think you'll be better off not doing this — but since Stack Overflow answers are intended to be useful for more people than just the original poster, an example of how you can do this is below. I encourage the OP not to read on.
fil='| grep LSmonitor'
log=/var/log/sys.log
string="cat $log $fil | wc -l"
lines="$(eval "$string")"
echo "$lines"
Try using eval (taken from https://stackoverflow.com/a/11531431/2687324).
It looks like it's interpreting | as a string, not a pipe, so when it reaches -l, it treats it as if you're trying to pass in -l to cat instead of wc.
The other answers outline why you shouldn't do it this way.
grep LSmonitor /var/log/syslog | wc -l will do what you're looking for.
What is the construct in bash called where you can take wrap a command that outputs to stdout, such that the output itself is treated like a stream? In case I'm not describing that so well, maybe an example will do best, and this is what I typically use it for: applying diff to output that does not come from a file, but from other commands, where
cmd
is wrapped as
<(cmd)
By wrapping a command in such a manner, in the example below I determine that there a difference of one between the two commands that I am running, and then I am able to determine that one precise difference. What is the construct/technique of wrapping a command as <(cmd) called? Thanks
[builder#george v6.5 html]$ git status | egrep modified | awk '{print $3}' | wc -l
51
[builder#george v6.5 html]$ git status | egrep modified | awk '{print $3}' | xargs grep -l 'Ext\.define' | wc -l
50
[builder#george v6.5 html]$ diff <(git status | egrep modified | awk '{print $3}') <(git status | egrep modified | awk '{print $3}' | xargs grep -l 'Ext\.define')
39d38
< javascript/reports/report_initiator.js
ADDENDUM
The revised command using the advice for using git's ls-file should be as follows (untested):
diff <(git ls-files -m) <(git ls-files -m | xargs grep -l 'Ext\.define')
It is called process substitution.
This is called Process Substitution
This is process substitution, as you have been told. I'd just like to point out that this also works in the other direction. Process substitution with >(cmd) allows you to take a command that writes to a file and instead have that output redirected to another command's stdin. It's very useful for inserting something into a pipeline that takes an output filename as an argument. You don't see it as much because pretty much every standard command will write to stdout already, but I have used it often with custom stuff. Here is a contrived example:
$ echo "hello world" | tee >(wc)
hello world
1 2 12
I'm trying to tail apache error logs through a few filters.
This works perfectly:
tail -fn0 /var/log/apache2/error.log | egrep -v "PHP Notice|File does not exist"
but there are some literal "\n" in the output which I want to replace with an actual new line so I pipe into perl:
tail -fn0 /var/log/apache2/error.log | egrep -v "PHP Notice|File does not exist" | perl -ne 's/\\n/\n/g; print"$_"'
This seems to have some caching issue (first page hit produces nothing, second page hit and two loads of debugging info comes out), It also seems a bit tempramental.
So I tried sed:
tail -fn0 /var/log/apache2/error.log | egrep -v "PHP Notice|File does not exist" | sed 's/\\n/\n/g'
which seems to suffer the same problem.
Correct, when you use most programs to file or pipe they buffer output. You can control this in some cases: the GNU grep family accepts the --line-buffered option, specifically for use in pipelines like this. Also, in Perl you can use $| = 1; for the same effect. (sed doesn't have any such option that I'm aware of.)
It's the stuff at the beginning or middle of the pipeline that will be buffering, not the end (which is talking to your terminal so it will be line buffered) so you want to use egrep --line-buffered.
Looks like you can use -u for sed as in:
tail -f myLog | sed -u "s/\(joelog\)/^[[46;1m\1^[[0m/g" | sed -u 's/\\n/\n/g'
which tails the log, highlights 'joelog', and then adds linebreaks where there are '\n'
source:
http://www-01.ibm.com/support/docview.wss?uid=isg1IZ42070
Basically I'm wondering why this doesn't output anything:
tail --follow=name file.txt | grep something | grep something_else
You can assume that it should produce output I have run another line to confirm
cat file.txt | grep something | grep something_else
It seems like you can't pipe the output of tail more than once!? Anyone know what the deal is and is there a solution?
EDIT:
To answer the questions so far, the file definitely has contents that should be displayed by the grep. As evidence if the grep is done like so:
tail --follow=name file.txt | grep something
Output shows up correctly, but if this is used instead:
tail --follow=name file.txt | grep something | grep something
No output is shown.
If at all helpful I am running ubuntu 10.04
You might also run into a problem with grep buffering when inside a pipe.
ie, you don't see the output from
tail --follow=name file.txt | grep something > output.txt
since grep will buffer its own output.
Use the --line-buffered switch for grep to work around this:
tail --follow=name file.txt | grep --line-buffered something > output.txt
This is useful if you want to get the results of the follow into the output.txt file as rapidly as possible.
Figured out what was going on here. It turns out that the command is working it's just that the output takes a long time to reach the console (approx 120 seconds in my case). This is because the buffer on the standard out is not written each line but rather each block. So instead of getting every line from the file as it was being written I would get a giant block every 2 minutes or so.
It should be noted that this works correctly:
tail file.txt | grep something | grep something
It is the following of the file with --follow=name that is problematic.
For my purposes I found a way around it, what I was intending to do was capture the output of the first grep to a file, so the command would be:
tail --follow=name file.txt | grep something > output.txt
A way around this is to use the script command like so:
script -c 'tail --follow=name file.txt | grep something' output.txt
Script captures the output of the command and writes it to file, thus avoiding the second pipe.
This has effectively worked around the issue for me, and I have explained why the command wasn't working as I expected, problem solved.
FYI, These other stackoverflow questions are related:
Trick an application into thinking its stdin is interactive, not a pipe
Force another program's standard output to be unbuffered using Python
You do know that tail starts by default with the last ten lines of the file? My guess is everything the cat version found is well into the past. Try tail -n+1 --follow=name file.txt to start from the beginning of the file.
works for me on Mac without --follow=name
bash-3.2$ tail delme.txt | grep po
position.bin
position.lrn
bash-3.2$ tail delme.txt | grep po | grep lr
position.lrn
grep pattern filename | grep pattern | grep pattern | grep pattern ......