I have a script that we use on Ubuntu (Linux) and I'd like to convert it to be used on both Ubuntu (Linux) and MacOS X. grep on Linux is different than grep on FreeBSD (i.e. MacOS X); grep on MacOS X doesn't support the -P option. Unfortunately, using the -E option on both platforms doesn't give the same results. Consider the following code that works on Linux:
wip_scenarios=$(grep -oP "^\d+ scenarios?" log/report.log | grep -oP "\d+")
echo "\n"
echo $wip_scenarios
This returns a 0 on Linux. Replacing all the -P with -E makes this work on MacOS X, but on Linux, this just returns a null which doesn't help the rest of my script when I use conditionals like this:
if [ $wip_scenarios != 0 ];then
One solution is to put a flag at the front and use the appropriate option set depending on the platform, but I was hoping for a cross-platform solution. Is there a way to do this?
For the regex you gave here, this is simple: Change \d to [[:digit:]].
Thus:
wip_scenarios=$(grep -Eo '^[[:digit:]]+ scenarios[?]' <report.log | grep -Eo '[[:digit:]]+')
If your script starts with #!/bin/bash (and thus will only ever be run with bash), I'd also consider skipping the dependency on the non-standard extension grep -o, and instead depending on bash itself to separate out the numbers you care about:
# This works with any POSIX-compliant grep, but requires that the shell be bash
wip_scenarios_re='([[:digit:]]+) scenarios[?]'
wip_scenarios_line=$(grep -E '^[[:digit:]]+ scenarios[?]' <report.log)
[[ $wip_scenarios_line =~ $wip_scenarios_re ]] && {
wip_scenarios=${BASH_REMATCH[1]}
}
How can I check latest kernel version by bash?
Is there any command to check latest kernel from https://www.kernel.org/ ?
If you are looking for the latest kernel version on the website and not the one on your system, you can use this command. It will work fine unless they change their page layout later. If they do, in that case, you will have to tweak your command:
[root#slave2 gc]# curl -s https://www.kernel.org/ | grep -A1 'mainline:' | grep -oP '(?<=strong>).*(?=</strong.*)'
3.16-rc7
It will return you the 'mainline' release. You can search for 'stable' release using the same logic.
Explanation:
-o Option to print only what matches the pattern.
-P Interpret the pattern as a Perl regular expression.
(?=pattern) A zero-width positive look-ahead assertion. To put it in simple words using an example, q(?=u) matches a q that is followed by a u.
(?<=pattern) A zero-width positive look-behind assertion. To put it in simple words using an example, (?<=a)b matches the b (and only the b) in cab, but does not match bed or debt
So, whatever pattern is matched is actually removed from the output and that's how we get the result. :)
You can refer these links for more detail:
http://perldoc.perl.org/perlre.html#Extended-Patterns
http://www.regular-expressions.info/lookaround.html
worked since 2017
curl -s https://www.kernel.org | grep -A1 latest_link | tail -n1 | egrep -o '>[^<]+' | egrep -o '[^>]+'
I am using a bison parser in my project. When I run the following command:
sed -i y.tab.c -e "s/ __attribute__ ((__unused__))$/# ifndef __cplusplus\n __attribute__ ((__unused__));\n# endif/"
I get this error
sed: -i may not be used with stdin
The command works fine in linux machines. I am using Mac OS X 10.9. It throws an error only on mac os x. I am not sure why. Can anyone help?
Thanks
The problem is that Mac OS X uses the BSD version of sed, which treats the -i option slightly differently. The GNU version used in Linux takes an optional argument with -i: if present, sed makes a backup file whose name consists of the input file plus the argument. Without an argument, sed simply modifies the input file without saving a backup of the original.
In BSD sed, the argument to -i is required. To avoid making a backup, you need to provide a zero-length argument, e.g. sed -i '' y.tab.c ....
Your command, which simply edits y.tab.c with no backup in Linux, would attempt to save a backup file using 'y.tab.c' as an extension. But now, with no other file in the command line, sed thinks you want to edit standard input in-place, something that is not allowed.
From the sed manpage:
-i extension
Edit files in-place, saving backups with the specified extension.
If a zero-length extension is given, no backup will be saved. It
is not recommended to give a zero-length extension when in-place
editing files, as you risk corruption or partial content in situ-
ations where disk space is exhausted, etc.
The solution is to send a zero-length extension like this:
sed -i '' 's/apples/oranges/' file.txt
You need to put the input file as the last parameter.
sed -i -e "s/ __attribute__ ((__unused__))$/# ifndef __cplusplus\n __attribute__ ((__unused__));\n# endif/" y.tab.c
Piggy-backing off of #chepner's explanation for a quick-and-dirty solution:
Install the version of sed that'll get the job done with brew install gnu-sed, then replace usages of sed in your script with gsed.
(The homebrew community is fairly cognizant of issues that can arise of OS X built-ins are overridden unexpectedly and has worked to not do that for most alternate-distro commands.)
I'm trying to come up with a way to find a specific flag in a man-page. Usually, I type '/'
to search for something, followed by something like '-Werror' to find a specific flag.
The thing is though that there are man-pages (gcc is the one motivating me right now) that
have a LOT of references to flags in their text, so there are a lot of occurrences.
It's not that big of a deal, but maybe it can be done a bit better. I thought of looking for
something like '-O\n' but it didn't work (probably because the man program doesn't use C escapes?)
Then I've tried something like man gcc | grep $'-O\n', since I kind of recall that a
single-quoted string preceded by a dollar sign haves bash interpret common C escapes...
It' didn't work, grep echoed the whole man-page.
That's what has brought me here: why? or rather, can this be done?
rici's helpful answer explains the problem with the original approach well.
However, there's another thing worth mentioning:
man's output contains formatting control characters, which interfere with text searches.
If you pipe to col -b before searching, these control characters are removed - note the side effect that the search results will be plain-text too.
However, grep is not the right tool for this job; I suggest using awk as follows to obtain the description of -O:
man gcc | col -b | awk -v RS= '/^\s+-O\n/'
RS= (an empty input-record separator) is an awk idiom that breaks the input into blocks of non-empty lines, so matching the option at the start of such a block ensures that all lines comprising the description of the option are returned.
If you have a POSIX-features-only awk such as BSD/OSX awk, use this version:
man gcc | col -b | awk -v RS= '/^[[:blank:]]+-O\n/'
Obviously, such a command is somewhat cumbersome to type, so find generic bash function manopt below, which returns the description of the specified option for the specified command from its man page. (There can be false positives and negatives, but overall it works pretty well.)
Examples:
manopt gcc O # search `man gcc` for description of `-O`
manopt grep regexp # search `man grep` for description of `--regexp`
manopt find '-exec.*' # search `man find` for all actions _starting with_ '-exec'
bash function manopt() - place in ~/.bashrc, for instance:
# SYNOPSIS
# manopt command opt
#
# DESCRIPTION
# Returns the portion of COMMAND's man page describing option OPT.
# Note: Result is plain text - formatting is lost.
#
# OPT may be a short option (e.g., -F) or long option (e.g., --fixed-strings);
# specifying the preceding '-' or '--' is OPTIONAL - UNLESS with long option
# names preceded only by *1* '-', such as the actions for the `find` command.
#
# Matching is exact by default; to turn on prefix matching for long options,
# quote the prefix and append '.*', e.g.: `manopt find '-exec.*'` finds
# both '-exec' and 'execdir'.
#
# EXAMPLES
# manopt ls l # same as: manopt ls -l
# manopt sort reverse # same as: manopt sort --reverse
# manopt find -print # MUST prefix with '-' here.
# manopt find '-exec.*' # find options *starting* with '-exec'
manopt() {
local cmd=$1 opt=$2
[[ $opt == -* ]] || { (( ${#opt} == 1 )) && opt="-$opt" || opt="--$opt"; }
man "$cmd" | col -b | awk -v opt="$opt" -v RS= '$0 ~ "(^|,)[[:blank:]]+" opt "([[:punct:][:space:]]|$)"'
}
fish implementation of manopt():
Contributed by Ivan Aracki.
function manopt
set -l cmd $argv[1]
set -l opt $argv[2]
if not echo $opt | grep '^-' >/dev/null
if [ (string length $opt) = 1 ]
set opt "-$opt"
else
set opt "--$opt"
end
end
man "$cmd" | col -b | awk -v opt="$opt" -v RS= '$0 ~ "(^|,)[[:blank:]]+" opt "([[:punct:][:space:]]|$)"'
end
I suspect you didn't actually use grep $'-O\n', but rather some flag recognized by grep.
From grep's point of view, you are simply passing an argument, and that argument starts with a - so it's going to be interpreted as an option. You need to do something like grep -- -O$ to explicitly flag the end of the list of options, or grep -e -O$ to explicitly flag the pattern as a pattern. In any event, you cannot include a newline in a pattern because grep patterns are actually lists of patterns separated by newline characters, so the argument $'foo\n' is actually two patterns, foo and the empty string, and the empty string will match every line.
Perhaps you searched for the flag -e since that takes a pattern as an argument, and giving it a newline as an argument will cause grep to find every line in the whole file.
For most GNU programs, such as gcc, you might find the info interface easier to navigate in, since it includes reference links, tables of contents, and even indices. The info gcc document includes an index of options, which is very useful. In some linux distributions, and somewhat surprisingly since they call themselves GNU/linux distributions, it's necessary to separately install info packages although man files are distributed with the base software. The debian/ubuntu package containing the gcc info files is called gcc-doc, for example. (The use of the -doc suffix to the package name is quite common.)
In the case of gcc you can rapidly find an option using a command like:
info gcc "option index" O
or
info gcc --index-search=funroll-loops
For programs with fewer options, it's usually good enough to use info's -O option:
info -O gawk
The thing is that 'man' uses a pager, commonly 'less', whose man-page states:
/pattern
Search forward in the file for the N-th line containing the pattern.
N defaults to 1. The pattern is a regular expression, as recognized by the
regular expression library supplied by your system. The search starts at the
first line displayed (but see the -a and -j options, which change this).
So one could try and look for '-O$' in a man-page to find a flag that lives alone in it's
own line. Although, it is common for a flag to be followed by text in the very same line,
so this is not guaranteed to work.
The issue with grep and $'-O\n' is still a mystery though.
man gcc | grep "\-"
This works pretty well, as it displays all flags and usually not much more.
Edit: I notice I didn't completely answer your question, but I hope my suggestion can be considered as a nice alternative.
I use folowing:
man some_command | col -b | grep -A5 -- 'your_request'
Examples:
man man | col -b | grep -A5 -- '-K'
man grep | col -b | grep -A5 -- '-e patt'
You can make alias for it.
The manly Python utility is very convenient for getting a quick explanation of all options used in a given command.
Note that it only outputs the first paragraph of the option descriptions.
pip install manly
$ manly blkid /dev/sda -o value -p
blkid - locate/print block device attributes
============================================
-o, --output format
Use the specified output format. Note that the order of vari‐
ables and devices is not fixed. See also option -s. The format
parameter may be:
-p, --probe
Switch to low-level superblock probing mode (bypassing the
cache).
a double dash (--) is used in most bash built-in commands and many other commands to signify the end of command options
https://unix.stackexchange.com/a/11382/204245
Without the double-dash, grep is trying to use whatever flag you are looking for:
$ man curl | grep -c # Looks for this c flag, but can't find one so throws the error below.
usage: grep [-abcDEFGHhIiJLlmnOoqRSsUVvwxZ] [-A num] [-B num] [-C[num]]
[-e pattern] [-f file] [--binary-files=value] [--color=when]
[--context[=num]] [--directories=action] [--label] [--line-buffered]
[--null] [pattern] [file ...]
If you use double-dash to signify the end of input to grep, it works a bit better, but you still end up with every occurrence of the match:
$ man curl | grep -- -c
--cacert <file>
certs file named 'curl-ca-bundle.crt', either in the same direc-
--capath <dir>
curl to make SSL-connections much more efficiently than using
--cacert if the --cacert file contains many CA certificates.
--cert-status
--cert-type <type>
-E, --cert <certificate[:password]>
--ciphers <list of ciphers>
# ...many more matches......
So simply wrap the flag in quotes and throw a space before it to only match the -c flag:
$ man curl | grep -- " -c"
-c, --cookie-jar <filename>
This has driven me insane for years. Hope this helps.
man is based on an environment variable (EDITOR if I'm not mistaking). You can change this from more (the default value) to, e.g., emacs, and then while using man an emacs session gets opened on your system, where you can search and browse as you like.
I need to change a directive in a config file and got it working in Linux but in Solaris, it says command garbled.
Here is the directive
enable-cache passwd yes
I need to simply change the yes to no. How can I do with with sed that will work for Solaris, HPUX and Linux?
Here is the sed command that worked in Linux. Solaris doesn't like the -r
sed -r 's/^([[:space:]]*check-files[[:space:]]+passwd[[:space:]]+)yes([[:space:]]*)$/\1no\2/' inputfile
The end goal is to put this command in a script and run it across the enterprise.
Thanks
Greg
I also posted something similar yesterday which worked for Linux but not for the others.
Solaris has /usr/bin/sed and /usr/xpg4/bin/sed. None of these support an -r option, which option for Linux is to use an extended regex. sed in Solaris does not have any option to set the regex like that. You can use other tools, specifically awk, if you want simpler portability. Or you will have to use two flavors of regex, one with -r and an extended regex, one without -r and a different regex. And you probably want to specify /usr/xpg4/bin/sed on Solaris boxes only:
#!/bin/bash
sun=`expr index Solaris $(uname -a)`
if [ $sun -ne 0 ] ; then
/usr/xpg4/bin/sed [longer regex here ]
else
/usr/bin/sed -r [ extended regex here ]
fi
This is not strictly equivalent as :space: match more characters but I assume only space and tab are to be expected in your file. I'm only using standard shell and sed commands so this should work on Solaris, Linux and HP-UX:
space=$(printf " \t")
sed 's/^\(['"$space"']*check-files['"$space"']+passwd['"$space"']+\)yes\(['"$space"']*\)$/\1no\2/' inputfile
Note that your script doesn't match your sample directive as it expects check-files but is given enable-cache.
The GNU [[:space:]] is usually equivalent to just [ \t]. And you need to escape the parentheses. And + is not supported. So with these replacements, your working sed command becomes:
sed 's/^\([ \t]*check-files[ \t][ \t]*passwd[ \t][ \t]*\)yes\([ \t]*\)$/\1no\2/' inputfile
Further note: The older sed's don't have a -i option for doing in-place changes, so you might first have to copy your target to a temporary file, and apply sed to it, redirecting the output to the target.