I created the following bash pipeline that will take the output of "who" and modify it to meet an assignment's requirements
This is the pipline:
who | sed -e "s/\b\(.\)/\u\1/g" | sed 's/[.]/ /g' | sed 's/ Pts\// TTY /g' | sed '1d' | sed -n 's/ .*$/ /gp'
After putting this into a sed file that looks like this:
s/\b\(.\)/\u\1/g
s/[.]/ /g
s/ Pts\// TTY /g
1d
s/ .*$/ /gp
And then running it like such:
who | sed -f sedfile
The output is correct in that everything is in the format of:
firstName lastName TTY (a number)
However each line is printed twice, where the pipeline properly printed each line once
Would anyone happen to know the issue please?
It's the gp at the last line. You're not running with sed -n (no-print), so by default you're running with sed -yesprint (or whatever). Then you hit that gp, which prints, and you get two copies.
Convert to sed -n, or change the gp to just g. Or better still, get rid of it, since the match pattern contains $, so it will only ever run in one place - the end of the line.
Related
I want sed to give me a single line output irrespective of whether the matched pattern is found and substituted, or even if there is no pattern match, with same command options.
1. echo "700K" | sed -n 's/[A-Z]//gp' // gives one output
2. echo "700" | sed -n 's/[A-Z]//gp' // no output
Is there any way in sed i can get a single output for second case without removing the "-n" option, forcing it to print the input irrespective of substitution made or not?
It is not clear for me why you need to keep the -n option but if you really do need to keep it you can use the following sed command:
echo "700" | sed -n 's/[A-Z]//g;p'
this will first make the substitution if possible then print the line.
output:
You don't need to mess with all these sed options. Use sed in it's simpliest format which will make a substitution if pattern is found:
$ echo "700K" | sed 's/[A-Z]//g'
700
$ echo "700" | sed 's/[A-Z]//g'
700
$ sed --version
sed (GNU sed) 4.4
$ sed 's/[A-Z]//g' <<<$'700\n700K\n500\n3500A'
700
700
500
3500
I am trying to use sed to insert sed command at the end of each line in a file using -
sed -r 's/$/ | sed -r s\/abc\/xyz\/ /' filename.extension
What I want next is to have single quotes around the inner sed. So that it will look something like-
sed -r 's/$/ | sed -r 's\/abc\/xyz\/' /' filename.extension
I tried escaping the inner single quotes, but no use.
Basically, I want the following lines -
line 1
line 2
line 3
to turn into-
line 1 | sed -r 's/abc/xyz/'
line 2 | sed -r 's/abc/xyz/'
line 3 | sed -r 's/abc/xyz/'
I am unable to get the single quotes, even with the escape characters.
sed -e "s:$: | sed -r 's/abc/xyz/':" yourfile
Your problem is an example of the general case of nesting shell expressions. There are a number of ways to do this.
Use alternate delimiters. That's what I did here.
Assign subexpressions to variables, and then expand them.
Use lots of \ escapes.
Put your subexpression in a file and read it.
Use alternative delimiter in inner sed and double quote in outer sed to simplify your command:
sed "s/$/ | sed -r 's~abc~xyz~'/" file.ext
btw -r is not really needed in inner sed
This might work for you (GNU sed):
sed 's/$/ | sed -r '\''s\/abc\/xyz\/'\''/' file
Use '\'' to end the current single quote, then in the shell use \' to quote a single quote and finally a single quote to start the quoting of the sed command. Use \/ to quote the forward slash in the sed command.
As the substitution command can use any delimiter:
sed 's#$# | sed -r '\''s/abc/xyz/'\''#' file
reduces the amount of quoting and:
sed "s#$# | sed -r 's/abc/xyz/'#" file
reduces it further. However double quoting sed commands (or any utility) can have unwanted side effects i.e. metacharacters can be evaluated by the shell, so it is best to single quote and live with the "hole-like" mechanism '\''.
Say for instance I'm searching a line that is like this:
Color asdf
and I use grep to find that line, like grep asdf file.txt
How would I then display Color? Learning linux is hard.
With the command line tool sed you can replace stings by using regular expressions:
echo "Color asdf" | sed 's/\([^ ]*\).*/\1/'
This part: \([^ ]*\).* is a regular expresion. The first part of the regex: [^ ]*, matches any character except a space as many times as possible and what's between the \( and \) is being captured in the variable \1. Then you also match the remaining part of the string with .* and replace all of that with only the first word which was captured by \([^ ]*\) by using \1 in the replace part of the sed command.
Here some more info about sed:
http://linux.about.com/od/commands/a/Example-Uses-Of-Sed-Cmdsedxa.htm
You could use sed:
sed -n 's/[[:space:]][[:space:]]*asdf$//p' file.txt
Details:
The -n option tells sed not to print the pattern space automatically. Basically, it doesn't output anything unless you tell it to.
The s command of sed replaces text. Here, if a line ends with asdf, preceded by at least one whitespace character, we replace all of that with nothing and then print the line (notice the p flag at the end of the s command). The printing is only done if something was actually replaced. More information about the s command can be found e. g. in the GNU sed manual.
Edit for clarity: When using single quotes, parameter expansion does not work and thus, variables won't be replaced. To use variables, use double quotes:
search=asdf
sed -n "s/[[:space:]][[:space:]]*${search}\$//p" file.txt
If you'd really like to use grep here, you could pipe the output from grep into cut:
grep -h asdf *.txt | cut -s -d -f 1
Note that there have to be two spaces after the -d option to cut - the first tells cut to use a blank as the field delimiter (I'm assuming your fields are blank-delimited rather than tab-delimited), while the second separates the -d option from the following option (-f).
But, yeah, sed or awk are probably your friends here... :-)
you can color pattern in the line using grep
grep --colour -o 'asdf' file.txt
edit: the -o option will print only the patterns
How to use sed to find lines with the word linux? As later display a first line 10 with the word linux?
EX.:
cat file | sed -e '/linux/!d' -e '10!d' ### I can not display the first 10 lines of the word linux
cat file | sed '/linux/!d' | sed '10!d' ### It is well
How to make it work with one sed?
cat file | sed -e '/linux/!d; ...?; 10!d'
...? - storing of the buffer linux? 10 later cut the lines?
Someone explain to me?
I would use awk:
awk '/linux/ && c<10 {print;c++} c==10 {exit}' file
This might work for you (GNU sed):
sed -nr '/linux/{p;G;/(.*\n){10}/q;h}' file
Print the line if it contains the required string. If the required number of lines has already been printed quit, otherwise store the line and previous lines in the hold space.
You could use perl:
perl -ne 'if (/linux/) {print; ++$a;}; last if $a==10' inputfile
Using GNU sed:
sed -rn "/linux/{p;x;s/^/P/;ta;:a;s/^P{10}$//;x;Tb;Q;:b}" filename
Thanks. You are great. All of the examples look very nice. Wooow :) It is a pity that I can not do that.
I have not seen for 'r' option in sed. I need to learn.
echo -e 'windows\nlinux\nwindows\nlinux\nlinux\nwindows' | sed -nr '/linux/{p;G;/(.*\n){2}/q;h}'
It works very well.
echo -e 'windows\nlinux\nwindows\nlinux\nlinux\nwindows' | sed -nr '/linux/{p;G;/(.*\n){2}/q;h}' | sed '2s/linux/debian/'
Can I ask you one more example? How to get a result at one sed?
I need some assistance trying to build up a variable using a list of exclusions in a file.
So I have a exclude file I am using for rsync that looks like this:
*.log
*.out
*.csv
logs
shared
tracing
jdk*
8.6_Code
rpsupport
dbarchive
inarchive
comms
PR116PICL
**/lost+found*/
dlxwhsr*
regression
tmp
working
investigation
Investigation
dcsserver_weblogic_
dcswebrdtEAR_weblogic_
I need to build up a string to be used as a variable to feed into egrep -v, so that I can use the same exclusion list for rsync as I do when egrep -v from a find -ls.
So I have created this so far to remove all "*" and "/" - and then when it sees certain special characters it escapes them:
cat exclude-list.supt | while read line
do
echo $line | sed 's/\*//g' | sed 's/\///g' | 's/\([.-+_]\)/\\\1/g'
What I need the ouput too look like is this and then export that as a variable:
SEXCLUDE_supt="\.log|\.out|\.csv|logs|shared|PR116PICL|tracing|lost\+found|jdk|8\.6\_Code|rpsupport|dbarchive|inarchive|comms|dlxwhsr|regression|tmp|working|investigation|Investigation|dcsserver\_weblogic\_|dcswebrdtEAR\_weblogic\_"
Can anyone help?
A few issues with the following:
cat exclude-list.supt | while read line
do
echo $line | sed 's/\*//g' | sed 's/\///g' | 's/\([.-+_]\)/\\\1/g'
Sed reads files line by line so cat | while read line;do echo $line | sed is completely redundant also sed can do multiple substitutions by either passing them as a comma separated list or using the -e option so piping to sed three times is two too many. A problem with '[.-+_]' is the - is between . and + so it's interpreted as a range .-+ when using - inside a character class put it at the end beginning or end to lose this meaning like [._+-].
A much better way:
$ sed -e 's/[*/]//g' -e 's/\([._+-]\)/\\\1/g' file
\.log
\.out
\.csv
logs
shared
tracing
jdk
8\.6\_Code
rpsupport
dbarchive
inarchive
comms
PR116PICL
lost\+found
dlxwhsr
regression
tmp
working
investigation
Investigation
dcsserver\_weblogic\_
dcswebrdtEAR\_weblogic\_
Now we can pipe through tr '\n' '|' to replace the newlines with pipes for the alternation ready for egrep:
$ sed -e 's/[*/]//g' -e 's/\([._+-]\)/\\\1/g' file | tr "\n" "|"
\.log|\.out|\.csv|logs|shared|tracing|jdk|8\.6\_Code|rpsupport|dbarchive|...
$ EXCLUDE=$(sed -e 's/[*/]//g' -e 's/\([._+-]\)/\\\1/g' file | tr "\n" "|")
$ echo $EXCLUDE
\.log|\.out|\.csv|logs|shared|tracing|jdk|8\.6\_Code|rpsupport|dbarchive|...
Note: If your file ends with a newline character you will want to remove the final trailing |, try sed 's/\(.*\)|/\1/'.
This might work for you (GNU sed):
SEXCLUDE_supt=$(sed '1h;1!H;$!d;g;s/[*\/]//g;s/\([.-+_]\)/\\\1/g;s/\n/|/g' file)
This should work but I guess there are better solutions. First store everything in a bash array:
SEXCLUDE_supt=$( sed -e 's/\*//g' -e 's/\///g' -e 's/\([.-+_]\)/\\\1/g' exclude-list.supt)
and then process it again to substitute white space:
SEXCLUDE_supt=$(echo $SEXCLUDE_supt |sed 's/\s/|/g')