I've searched all over stackoverflow (perhaps I just suck at searching) but I cannot find the answer to my problem. I'm trying to insert a word or a string in between two patterns in the same line using sed.
I know how to insert a word AFTER a searched pattern using
sed -e "s/pattern/& new_word/g"
with an ampersand (&).
But this command inserts 'new_word' in every occurrence of searched pattern so I'm trying to specify it so that it inserts 'new_word' only in between two patterns.
For example,
Some words = [want to insert words here];
How do I insert it between "Some words (multiple whitespaces here) =" and ";"?
What is the syntax for this kind of command? Also, what resources do you guys use to learn sed? Many of the sed tutorials that I've searched are very basic and doesn't go into details of usage of different options and flags.
Thank you.
Use capture groups.
sed -e 's/(pattern1)(pattern2)/\1new_word\2/'
\1 is replaced with whatever matched the first pattern, \2 gets whatever matched the second pattern.
Related
I have a large taxonomy file that I need to edit. There is an issue with the file as "Candida" is listed as both Candida and [Candida]. What I want to do is change every case of [Candida] to Candida within the file.
I have tried doing this several ways but never get the output I am after. This is the first few lines of the taxonomy file:
Penicillium;marneffei;NW_002197112.1
Penicillium;marneffei;NW_002197111.1
Penicillium;marneffei;NW_002197110.1
Penicillium;marneffei;NW_002197109.1
Penicillium;marneffei;NW_002197108.1
Using sed gives me this output:
$ sed -i -e 's/[Candida]/Candida/g' Full_HMS_Taxonomy.txt
PeCandidaCandidacCandidallCandidaum;mCandidarCandidaeffeCandida;NW_002197112.1
PeCandidaCandidacCandidallCandidaum;mCandidarCandidaeffeCandida;NW_002197111.1
PeCandidaCandidacCandidallCandidaum;mCandidarCandidaeffeCandida;NW_002197110.1
PeCandidaCandidacCandidallCandidaum;mCandidarCandidaeffeCandida;NW_002197109.1
PeCandidaCandidacCandidallCandidaum;mCandidarCandidaeffeCandida;NW_002197108.1
Using awk gives me this output:
$ awk '{gsub(/[Candida]/,"Candida")}1' Full_HMS_Taxonomy.txt
PeCandidaCandidacCandidallCandidaum;mCandidarCandidaeffeCandida;NW_002197112.1
PeCandidaCandidacCandidallCandidaum;mCandidarCandidaeffeCandida;NW_002197111.1
PeCandidaCandidacCandidallCandidaum;mCandidarCandidaeffeCandida;NW_002197110.1
PeCandidaCandidacCandidallCandidaum;mCandidarCandidaeffeCandida;NW_002197109.1
PeCandidaCandidacCandidallCandidaum;mCandidarCandidaeffeCandida;NW_002197108.1
In both cases it is adding Candida to multiple places and multiple lines, instead of just replacing each instance of [Candida]. Any ideas on what I am doing wrong?
[] are special characters in regexp, so you should escape them like that:
's/\[Candida\]/Candida/g'
Brackets are treated specially by regular expression parsers, matching each character listed inside them. So, [Candida] matches any of the characters inside it (C, a, n...). That's why you get a lot of substitutions.
You need to tell those utilities that you want literal brackets by escaping them with backslashes, e.g. with sed:
sed -i 's/\[Candida\]/Candida/g' Full_HMS_Taxonomy.txt
I am trying to replace string between two strings in a file with the command below. There could be any number of such patterns in the file. This is just an example.
sed 's/word1.*word2/word1/' 1.txt
There are two instances where 'word1' followed by 'word2' occurs in the sample source file I'm testing. Content of the 1.txt file
word1---sjdkkdkjdk---word2 I want this text----word1---jhfnkfnsjkdnf----word2 I need this also
Result is as below.
word1 I need this also
Expected Output :
word1 I want this text----word1 I need this also
Can anybody help me with this please?
I looked at other stack-overflow questionnaire but they discuss about replacing only one instance of the pattern.
Regular expressions are greedy - they match the longest possible string, so everything from the first 'word1' to the last 'word2'. Not sure if any version of sed supports non-greedy regexps... you could just use perl, though, which does:
perl -pe 's/word1.*?word2/word1/g' 1.txt
should do the trick. That ? changes the meaning of the prior * from 'match as many times as possible as long as the rest of the pattern matches' to 'match as few times as possible as long as the rest of the pattern matches'.
$ sed 's/#/#A/g; s/{/#B/g; s/}/#C/g; s/word1/{/g; s/word2/}/g; s/{[^{}]*}/word1/g; s/}/word2/g; s/{/word1/g; s/#C/}/g; s/#B/{/g; s/#A/#/g' file
word1 I want this text----word1 I need this also
It's lengthy and looks complicated but it's a technique that is used fairly often and is really just a series of simple steps to robustly convert word1 to { and word2 to } so you're dealing with characters instead of strings in the actual substitution s/{[^{}]*}/word1/g and so can use a negated bracket expression to avoid the greedy regexp taking up too much of the line.
See https://stackoverflow.com/a/35708616/1745001 for more info on the general approach used here to be able to turn strings into characters that cannot be present in the input by the time the real work takes place and then restore them again afterwards.
If you only have two instances of the word1-word2 pattern on a line, this should work:
sed 's/\(word1\).*word2\(.*\)\(word1\).*word2\(.*\)/\1\2\3\4/' 1.txt
I grab the parts we want to keep inside escaped brackets \( and \) then I can refer to those parts as \1 \2 and so on.
I have a file with many different lines in it and I need a way to find specific lines:
user=omega group=plasma account=test jobname=STDIN queue=omega_node
I want to use sed to search for "account=test" then replace "group=plasma" with "group=solid". I know how to do a "sed -i" and do the replacement but I can't figure out a way to do a search then pass replacement parameters.
You can make a conditional substitution using this syntax:
sed '/account=test/s/group=plasma/group=solid/' file
This only performs the substitution on lines which match the pattern account=test.
I'm writing a shell script to modify a file and I have a line something like this in it:
sed s/here \(.*\n\)/gone \1/g
Unfortunately, the search seems to match the longest string (i.e., it goes all the way to the last \n -- thus giving me just one replacement) but I want it to match only up to the first \n it finds (so I can get replacements on every line).
Is this possible?
Thanks for your help!
Looks like you want the feature called non-greedy (or lazy) match. Unfortunately sed does not provide such feature. To emulate it you need to search for anything except separator match until separator match. Like this:
s/here \([^\n]*\n\)/gone \1/g
I'm trying to write a grep (or egrep) command that will find and print any lines in "words.txt" which contain the same lower-case letter three times in a row. The three occurrences of the letter may appear consecutively (as in "mooo") or separated by one or more spaces (as in "x x x") but not separated by any other characters.
words.txt contains:
The monster said "grrr"!
He lived in an igloo only in the winter.
He looked like an aardvark.
Here's what I think the command should look like:
grep -E '\b[^ ]*[[:alpha:]]{3}[^ ]*\b' 'words.txt'
Although I know this is wrong, but I don't know enough of the syntax to figure it out. Using grep, could someone please help me?
Does this work for you?
grep '\([[:lower:]]\) *\1 *\1'
It takes a lowercase character [[:lower:]] and remembers it \( ... \). It than tries to match any number of spaces _* (0 included), the rememberd character \1, any number of spaces, the remembered character. And that's it.
You can try running it with --color=auto to see what parts of the input it matched.
Try this. Note that this will not match "mooo", as the word boundary (\b) occurs before the "m".
grep -E '\b([[:alpha:]]) *\1 *\1 *\b' words.txt
[:alpha:] is an expression of a character class. To use as a regex charset, it needs the extra brackets. You may have already known this, as it looks like you started to do it, but left the open bracket unclosed.