I need to write a script, that goes through a bunch of text files and replaces a certain string with another. The other one is located two lines below the certain string. Like this:
some text
this is the first line (helloWorld).
this is the second line.
This is the third line (patternxxx).
more text
And I want this:
some text
this is the first line (helloxxxWorld).
this is the second line.
This is the third line (patternxxx).
more text
I'm under Linux.
awk '
BEGIN { mr=3 }
/helloWorld/{ mr=0; gsub("helloWorld","helloxxxWorld");print; };
mr==2 { gsub("pattern", "patternxxx");}
mr++
'
test:
$ cat file
some text
this is the first line (helloWorld).
this is the second line.
This is the third line (pattern).
more text
some text
this is the first line (helloWorld).
this is the second line.
this is the second line.
This is the third line (pattern).
more text
$ awk '
BEGIN { mr=3 }
/helloWorld/{ mr=0; gsub("helloWorld","helloxxxWorld");print; };
mr==2 { gsub("pattern", "patternxxx");}
mr++
' file
some text
this is the first line (helloxxxWorld).
this is the second line.
This is the third line (patternxxx).
more text
some text
this is the first line (helloxxxWorld).
this is the second line.
this is the second line.
This is the third line (pattern).
more text
You can achieve this using sed, see below command. this will permanently make changes in the files.
sed -i 's/helloWorld/helloxxxWorld/g' file1 file2 ...filen
You can loop files one by one also.
for file in `ls`;
do
sed -i 's/helloWorld/helloxxxWorld/g' $file;
echo "$file completed";
done
make sure you have backups for the files.
This might work for you (GNU sed):
sed -ri '1N;N;s/(hello)(World.*\n.*\n.*pattern(xxx))/\1\3\2/;P;D' file1 file2 ...
Open a three line window in the file and pattern match on the first and third lines throughout the file.
Related
I'm trying to perform something that should be simple but I'm not quite understanding how to do it. I have a file that looks like this:
#A01182:104:HKNG5DSX3:3:1101:3947:1031 1:N:0:CATTGCCT+NATCTCAG
CNTCATAGCTGGTTGCACAGTTAACGTCGTTCAGGCCACGTTCCAGACCGTAGTTTGCCAGCGTCAGATCATAAACGGTGGTCACCAGGGCGGTGCTGCCA
+
F#FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFF,FFFFFFFFFFFFFFFFFFFFFFFFFF
#A01182:104:HKNG5DSX3:3:1101:7997:1031 1:N:0:CATTGCCT+NATCTCAG
GNCGATCCCTTCGCTGCTGCTGGCAATTATCGTTGTAGCGTTTGCCGGACCGAGTTTGTCTCACGCCATGTTTGCTGTCTGGCTGGCGCTGCTGCCGCGTA
+
F#FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:,FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
#A01182:104:HKNG5DSX3:3:1101:5547:1047 1:N:0:CATTGCCT+NATCTCAG
GGTGATGATTGTCTTTGGCGCAACGTTAATGAAAGATGCGCCGAAGCAGGAAGTGAAAACCAGCAATGGTGTGGTGGAGAAGGACTACACCCTGGCAGAGT
+
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFF
#A01182:104:HKNG5DSX3:3:1101:20726:1063 1:N:0:CATTGCCT+GATCTCAG
GGGACGCCCATTACGCTGGTGAATCTGGCAACCCATACCAGCGCCCTGCCCCGTGAACAGCCCGGTGGCGCGGCACATCGTCCGGTATTTGTCTGGCCAAC
+
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFF
and goes on for a lot of lines (The actual file is 2.5 Gb). What I want to do is to replace every fourth line (all those that have a lot of F's) for another string, the same for all.
I have tried with sed but I don't seem to be able to get the script right since I produce and output without changes.
Any help would be really appreciated!
This might work for you (GNU sed):
sed -i '4~4s/.*/another string/' file(s)
Starting at the 4th line and every 4 lines thereafter, replace the whole line with another string.
I'd use awk for this
awk '
NR % 4 == 0 {print "new string"; next}
{print}
' file > file.new && mv file.new file
I have file with the below info
testing
testing
testing
I want to insert a word(tested) before the first testing word using sed or any linux command
need to get output like
tested
testing
testing
testing
Thanks
For lines consisting of "testing" exactly:
sed '0,/^testing$/s/^testing$/tested\n&/' file
For lines containing "testing":
sed '0,/.*testing.*/s/.*testing.*/tested\n&/' file
For Lines Starting with "testing"
sed '0,/^testing.*/s/^testing.*/tested\n&/' file
For lines ending with "testing":
sed '0,/.*testing$/s/.*testing$/tested\n&/' file
To update the content of the file with the result add "-i", example:
sed -i '0,/testing/s/testing/tested\n&/' file
To provide an awk-based alternative that is easier to understand:
awk '!found && /testing/ { print "tested"; found=1 } 1' file
found is used to keep track of whether the first instance of testing has been found (variable found, as any Awk variable, defaults to 0, i.e., false in a Boolean context).
/testing/ therefore matches the first line containing testing, and processes the associated block:
{ print "tested"; found=1 } prints the desired text and sets the flag that the first testing line has been found
1 is a common shorthand for { print }, i.e., simply printing the current input line as is.
This might work for you (GNU sed):
sed -e '/testing/{itested' -e ':a;n;ba}' file
Insert tested before the first match of testing and then use a loop to read/print the remainder of the file.
Or use the GNU specific:
sed '0,/testing/itested' file
Consider file:
apple
banana
bar TEST qux
TEST foo
baz TEST
TEST
TEST
Insert a line before the first occurrence of a line consisting solely of the string 'TEST':
sed '0,/^TEST$/s//INSERTED LINE\n&/' <file
Result:
apple
banana
bar TEST qux
TEST foo
baz TEST
INSERTED LINE
TEST
TEST
0,/^TEST$ is an address range. This range begins at the first line of input and extends until a line beginning and ending with TEST is reached. The use of a range restricts the region of applicability of the following command, a substitution command in our case.
A note about the meaning of //:
The empty regular expression ‘//’ repeats the last regular expression match
– https://www.gnu.org/software/sed/manual/html_node/Regexp-Addresses.html
A note about the meaning of the ampersand, &, used in the replacement portion of the s command:
the replacement can contain unescaped & characters which reference the whole matched portion of the pattern space
– https://www.gnu.org/software/sed/manual/html_node/The-_0022s_0022-Command.html
For completeness, here's three additional cases to consider...
Insert a line before the first occurrence of a line containing 'TEST':
sed '0,/.*TEST/s//INSERTED LINE\n&/' <file
Insert a line before the first occurrence of a line beginning with 'TEST':
sed '0,/^TEST/s//INSERTED LINE\n&/' <file
Insert a line before the first occurrence of a line ending with 'TEST':
sed '0,/.*TEST$/s//INSERTED LINE\n&/' <file
How to insert string to the beginning of the last line?
I want to add a time stamp to a text file which contains multiple lines
var1 = `date`
LINE1
LINE2
LINE3
...
(INSERT var1 here) LASTLINE
sed 's/^/test line /g' textfile inserts characters to the beginning of every line but how can I specifically modify the last line only?
Thanks
Going forward:
sed '$s/^/sample text /' textfile works, but only when inserting regular strings. If I try
var1 = "sample text"
and use substition, here are the problems I encounter
using single quotes in sed does not expand variables, so sed '$s/^/$var1/' textfile will insert the string $var1 into the beginning of the last line.
To enable variable substitution I tried using double quotes. It works when I specify the exact line number. something like:
sed "5s/^/$var1/" textfile
But when I try sed "$s/^/$var1" text file, it returns an error:
sed: -e expression #1, char 5: extra characters after command
Can someone help me please?
Like this:
sed '$s/^/test line /' textfile
$ indicates last line. Similarly, you can insert into a any specific line by putting the line number in place of $
But when I try sed "$s/^/$var1" text file, it returns an error:
It returns an error because the shell attempts to expand $s since you've used double quotes. You need to escape the $ in $s.
sed "\$s/^/$var1/" filename
sedshould be the best tool, but awk can do this too:
awk '{a[++t]=$0} END {for (i=1;i<t;i++) print a[i];print v$0}' v="$var1" file
It will insert value of var1 in front of last line
Another variation
awk 'NR==f {$0=v$0}1' v="$var1" f=$(wc -l file)
PS you do not need to specify file after awk, not sure why. If you do so, it reads it double.
This command would work for you:
sed -i "5s/^/$var1 /" text file
In Linux what command can I use to replace a single line of text with new multiple lines? I want to look for a keyword on a line and delete this line and replace it with multiple new lines. So in the text shown below I want to search for the line that contains "keyword" and replace the entire line with 3 new lines of text as shown.
For example replacing the line containing the keyword,
This is Line 1
This is Line 2 that has keyword
This is Line 3
changed to this:
This is Line 1
Inserted is new first line
Inserted is new second line
Inserted is new third line
This is Line 3
$ sed '/keyword/c\
> Inserted is new first line\
> Inserted is new second line\
> Inserted is new third line' input.txt
This is Line 1
Inserted is new first line
Inserted is new second line
Inserted is new third line
This is Line 3
$ and > are bash prompt
Create a file, script.sed, containing:
/keyword/{i\
Inserted is new first line\
Inserted is new second line\
Inserted is new third line
d
}
Apply it to your data:
sed -f script.sed your_data
There are numerous variations on how to do it, using the c and a commands instead of i and/or d, but this is reasonably clean. It finds the keyword, inserts three lines of data, and then deletes the line containing the keyword. (The c command does that all, but I didn't remember that it existed, and the a command appends the text and is essentially synonymous with i in this context.)
you can do it using shell builtins too:
STRING1_WITH_MULTIPLE_LINES="your
text
here"
STRING2_WITH_MULTIPLE_LINES="more
text"
OUTPUT=""
while read LINE || [ "$LINE" ]; do
case "$LINE" in
"Entire line matches this")OUTPUT="$OUTPUT$STRING1_WITH_MULTIPLE_LINES
";;
*"line matches this with extra before and/or after"*)OUTPUT="$OUTPUT$STRING2_WITH_MULTIPLE_LINES
";;
*)OUTPUT="$OUTPUT$LINE
";;
esac
done < file
echo "$OUTPUT" >file
UPDATED:
Using sed, how can I insert (NOT SUBSTITUTE) a new line on only the first match of keyword for each file.
Currently I have the following but this inserts for every line containing Matched Keyword and I want it to only insert the New Inserted Line for only the first match found in the file:
sed -ie '/Matched Keyword/ i\New Inserted Line' *.*
For example:
Myfile.txt:
Line 1
Line 2
Line 3
This line contains the Matched Keyword and other stuff
Line 4
This line contains the Matched Keyword and other stuff
Line 6
changed to:
Line 1
Line 2
Line 3
New Inserted Line
This line contains the Matched Keyword and other stuff
Line 4
This line contains the Matched Keyword and other stuff
Line 6
You can sort of do this in GNU sed:
sed '0,/Matched Keyword/s//New Inserted Line\n&/'
But it's not portable. Since portability is good, here it is in awk:
awk '/Matched Keyword/ && !x {print "Text line to insert"; x=1} 1' inputFile
Or, if you want to pass a variable to print:
awk -v "var=$var" '/Matched Keyword/ && !x {print var; x=1} 1' inputFile
These both insert the text line before the first occurrence of the keyword, on a line by itself, per your example.
Remember that with both sed and awk, the matched keyword is a regular expression, not just a keyword.
UPDATE:
Since this question is also tagged bash, here's a simple solution that is pure bash and doesn't required sed:
#!/bin/bash
n=0
while read line; do
if [[ "$line" =~ 'Matched Keyword' && $n = 0 ]]; then
echo "New Inserted Line"
n=1
fi
echo "$line"
done
As it stands, this as a pipe. You can easily wrap it in something that acts on files instead.
If you want one with sed*:
sed '0,/Matched Keyword/s//Matched Keyword\nNew Inserted Line/' myfile.txt
*only works with GNU sed
This might work for you:
sed -i -e '/Matched Keyword/{i\New Inserted Line' -e ':a;n;ba}' file
You're nearly there! Just create a loop to read from the Matched Keyword to the end of the file.
After inserting a line, the remainder of the file can be printed out by:
Introducing a loop place holder :a (here a is an arbitrary name).
Print the current line and fetch the next into the pattern space with the ncommand.
Redirect control back using the ba command which is essentially a goto to the a place holder. The end-of-file condition is naturally taken care of by the n command which terminates any further sed commands if it tries to read passed the end-of-file.
With a little help from bash, a true one liner can be achieved:
sed $'/Matched Keyword/{iNew Inserted Line\n:a;n;ba}' file
Alternative:
sed 'x;/./{x;b};x;/Matched Keyword/h;//iNew Inserted Line' file
This uses the Matched Keyword as a flag in the hold space and once it has been set any processing is curtailed by bailing out immediately.
If you want to append a line after first match only, use AWK instead of SED as below
awk '{print} /Matched Keyword/ && !n {print "New Inserted Line"; n++}' myfile.txt
Output:
Line 1
Line 2
Line 3
This line contains the Matched Keyword and other stuff
New Inserted Line
Line 4
This line contains the Matched Keyword and other stuff
Line 6