I am trying to replace string using sed from sh file.
Issue: After 'connection' it has blank line and its '-url' string comes in the next line, in addition requires to replace port number and password string as well. Using sed I am not able to remove blank line after connection.
Original String:
connection
-url>jdbc:oracle:thin:#10.10.10.11\:1551/password1 /connection-url
Replace with:
connection-url>jdbc:oracle:thin:#10.10.10.90\:1555/password2 /connection-url
I tried below commands which didn't worked:
sed -i 's/connection[\t ]+/,/g' sed-script.sh
sed 's/\connection*-\connection*/-/g' sed-script.sh
Tested with GNU awk.
awk -v RS="\n+{n}" '{$1=$1} 1' Input_file
connection -url>jdbc:oracle:thin:#10.10.10.11\:1551/password1 /connection-url
Could you please try following once.
awk '/^connection/{val=$0;next} NF && /^-url/{print val $0;val=""}' Input_file
Output will be as follows.
connection-url>jdbc:oracle:thin:#10.10.10.11\:1551/password1 /connection-url
You can remove the blank line after 'connection' using tr.
echo <input string> | tr -d "\n"
Where we can see that we want the \n character by running the string through od -c:
0000000 c o n n e c t i o n \n \n - u r l
0000020 > j d b c : o r a c l e : t h i
0000040 n : # 1 0 . 1 0 . 1 0 . 1 1 \ :
0000060 1 5 5 1 / p a s s w o r d 1 /
0000100 c o n n e c t i o n - u r l \n
With sed -
sed -E '
/connection$/,/^-url/ {
/connection$/ { h; d; }
/^$/ d
/^-url/ { H; s/.*//; x; s/\n//g; }
}
' old > new
Assumes no stray whitespace, and that a connection on a line by itself should be followed by a line that starts with -url...
sed processes a line at a time by default; if you want to check whether an empty line follows another line, you have to write a sed script to implement that.
I would go with Awk or Perl instead for this particular task.
perl -p0777 -i -e 's/connection\n\n-url/connection-url/' file
awk '/^connection/ { c=1; next }
c && /^$/ { c++; next }
c && /^-url/ { $1="connection" $1; c=0 }
c { print "connection-";
while(--c) print "" }
1' file >file.new
Perl, like sed, has an -i option to replace the file in-place. GNU Awk has an extension to do the same (look for -i inplace) but it's not portable to lesser Awks.
The Perl -0777 option causes the whole file to be slurped into memory as a single "line", line feeds (\n) and all. If the file is very big, this will obviously be problematic.
The Awk script takes care to put back the lines it skipped if it turned out to be a false match after all.
Related
I have a text file that has some lines like this (hyphen repeated)
-------------------------------------------------------
I need to replace these lines with character 'B' repeated 1500 times. For example, like
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
Any suggestions using 'sed' or 'awk' command?
With awk:
$ awk '/^-+$/ {s = sprintf("% 1500s", ""); gsub(/ /,"B",s); print s; next} 1' file
Or, maybe a bit more efficient if you have many such lines:
$ awk 'BEGIN {s = sprintf("% 1500s", ""); gsub(/ /,"B",s)} \
/^-+$/ {print s; next} 1' file
I think
perl -pe 'my $bb = "B"x1500; s/^-+$/$bb/g'
should do it.
printf+sed variant:
$ cat file
1111
----
2222
$ sed -r 's/^-+$/'"$(printf -- "%.1s" B{1..150})/" file
1111
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
2222
Here sed is used only for replacement.
printf is used for generating 1500 times B. (The above scriptlet has 150 instead of 1500, because it required too much scrolling.)
Hello let say I have a file such as :
$OUT some text
some text
some text
$OUT
$OUT
$OUT
how can I use sed in order to replace the 3 $OUT into "replace-thing" ?
and get
$OUT some text
some text
some text
replace-thing
With sed:
sed -n '1h; 1!H; ${g; s/\$OUT\n\$OUT\n\$OUT/replace-thing/g; p;}' file
GNU sed does not require the semicolon after p.
With commentary
sed -n ' # without printing every line:
# next 2 lines read the entire file into memory
1h # line 1, store current line in the hold space
1!H # not line 1, append a newline and current line to hold space
# now do the search-and-replace on the file contents
${ # on the last line:
g # replace pattern space with contents of hold space
s/\$OUT\n\$OUT\n\$OUT/replace-thing/g # do replacement
p # and print the revised contents
}
' file
This is the main reason I only use sed for very simple things: once you start using the lesser-used commands, you need extensive commentary to understand the program.
Note the commented version does not work on the BSD-derived sed on MacOS -- the comments break it, but removing them is OK.
In plain bash:
pattern=$'$OUT\n$OUT\n$OUT' # using ANSI-C quotes
contents=$(< file)
echo "${contents//$pattern/replace-thing}"
And the perl one-liner:
perl -0777 -pe 's/\$OUT(\n\$OUT){2}/replace-thing/g' file
for this particular task, I recommend to use awk instead. (hope that's an option too)
Update: to replace all 3 $OUT use: (Thanks to #thanasisp and #glenn jackman)
cat input.txt | awk '
BEGIN {
i = 0
p = "$OUT" # Pattern to match
n = 3 # N matches
r = "replace-thing"
}
$0 == p {
++i
if(i == n){
print(r)
i = 0 #reset counter (optional)
}
}
$0 != p {
i = 0
print($0)
}'
If you just want to replace the 3th $OUT usage, use:
cat input.txt | awk '
BEGIN {
i = 0
p = "\\$OUT" # Pattern to match
n = 3 # Nth match
r = "replace-thing"
}
$0 ~ p {
++i
if(i == n){
print(r)
}
}
i <= n || $0 !~ p {
print($0)
}'
This might work for you (GNU sed):
sed -E ':a;N;s/[^\n]*/&/3;Ta;/^(\$OUT\n?){3}$/d;P;D' file
Gather up 3 lines in the pattern space and if those 3 lines each contain $OUT, delete them. Otherwise, print/delete the first line and repeat.
I have an output of an analysis and I would like to grep a keyword "X" -which always appears- every time a phrase "Y" occurs. The keyword "X" appears many times but I would like to get only the subsequent after "Y".
For example, I would like to get the subsequent Folder name every time Iter = 10 occurs, i.e. F1, F4.
Iter = 10
Folder = F1
Iter = 5
Folder = F2
Iter = 6
Folder = F3
Iter = 10
Folder = F4
Any ideas?
Hexdump -c output of file (as requested by #Inian):
0000000 I t e r = 1 0 \n F o l d
0000010 e r = F 1 \n \n I t e r
0000020 = 5 \n F o l d e r = F 2 \n
0000030 \n I t e r = 6 \n F o l d
0000040 e r = F 3 \n \n I t e r
0000050 = 1 0 \n F o l d e r = F 4
0000060 \n
0000061
You could use awk for this requirement. It works on a /pattern/{action} based rule on each line of the input file. So in our case we first match the string Iter = 10 and enable a flag so that on the next match of the string starting with Folder, we extract the last space de-limited column, which in awk is represented by $NF and we reset the flag for subsequent matches.
awk '/\<Iter = 10\>/{flag=1; next} flag && /^Folder/{print $NF; flag=0;}' file
or without the <> try
awk '/Iter = 10/{flag=1; next} flag && /^Folder/{print $NF; flag=0;}' file
You could also you grep:
$ grep -A 1 Iter.*10 file | grep Folder | grep -o "[^ ]*$"
F1
F4
Explained:
grep -A 1 Iter.*10 file search for desired pattern and get some trailing context (-A 1, just one line)
grep Folder next search for keyword Folder
grep -o "[^ ]*$" get the last part of previous output
If there is noise between Iter and Folder lines you could remove that with grep "\(Iter.*10\|Folder\)" file first.
Above expects for Iter line to appear before Folder line. If that is not the case, awk is the cure. For example, data (line orders vary, there is noise):
Folder = F1
Foo = bar
Iter = 10
Iter = 5
Foo = bar
Folder = F2
$ awk -v RS="" -F"\n" ' # record separated by empty line
/Iter/ && / 10$/ { # look for record with Iter 10
for(i=1;i<=NF;i++) # iterate all fields (lines within record)
if(split($i,a," *") && a[1]=="Folder") # split Folder line to components
print a[3] # output value
}
' file
F1
grep is simply regrex search.
For, doing more complex operation, you can use awk.
E.g.
awk '/Iter = 10/ { getline; print $0 }' /path/to/file
where /path/to/file is the file containing your text to be searched
EDIT:
Just after posting my answer I read Inian's answer and it is more elaborate and accurate.
I have a file that looks like:
ignoretext
START
a b
c d
e
END
ignoretext
START
f g h
i
END
ignoretext
I want to translate that into rows of:
a b c d e
f g h i
Here is one way to do it with awk
awk '/END/ {ORS=RS;print "";f=0} f; /START/ {ORS=" ";f=1}' file
a b c d e
f g h i
Added a version that does not give space at the end of line. It may be shorter way to do this
awk 'a && !/END/ {printf FS} /END/ {print "";f=a=0} f {printf "%s",$0;a++} /START/ {f=1}'
a b c d e
f g h i
Here is another variant using GNU sed:
sed -n '/START/,/END/{:a;/START/d;/END/!{N;ba};s/\n/ /g;s/ END//;p}' file
a b c d e
f g h i
In a more readable format with explaination:
sed -n ' # Suppress default printing
/START/,/END/ { # For the range between /START/ and /END/
:a; # Create a label a
/START/d # If the line contains START, delete it
/END/! { # Until a line with END is seen
N # Append the next line to pattern space
ba # Branch back to label a to repeat
}
s/\n/ /g # Remove all new lines
s/ END// # Remove the END tag
p # Print the pattern space
}' file
Jotne's awk solution is probably the cleanest, but here's one way you can do it with GNU's version of sed:
sed -ne '/START/,/END/{/\(START\|END\)/!H}' \
-e '/END/{s/.*//;x;s/\n/ /g;s/^ *\| *$//\p}'
$ awk 'f{ if (/END/) {print rec; rec=sep=""; f=0} else {rec = rec sep $0; sep=" "} } /START/{f=1}' file
a b c d e
f g h i
I have a file containing lines like the following:
a b c patch/sample/upgrade.sql
a b c demo/sample/script.sh
I want to be able to copy everything starting from the position after "c" to the last "/" and append it to the end of each line in the file. For example:
a b c patch/sample/upgrade.sql patch/sample
a b c demo/sample/script.sh demo/sample
Does anyone know how I can do this?
if every line starts with a b c, that should do:
$> sed 's#a b c \(.*\)/[^/]*#& \1#g' foo.txt
a b c patch/sample/upgrade.sql patch/sample
a b c demo/sample/script.sh demo/sample
Otherwise, just adapt the first part.
If you can use awk then you can try:
awk '
{
last=x
n=split($4,tmp,/[/]/)
for(i=1;i<n;i++) {
last=last sep tmp[i];
sep="/"
}
$0=$0 FS last
}1' file
$ cat file
a b c patch/sample/upgrade.sql
a b c demo/sample/script.sh
$ awk '
{
last=x
n=split($4,tmp,/[/]/)
for(i=1;i<n;i++) {
last=last sep tmp[i];
sep="/"
}
$0=$0 FS last
}1' file
a b c patch/sample/upgrade.sql patch/sample
a b c demo/sample/script.sh /demo/sample
Taking inspiration from #fredtantini's answer, try the following in a terminal
sed -r 's|(\w+ \w+ \w+ /((.*)?/)*(.*))|\1 \3|' foo.txt
If you want to output this to a file,
sed -r 's|(\w+ \w+ \w+ /((.*)?/)*(.*))|\1 \3|' foo.txt > newFoo.txt