I am trying to use sed to insert sed command at the end of each line in a file using -
sed -r 's/$/ | sed -r s\/abc\/xyz\/ /' filename.extension
What I want next is to have single quotes around the inner sed. So that it will look something like-
sed -r 's/$/ | sed -r 's\/abc\/xyz\/' /' filename.extension
I tried escaping the inner single quotes, but no use.
Basically, I want the following lines -
line 1
line 2
line 3
to turn into-
line 1 | sed -r 's/abc/xyz/'
line 2 | sed -r 's/abc/xyz/'
line 3 | sed -r 's/abc/xyz/'
I am unable to get the single quotes, even with the escape characters.
sed -e "s:$: | sed -r 's/abc/xyz/':" yourfile
Your problem is an example of the general case of nesting shell expressions. There are a number of ways to do this.
Use alternate delimiters. That's what I did here.
Assign subexpressions to variables, and then expand them.
Use lots of \ escapes.
Put your subexpression in a file and read it.
Use alternative delimiter in inner sed and double quote in outer sed to simplify your command:
sed "s/$/ | sed -r 's~abc~xyz~'/" file.ext
btw -r is not really needed in inner sed
This might work for you (GNU sed):
sed 's/$/ | sed -r '\''s\/abc\/xyz\/'\''/' file
Use '\'' to end the current single quote, then in the shell use \' to quote a single quote and finally a single quote to start the quoting of the sed command. Use \/ to quote the forward slash in the sed command.
As the substitution command can use any delimiter:
sed 's#$# | sed -r '\''s/abc/xyz/'\''#' file
reduces the amount of quoting and:
sed "s#$# | sed -r 's/abc/xyz/'#" file
reduces it further. However double quoting sed commands (or any utility) can have unwanted side effects i.e. metacharacters can be evaluated by the shell, so it is best to single quote and live with the "hole-like" mechanism '\''.
Related
I am trying to insert line in a file which will have a sed command to insert a line in another file
Like below for eg :
want to add this line sed -i '1s/^/insideFile\n/' secondFile.sh to 65th line of firstfile.txt
I tried:
sed -i '65s/^/\'sed -i \'1s/^/insideFile\n/\' secondFile.sh\'\n/' firstfile.sh
but not able to escape the '
also tried
sed -i "65s/^/'sed -i "1s/^/topLine\n/" $FILE_HOME_LOCAL_FILE'\n/" secondFile.sh
but got
sed: -e expression #1, char 18: unknown option to `s
You may use this sed:
sed -i.bak "65s~^~sed -i '1s/^/insideFile\\\\n/' secondFile.sh\\n~" firstfile.sh
Note:
Use of alternate delimiter ~ to avoid escaping /
use of double quotes for your sed command to avoid escaping single quotes
use of \\\\ to insert a single \
use of \\n inside double quotes to insert a line break
Alternatively with i (insert) instead of s (substitute):
sed -i.bak "65i\\
sed -i '1s/^/insideFile\\\\n/' secondFile.sh\\
" firstfile.sh
Cleanest solution would be to create a sed script like this thus avoiding all quoting and extra escaping:
cat ins.sed
65i\
sed -i '1s/^/insideFile\\n/' secondFile.sh
Then use it as:
sed -i.bak -f ins.sed firstfile.sh
While sed allows you to do these things, it becomes a bit cumbersome when slashes and other control characters are involved. So it might be beneficial to use awk instead for this one:
$ awk '(FNR==65){print "sed -i \0421s/^/insideFile\\n/\042 secondFile.sh"}1' firstfile.txt > firstfile.new
$ mv firstfile.new > firstfile.txt
There are inplace possibilities, but if you come to think of it. Inplace is nothing more than making a temp to rename afterwards. This is exactly what sed is doing.
I want to add this line sed -i '1s/^/insideFile\n/' secondFile.sh to 65th line of firstfile.txt
A scripted solution:
#!/bin/bash
## uncomment to make sure firstfile.txt has at least 65 (e.g. 66) lines
# for index in {1..66}; do
# echo -e "$index" >> firstfile.txt
# done
# prepare the insert text, escaping the newline character twice
text="sed -i '1s/^/insideFile\\\\n/'"
# append to 65th line
sed -i '' "65i\\
$text
" firstfile.txt
I created the following bash pipeline that will take the output of "who" and modify it to meet an assignment's requirements
This is the pipline:
who | sed -e "s/\b\(.\)/\u\1/g" | sed 's/[.]/ /g' | sed 's/ Pts\// TTY /g' | sed '1d' | sed -n 's/ .*$/ /gp'
After putting this into a sed file that looks like this:
s/\b\(.\)/\u\1/g
s/[.]/ /g
s/ Pts\// TTY /g
1d
s/ .*$/ /gp
And then running it like such:
who | sed -f sedfile
The output is correct in that everything is in the format of:
firstName lastName TTY (a number)
However each line is printed twice, where the pipeline properly printed each line once
Would anyone happen to know the issue please?
It's the gp at the last line. You're not running with sed -n (no-print), so by default you're running with sed -yesprint (or whatever). Then you hit that gp, which prints, and you get two copies.
Convert to sed -n, or change the gp to just g. Or better still, get rid of it, since the match pattern contains $, so it will only ever run in one place - the end of the line.
I have a flat file where I have multiple occurrences of strings that contains single quote, e.g. hari's and leader's.
I want to replace all occurrences of the single quote with space, i.e.
all occurences of hari's to hari s
all occurences of leader's to leader s
I tried
sed -e 's/"'"/ /g' myfile.txt
and
sed -e 's/"'"/" "/g' myfile.txt
but they are not giving me the expected result.
Try to keep sed commands simple as much as possible.
Otherwise you'll get confused of what you'd written reading it later.
#!/bin/bash
sed "s/'/ /g" myfile.txt
This will do what you want to
echo "hari's"| sed 's/\x27/ /g'
It will replace single quotes present anywhere in your file/text. Even if they are used for quoting they will be replaced with spaces. In that case(remove the quotes within a word not at word boundary) you can use the following:
echo "hari's"| sed -re 's/(\<.+)\x27(.+\>)/\1 \2/g'
HTH
Just go leave the single quote and put an escaped single quote:
sed 's/'\''/ /g' input
also possible with a variable:
quote=\'
sed "s/$quote/ /g" input
Here is based on my own experience.
Please notice on how I use special char ' vs " after sed
This won't do (no output)
2521 #> echo 1'2'3'4'5 | sed 's/'/ /g'
>
>
>
but This would do
2520 #> echo 1'2'3'4'5 | sed "s/'/ /g"
12345
The -i should replace it in the file
sed -i 's/“/"/g' filename.txt
if you want backups you can do
sed -i.bak 's/“/"/g' filename.txt
I had to replace "0x" string with "32'h" and resolved with:
sed 's/ 0x/ 32\x27h/'
I need some assistance trying to build up a variable using a list of exclusions in a file.
So I have a exclude file I am using for rsync that looks like this:
*.log
*.out
*.csv
logs
shared
tracing
jdk*
8.6_Code
rpsupport
dbarchive
inarchive
comms
PR116PICL
**/lost+found*/
dlxwhsr*
regression
tmp
working
investigation
Investigation
dcsserver_weblogic_
dcswebrdtEAR_weblogic_
I need to build up a string to be used as a variable to feed into egrep -v, so that I can use the same exclusion list for rsync as I do when egrep -v from a find -ls.
So I have created this so far to remove all "*" and "/" - and then when it sees certain special characters it escapes them:
cat exclude-list.supt | while read line
do
echo $line | sed 's/\*//g' | sed 's/\///g' | 's/\([.-+_]\)/\\\1/g'
What I need the ouput too look like is this and then export that as a variable:
SEXCLUDE_supt="\.log|\.out|\.csv|logs|shared|PR116PICL|tracing|lost\+found|jdk|8\.6\_Code|rpsupport|dbarchive|inarchive|comms|dlxwhsr|regression|tmp|working|investigation|Investigation|dcsserver\_weblogic\_|dcswebrdtEAR\_weblogic\_"
Can anyone help?
A few issues with the following:
cat exclude-list.supt | while read line
do
echo $line | sed 's/\*//g' | sed 's/\///g' | 's/\([.-+_]\)/\\\1/g'
Sed reads files line by line so cat | while read line;do echo $line | sed is completely redundant also sed can do multiple substitutions by either passing them as a comma separated list or using the -e option so piping to sed three times is two too many. A problem with '[.-+_]' is the - is between . and + so it's interpreted as a range .-+ when using - inside a character class put it at the end beginning or end to lose this meaning like [._+-].
A much better way:
$ sed -e 's/[*/]//g' -e 's/\([._+-]\)/\\\1/g' file
\.log
\.out
\.csv
logs
shared
tracing
jdk
8\.6\_Code
rpsupport
dbarchive
inarchive
comms
PR116PICL
lost\+found
dlxwhsr
regression
tmp
working
investigation
Investigation
dcsserver\_weblogic\_
dcswebrdtEAR\_weblogic\_
Now we can pipe through tr '\n' '|' to replace the newlines with pipes for the alternation ready for egrep:
$ sed -e 's/[*/]//g' -e 's/\([._+-]\)/\\\1/g' file | tr "\n" "|"
\.log|\.out|\.csv|logs|shared|tracing|jdk|8\.6\_Code|rpsupport|dbarchive|...
$ EXCLUDE=$(sed -e 's/[*/]//g' -e 's/\([._+-]\)/\\\1/g' file | tr "\n" "|")
$ echo $EXCLUDE
\.log|\.out|\.csv|logs|shared|tracing|jdk|8\.6\_Code|rpsupport|dbarchive|...
Note: If your file ends with a newline character you will want to remove the final trailing |, try sed 's/\(.*\)|/\1/'.
This might work for you (GNU sed):
SEXCLUDE_supt=$(sed '1h;1!H;$!d;g;s/[*\/]//g;s/\([.-+_]\)/\\\1/g;s/\n/|/g' file)
This should work but I guess there are better solutions. First store everything in a bash array:
SEXCLUDE_supt=$( sed -e 's/\*//g' -e 's/\///g' -e 's/\([.-+_]\)/\\\1/g' exclude-list.supt)
and then process it again to substitute white space:
SEXCLUDE_supt=$(echo $SEXCLUDE_supt |sed 's/\s/|/g')
How i can delete whitespace in each line of file, using bash
For instance, file1.txt. Before:
gg g
gg g
t ttt
after:
gg g
gg g
t ttt
sed -i 's/ //g' your_file will do it, modifying the file inplace.
To delete only the whitespaces at the beginning of one single line, use sed -i 's/^ *//' your_file
In the first expression, we replace all spaces with nothing.
In the second one, we replace at the beginning using the ^ keyword
tr(delete all whitespaces):
$ tr -d ' ' <input.txt >output.txt
$ mv output.txt input.txt
sed(delete leading whitespaces)
$ sed -i 's/^ *//' input.txt
use can use perl -i for in place replacement.
perl -p -e 's/^ *//' file
To delete the white spaces before start of the line if the pattern matches. Use the following command.
For example your foo.in has pattern like this
This is a test
Lolll
blaahhh
This is a testtt
After issuing following command
sed -e '/This/s/ *//' < foo.in > foo.out
The foo.out will be
This is a test
Lolll
blaahhh
This is a testtt
"Whitespace" can include both spaces AND tabs. The solutions presented to date will only match and operate successfully on spaces; they will fail if the whitespace takes the form of a tab.
The below has been tested on the OP's specimen data set with both spaces AND tabs, matching successfully & operating on both:
sed 's/^[[:blank:]]*//g' yourFile
After testing, supply the -i switch to sed to make the changes persistent-