I have a flat file where I have multiple occurrences of strings that contains single quote, e.g. hari's and leader's.
I want to replace all occurrences of the single quote with space, i.e.
all occurences of hari's to hari s
all occurences of leader's to leader s
I tried
sed -e 's/"'"/ /g' myfile.txt
and
sed -e 's/"'"/" "/g' myfile.txt
but they are not giving me the expected result.
Try to keep sed commands simple as much as possible.
Otherwise you'll get confused of what you'd written reading it later.
#!/bin/bash
sed "s/'/ /g" myfile.txt
This will do what you want to
echo "hari's"| sed 's/\x27/ /g'
It will replace single quotes present anywhere in your file/text. Even if they are used for quoting they will be replaced with spaces. In that case(remove the quotes within a word not at word boundary) you can use the following:
echo "hari's"| sed -re 's/(\<.+)\x27(.+\>)/\1 \2/g'
HTH
Just go leave the single quote and put an escaped single quote:
sed 's/'\''/ /g' input
also possible with a variable:
quote=\'
sed "s/$quote/ /g" input
Here is based on my own experience.
Please notice on how I use special char ' vs " after sed
This won't do (no output)
2521 #> echo 1'2'3'4'5 | sed 's/'/ /g'
>
>
>
but This would do
2520 #> echo 1'2'3'4'5 | sed "s/'/ /g"
12345
The -i should replace it in the file
sed -i 's/“/"/g' filename.txt
if you want backups you can do
sed -i.bak 's/“/"/g' filename.txt
I had to replace "0x" string with "32'h" and resolved with:
sed 's/ 0x/ 32\x27h/'
Related
I am trying to use sed to insert sed command at the end of each line in a file using -
sed -r 's/$/ | sed -r s\/abc\/xyz\/ /' filename.extension
What I want next is to have single quotes around the inner sed. So that it will look something like-
sed -r 's/$/ | sed -r 's\/abc\/xyz\/' /' filename.extension
I tried escaping the inner single quotes, but no use.
Basically, I want the following lines -
line 1
line 2
line 3
to turn into-
line 1 | sed -r 's/abc/xyz/'
line 2 | sed -r 's/abc/xyz/'
line 3 | sed -r 's/abc/xyz/'
I am unable to get the single quotes, even with the escape characters.
sed -e "s:$: | sed -r 's/abc/xyz/':" yourfile
Your problem is an example of the general case of nesting shell expressions. There are a number of ways to do this.
Use alternate delimiters. That's what I did here.
Assign subexpressions to variables, and then expand them.
Use lots of \ escapes.
Put your subexpression in a file and read it.
Use alternative delimiter in inner sed and double quote in outer sed to simplify your command:
sed "s/$/ | sed -r 's~abc~xyz~'/" file.ext
btw -r is not really needed in inner sed
This might work for you (GNU sed):
sed 's/$/ | sed -r '\''s\/abc\/xyz\/'\''/' file
Use '\'' to end the current single quote, then in the shell use \' to quote a single quote and finally a single quote to start the quoting of the sed command. Use \/ to quote the forward slash in the sed command.
As the substitution command can use any delimiter:
sed 's#$# | sed -r '\''s/abc/xyz/'\''#' file
reduces the amount of quoting and:
sed "s#$# | sed -r 's/abc/xyz/'#" file
reduces it further. However double quoting sed commands (or any utility) can have unwanted side effects i.e. metacharacters can be evaluated by the shell, so it is best to single quote and live with the "hole-like" mechanism '\''.
Below is my input
54.243.94.244, 54.243.113.63
and I want the out like below,
54.243.94.244
54.243.113.63
i.e. after comma I need to add newline.
How to achieve it in Unix? Please suggest some commands.
Another option is tr
tr ',' '\n'
sed will do the trick:
$ echo '54.243.94.244, 54.243.113.63' | sed 's/, /\n/g'
54.243.94.244
54.243.113.63
The sed command s/, /\n/g will replace all occurrences of a comma followed by a space in the input with a newline.
A simple example would be
VAR1=a
VAR1="$VAR1"$'\n'b
echo "$VAR"
This would give output like
a
b
I could do the following in python to clean and strip unwanted whitespaces, but can it be done just through the terminal by other means like sed , grep or something?
outfile = open('textstripped.txt','w+','utf8')
for i in open('textfile.txt','r','utf8'):
print>>outfile, i.strip()
Using perl on the command line:
perl -lpe 's/^\s+//; s/\s+$//' file.txt > stripped.txt
This solution is based on sed man page:
sed 'y/\t/ /;s/^ *//;s/ *$//' input > output
http://www.gnu.org/software/sed/manual/sed.html#Centering-lines
Description:
y\t/ / replaces tabs with spaces
s/^ *// removes leading spaces
s/ *$// removes trailing spaces
$ cat input.txt | sed 's/^[ \t]*//;s/[ \t]*$//' > output.txt
This gets rid of the leading and trailing white spaces..
EDIT: sed -e "s/^[ \t]+//; s/[ \t]+$//" -i .bk input.txt
This does in place file editing, and saves backup to input.txt.bk (and saves a process as some suggested)
sed -E "s/(^[ \t]+|[ \t]+$)//" < input > output
Or if you have a GNU-compliant version of SED:
sed -E "s/^\s+|\s+$//g" < in > out
If you have a Mac, I recommed getting homebrew and installing gnu-sed.
Then, alias sed=gsed.
How i can delete whitespace in each line of file, using bash
For instance, file1.txt. Before:
gg g
gg g
t ttt
after:
gg g
gg g
t ttt
sed -i 's/ //g' your_file will do it, modifying the file inplace.
To delete only the whitespaces at the beginning of one single line, use sed -i 's/^ *//' your_file
In the first expression, we replace all spaces with nothing.
In the second one, we replace at the beginning using the ^ keyword
tr(delete all whitespaces):
$ tr -d ' ' <input.txt >output.txt
$ mv output.txt input.txt
sed(delete leading whitespaces)
$ sed -i 's/^ *//' input.txt
use can use perl -i for in place replacement.
perl -p -e 's/^ *//' file
To delete the white spaces before start of the line if the pattern matches. Use the following command.
For example your foo.in has pattern like this
This is a test
Lolll
blaahhh
This is a testtt
After issuing following command
sed -e '/This/s/ *//' < foo.in > foo.out
The foo.out will be
This is a test
Lolll
blaahhh
This is a testtt
"Whitespace" can include both spaces AND tabs. The solutions presented to date will only match and operate successfully on spaces; they will fail if the whitespace takes the form of a tab.
The below has been tested on the OP's specimen data set with both spaces AND tabs, matching successfully & operating on both:
sed 's/^[[:blank:]]*//g' yourFile
After testing, supply the -i switch to sed to make the changes persistent-
File1:
hello
world
How would one delete the leading/trailing blank spaces within this file using sed - using one command (no intermediate files)?
I've currently got:
sed -e 's/^[ \t]*//' a > b
For leading spaces.
sed 's/ *$//' b > c
And this for trailing spaces.
You almost got it:
sed -e 's/^[ \t]*//;s/[ \t]*$//' a > c
Moreover on some flavours of sed, there is also an option for editing inline:
sed -i -e 's/^[ \t]*//;s/[ \t]*$//' a
easier way, using awk
awk '{$1=$1}1' file
or
awk '{gsub(/^ +| +$/,"")}1' file
perl -lape 's/^\s+|\s+$//g'
Honestly, I know perl regexps the best, so I find perl -lape much easier to use than sed -e.
Also, to answer the original question, you can have sed execute multiple operations like this:
sed -e 's/something/something else/' -e 's/another substitution/another replacement/'
Apparently you can also put the two substitutions in one string and separate them with a semicolon, as indicated in another answer.
Note that in the more general case of applying several filters in a row to an input file without using intermediate files, the solution is to use pipes:
sed -e 's/^[ \t]*//' a | sed -e 's/ *$//' > c
Obviously they are not required here because one invocation of sed is sufficient, but if the second sed command was something different, like uniq or sort, then this pattern is the right one.