What is the proper way to insert tab in sed? I'm inserting a header line into a stream using sed. I could probably do a replacement of some character afterward to put in tab using regular expression, but is there a better way to do it?
For example, let's say I have:
some_command | sed '1itextTABtext'
I would like the first line to look like this (text is separated by a tab character):
text text
I have tried substituting TAB in the command above with "\t", "\x09", " " (tab itself). I have tried it with and without double quotes and I can't get sed to insert tab in between the text.
I am trying to do this in SLES 9.
Assuming bash (and maybe other shells will work too):
some_command | sed $'1itext\ttext'
Bash will process escapes, such as \t, inside $' ' before passing it as an arg to sed.
You can simply use the sed i command correctly:
some_command | sed '1i\
text text2'
where, as I hope it is obvious, there is a tab between 'text' and 'text2'. On MacOS X (10.7.2), and therefore probably on other BSD-based platforms, I was able to use:
some_command | sed '1i\
text\ttext2'
and sed translated the \t into a tab.
If sed won't interpret \t and inserting tabs at the command line is a problem, create a shell script with an editor and run that script.
As most answers say, probably literal tab char is the best.
info sed saying "\t is not portable." :
...
'\CHAR'
Matches CHAR, where CHAR is one of '$', '*', '.', '[', '\', or '^'.
Note that the only C-like backslash sequences that you can
portably assume to be interpreted are '\n' and '\\'; in particular
'\t' is not portable, and matches a 't' under most implementations
of 'sed', rather than a tab character.
...
Sed can do this, but it's awkward:
% printf "1\t2\n3\t4\n" | sed '1i\\
foo bar\\
'
foo bar
1 2
3 4
$
(The double backslashes are because I'm using tcsh as my shell; if you use bash, use single backslashes)
The space between foo and bar is a tab, which I typed by prepending it with CtrlV. You'll also need to prepend the newlines inside your single quotes with a CtrlV.
It would probably be simpler/clearer to do this with awk:
$ printf "1\t2\n3\t4\n" | awk 'BEGIN{printf("foo\tbar\n");} {print;}'
escape the tab character:
sed -i '/<setup>/ a \\tmy newly added line' <file_name>
NOTE: above we have two backslashes (\) first one is for escaping () and the next one is actual tab char (\t)
To illustrate the fact the BRE syntax for sed does mention that \t is not portable, Git 2.13 (Q2 2017) gets rid of it.
See commit fba275d (01 Apr 2017) by Junio C Hamano (gitster).
(Merged by Junio C Hamano -- gitster -- in commit 3c833ca, 17 Apr 2017)
contrib/git-resurrect.sh: do not write \t for HT in sed scripts
Just like we did in 0d1d6e5 ("t/t7003: replace \t with literal tab
in sed expression", 2010-08-12, Git 1.7.2.2), avoid writing "\t" for HT in sed scripts, which is not portable.
- sed -ne 's~^\([^ ]*\) .*\tcheckout: moving from '"$1"' .*~\1~p'
+ sed -ne 's~^\([^ ]*\) .* checkout: moving from '"$1"' .*~\1~p'
^^^^
|
(literal tab)
I found an alternate way to insert a tab by using substitution.
some_command | sed '1s/^/text\ttext\n/'
I still do not know of a way to do it using the insert method.
This command replace old to new in file.txt:
sed -i '' 's/old/new/' file.txt
This command will add a tab for new:
sed -i '' $'s/old/\tnew/' file.txt
This command replaces an entire string:
sed -i '' 's/.*old.*/new/' file.txt
Related
How to escape a single quote in a sed expression that is already surrounded by quotes?
For example:
sed 's/ones/one's/' <<< 'ones thing'
Quote sed codes with double quotes:
$ sed "s/ones/one's/"<<<"ones thing"
one's thing
I don't like escaping codes with hundreds of backslashes – hurts my eyes. Usually I do in this way:
$ sed 's/ones/one\x27s/'<<<"ones thing"
one's thing
One trick is to use shell string concatenation of adjacent strings and escape the embedded quote using shell escaping:
sed 's/ones/two'\''s/' <<< 'ones thing'
two's thing
There are 3 strings in the sed expression, which the shell then stitches together:
sed 's/ones/two'
\'
's/'
Escaping single quote in sed: 3 different ways:
From fragile to solid...
Note: This answer is based on GNU sed!!
1. Using double-quotes to enclose sed script:
Simpliest way:
sed "s/ones/one's/" <<< 'ones thing'
But using double-quote lead to shell variables expansion and backslashes to be considered as shell escape before running sed.
1.1. Specific case without space and special chars
In this specific case, you could avoid enclosing at shell level (command line):
sed s/ones/one\'s/ <<<'ones thing'
will work until whole sedscript don't contain spaces, semicolons, special characters and so on... (fragile!)
2. Using octal or hexadecimal representation:
This way is simple and efficient, if not as readable as next one.
sed 's/ones/one\o047s/' <<< 'ones thing'
sed 's/ones/one\x27s/' <<< 'ones thing'
And as following character (s) is not a digit, you coul write octal with only 2 digits:
sed 's/ones/one\o47s/' <<< 'ones thing'
3. Creating a dedicated sed script
cat <<eosedscript >sampleSedWithQuotes.sed
#!$(which sed) -f
s/ones/one's/;
eosedscript
chmod +x sampleSedWithQuotes.sed
From there, you could run:
./sampleSedWithQuotes.sed <<<'ones thing'
one's thing
This is the strongest and simpliest solution as your script is the most readable:$ cat sampleSedWithQuotes.sed
#!/bin/sed -f
s/ones/one's/;
3.1 You coud use -i sed flag:
As this script use sed in shebang, you could use sed flags on command line. For editing file.txt in place, with the -i flag:
echo >file.txt 'ones thing'
./sampleSedWithQuotes.sed -i file.txt
cat file.txt
one's thing
3.2 Mixing quotes AND double quotes
Using dedicated script may simplify mixing quotes and double quotes in same script.
Adding a new operation in our script to enclose the word thing in double quotes:
echo >>sampleSedWithQuotes.sed 's/\bthing\b/"&"/;'
( now our script look like:
#!/bin/sed -f
s/ones/one's/;
s/\bthing\b/"&"/;
)
then
./sampleSedWithQuotes.sed <<<'ones thing'
one's "thing"
The best way is to use $'some string with \' quotes \''
eg:
sed $'s/ones/two\'s/' <<< 'ones thing'
Just use double quotes on the outside of the sed command.
$ sed "s/ones/one's/" <<< 'ones thing'
one's thing
It works with files too.
$ echo 'ones thing' > testfile
$ sed -i "s/ones/one's/" testfile
$ cat testfile
one's thing
If you have single and double quotes inside the string, that's ok too. Just escape the double quotes.
For example, this file contains a string with both single and double quotes. I'll use sed to add a single quote and remove some double quotes.
$ cat testfile
"it's more than ones thing"
$ sed -i "s/\"it's more than ones thing\"/it's more than one's thing/" testfile
$ cat testfile
it's more than one's thing
This is kind of absurd but I couldn't get \' in sed 's/ones/one\'s/' to work. I was looking this up to make a shell script that will automatically add import 'hammerjs'; to my src/main.ts file with Angular.
What I did get to work is this:
apost=\'
sed -i '' '/environments/a\
import '$apost'hammerjs'$apost';' src/main.ts
So for the example above, it would be:
apost=\'
sed 's/ones/one'$apost's/'
I have no idea why \' wouldn't work by itself, but there it is.
Some escapes on AppleMacOSX terminals fail so:
sed 's|ones|one'$(echo -e "\x27")'s|1' <<<'ones thing'
I know this is going to sound like a cop out but I could never get sed working when there were both single and double quotes in the string. To help any newbies like me that are having trouble, one option is to split up the string. I had to replace code in over 100 index.hmtl files. The strings had both single and double quotes so I just split up the string and replaced the first block with
<!-- and the second block with -->. It made a mess of my index.html files but it worked.
use an alternative string seperator like ":" to avoid confusion with different slashes
sed "s:ones:one's:" <<< 'ones thing'
or if you wish to highligh the single quote
sed "s:ones:one\'s:" <<< 'ones thing'
both return
one's thing
This question already has answers here:
Using different delimiters in sed commands and range addresses
(3 answers)
Closed 6 years ago.
I'm trying to replace text in a file with the output of another command. Unfortunately, the outputted text contains characters bash expands. For example, I'm running the following script to change the file (somestring references output that would break the sed command):
#!/bin/bash
somestring='$6$sPnfj/lnXwZVrec7$fCnL9uy1oWIMZduInKTHBAxhsQxGCsBpm2XfVFFqDPHKidrd93yfjbYvKgYexXHVcvkKdu9lbfy16Ek5GvKy/1'
sed '0,/^title/s/^title*/'"$somestring"'\n&/' $HOME/example.txt
sed fails with this error:
sed: -e expression #1, char 30: unknown option to `s'
I think bash is substuting the contents of $somestring when building the sed command, but is then trying to expand the resulting text. I can't put the entire sed script in single quotes, I need bash to expand it the first time, just not the second. Any suggestions? Thanks
here the forward slash / is the problem. If it's the only issue you can set sed to use a different delimiter.
for example
$ somestring="abc/def"; echo xxx | sed 's/xxx/'"$somestring"'/'
sed: -e expression #1, char 11: unknown option to `s'
$ somestring="abc/def"; echo xxx | sed 's_xxx_'"$somestring"'_'
abc/def
you also need to worry about & and \ chars and escape them if can appear in the replacement text.
If you can't control the the replacement string, either you have to sanitize with another sed script or, alternatively use r command to read it from a file. For example,
$ seq 5 | sed -e '/3/{r replace' -e 'd}'
1
2
3slashes///1ampersand&and2backslashes\\end
4
5
where
$ cat replace
3slashes///1ampersand&and2backslashes\\end
You have several errors here:
the string somestring has characters that are significative for sed command (the most important being '/' that you are using as a delimiter) You can escape it, by substituting it with a previous
somestring=$(echo "$somestring" | sed -e 's/\//\\\//g')
that will convert your / chars to \/ sequences.
you are using sed '0,/^title/s/^title*/'"$somestring"'\n&/' $HOME/example.txt which is looking to substitute the string titl followed by any number of e characters by that $somestring value, followed by a new line and the original one. Unfortunately, sed(1) doesn't allow you to use newline characters in the pattern substitution side of the s command, but you can afford the result by using the i command with a text consisting of you pattern (preceding any new line by a \ to interpret it as literal):
Finally the script leads to:
#!/bin/bash
somestring='$6$sPnfj/lnXwZVrec7$fCnL9uy1oWIMZduInKTHBAxhsQxGCsBpm2XfVFFqDPHKidrd93yfjbYvKgYexXHVcvkKdu9lbfy16Ek5GvKy/1'
somestring=$(echo "$somestring" | sed -e 's/\//\\\//g')
sed '/^title/i\
'"$somestring\\
" $HOME/example.txt
If your shell is Bash, you can use parameter substitution to replace the problematic /:
somestring="{somestring//\//\\/}"
That looks scary, but is easier to understand if you look at the version that replaces x with __:
somestring="${somestring//x/__}"
It might be easier to use (say) underscore as the delimiter for your sed s command, and then the substitution above would be
somestring="${somestring//_/\\_}"
If you already have backslashes, you'll need to first replace those:
somestring="${somestring//\\/\\\\}"
somestring="{somestring//\//\\/}"
If there were other characters that needed escaping (e.g. on the search side of s///), then you could extend the above appropriately.
This URL provides the cleanest answer:
Command to escape a string in bash
printf "%q" "$someVariable"
will escape any characters you need escaped for you.
I am trying to convert 2015-06-03_18-05-30 to 20150603180530 using sed.
I have this:
$ var='2015-06-03_18-05-30'
$ echo $var | sed 's/\-\|\_//g'
$ echo $var | sed 's/-|_//g'
None of these are working. Why is the alternation not working?
As long as your script has a #!/bin/bash (or ksh, or zsh) shebang, don't use sed or tr: Your shell can do this built-in without the (comparatively large) overhead of launching any external tool:
var='2015-06-03_18-05-30'
echo "${var//[-_]/}"
That said, if you really want to use sed, the GNU extension -r enables ERE syntax:
$ sed -r -e 's/-|_//g' <<<'2015-06-03_18-05-30'
20150603180530
See http://www.regular-expressions.info/posix.html for a discussion of differences between BRE (default for sed) and ERE. That page notes, in discussing ERE extensions:
Alternation is supported through the usual vertical bar |.
If you want to work on POSIX platforms -- with /bin/sh rather than bash, and no GNU extensions -- then reformulate your regex to use a character class (and, to avoid platform-dependent compatibility issues with echo[1], use printf instead):
printf '%s\n' "$var" | sed 's/[-_]//g'
[1] - See the "APPLICATION USAGE" section of that link, in particular.
Something like this ought to do.
sed 's/[-_]//g'
This reads as:
s: Search
/[-_]/: for any single character matching - or _
//: replace it with nothing
g: and do that for every character in the line
Sed operates on every line by default, so this covers every instance in the file/string.
I know you asked for a solution using sed, but I offer an alternative in tr:
$ var='2015-06-03_18-05-30'
$ echo $var | tr -d '_-'
20150603180530
tr should be a little faster.
Explained:
tr stands for translate and it can be used to replace certain characters with another ones.
-d option stands for delete and it removes the specified characters instead of replacing them.
'_-' specifies the set of characters to be removed (can also be specified as '\-_' but you need to escape the - there because it's considered another option otherwise).
Easy:
sed 's/[-_]//g'
The character class [-_] matches of the characters from the set.
sed 's/[^[:digit:]]//g' YourFile
Could you tell me what failed on echo $var | sed 's/\-\|\_//g', it works here (even if escapping - and _ are not needed and assuming you use a GNU sed due to \| that only work in this enhanced version of sed)
I have a large file that I need to edit in Linux.
the file has data fields enclosed by double quotes ( "" ). But when I open the file using notepad++ I see SOH character between the double quotes (ie. "filed1"SOH"field2"SOHSOH"field3"SOH"field4")
And when I open the same file in vim I see the double quotes followed by ^A character. (ie. "filed1"^A"field2"^A^A"field3"^A"field4")
Then when I execute this command in the command line
cat filename.txt | od -c | more
I see that the character is shown as 001 (ie. "filed1"001"field2"001001"field3"001"field4")
I have tried the following via vim
:s%/\\001//g
I also tried this command
sed -e s/\001//g filename.text > filename_new.txt
sed -e s/\\001//g filename.text > filename_new.txt
I need to remove those characters from that file.
How can I do that?
Your attempts at escaping the SOH character with \001 were close.
GNU sed has an extension to specify a decimal value with \d001 (there are also octal and hexadecimal variants):
$ sed -i -e 's/\d001//g' file.txt
In Vim, the regular expression atom looks slightly different: \%d001; alternatively, you can directly enter the character in the :%s command-line via Ctrl + V followed by 001; cp. :help i_CTRL-V_digit.
Use echo -e to get a literal \001 character into your sed command:
$ sed -i -e $(echo -e 's/\001//g') file.txt
(-i is a GNU sed extension to request in-place editing.)
just keep it simple with awk instead of having to fuss with quotation formatting issues :
mawk NF=NF FS='\1' OFS=
"filed1""field2""field3""field4"
Is it possible to use cut and have unprintable characters be the delimiter? For example I'd like to have the "^A" characters (also represented as \001) be the delimiter.
If you're using Bash,
cut -d $'\001' ...
works (see Bash Reference Manual # 3.1.2.4 ANSI-C Quoting).
Other (more portable) options,
cut -d `echo -e '\001'` ...
FS=`echo -e '\001'`
cut -d $FS ...
or inserting the control character directly using ^V as mentioned by Alnitak and etlerant -- on the shell command line, and in editors such as vi, this means "don't treat the next thing I type specially".
Yes, it's perfectly possible.
If typing in a shell, press ^V and then ^A to insert the ^A verbatim in the current line rather than have it treated as the normal 'go to start of line' command:
% cat -v foo
abc^Adef^Aghi
% cut -d^A -f2 foo
def
If for example you unprintable delimiter is tab which is equivalent of \t and you want to find the second to the end item of each line separated by tab you can use this:
cut -d $'\t' -f2- tablimited.csv
CTRL-V CTRL-A ?