I looked at a very similar question but was unable to resolve the issue Replace comma with newline in sed
I am trying to convert : characters in a string. This is what I tried:
echo -e 'this:is:a:test' | sed "s/\:/'\n'/g"
but this replaces : with n. I tried tr too but had the same result. I believe the -e is not seen after being piped so new line is not recognized.
Any help is appreciated.
echo 'this:is:a:test' | tr : \\n
Any POSIX-compliant tr will support the \n escape sequence. You need to take care to quote or escape the escape sequence, however (double backslash above).
The -e argument to echo has no effect on your argument to echo.
I'll presume that you have the string in a variable already. This uses the parameter expansion substitution operator to replace every : with a newline, which is specified using a $'...'-quoted string. Both features are bash extensions to the standard, and may not work in another shell.
$ foo="this:is:a:test"
$ bar="${foo//:/$'\n'}"
$ echo "$bar"
this
is
a
test
Perhaps Perl is an option?
echo -e 'this:is:a:test' | perl -p -e 's/:/\n/g'
You do not need echo -e because you have \n in sed, not in echo statement.
So, the following should work (note that I have changed '\n' to \n):
echo -e 'this:is:a:test' | sed "s/\:/\n/g"
or
echo 'this:is:a:test' | sed "s/\:/\n/g"
Also note that you do not need to escape : so the following will work too (thanks to #anishsane)
echo 'this:is:a:test' | sed "s/:/\n/g"
Below is just to reiterate why you need -e for echo
$ echo -e "hello \n"
hello
$ echo "hello \n"
hello \n
If Python is an option:
echo 'this:is:a:test' | python3 -c 'print(input().replace(":", "\n"))'
output:
this
is
a
test
Related
Here are my attempts to replace a b character with a newline using sed while running bash
$> echo 'abc' | sed 's/b/\n/'
anc
no, that's not it
$> echo 'abc' | sed 's/b/\\n/'
a\nc
no, that's not it either. The output I want is
a
c
HELP!
Looks like you are on BSD or Solaris. Try this:
[jaypal:~/Temp] echo 'abc' | sed 's/b/\
> /'
a
c
Add a black slash and hit enter and complete your sed statement.
$ echo 'abc' | sed 's/b/\'$'\n''/'
a
c
In Bash, $'\n' expands to a single quoted newline character (see "QUOTING" section of man bash). The three strings are concatenated before being passed into sed as an argument. Sed requires that the newline character be escaped, hence the first backslash in the code I pasted.
You didn't say you want to globally replace all b. If yes, you want tr instead:
$ echo abcbd | tr b $'\n'
a
c
d
Works for me on Solaris 5.8 and bash 2.03
In a multiline file I had to pipe through tr on both sides of sed, like so:
echo "$FILE_CONTENTS" | \
tr '\n' ¥ | tr ' ' ∑ | mySedFunction $1 | tr ¥ '\n' | tr ∑ ' '
See unix likes to strip out newlines and extra leading spaces and all sorts of things, because I guess that seemed like the thing to do at the time when it was made back in the 1900s. Anyway, this method I show above solves the problem 100%. Wish I would have seen someone post this somewhere because it would have saved me about three hours of my life.
echo 'abc' | sed 's/b/\'\n'/'
you are missing '' around \n
trying to grep "#‘om" but not able to escape or account for the quote char. I tried grep -F, grep -e, grep -n or simply grep "#\‘om" to no avail.
That quote is not the simple quote character it appears to be. It's not clear whether copying-and-pasting the quote character from this website is accurate.
$ echo '‘' | cat -v
M-bM-^#M-^X
$ echo '‘' | xxd
$ 00000000: e280 980a ....
So, it appears the problem is one of character sets.
Note, however, that the following works for me:
$ echo '‘' | grep -F '‘'
‘
As does the following:
$ echo '#‘om' | grep -F '#‘om'
#‘om
It would help to see exactly what is being tried. Perhaps use xxd to confirm what bytes are making up that quote.
I want sed to give me a single line output irrespective of whether the matched pattern is found and substituted, or even if there is no pattern match, with same command options.
1. echo "700K" | sed -n 's/[A-Z]//gp' // gives one output
2. echo "700" | sed -n 's/[A-Z]//gp' // no output
Is there any way in sed i can get a single output for second case without removing the "-n" option, forcing it to print the input irrespective of substitution made or not?
It is not clear for me why you need to keep the -n option but if you really do need to keep it you can use the following sed command:
echo "700" | sed -n 's/[A-Z]//g;p'
this will first make the substitution if possible then print the line.
output:
You don't need to mess with all these sed options. Use sed in it's simpliest format which will make a substitution if pattern is found:
$ echo "700K" | sed 's/[A-Z]//g'
700
$ echo "700" | sed 's/[A-Z]//g'
700
$ sed --version
sed (GNU sed) 4.4
$ sed 's/[A-Z]//g' <<<$'700\n700K\n500\n3500A'
700
700
500
3500
Can this usage of xargs argument enumaration be optimized better?
The aim is to inject single argument in the middle of the actual command.
I do:
echo {1..3} | xargs -I{} sh -c 'for i in {};do echo line $i here;done'
or
echo {1..3} | for i in $(xargs -n1);do echo line $i here; done
I get:
line 1 here
line 2 here
line 3 here
which is what I need but I wondered if loop and temporary variable could be avoided?
You need to separate the input to xargs by newlines:
echo {1..3}$'\n' | xargs -I% echo line % here
For array expansions, you can use printf:
ar=({1..3})
printf '%s\n' "${ar[#]}" | xargs -I% echo line % here
(and if it's just for output, you can use it without xargs:
printf 'line %s here\n' "${ar[#]}"
)
Try without xargs. For most situations xargs is overkill.
Depending on what you really want you can choose a solution like
# Normally you want to avoid for and use while, but here you want the things splitted.
for i in $(echo {1 2 3} );do
echo line $i here;
done
# When you want 1 line turned into three, `tr` can help
echo {1..3} | tr " " "\n" | sed 's/.*/line & here/'
# printf will repeat itself when there are parameters left
printf "line %s here\n" $(echo {1..3})
# Using the printf feature you can avoid the echo
printf "line %s here\n" {1..3}
Maybe this?
echo {1..3} | tr " " "\n" | xargs -n1 sh -c ' echo "line $0 here"'
The tr replaces the spaces with newlines, so xargs sees three lines. I would not be surprised if there were a better (more efficient) solution, but this one is quite simple.
Please note I have modified my previous answer to remove the use of {}, which was suggested in the comments to eliminate a potential code injection vulnerability.
There is a not well known feature of GNU sed. You can add the e flag to the s command and then sed executes whatever is in the pattern space and replaces the pattern space with the output if that command.
If you are really only interested in the output of the echo commands, you might try this GNU sed example, which eliminates the temporary variable, the loop (and the xargs as well):
echo {1..3} | sed -r 's/([^ ])+/echo "line \1 here"\n/ge
it fetches one token (i.e. whatever is separated by the spaces)
replaces it with echo "line \1 here"\n command, with \1 replaced by the token
then executes echo
puts the output of the echo command back into pattern space
that means it outputs the result of the three echos
But an even better way to get the desired output is to skip the execution and do the transformation directly in sed, like this:
echo {1..3} | sed -r 's/([^ ])+ ?/line \1 here\n/g'
I need some assistance trying to build up a variable using a list of exclusions in a file.
So I have a exclude file I am using for rsync that looks like this:
*.log
*.out
*.csv
logs
shared
tracing
jdk*
8.6_Code
rpsupport
dbarchive
inarchive
comms
PR116PICL
**/lost+found*/
dlxwhsr*
regression
tmp
working
investigation
Investigation
dcsserver_weblogic_
dcswebrdtEAR_weblogic_
I need to build up a string to be used as a variable to feed into egrep -v, so that I can use the same exclusion list for rsync as I do when egrep -v from a find -ls.
So I have created this so far to remove all "*" and "/" - and then when it sees certain special characters it escapes them:
cat exclude-list.supt | while read line
do
echo $line | sed 's/\*//g' | sed 's/\///g' | 's/\([.-+_]\)/\\\1/g'
What I need the ouput too look like is this and then export that as a variable:
SEXCLUDE_supt="\.log|\.out|\.csv|logs|shared|PR116PICL|tracing|lost\+found|jdk|8\.6\_Code|rpsupport|dbarchive|inarchive|comms|dlxwhsr|regression|tmp|working|investigation|Investigation|dcsserver\_weblogic\_|dcswebrdtEAR\_weblogic\_"
Can anyone help?
A few issues with the following:
cat exclude-list.supt | while read line
do
echo $line | sed 's/\*//g' | sed 's/\///g' | 's/\([.-+_]\)/\\\1/g'
Sed reads files line by line so cat | while read line;do echo $line | sed is completely redundant also sed can do multiple substitutions by either passing them as a comma separated list or using the -e option so piping to sed three times is two too many. A problem with '[.-+_]' is the - is between . and + so it's interpreted as a range .-+ when using - inside a character class put it at the end beginning or end to lose this meaning like [._+-].
A much better way:
$ sed -e 's/[*/]//g' -e 's/\([._+-]\)/\\\1/g' file
\.log
\.out
\.csv
logs
shared
tracing
jdk
8\.6\_Code
rpsupport
dbarchive
inarchive
comms
PR116PICL
lost\+found
dlxwhsr
regression
tmp
working
investigation
Investigation
dcsserver\_weblogic\_
dcswebrdtEAR\_weblogic\_
Now we can pipe through tr '\n' '|' to replace the newlines with pipes for the alternation ready for egrep:
$ sed -e 's/[*/]//g' -e 's/\([._+-]\)/\\\1/g' file | tr "\n" "|"
\.log|\.out|\.csv|logs|shared|tracing|jdk|8\.6\_Code|rpsupport|dbarchive|...
$ EXCLUDE=$(sed -e 's/[*/]//g' -e 's/\([._+-]\)/\\\1/g' file | tr "\n" "|")
$ echo $EXCLUDE
\.log|\.out|\.csv|logs|shared|tracing|jdk|8\.6\_Code|rpsupport|dbarchive|...
Note: If your file ends with a newline character you will want to remove the final trailing |, try sed 's/\(.*\)|/\1/'.
This might work for you (GNU sed):
SEXCLUDE_supt=$(sed '1h;1!H;$!d;g;s/[*\/]//g;s/\([.-+_]\)/\\\1/g;s/\n/|/g' file)
This should work but I guess there are better solutions. First store everything in a bash array:
SEXCLUDE_supt=$( sed -e 's/\*//g' -e 's/\///g' -e 's/\([.-+_]\)/\\\1/g' exclude-list.supt)
and then process it again to substitute white space:
SEXCLUDE_supt=$(echo $SEXCLUDE_supt |sed 's/\s/|/g')