How to replace a string between two commas in linux using sed - linux

execute PKG_SP_MAINTENANCE.MoveAccount(91, 129031, 958408630); Lowes
From the above statement I am trying to get the content between the first comma and second comma i.e., 129031 and replace it with a new string which is passed as a parameter to the script. For now let's replace with N . I tried the following sed command ended up getting an error. Could someone please help?
04:24:01 Tue Sep 19 [serviceb#LQASRDSBMST002V:~/isops/tmp] cat Navya | sed 's/,^.\{*\},/N/'
sed: -e expression #1, char 14: Invalid content of \{\}

$: echo "start,middle,end" | sed 's/,[^,]*,/,NEW,/g'
start,NEW,end
Is this what you mean? This simply matches the inner-most commas and replaces the text.
Depending how you want to handle strings with more than two commas, you could do something like this to match the outer-most instead:
$: echo "start,middle,end" | sed 's/,.*,/,NEW,/g'
start,NEW,end

Related

Using SED to replace long string - but got unterminated substitute in regular expression

hi trying to replace the following string with a long one :
#x#
with string that I got from the command line:
read test
sed -i --backup 's/#x#/'${test}'/g' file.json README.md
but it is working only for 1 word, it is not working if there is space between word . even between quotes
sed: 1: "s/#x#/string test string: unterminated substitute in regular expression
if case you run it on MacOS and struggling with "unterminated substitute in regular expression", there is an easier explanation for this:
MacOS has slightly other version of sed than usually is on linux. -i requires a parameter. If you have none, just add "" after -i
sed -i "" --backup 's/#x#/'${test}'/g' file.json README.md
or for example if you just have to delete dome line, this works on linux, but brings “invalid command code” on MacOS
sed -i 39d filenamehere.log
and this works on MacOS
sed -i "" 39d filenamehere.log
The problem originates from the way you are using the single-quotes. Currently you are terminating your input behind the 2. single-quote. See the Error message, it makes you aware of the fact that it is missing something.
If you have a file with the following content:
foo #x# foo
Than you can replace the content e.g. with the following command:
sed 's/#x#/bar foo bar/' foo.txt > foo2.txt
And get:
foo bar foo bar foo
If you need to pass in a variable the comment from Gordon Davisson shows you the right way.
By the way, if you want to use the inplace option, on my linux you would need to use the command like this:
sed -i.old "s/#x#/${test}/" foo.txt
But I think this might depends on your enviroment (mac?).
sed doesn't understand strings where a string is a series of literal characters. It replaces a regexp (not a string) with a backreference-enabled "string" (also not a string) all within a set of delimiters (which ALSO require careful handling in both the regexp and the replacement). See Is it possible to escape regex metacharacters reliably with sed for more info.
To replace a string with another string the simplest approach is to just use a tool that understands strings such as awk:
$ cat file
before stuff
foo #x# bar
after stuff
$ cat tst.awk
BEGIN {
old = ARGV[1]
new = ARGV[2]
ARGV[1] = ARGV[2] = ""
}
s = index($0,old) { $0 = substr($0,1,s-1) new substr($0,s+length(old)) }
{ print }
$ test='a/\t/&"b'
$ awk -f tst.awk '#x#' "$test" file
before stuff
foo a/\t/&"b bar
after stuff
The above will work no matter what characters test contains, even newlines:
$ test='contains a
newline'
$ awk -f tst.awk '#x#' "$test" file
before stuff
foo contains a
newline bar
after stuff

Remove path prefix of space separated paths

Given a list of paths separated by a single space:
/home/me/src/test /home/me/src/vendor/a /home/me/src/vendor/b
I want to remove the prefix /home/me/src/ so that the result is:
test vendor/a vendor/b
For a single path I would do: ${PATH#/home/me/src/} but how do I apply it to this series?
You can use // to replace all occurrences of substring. Replace it with null string to remove them.
$ path="/home/me/src/test /home/me/src/vendor/a /home/me/src/vendor/b"
$ echo ${path//\/home\/me\/src\/}
test vendor/a vendor/b
Reference: ${parameter/pattern/string} in Bash reference manual
Using shell parameter expansion doesn't seem to be the solution for this, since it would remove everything up to / from a given point is useful, as nu11p01n73R's answer reveals.
For clarity, I would use sed with the syntax sed 's#pattern#replacement#g':
$ str="/home/me/src/test /home/me/src/vendor/a /home/me/src/vendor/b"
$ sed 's#/home/me/src/##g' <<< "$str"
test vendor/a vendor/b
Like always a grep solution from my side :
echo 'your string' | grep -Po '^/([^ /]*/)+\K.+'
Please note that the above regex do this for any string like /x/y/z/test ... But if you are interested only in replacing /home/me/src/, try the following :
echo 'your string' | grep -Po '^/home/me/src/\K.+' --color

Two pattern match on same sed command

I have the following sed command:
sed -n '/^out(/{n;p}' ${filename} | sed -n '/format/ s/.*format=//g; s/),$//gp; s/))$//gp'
I tried to do it as one line as in:
sed -n '/^out(/{n;}; /format/ s/.*format=//g; s/),$//gp; s/))$//gp' ${filename}
But that also display the lines I don't want (those that do not match).
What I have is a file with some strings as in:
entry(variable=value)),
format(variable=value)),
entry(variable=value)))
out(variable=value)),
format(variable=value)),
...
I just want the format lines that came right after the out entry. and remove those trailing )) or ),
You can use this sed command:
sed -nr '/^out[(]/ {n ; s/.*[(]([^)]+)[)].*/\1/p}' your_file
Once a out is found, it advanced to the next line (n) and uses the s command with p flag to extract only what is inside parenthesises.
Explanation:
I used [(] instead of \(. Outside brackets a ( usually means grouping, if you want a literal (, you need to escape it as \( or you can put it inside brackets. Most RE special characters dont need escaping when put inside brackets.
([^)]+) means a group (the "(" here are RE metacharacters not literal parenthesis) that consists of one or more (+) characters that are not (^) ) (literal closing parenthesis), the ^ inverts the character class [ ... ]

bash string manipulation - Display the value, not the variable name

I'm writing a script to process inbound data files. The inbound file names all follow the same pattern:
word1_word2_word3_YYYYMMDD.txt
My script takes the name of the inbound file, strips the file extension, strips out the date, replaces all underscores with spaces and appends the resulting string to each line in the original file. I can succesfully create the desired string and have assigned it to a variable "STR"
The last step is to append the value of $STR to each line in the file so that the data lines within the file end up looking like this:
casenumber1"|"word1 word2 word3
casenumber2"|"word1 word2 word3
casenumber3"|"word1 word2 word3
My problem is that for the life of me I cannot get bash to display the variable value, it always displays the variable name.
This is the line I use to create the string needed from the file name:
STR=`echo $DATAFILENAME | cut -d '.' -f 1 | sed 's/[0-9]*//g'|sed 's/_/ /g' | sed 's/[[:blank:]]*$//'`
I'm trying to use a typical sed replace command:
sed 's/$/`echo "$STR"`/g' inputfile > outputfile
But keep getting the variable name instead of the variable value:
example output:
1000056|$"STR"
1000057|$"STR"
...
desired output:
1000056|Closed With Notification
1000057|Closed With Notification
What am I doing wrong? Thanks, Vic
The gist of your question is that you need to add a string to a file using sed and the value of that string is contained in a variable, which you call "a", as we read in the final list.
Then you need use this combination, which is missing from your list above:
sed "s/$/| $a/g" $DATAFILE > datfile99
The problem is that the single quotes around your command prevent the interpolation of the variable $a.
If you wrap the command in double quotes the whole string will be passed to sed after that the shell replaces $a to its current value.
Try replacing your ' with " this will tell your shell to substitute any shell variables
sed -i "s/$/echo $STR/g"
Note -i option will make actual changes to your file, hence it is wise to backup.
EDIT: instead of using this
STR=`echo $DATAFILENAME | cut -d '.' -f 1 | sed 's/[0-9]*//g'|sed 's/_/ /g' | sed 's/[[:blank:]]*$//'`
You can try this
sed -i -r "s/(.*)[.][a-zA-Z]+$/\\1/g;s/[._]/ /g" <<< "$DATAFILENAME"

SED: How to insert string to the beginning of the last line

How to insert string to the beginning of the last line?
I want to add a time stamp to a text file which contains multiple lines
var1 = `date`
LINE1
LINE2
LINE3
...
(INSERT var1 here) LASTLINE
sed 's/^/test line /g' textfile inserts characters to the beginning of every line but how can I specifically modify the last line only?
Thanks
Going forward:
sed '$s/^/sample text /' textfile works, but only when inserting regular strings. If I try
var1 = "sample text"
and use substition, here are the problems I encounter
using single quotes in sed does not expand variables, so sed '$s/^/$var1/' textfile will insert the string $var1 into the beginning of the last line.
To enable variable substitution I tried using double quotes. It works when I specify the exact line number. something like:
sed "5s/^/$var1/" textfile
But when I try sed "$s/^/$var1" text file, it returns an error:
sed: -e expression #1, char 5: extra characters after command
Can someone help me please?
Like this:
sed '$s/^/test line /' textfile
$ indicates last line. Similarly, you can insert into a any specific line by putting the line number in place of $
But when I try sed "$s/^/$var1" text file, it returns an error:
It returns an error because the shell attempts to expand $s since you've used double quotes. You need to escape the $ in $s.
sed "\$s/^/$var1/" filename
sedshould be the best tool, but awk can do this too:
awk '{a[++t]=$0} END {for (i=1;i<t;i++) print a[i];print v$0}' v="$var1" file
It will insert value of var1 in front of last line
Another variation
awk 'NR==f {$0=v$0}1' v="$var1" f=$(wc -l file)
PS you do not need to specify file after awk, not sure why. If you do so, it reads it double.
This command would work for you:
sed -i "5s/^/$var1 /" text file

Resources