Im trying to understand sed command and loops.
I need to take part of the text (20 lines) and append it to the csv with the file name.
here is my code
for i in ~/mspl/*.doc
do
catdoc "$i" > auto.txt
sed -e '1,20!d' auto.txt > auto1.txt
sed -e '1s/^/"$i" ;/' auto1.txt > auto2.txt
sed -e '20s/$/~/' auto2.txt > auto3.txt
cat auto3.txt >> lines.csv
done
the problem is that my second "i" argument doesn't convert to the file name in csv.
in the line
sed -e '1s/^/"$i" ;/' auto1.txt > auto2.txt
Please tell what is my mistake here?
The issue is that variables are not expanded within single quotes '. Use double quotes instead ":
# this is assuming you did want to add explicit " around your $i
sed -e "1s/^/\"$i\" ;/" auto1.txt > auto2.txt
If your double-quoted string needs to contain explicit double quotes, you have to escape them (\"). If using double quotes is not an option (e.g. if you have a very complex command with multiple levels of quoting already) you can always exit the quoting around your variable:
sed -e '1s/^/"'$i'" ;/' auto1.txt > auto2.txt
You might find it useful to run this script with the -x setting, to see what's actually going on. (Just do set -x.)
But just to be blunt, what's going on is that your $i is inside single quotes, so it doesn't get expanded.
Related
I am using the below code for replacing a string
inside a shell script.
echo $LINE | sed -e 's/12345678/"$replace"/g'
but it's getting replaced with $replace instead of the value of that variable.
Could anybody tell what went wrong?
If you want to interpret $replace, you should not use single quotes since they prevent variable substitution.
Try:
echo $LINE | sed -e "s/12345678/${replace}/g"
Transcript:
pax> export replace=987654321
pax> echo X123456789X | sed "s/123456789/${replace}/"
X987654321X
pax> _
Just be careful to ensure that ${replace} doesn't have any characters of significance to sed (like / for instance) since it will cause confusion unless escaped. But if, as you say, you're replacing one number with another, that shouldn't be a problem.
you can use the shell (bash/ksh).
$ var="12345678abc"
$ replace="test"
$ echo ${var//12345678/$replace}
testabc
Not specific to the question, but for folks who need the same kind of functionality expanded for clarity from previous answers:
# create some variables
str="someFileName.foo"
find=".foo"
replace=".bar"
# notice the the str isn't prefixed with $
# this is just how this feature works :/
result=${str//$find/$replace}
echo $result
# result is: someFileName.bar
str="someFileName.sally"
find=".foo"
replace=".bar"
result=${str//$find/$replace}
echo $result
# result is: someFileName.sally because ".foo" was not found
Found a graceful solution.
echo ${LINE//12345678/$replace}
Single quotes are very strong. Once inside, there's nothing you can do to invoke variable substitution, until you leave. Use double quotes instead:
echo $LINE | sed -e "s/12345678/$replace/g"
Let me give you two examples.
Using sed:
#!/bin/bash
LINE="12345678HI"
replace="Hello"
echo $LINE | sed -e "s/12345678/$replace/g"
Without Using sed:
LINE="12345678HI"
str_to_replace="12345678"
replace_str="Hello"
result=${str//$str_to_replace/$replace_str}
echo $result
Hope you will find it helpful!
echo $LINE | sed -e 's/12345678/'$replace'/g'
you can still use single quotes, but you have to "open" them when you want the variable expanded at the right place. otherwise the string is taken "literally" (as #paxdiablo correctly stated, his answer is correct as well)
To let your shell expand the variable, you need to use double-quotes like
sed -i "s#12345678#$replace#g" file.txt
This will break if $replace contain special sed characters (#, \). But you can preprocess $replace to quote them:
replace_quoted=$(printf '%s' "$replace" | sed 's/[#\]/\\\0/g')
sed -i "s#12345678#$replace_quoted#g" file.txt
I had a similar requirement to this but my replace var contained an ampersand. Escaping the ampersand like this solved my problem:
replace="salt & pepper"
echo "pass the salt" | sed "s/salt/${replace/&/\&}/g"
use # if you want to replace things like /. $ etc.
result=$(echo $str | sed "s#$oldstr#$newstr#g")
the above code will replace all occurrences of the specified replacement term
if you want, remove the ending g which means that the only first occurrence will be replaced.
Use this instead
echo $LINE | sed -e 's/12345678/$replace/g'
this works for me just simply remove the quotes
I prefer to use double quotes , as single quptes are very powerful as we used them if dont able to change anything inside it or can invoke the variable substituion .
so use double quotes instaed.
echo $LINE | sed -e "s/12345678/$replace/g"
I have a sed command in my makefile like following ,purpose of the command is, it should replace string "listen $address" to "",but as a result it is replacing "listen $address" to "$address"
sed -i -e "s/listen $$address//" file.txt
Please suggest any solution.
I have checked many posts but there $ is used as regex variable ,in my case it needs to be treated as hardcoded string
Enclose your sed command in single quotes to avoid variable expansion within double quotes:
sed -e -i 's/listen $address//' file
You can use double quotes though by escaping the $:
sed -e -i "s/listen \$address//" file
I am new to shell scripting. I am using ksh.
I have this particular line in my script which I use to append text in a variable q to the end of a particular line given by the variable a
containing the line number .
sed -i ''$a's#$#'"$q"'#' test.txt
Now the variable q can contain a large amount of text, with all sorts of special characters, such as !##$%^&*()_+:"<>.,/;'[]= etc etc, no exceptions. For now, I use a couple of sed commands in my script to remove any ' and " in this text (sed "s/'/ /g" | sed 's/"/ /g'), but still when I execute the above command I get the following error
sed: -e expression #1, char 168: unterminated `s' command
Any sed, awk, perl, suggestions are very much appreciated
The difficulty here is to quote (escape) the substitution separator characters # in the sed command:
sed -i ''$a's#$#'"$q"'#' test.txt
For example, if q contains # it will not work. The # will terminate the replacement pattern prematurely. Example: q='a#b', a=2, and the command expands to
sed -i 2s#$#a#b# test.txt
which will not append a#b to the end of line 2, but rather a#.
This can be solved by escaping the # characters in q:
sed -i 2s#$#a\#b# test.txt
However, this escaping could be cumbersome to do in shell.
Another approach is to use another level of indirection. Here is an example of using a Perl one-liner. First q is passed to the script in quoted form. Then, within the script the variable assigned to a new internal variable $q. Using this approach there is no need to escape the substitution separator characters:
perl -pi -E 'BEGIN {$q = shift; $a = shift} s/$/$q/ if $. == $a' "$q" "$a" test.txt
Do not bother trying to sanitize the string. Just put it in a file, and use sed's r command to read it in:
echo "$q" > tmpfile
sed -i -e ${a}rtmpfile test.txt
Ah, but that creates an extra newline that you don't want. You can remove it with:
sed -e ${a}rtmpfile test.txt | awk 'NR=='$a'{printf $0; next}1' > output
Another approach is to use the patch utility if present in your system.
patch test.txt <<-EOF
${a}c
$(sed "${a}q;d" test.txt)$q
.
EOF
${a}c will be replaced with the line number followed by c which means the operation is a change in line ${a}.
The second line is the replacement of the change. This is the concatenated value of the original text and the added text.
The sole . means execute the commands.
I want to add the string %%% to the beginning of some specific lines in a text file.
This is my script:
#!/bin/bash
a="c:\Temp"
sed "s/$a/%%%$a/g" <File.txt
And this is my File.txt content:
d:\Temp
c:\Temp
e:\Temp
But nothing changes when I execute it.
I think the 'sed' command is not finding the pattern, possibly due to the \ backslashes in the variable a.
I can find the c:\Temp line if I use grep with -F option (to not interpret strings):
cat File.txt | grep -F "$a"
But sed seems not to implement such '-F` option.
Not working neither:
sed 's/$a/%%%$a/g' <File.txt
sed 's/"$a"/%%%"$a"/g' <File.txt
I have found similar threads about replacing with sed, but they don't refer to variables.
How can I replace the desired lines by using a variable adding them the %%% char string?
EDIT: It would be fine that the $a variable could be entered via parameter when calling the script, so it will be assigned like:
a=$1
Try it like this:
#!/bin/sh
a='c:\\Temp' # single quotes
sed "s/$a/%%%$a/g" <File.txt # double quotes
Output:
Johns-MacBook-Pro:sed jcreasey$ sh x.sh
d:\Temp
e:\Temp
%%%c:\Temp
You need the double slash '\' to escape the '\'.
The single quotes won't expand the variables.
So you escape the slash in single quotes and pass it into the double quotes.
Of course you could also just do this:
#!/bin/sh
sed 's/\(.*Temp\)/%%%&/' <File.txt
If you want to get input from the command line you have to allow for the fact that \ is an escape character there too. So the user needs to type 'c:\\' or the interpreter will just wait for another character. Then once you get it, you will need to escape it again. (printf %q).
#!/bin/sh
b=`printf "%q" $1`
sed "s/\($b\)/%%% &/" < File.txt
The issue you are having has to do with substitution of your variable providing a regular expression looking for a literal c:Temp with the \ interpreted as an escape by the shell. There are a number of workarounds. Seeing the comments and having worked through the possibilities, the following will allow the unquoted entry of the search term:
#!/bin/bash
## validate that needed input is given on the command line
[ -n "$1" -a "$2" ] || {
printf "Error: insufficient input. Usage: %s <term> <file>\n" "${0//*\//}" >&2
exit 1
}
## validate that the filename given is readable
[ -r "$2" ] || {
printf "Error: file not readable '%s'\n" "$2" >&2
exit 1
}
a="$1" # assign a
filenm="$2" # assign filename
## test and fix the search term entered
[[ "$a" =~ '/' ]] || a="${a/:/:\\}" # test if \ removed by shell, if so replace
a="${a/\\/\\\\}" # add second \
sed -e "s/$a/%%%$a/g" "$filenm" # call sed with output to stdout
Usage:
$ bash sedwinpath.sh c:\Temp dat/winpath.txt
d:\Temp
%%%c:\Temp
e:\Temp
Note: This allows both single-quoted or unquoted entry of the dos path search term. To edit in place use sed -i. Additionally, the [[ operator and =~ operator are limited to bash.
I could have sworn the original question said replace, but to append, just as you suggest in the comments. I have updated the code with:
sed -e "s/$a/%%%$a/g" "$filenm"
Which provides the new output:
$ bash sedwinpath.sh c:\Temp dat/winpath.txt
d:\Temp
%%%c:\Temp
e:\Temp
Remember: If you want to edit the file in place use sed -i or sed -i.bak which will edit the actual file (and if -i.bak is given create a backup of the original in originalname.bak). Let me know if that is not what you intended and I'm happy to edit again.
Creating your script with a positional parameter of $1
#!/bin/bash
a="$1"
cat <file path>|sed "s/"$1"/%%%"$1"/g" > "temporary file"
Now whenever you want sed to find "c:\Temp" you need to use your script command line as follows
bash <my executing script> c:\\\\Temp
The first backslash will make bash interpret any backslashes that follows therefore what will be save in variable "a" in your executing script is "c:\\Temp". Now substituting this variable in sed will cause sed to interpret 1 backlash since the first backslash in this variable will cause sed to start interpreting the other backlash.
when you Open your temporary file you will see:
d:\Temp
%%%c:\Temp
e:\Temp
Hi I have following line in shellscript
sed 's_$org_$repl_g' $i > $temp_file
In this $org denotes the name to be change and $repl denotes replacement. I have done echo for both and both are write. $i represent the file name. when I echo below
echo $(sed "s/$org/$repl_g" $i)
Then also it does not replace the word . while when I try this with terminal directly as below
sed 's_Dilip_Agarwal_g' test.txt
then it give correct output by replacing the original one.
Can any body help me where I am wrong in this.
Thanks
Do not use command substitution on it. And use double quotes instead of single quotes. Single quotes do not expand parameters. You also have to fix your parameter expansion. _ is also a valid parameter character so you need to use braces to isolate the only characters that would identify the parameters.
sed "s_${org}_${repl}_g" "$i" > "$temp_file"
You can also just use another delimeter that's not a parameter character:
sed "s|$org|$repl|g" "$i" > "$temp_file"