I want to make a sed command to change strings that each comma separated part will be in double quotes. The problem is that some values have already double quotes.
Change from:
ABZ00016,ABZ,"449,9",450,445,449,"-0,21",405,13,"182,15",0,0,0
to
"ABZ00016","ABZ","449,9","450","445","449","-0,21","405","13","182,15","0","0","0"
I prepared two sed commands:
First command excludes values with double quotes
sed -e 's/"[^"]*"//g'
Second adds double quotes to each part
sed -e 's/\([^,]*\),/"\1",/g'
And now I wanted to exclude results from first command and make changes using second command:
sed -e '/"[^"]*"/!s/\([^,]*\),/"\1",/g'
But it doesn't work...
Using gnu-awk you can do this:
awk -v FPAT='"[^"]*"|[^,]*' -v OFS=, '{
for(i=1; i<=NF; i++) {
gsub(/^"|"$/, "", $i)
$i = "\"" $i "\""
}
} 1' file
Output:
"ABZ00016","ABZ","449,9","450","445","449","-0,21","405","13","182,15","0","0","0"
This might work for you (GNU sed):
sed -r 's/^/\n/;:a;s/\n$//;s/\n("[^"]*",?)/\1\n/;s/\n([^,]*)(,?)/"\1"\2\n/;ta' file
Introduce a newline as a marker then: remove the marker when all fields have been processed, skip over quoted fields and surround other fields with quotes. With each substitution advance the marker down the line.
Use sed with regex mate:
sed -r -e 's/[-.a-zA-Z0-9]+/"&"/g' -e 's/""/"/g'
Edit: Updated to Your request, works as expected:
sed -r -e 's/"([^,]+)(,)([0-9]+)"/\1\.\3/g' -e 's/[-.a-zA-Z0-9]+/"&"/g' -e 's/""/"/g' -e 's/\./,/g'
Related
Need help in making a sed script to find and replace user input along with single quotes. Input file admins.py:
Script:
read adminsid
while [[ $adminsid == "" ]];
do
echo "You did not enter anything. Please re-enter AdminID"
read adminsid
done
## Please enter Admin's ID
9999999999,8888888888,1111111111
## Script To Replace ADMIN_IDS = [] to ADMIN_IDS = ['9999999999,8888888888,1111111111'] in file
sed -i "s|ADMIN_IDS = \[.*\]|ADMIN_IDS = ['$adminsid']|g" $file
## Current results:
ADMIN_IDS = ['9999999999,8888888888,1111111111']
## Expected results:
ADMIN_IDS = ['9999999999','8888888888','1111111111']
Assign the variable to the data
adminsid=9999999999,8888888888,1111111111
Then use sed -e (script) option to add the quoting, and square brackets.
echo "$adminsid" | sed -e "s/,/\',\'/g" -e "s/^/[\'/" -e "s/$/\']/"
or to apply changes to a file (filename in $file):
sed -i "$file" -e "s/,/\',\'/g" -e "s/^/[\'/" -e "s/$/\']/"
You can do this with awk too:
Suppose you have assigned the variable as :
adminsid=9999999999,8888888888,1111111111
Then the solution:
echo "$adminsid"| awk -F"," -v quote="'" -v OFS="','" '$1=$1 {print "["quote $0 quote"]"}'
-F"," -v OFS="','" :: Replacing separator (,) with (',')
print "["quote $0 quote"]" :: Add single quotes(') and ([) and (]) to the begin and end of line
This might work for you (GNU sed & bash):
<<<"$adminsid" sed 's/[^,]\+/'\''&'\''/g;s/.*/[&]/'
Surround all non-comma characters by single quotes and then surround the entire string by square brackets.
Replace the , with ',' in the variable and add characters at the beginning and at the end.
sed "s/.*/['&']/" <<< "${adminsid//,/','}"
echo "('${adminsid//,/\\',\\'}')"
I'm trying to use a list of patterns to search in 4 large files, and remove the line that contains the regex.
I tried to specify the file path but it didn't work
sed -n '/{home/dirco/shut}/p' rimco rimco2 aval aval2
I tried to use sed option -f but it didn't work either
sed -f home/dirco/shut rimco rimco2 aval aval2
ultimately the goal will be to sed in place by removing that line if the pattern is found.
This might work for you (GNU sed):
sed 's#/#\\/#g;s#.*#/&/p#g' patternFile | sed -nf - file1 file2 file3 ...
Turn the patternFile into a sed script and run it against the data files.
N.B. The sed delimiter / is first quoted and the each line of the patternFile is turned into an address which is printed /pattern/p.
try this:
cmd=$(
echo -n "sed -i '"
while read -r line; do
echo -n "/$line/d;"
done < patternfile.txt
echo "'"
)
"$cmd" rimco rimco2 aval aval2
Here's how to do what you want efficiently and robustly by using GNU awk for inplace editing (assuming your list of regexps in regexpsfile isn't massive):
awk -i inplace 'NR==FNR{re=re sep "(" $0 ")"; sep="|"} NR!=FNR && $0~re{next} 1' regexpsfile rimco rimco2 aval aval2
I have a text file which contains some lines as the following:
ASDASD2W 3ASGDD12 SDADFDFDDFDD W11 ACC=PNO23 DFSAEFEA EAEDEWRESAD ASSDRE
AERREEW2 3122312 SDADDSADADAD W12 ACC=HH34 23SAEFEA EAEDEWRESAD ASEEWEE
A15ECCCW 3XCXXF12 SDSGTRERRECC W43 ACC=P11 XXFSAEFEA EAEDEWRESAD ASWWWW
ASDASD2W 3122312 SDAFFFDEEEEE SD3 ACC=PNI22 ABCEFEA EAEDEWRESAD ASWEDSSAD
...
I have to extract the substring between the '=' character and the following blank space for each line , i.e.
PNO23
HH34
P11
PNI22
I've been using the sed command but cannot figure out how to ignore all characters following the blank space.
Any help?
Use the right tool for the job.
$ awk -F '[= ]+' '{ print $6 }' input.txt
PNO23
HH34
P11
PNI22
Sorry, but have to add another one because I feel the existing answers are just to complicated
sed 's/.*=//; s/ .*//;' inputfile
This might work for you:
sed -n 's/.*=\([^ ]*\).*/\1/p' file
or, if you prefer:
sed 's/.*=\([^ ]*\).*/\1/p;d' file
Put the string you want to capture in a backreference:
sed 's/.*=\([^ =]*\) .*/\1/'
or do the substitution piecemeal;
sed -e 's/.*=//' -e 's/ .*//'
sed 's/[^=]*=\([^ ]*\) .*/\1/' inputfile
Match all the non-equal-sign characters and an equal sign. Capture a sequence of non-space characters. Match a space and the rest of the line. Substitute the captured string.
A chain of grep can do the trick.
grep -o '[=][a-zA-Z0-9]*' file | grep -o '[a-zA-Z0-9]*'
I have a command output from which I want to remove the double quotes ".
Command:
strings -a libAddressDoctor5.so |\
grep EngineVersion |\
awk '{if(NR==2)print}' |\
awk '{print$2}'
Output:
EngineVersion="5.2.5.624"
I'd like to know how to remove unwanted characters with awk or sed.
Use sed's substitution: sed 's/"//g'
s/X/Y/ replaces X with Y.
g means all occurrences should be replaced, not just the first one.
Using just awk you could do (I also shortened some of your piping):
strings -a libAddressDoctor5.so | awk '/EngineVersion/ { if(NR==2) { gsub("\"",""); print $2 } }'
I can't verify it for you because I don't know your exact input, but the following works:
echo "Blah EngineVersion=\"123\"" | awk '/EngineVersion/ { gsub("\"",""); print $2 }'
See also this question on removing single quotes.
tr can be more concise for removing characters than sed or awk, especially when you want to remove multiple different characters from a string.
Removing double quotes:
echo '"Hi"' | tr -d \"
# Prints Hi without quotes
Removing different kinds of brackets:
echo '[{Hi}]' | tr -d {}[]
# Prints Hi without brackets
-d stands for "delete".
I am wrinting a shell script and have a variable like this: something-that-is-hyphenated.
I need to use it in various points in the script as:
something-that-is-hyphenated, somethingthatishyphenated, SomethingThatIsHyphenated
I have managed to change it to somethingthatishyphenated by stripping out - using sed "s/-//g".
I am sure there is a simpler way, and also, need to know how to get the camel cased version.
Edit: Working function derived from #Michał's answer
function hyphenToCamel {
tr '-' '\n' | awk '{printf "%s%s", toupper(substr($0,1,1)), substr($0,2)}'
}
CAMEL=$(echo something-that-is-hyphenated | hyphenToCamel)
echo $CAMEL
Edit: Finally, a sed one liner thanks to #glenn
echo a-hyphenated-string | sed -E "s/(^|-)([a-z])/\u\2/g"
a GNU sed one-liner
echo something-that-is-hyphenated |
sed -e 's/-\([a-z]\)/\u\1/g' -e 's/^[a-z]/\u&/'
\u in the replacement string is documented in the sed manual.
Pure bashism:
var0=something-that-is-hyphenated
var1=(${var0//-/ })
var2=${var1[*]^}
var3=${var2// /}
echo $var3
SomethingThatIsHyphenated
Line 1 is trivial.
Line 2 is the bashism for replaceAll or 's/-/ /g', wrapped in parens, to build an array.
Line 3 uses ${foo^}, which means uppercase (while ${foo,} would mean 'lowercase' [note, how ^ points up while , points down]) but to operate on every first letter of a word, we address the whole array with ${foo[*]} (or ${foo[#]}, if you would prefer that).
Line 4 is again a replace-all: blank with nothing.
Line 5 is trivial again.
You can define a function:
hypenToCamel() {
tr '-' '\n' | awk '{printf "%s%s", toupper(substr($0,0,1)), substr($0,2)}'
}
CAMEL=$(echo something-that-is-hyphenated | hypenToCamel)
echo $CAMEL
In the shell you are stuck with being messy:
aa="aaa-aaa-bbb-bbb"
echo " $aa" | sed -e 's/--*/ /g' -e 's/ a/A/g' -e 's/ b/B/g' ... -e 's/ *//g'
Note the carefully placed space in the echo and the double space in the last -e.
I leave it as an exercise to complete the code.
In perl it is a bit easier as a one-line shell command:
perl -e 'print map{ $a = ucfirst; $a =~ s/ +//g; $a} split( /-+/, $ARGV[0] ), "\n"' $aa
For the records, here's a pure Bash safe method (that is not subject to pathname expansion)—using Bash≥4:
var0=something-that-is-hyphenated
IFS=- read -r -d '' -a var1 < <(printf '%s\0' "${var0,,}")
printf '%s' "${var1[#]^}"
This (safely) splits the lowercase expansion of var0 at the hyphens, with each split part in array var1. Then we use the ^ parameter expansion to uppercase the first character of the fields of this array, and concatenate them.
If your variable may also contain spaces and you want to act on them too, change IFS=- into IFS='- '.