I am trying to change the column names to lowercase in a csv file. I found the code to do that online but I dont know how to replace the old column names(uppercase) with new column names(lowercase) in the original file. I did something like this:
$cat head -n1 xxx.csv | tr "[A-Z]" "[a-z]"
But it simply just prints out the column names in lowercase, which is not enough for me.
I tried to add sed -i but it did not do any good. Thanks!!
Using awk (readability winner) :
concise way:
awk 'NR==1{print tolower($0);next}1' file.csv
or using ternary operator:
awk '{print (NR==1) ? tolower($0): $0}' file.csv
or using if/else statements:
awk '{if (NR==1) {print tolower($0)} else {print $0}}' file.csv
To change the file for real:
awk 'NR==1{print tolower($0);next}1' file.csv | tee /tmp/temp
mv /tmp/temp file.csv
For your information, sed using the in place edit switch -i do the same: it use a temporary file under the hood.
You can check this by using :
strace -f -s 800 sed -i'' '...' file
Using perl:
perl -i -pe '$_=lc() if $.==1' file.csv
It replace the file on the fly with -i switch
You can use sed to tell it to replace the first line with all lower-case and then print the rest as-is:
sed '1s/.*/\L&/' ./xxx.csv
Redirect the output or use -i to do an in-place edit.
Proof of Concept
$ echo -e "COL1,COL2,COL3\nFoO,bAr,baZ" | sed '1s/.*/\L&/'
col1,col2,col3
FoO,bAr,baZ
I have a text file with layout as:
tableName1|counterVariable1
tableName2|counterVariable2
I want to replace the counterVariable1 with some other variable say counterVariableNew.
How can I accomplish this?
I have tried various SED/AWK approaches, closest one is mentioned below:
cat $fileName | grep -w $tableName | sed -i 's/$tableName\|counterVariable/$tableName\|counterVariableNew'
But all the 3 commands are not merging properly, please help!
Your script is an example of [ useless use of cat ]. But the key point here is to escape the pipe delimiter which has a special meaning(it stands for OR) when used with awk FS. So below script should do
# cat 42000479
tableName1|counterVariable1
tableName2|counterVariable2
tableName3|counterVariable2
# awk -F\| '$1=="tableName2"{$2="counterVariableNew"}1' 42000479
tableName1|counterVariable1
tableName2 counterVariableNew
tableName3|counterVariable2
An alternate way of doing the same stuff is below
# awk -v FS='|' '$1=="tableName2"{$2="counterVariableNew"}1' 42000479
Stuff inside the single quote will not be expanded.
awk -F'|' -v OFS='|' '/tableName1/ {$2="counterVariableNew"}1' file
tableName1|counterVariableNew
tableName2|counterVariable2
This will search for A (tableName1) and replace B (counterVariable1) to counterVariableNew.
Or by using sed :
sed -r '/tableName1/ s/(^.*\|)(.*)/\1counterVariableNew/g' file
tableName1|counterVariableNew
tableName2|counterVariable2
For word bounded search: Enclose the pattern inside \< and \> .
sed -r '/\<tableName1\>/ s/(^.*\|)(.*)/\1counterVariableNew/g' file
awk -F'|' -v OFS='|' '/\<tableName1\>/ {$2="counterVariableNew"}1' file
I have a file containing consecutive symbols (as pipe "|") like
ANKRD54,LIAR,allergy,|||
ANKRD54,LIAR,asthma,||20447076||
ANKRD54,LIAR,autism,||||
ANKRD54,LIAR,cancer,|||
ANKRD54,LIAR,chronic_obstructive_pulmonary_disease,|||
ANKRD54,LIAR,dental_caries,||||
Now using shell or a sed command in shell is it possible to replace multiple pipe with one pipe like
ANKRD54,LIAR,allergy,|
ANKRD54,LIAR,asthma,|20447076|
ANKRD54,LIAR,autism,|
ANKRD54,LIAR,cancer,|
ANKRD54,LIAR,chronic_obstructive_pulmonary_disease,|
ANKRD54,LIAR,dental_caries,|
I guess the easiest way is use built-in commands: cat your_file | tr -s '|'
Pass your text to sed (e.g. via a pipe)
cat your_file | sed "s/|\+/|/g"
You can do that with a simple awk gsub as:-
awk -F"," -v OFS="," '{gsub(/[|]+/,"|",$4)}1' file
See it in action:-
$ cat file
ANKRD54,LIAR,allergy,|||
ANKRD54,LIAR,asthma,||20447076||
ANKRD54,LIAR,autism,||||
ANKRD54,LIAR,cancer,|||
ANKRD54,LIAR,chronic_obstructive_pulmonary_disease,|||
ANKRD54,LIAR,dental_caries,||||
$ awk -F"," -v OFS="," '{gsub(/[|]+/,"|",$4)}1' file
NKRD54,LIAR,allergy,|
ANKRD54,LIAR,asthma,|20447076|
ANKRD54,LIAR,autism,|
ANKRD54,LIAR,cancer,|
ANKRD54,LIAR,chronic_obstructive_pulmonary_disease,|
ANKRD54,LIAR,dental_caries,|
I am trying to extract text between pattern1 (fixed) and pattern2 (this can be p2-1/p2-2).
can you please tell me how to achieve this in a single command?
A file starts with start and ends with either end or close
File1:
======
junktest
data
start
stackoverflow
sed
close
File2:
======
data2
start
stackoverflow
end
I can extract text from File1 with
sed -n "/start/,/close/p"
And from File2 with
sed -n "/start/,/end/p"
I need a single sed command to achieve both..
something like:
sed -n "/start/, /close or end /p"
Both GNU sed and BSD sed:
sed -nE '/start/,/close|end/p' file
This awk looks better
awk '/start/,/end|close/' file
sed -n -E "/Word1/,/Word2-1/p" | sed -n -E "/Word1/,/Word2-2/p"
Easy with awk:
$ awk '/start/{p=1}p{print}/end|close/{p=0}' file
I have file, with lines, contains ip with netmask
a.b.c.d/24
w.x.y.z/32
etc
How to delete delete specific row?
i'm using
sed -ie "s#a.b.c.d/24##g" %filname%
but after the removal is an empty string in file.
It should run inside a script, with ip as parameter and also work in freebsd under sh.
Sed solution
sed -i '/<pattern-to-match-with-proper-escape>/d' data.txt
-i option will change the original file.
Awk solution
awk '!/<pattern-to-match-with-proper-escape>/' data.txt
Using sed:
sed -i '\|a.b.c.d/24|d' file
Command line arg:
For the input being command line argument, say 1st argument($1):
sed -i "\|$1|d" file
Replace $1 with appropriate argument number as is your case.
You should use d (delete) not g. Also do not use s (replacement).
sed -ie '/a.b.c.d\/24/d' %filename%
In a script you should using it in this way
IP=$1
IPA=${IP////\\/}
sed -i /"${IPA}"/d %filename%
And the script parameter should be called in this way:
./script.sh a.b.c.d/24
perl -i -lne 'print unless(/a.b.c.d\/24/)' your_file
or in awk if you donot want to do inplace editing:
awk '$0!~/a.b.c.d\/24/' your_file