I have a file values.properties which contain data, like:
$ABC=10
$XYZ=20
I want to create a shell script that will take each element one by one from above file.
Say $ABC, then go to file ABC.txt & replace the value of $ABC with 10.
Similarly, then go to file XYZ.txt and replace $XYZ with 20.
I think maybe this should be in the Unix and Linux section, the solution I've hacked together is as follows:
cat values.properties | grep "=" | cut -d "$" -f2 | awk -F "=" '{print "s/$"$1"/"$2"/g "$1".txt"}' | xargs -n2 sed -i
The flow is like so:
Filter out all the value assignments via: grep "="
Remove the '$' via: cut -d "$" -f2
Use awk to split the variable name and value and construct sed replacement command
Use xargs to pull in the replacement parameter and target file via: xargs -n2
Finally pass sed to as the command to xargs: xargs -n2 sed
Related
I have this content in file.csv
cat file.csv
QUOTA,landscape=test,region=europe,limit=N2_CPUS quota=24.0,quota_used=0.0,quota_used_percent=0
QUOTA,landscape=test,region=europe,limit=COMMITTED_N2_CPUS quota=0.0,quota_used=0.0,quota_used_percent=0
QUOTA,landscape=test,region=europe,limit=COMMITTED_C2_CPUS quota=0.0,quota_used=0.0,quota_used_percent=0
QUOTA,landscape=test,region=europe,limit=RESERVATIONS quota=100.0,quota_used=0.0,quota_used_percent=0
I need to remove values which contain strings "RESERVATIONS" and "N2_CPUS" and the variables can be random
variable=("RESERVATIONS","N2_CPUS")
I am able to do when i use one value as variable using
cat file.csv | grep -v $variable
When there are more values in a variable, even loops are not working as expected. Could you please suggest?
I would use egrep (or grep -E, depending on your flavor of linux)
variable="RESERVATIONS|N2_CPUS"
cat file.csv | egrep -v $variable
or
cat file.csv | grep -Ev $variable
Note, though, in your example, the cat is not required:
grep -Ev "${variable}" file.csv
Notice the quotes around the variable, you may need those as well, depending on your shell & Linux version.
egrep (or grep -E) is an grep with Extended Regular Expression. The vertical bar, or pipe | separates the values. Effectively it is saying OR. Thus,
egrep -Ev "A|B" means look for 'A' or 'B' and remove them.
Use grep -E so you can use an extended regular expression, and then use | in the regexp to match multiple strings.
variable=RESERVATIONS|N2_CPUS
grep -v -E "$variable" file.csv
I've got a command to perform a series of commands that produce a variable output string such as 123456. I want to pipe that to a sed command replacing a known string in a csv file that looks like this:
Fred,Wilma,Betty,Barney
However, the command below does not work and I haven't found any other references to using pipe values as the variable for a replace.
How does this code change if the values in the csv are in a random order and I always want to change the second value?
Example code:
find / -iname awk 2>/dev/null | sha256sum | cut -c1-10 > test.txt |
sed -i -e '/Wilma/ r test.txt' -e 's/Wilma//' input.csv
Contents of input.csv should become: Fred,0d522cd316,Betty,Barney
Okay, in
find / -iname awk 2>/dev/null | sha256sum | cut -c1-10 > test.txt | sed -i -e '/Wilma/ r test.txt' -e 's/Wilma//' input.csv
you have a bug. That "> test.txt" after cut is going to eat your stdin on sed, so things go weird with that pipe afterwards taking stdin. You don't want a pipe there, or you don't want to redirect to a file.
The way to take piped stdin and use it as a parameter in a command is through xargs.
find / -iname awk 2>/dev/null | sha256sum | cut -c1-10 | xargs --replace=INSERTED -- sed -i -e 's/Wilma/INSERTED/' input.csv
(...though that find|shasum is suspect too, in that the order of files is random(ish) and it matters for a reliable sum. You prpobably mean to "|sort" after find.)
(Some would sed -i -e "s/Wilma/$(find|sort|shasum|cut)" f, but I ain't among them. Animals.)
For replacing a fixed string like "Wilma", try:
sed -i 's/Wilma/'"$(find / -iname awk 2>/dev/null |
sha256sum | cut -c1-10)"'/' input.csv
To replace the 2nd field no matter what's in it, try:
sed -i 's/[^,]*/'"$(find / -iname awk 2>/dev/null |
sha256sum | cut -c1-10)"'/2' input.csv
I want to move the output of the command:
ls -1 /${TMP_DIR}/*0000000221*.dbf | xargs | sed 's/ /,/g'
In the end of a command that come after it, like that:
ls -1 /${TMP_DIR}/*0000000221*.dbf | xargs | sed 's/ /,/g' | impdp sim/sim files=$1
For example:
execute ls -1 /${TMP_DIR}/*0000000221*.dbf | xargs | sed 's/ /,/g' will give me:
/tmp/a_0000000221.dbf,/tmp/a_00000002212.dbf,/tmp/b_0000000221.dbf
So I want the final command will look like:
impdp sim/sim files=/tmp/a_0000000221.dbf,/tmp/a_00000002212.dbf,/tmp/b_0000000221.dbf
EDIT:
Sorry I didnt write this from the beginning - I've variable in the command ${TMP_DIR}
You probably don't need that many pipes. You can use it like this:
printf "impdp sim/sim files=" && printf "%s," /tmp/*0000000221*.dbf
impdp sim/sim files=/tmp/a_0000000221.dbf,/tmp/a_00000002212.dbf,/tmp/b_0000000221.dbf,
ls is a bit redundant if you just want to get the file names.
You can get the shell to glob those and then use printf to put them one per line.
To separate those items with ',' rather than '\n', you can use paste
Finally, putting all that within $() will execute that in a subshell,
and output the result for the command in the current shell.
impdp sim/sim files=$(printf '%s\n' /${TMP_DIR}/*0000000221*.dbf | paste -d, -s)
You can try other order of commands:
impdp sim/sim files=$(ls -1 /tmp/*0000000221*.dbf | xargs | sed 's/ /,/g')
You can use globbing, an array and IFS to construct the parameter string:
$ ls -1
1.txt
2.txt
3.txt
$ echo impdp sim/sim files="$(a=(*.txt);IFS=',';echo "${a[*]}")"
impdp sim/sim files=1.txt,2.txt,3.txt
Obviously this will break on filenames with spaces or newlines.
To run, just remove the echo.
(all solutions including mine assumes your filenames do not contain spaces)
sed is a little overkill, you can use tr and avoid xargs too:
impdp sim/sim files=$(ls /tmp/*0000000221*.dbf | tr "\n" ",")
I need some assistance trying to build up a variable using a list of exclusions in a file.
So I have a exclude file I am using for rsync that looks like this:
*.log
*.out
*.csv
logs
shared
tracing
jdk*
8.6_Code
rpsupport
dbarchive
inarchive
comms
PR116PICL
**/lost+found*/
dlxwhsr*
regression
tmp
working
investigation
Investigation
dcsserver_weblogic_
dcswebrdtEAR_weblogic_
I need to build up a string to be used as a variable to feed into egrep -v, so that I can use the same exclusion list for rsync as I do when egrep -v from a find -ls.
So I have created this so far to remove all "*" and "/" - and then when it sees certain special characters it escapes them:
cat exclude-list.supt | while read line
do
echo $line | sed 's/\*//g' | sed 's/\///g' | 's/\([.-+_]\)/\\\1/g'
What I need the ouput too look like is this and then export that as a variable:
SEXCLUDE_supt="\.log|\.out|\.csv|logs|shared|PR116PICL|tracing|lost\+found|jdk|8\.6\_Code|rpsupport|dbarchive|inarchive|comms|dlxwhsr|regression|tmp|working|investigation|Investigation|dcsserver\_weblogic\_|dcswebrdtEAR\_weblogic\_"
Can anyone help?
A few issues with the following:
cat exclude-list.supt | while read line
do
echo $line | sed 's/\*//g' | sed 's/\///g' | 's/\([.-+_]\)/\\\1/g'
Sed reads files line by line so cat | while read line;do echo $line | sed is completely redundant also sed can do multiple substitutions by either passing them as a comma separated list or using the -e option so piping to sed three times is two too many. A problem with '[.-+_]' is the - is between . and + so it's interpreted as a range .-+ when using - inside a character class put it at the end beginning or end to lose this meaning like [._+-].
A much better way:
$ sed -e 's/[*/]//g' -e 's/\([._+-]\)/\\\1/g' file
\.log
\.out
\.csv
logs
shared
tracing
jdk
8\.6\_Code
rpsupport
dbarchive
inarchive
comms
PR116PICL
lost\+found
dlxwhsr
regression
tmp
working
investigation
Investigation
dcsserver\_weblogic\_
dcswebrdtEAR\_weblogic\_
Now we can pipe through tr '\n' '|' to replace the newlines with pipes for the alternation ready for egrep:
$ sed -e 's/[*/]//g' -e 's/\([._+-]\)/\\\1/g' file | tr "\n" "|"
\.log|\.out|\.csv|logs|shared|tracing|jdk|8\.6\_Code|rpsupport|dbarchive|...
$ EXCLUDE=$(sed -e 's/[*/]//g' -e 's/\([._+-]\)/\\\1/g' file | tr "\n" "|")
$ echo $EXCLUDE
\.log|\.out|\.csv|logs|shared|tracing|jdk|8\.6\_Code|rpsupport|dbarchive|...
Note: If your file ends with a newline character you will want to remove the final trailing |, try sed 's/\(.*\)|/\1/'.
This might work for you (GNU sed):
SEXCLUDE_supt=$(sed '1h;1!H;$!d;g;s/[*\/]//g;s/\([.-+_]\)/\\\1/g;s/\n/|/g' file)
This should work but I guess there are better solutions. First store everything in a bash array:
SEXCLUDE_supt=$( sed -e 's/\*//g' -e 's/\///g' -e 's/\([.-+_]\)/\\\1/g' exclude-list.supt)
and then process it again to substitute white space:
SEXCLUDE_supt=$(echo $SEXCLUDE_supt |sed 's/\s/|/g')
I have a csv file with 4 attributes in each line, delimited by comma. I'm trying to come up with a sed command to keep only the second attribute from each line. Any ideas on how to do it?
You'd be better off with cut:
cut -d "," -f 2 file.txt
If you want to remove dupes, and you don't mind the order of the entries, simply do:
cut -d "," -f 2 file.txt | sort -u
And to extend to attrs 1 and to, simply use:
cut -d "," -f 1,2 file.txt | sort -u
You do not need sed for this . The fastest is with cut:
cut -d, -f2 file
However , if you want sed , you can do it so:
sed '/[^,]*,\([^,]*\).*/ sxx\1x' file