I want to be able to read and write to a file with a bash script. I am able to read the variables from the file but I am not sure how to write to it.
The file data.txt contains the variable names and their values like this:
var1=2
var2=3
The shell script looks like this:
#!bin/bash
echo "Read"
var1=$(grep -Po "(?<=^var1=).*" data.txt)
echo "Do Something"
var2=${var1}
echo "Write"
# var2 should be saved in data.txt like this: var2=<Here is the Value>, e.g. var2=2
I can read the specific values of the variables with the grep command, which I got from this answer. But I am not sure how to implement the functionality to be able to save the values of the variables at the correct position (see the last comment in the bash script).
The variable names are unique in data.txt.
Any ideas how to achieve this?
I found a solution with the sed command:
echo "Write"
sed -i -e "s/$(grep -Po '(?<=^var2=).*' data.txt)/${var2}/g" data.txt
You can use sed to do the job.
!/bin/bash
echo "Read"
var1=$(grep -Po "(?<=^var1=).*" data.txt)
echo "Do Something"
var2=${var1}
echo "Write"
sed -i "/var2/ s/.*/var2=${var2}/" data.txt # replace the line contains 'var2' with var2=2
I assume you use GNU sed, for BSD sed, you need to feed an extra suffix to -i.
Related
So I am trying to make a script that contains egrep and accepts a numeric variable
#!/bin/bash
var=$1
list="egrep "^.{$var}$ /usr/share/dict/words"
cat list
For example, if var is 5, I would like this script to print out every line with 5 characters. For some reason the script does not do that. Help would be greatly appreciated!
Your script doesn't work because there are several problems with these lines:
list="egrep "^.{$var}$ /usr/share/dict/words"
cat list
The first line isn't complete, it's missing a closing quote,
Even if you fixed it, you're assigning a literal string to list, not the output of a command,
RE and filename should be separated
cat doesn't print a variable's content, echo does that.
So:
#!/bin/bash
var="$1"
list="$(egrep '^.{'"$var"'}$' /usr/share/dict/words)"
echo "$list"
should work.
Or even better, you can use just an awk command:
awk 'length==5' /usr/share/dict/words
with $1 or any other variable:
awk -v n="$1" 'length==n' /usr/share/dict/words
My first input file contains records name abc.txt:
abc#gmail.com
bscd#yahoo.co.in
abcd.21#gmail.com
1234#hotmail.com
My second file contains record name details.txt:
123456^atulsample^1203320
I want my final file having output to be Final.txt:
abc#gmail.com^123456^atulsample^1203320
bscd#yahoo.co.in^123456^atulsample^1203320
abcd.21#gmail.com^123456^atulsample^1203320
I have uses sed command but I am not getting my required output.
Kindly help as I don't have much knowledge in shell scripting.
try something like this;
#!/bin/bash
while read -r line
do
detail="$line"
sed '/^[ \t]*$/d' abc.txt | sed "s/$/^${detail}/" >> Final.txt
done < "details.txt"
this is to delete blank lines;
sed '/^[ \t]*$/d' abc.txt
this is to append from details.txt
sed "s/$/^${detail}/"
deploy.sh
USERNAME="Tom"
PASSWORD="abc123"
FILE="config.conf"
sed -i "s/\PLACEHOLDER_USERNAME/$USERNAME/g" $FILE
sed -i "s/\PLACEHOLDER_PASSWORD/$PASSWORD/g" $FILE
config.conf
deloy="PLACEHOLDER_USERNAME"
pass="PLACEHOLDER_PASSWORD"
This file puts my variables defined in deploy into my config file. I can't source the file so I want put my variables in this way.
Question
I want a command that is generic to work for all placeholder variables using some sort of while loop rather than needing one command per variable. This means any term starting with placeholder_ in the file will try to be replaced with the value of the variable defined already in deploy.sh
All variables should be set and not empty. I guess if there is the ability to print a warning if it can't find the variable that would be good but it isn't mandatory for this.
Basically, use shell code to write a sed script and then use sed -i .bak -f sed.script config.conf to apply it:
trap "rm -f sed.script; exit 1" 0 1 2 3 13 15
for var in USERNAME PASSWORD
do
echo "s/PLACEHOLDER_$var/${!var}/"
done > sed.script
sed -i .bak -f sed.script config.conf
rm -f sed.script
trap 0
The main 'tricks' here are:
knowing that ${!var} expands to the value of the variable named by $var, and
knowing that sed will take a script full of commands via -f sed.script, and
knowing how to use trap to ensure temporary files are cleaned up.
You could also use sed -e "s/.../.../" -e "s/.../.../" -i .bak config.conf too, but the script file is easier, I think, especially if you have more than 2 values to substitute. If you want to go down this route, use a bash array to hold the arguments to sed. A more careful script would use at least $$ in the script file name, or use mktemp to create the temporary file.
Revised answer
The trouble is, although much closer to being generic, it is still not generic since I have to manually put in what variables I want to change. Can it not be more like "for each placeholder_, find the variable in deploy.sh and add that variable, so it can work for any number of variables.
So, find what the variables are in the configuration file, then apply the techniques of the previous answer to solve that problem:
trap "rm -f $tmp; exit 1" 0 1 2 3 13 15
for file in "$#"
do
for var in $(sed 's/.*PLACEHOLDER_\([A-Z0-9_]*\).*/\1/' "$file")
do
value="${!var}"
[ -z "$value" ] && { echo "$0: variable $var not set for $file" >&2; exit 1; }
echo "s/PLACEHOLDER_$var/$value/"
done > $tmp
sed -i .bak -f $tmp "$file"
rm -f $tmp
done
trap 0
This code still pulls the values from the environment. You need to clarify what is required if you want to extract the settings from the shell script, but it can be done — the script will have to be sufficiently self-aware to find its source so it can search it for the names. But the basics are in this answer; the rest is a question of tinkering until it does what you need.
#!/bin/ksh
TemplateFile=$1
SourceData=$2
(sed 's/.*/#V0r:PLACEHOLDER_&:r0V#/' ${SourceData}; cat ${TemplateFile}) | sed -n "
s/$/²/
H
$ {
x
s/^\(\n *\)*//
# also reset t flag
t varxs
:varxs
s/^#V0r:\([a-zA-Z0-9_]\{1,\}\)=\([^²]*\):r0V#²\(\n.*\)\"\1\"/#V0r:\1=\2:r0V#²\3\2/
t varxs
# clean the line when no more occurance in text
s/^[^²]*:r0V#²\n//
# and next
t varxs
# clean the marker
s/²\(\n\)/\1/g
s/²$//
# display the result
p
}
"
call like this: YourScript.ksh YourTemplateFile YourDataSourceFile where:
YourTemplateFile is the file that contain the structure with generic value like deloy="PLACEHOLDER_USERNAME"
YourDataSourceFile is the file that contain all the peer Generic value = specific value like USERNAME="Tom"
I have been going over this for a few days and I am completely stuck. I put placeholder variables in my files which are in the following format: *$PLACEHOLDER_VAR* with the original variable being $VAR.
Original config.conf
TEST="$PLACEHOLDER_USERNAME"
TEST2="$PLACEHOLDER_PASSWORD"
What I want to do is have a deploy.sh which contains defined variables, and can run a command which will replace any placeholder variables with the defined variables, leaving it like this:
New config.conf
TEST="Tom"
TEST2="abc123"
I need a single command for my deploy.sh which will work for any placeholder variables.
$PLACEHOLDER_USERNAME -> $USERNAME -> Tom
An example (but not right) deploy.sh using username as an example
USERNAME="Tom"
PASSWORD="abc123"
FILE="config.conf"
sed -i s/STRING_TO_REPLACE/STRING_TO_REPLACE_IT/g $FILE
Can anyone please help me complete this script?
You could try something like this :
USERNAME="Tom"
PASSWORD="abc123"
FILE="config.conf"
sed -i "s/\$PLACEHOLDER_USERNAME/$USERNAME/g" $FILE
sed -i "s/\$PLACEHOLDER_PASSWORD/$PASSWORD/g" $FILE
# perl -p -i -e "s/\$PLACEHOLDER_PASSWORD/$PASSWORD/g" $FILE
I have also included a sample of the perl alternative to sed since im not sure if the sed above will do the trick.
Try #2
This should do the trick.
USERNAME="Tom"
PASSWORD="abc123"
LIST="USERNAME PASSWORD"
FILE="config.conf"
for item in $LIST; do
eval "sed -i \"s/\$PLACEHOLDER_$item/\$$item/g\" $FILE"
done
I have a text file that contains references to variables and lets a user set up the formatting they want around variables, say something like
The date is $DATE
The time is $TIME
I then want to read this text file in, replace the variables, and print the result to stdout using a bash script. The closest thing I've gotten is using "echo" to output it,
DATE="1/1/2010"
TIME="12:00"
TMP=`cat file.txt`
echo $TMP
However, the output ends up all on one line, and I don't want to have \n at the end of every line in the text file. I tried using "cat << $TMP", but then there are no newlines and the variables inside the text aren't getting replaced with values.
You can use eval to ensure that variables are expanded in your data file:
DATE="1/1/2010"
TIME="12:00"
while read line
do
eval echo ${line}
done < file.txt
Note that this allows the user with control over the file content to execute arbitrary commands.
If the input file is outside your control, a much safer solution would be:
DATE="1/1/2010"
TIME="12:00"
sed -e "s#\$DATE#$DATE#" -e "s#\$TIME#$TIME#" < file.txt
It assumes that neither $DATE nor $TIME contain the # character and that no other variables should be expanded.
Slightly more compact than Adam's response:
DATE="1/1/2010"
TIME="12:00"
TMP=`cat file.txt`
eval echo \""$TMP"\"
The downside to all of this is that you end up squashing quotes. Better is to use a real template. I'm not sure how to do that in shell, so I'd use another language if at all possible. The plus side to templating is that you can eliminate that "arbitrary command" hole that Adam mentions, without writing code quite as ugly as sed.
Just quote your variable to preserve newline characters.
DATE="1/1/2010"
TIME="12:00"
TMP=$(cat file.txt)
echo "$TMP"
For your modification of values in file with variables you can do -
while read -r line
do
sed -e "s#\$DATE#$DATE#" -e "s#\$TIME#$TIME#" <<< "$line"
done < file.txt
Test:
[jaypal:~/Temp] cat file.txt
The date is $DATE
The time is $TIME
[jaypal:~/Temp] DATE="1/1/2010"
[jaypal:~/Temp] TIME="12:00"
[jaypal:~/Temp] while read -r line; do sed -e "s#\$DATE#$DATE#" -e "s#\$TIME#$TIME#" <<< "$line"; done < file.txt
The date is 1/1/2010
The time is 12:00