Before I go ahead and create something I wanted to check does anyone know of a CLI or other tool to update the values within a .tfvars file?
Example vars file:
ecs_desired_capacity = 1
ecs_asg_min_size = 1
What I'd like:
./somecommand ./myvars.tfvars --set="ecs_asg_min_size=2,new_var=1"
Result edit file:
ecs_desired_capacity = 1
ecs_asg_min_size = 2
new_var = 1
Thanks in advance.
You can use HCLQ (command line tool for querying and manipulating HashiCorp HCL):
cat myvars.tfvars | hclq set 'ecs_asg_min_size' 2
you can add a new variable and then check its value or update it
echo "new_var = 1" >> myvars.tfvars
cat myvars.tfvars | hclq get 'new_var'
This seems to work
#!/bin/sh
# Example usage:
# ./update_tfvars.sh example.tfvars variable_name_here 123
set -e
tfvars_file=$1
var_name=$2
var_value=$3
sed -r -i -e "s/($var_name\s*=\s*).*/\1$var_value/" $tfvars_file
Doesn't add a new entry if it doesn't exist, which fits my needs.
Related
I have a file file.dat as follow:
1.1,2.1 1.4
3.1,2.1 2.4
2.4,4.5 11.5
..
And I want to select each time the whole line (string) and replace it in another file. So far I tried the following
#!/bin/csh
set FILENAME = 'file.dat' # file in which the strings are
set str = "229.8,230.9 230.36" # initialize the first string
set n = 1
while ( $n <= 3 ) # number of lines in the FILENAME
echo Testing the first string $str
set rep = $(head -n $n "$FILENAME")
# n++ # increment the index
end
When I tried to launch the script csh launch.sh I obtained the follow error message
Testing the first string 229.8,230.9 230.36
Illegal variable name. # connect with the rep definition(?)
The file in which I want to change the string str is as follow (this is btw a secondary problem which I could figure out by myself once I understand what's wrong in the first lines):
# Name Type Par
Mi FI 154.2355189465
So UN 229.8,230.9 230.36 # line to be changed
Za FI 0.8000020209
May somebody help me, please?
$(...) is Bash syntax for command substitution in Bash.
In C-shell you have to use backticks instead (yuck).
I am trying to read to id_rsa file into a variable var( set var=`cat id_rsa`) in tcsh to provide input to a program. But when i echo the variable ( echo "$var")new lines are gone, its a one line file content. So how do i correctly store and print the variable?
Don't use tcsh for this task, getting the output of a command into a variable in verbatim is unnecessarily difficult:
Some workarounds if you have to use tcsh are:
use redirection
% yourtool < id_rsa
Store the variable as base-16 (or something else) encoded stuff, so that it doesn't contain any newline characters that will get mangled by tcsh.
% set hex_contents = `<id_rsa xxd -l 16 -p`
Use a tempfile?
% set tempfile = `mktemp`
% program > tempfile
... later
% <tempfile other-program
I asked a similar question almost a year ago; https://unix.stackexchange.com/questions/284220/tcsh-preserve-newlines-in-command-substitution
In case you're curious this is how you get the verbatim contents (credit Stéphane Chazelas).
set temp = "`(some command; echo .) | paste -d . - /dev/null`"
set var = ""
set nl = '\
'
foreach i ($temp:q)
set var = $var:q$i:r:q$nl:q
end
set var = $var:r:q
So i would like to print my env variable which contains other variables.
For example i have:
MY_VARS="My var are:\nVAR1 = ${MY_VAR1}\nVAR2 = ${MY_VAR2}"
MY_VAR1=var1
MY_VAR2=var2
and i would like to make it possible in way like:
printf "${MY_VARS}" > my.conf
or
printf "$(echo ${MY_VARS})" > my.conf
to get sth like in my.conf file:
My var are:
VAR1 = var1
VAR2 = var2
but it dosent work.
Is ther possibility to do such thing ?
I need it to us it with Kubernetes so i can set env in my ReplicationController and use it with Kubernetes envs like SERVICE_HOST and SERVICE_PORT
There is another problem that Kubernetes is changing my MY_VARS variable to multiline inside container so it looks like:
MY_VARS=My var are:
VAR1 = ${MY_VAR1}
VAR2 = ${MY_VAR2}
Hope it's quite clear :)
My solution:
while read -r line
do
printf "$line\n" >> conf.tmp
done <<< "$CONFIG"
while read -r line
do
eval echo $line >> conf
done < conf.tmp
Where CONFIG variable is passed by me to the container and this variable contains Kubernetes service variables with IPs and Ports.
I had to make it twice because eval couldnt resolve \n.
With bash you can perform indirect references but it would not replace the value of your variable directly, only when you use it from a bash terminal.
You've got to define the vars first. The following script prints out what you want:
MY_VAR1=var1
MY_VAR2=var2
MY_VARS="My var are:\nVAR1 = ${MY_VAR1}\nVAR2 = ${MY_VAR2}"
printf "${MY_VARS}"
I have a csv file which im trying to loop through with the purpose to find out if an User Input is found inside the csv data. I wrote the following code which sometimes works and others doesn't. It always stops working when I try to compare to a 2+ digit number. It works OK for numbers 1 through 9, but once u enter lets say 56 , or 99 or 100, it stops working.
the csv data is comma delimited, i have about 300 lines they are just like this.
1,John Doe,Calculus I,5.0
1,John Doe,Calculus II,4.3
1,John Doe,Physics II,3.5
2,Mary Poppins,Calculus I,3.7
2,Mary Poppins,Calculus II,4.7
2,Mary Poppins,Physics I,3.7
Data is just like that, all the way down until ID #100 for a total of 300 lines. Both the sh file and csv file are in the same folder, I'm using a fresh installation of Ubuntu 12.04.3, using gedit as the text editor.
I tried Echoing the variables ID and inside the IF conditionals but it doesn't behave the way it should when testing for the same value. Could someone point me out in the right direction. Thanks
Here's the code:
#s!/bin/bash
echo "enter your user ID";
read user;
INPUT_FILE=notas.csv
while IFS="," read r- ID name asignature final;
do
if [$ID = $user]; then
userType=1;
else
userType=2;
fi
done < notas.csv
Well, your code as written has a few issues.
You have r- instead of -r on the read line - I assume that's a typo not present in your actual code or you wouldn't get very far.
Similarly, you need space around the [...] brackets: [$ID is a syntax error.
You need to quote the parameter expansions in your if clause, and/or switch bracket types. You probably make it a numeric comparison as #imp25 suggested, which I would do by using ((...)).
You probably don't want to set userType to 2 in an else clause, because that will set it to 2 for everyone except whoever is listed last in the file (ID 100, presumably). You want to set it to 2 first, outside the loop. Then, inside the loop when you find a match, set it to 1 and break out of the loop:
userType=2
while IFS=, read -r ID name asignature final; do
if (( $ID == $user )); then
userType=1;
break
fi
done < notas.csv
You could also just use shell tools like awk:
userType=$(awk -F, -vtype=2 '($1=="'"$user"'") {type=1}; END {print type}' notas.csv)
or grep:
grep -q "^$user," notas.csv
userType=$(( $? + 1 ))
etc.
You should quote your variables in the if test statement. You should also perform a numeric test -eq rather than a string comparison =. So your if statement should look like:
if [[ "$ID" -eq "$user" ]]
In Linux, say I have the following file (e.g. conf.properties):
HOST_URL=http://$HOSTNAME
STD_CONFIG=http://$HOSTNAME/config
USER_CONFIG=http://$HOSTNAME/config/$unconfigured
I want to create another file with all the environment variables replaced...e.g. say the environment variable $HOSTNAME is 'myhost' and $unconfigured is not set, a script should produce the following output:
HOST_URL=http://myhost
STD_CONFIG=http://myhost/config
USER_CONFIG=http://myhost/config/
I was thinking this could be done in a simple one-liner with some sort of sed/awk magic, but I'm no expert and my searches have been in vein, so appreciate any help.
Edit:
I should mention that the file can really be any format text file, for example xml. I just want to replace anything that looks like an env variable with whatever is currently set in the environment.
This is what envsubst is for.
echo 'Hello $USER'
Hello $USER
echo 'Hello $USER' | envsubst
Hello malvineous
You would probably use it more like this though:
envsubst < input.txt > output.txt
envsubst seems to be part of GNU gettext.
sed 's/$HOSTNAME/myhost/g;s/$unconfigured//g' yourfile.txt > another_file.txt
update:
Based on updates in your question, this won't be a good solution.
update2 :
This is based on an answer to a related question. I've hacked at it (I'm unfamiliar with perl) to remove undefined vars.
perl -p -e 's/\$\{([^}]+)\}/defined $ENV{$1} ? $ENV{$1} : $&/eg; s/\$\{([^}]+)\}//eg' yourfile.txt
Should work for any input text file, however you will need to define vars using the ${...} format which simplifies the string matching.
(rant regarding the evilness of eval moved to a separate post so as not to confuse readers)
"eval is evil"
This is not an answer, but a warning in response to using eval for this task. You really really really don't want to do that.
Exhibit 1: a malicious template file:
HOST_URL=http://$HOSTNAME
STD_CONFIG=http://$HOSTNAME/config
USER_CONFIG=http://$HOSTNAME/config/$unconfigured
&& cat /etc/redhat-release
An unsuspecting user:
[lsc#aphek]$ cat somefile | while read line; do echo $(eval echo `echo $line`); done
HOST_URL=http://xyz
STD_CONFIG=http://xyz/config
USER_CONFIG=http://xyz/config/
Red Hat Enterprise Linux WS release 4 (Nahant Update 9)
Note the last line!
Now, imagine the possibilities....
I'd do it like this:
# Set the $HOSTNAME and other variables
# Now evaluate the properties file as a shell script.
. config.properties
# Write the values
cat >somefile <<EOF
HOST_URL=$HOST_URL
STD_CONFIG=$STD_CONFIG
USER_CONFIG=$USER_CONFIG
EOF
Edit: Or this very nasty thing (I'm sure there's a better way)
for name in HOST_URL STD_CONFIG USER_CONFIG
echo "$name=$(eval echo `echo '$'$name`)" >>somefile
end
Thanks to #DarkDust I came up with this:
cat somefile | while read line; do echo $(eval echo `echo $line`); done > somefile.replaced
I used this oneliner to replace ${VARIABLE} style variables in a file:
TARGET_FILE=/etc/apache2/apache2.conf; for VARNAME in $(grep -P -o -e '\$\{\S+\}' ${TARGET_FILE} | sed -e 's|^\${||g' -e 's|}$||g' | sort -u); do sed -i "s|\${$(echo $VARNAME)}|${!VARNAME}|g" ${TARGET_FILE}; done
I'm pretty sure someone can do this in 1/3rd of the length using awk… feel challenged! ;)
Here is a snippet of Javascript that I like to have around for solving this exact problem:
// A Javascript version of envsubst for our builds
// Purpose: replace all ocurrences of ${VAR} with the equivalent var from the environment from stdin
var readline = require('readline');
var rl = readline.createInterface({
input: process.stdin,
output: process.stdout,
terminal: false
});
const environment = process.env;
rl.on('line', function(line) {
const newLine = line.replace(/\$\{([a-zA-Z0-9_]+)\}/g, function(_match, variable) {
const envVar = environment[variable];
return envVar ? envVar : '';
});
process.stdout.write(`${newLine}\n`);
});
Hopefully this helps somebody else.
Here's a short one-liner that uses python's curly brace formatting to safely do the magic:
contents=\"\"\"`cat $file`\"\"\"; python -c "import os;print $contents.format(**os.environ)"
avoids evil eval
allows outputting curly braces: use {{ instead of {
no need to specify vars explicitly when calling the script
For example, given properties file settings.properties:
# my properties file
someVar = {MY_ENV_VAR}
curlyBraceVar = has {{curly braces}}
Then, do the substitution with:
$ export MY_ENV_VAR="hello"
$ file=settings.properties
$ contents=\"\"\"`cat $file`\"\"\"; python -c "import os;print $contents.format(**os.environ)"
# my properties file
someVar = hello
curlyBraceVar = has {curly braces}
A script is here: https://raw.githubusercontent.com/aneilbaboo/machome/master/bin/substenv
if you have installed nodejs you can run
npx #utft/tt -e FOO=bar /path/to/input /path/to/output
or you can run it programmatically
https://github.com/utftufutukgyftryidytftuv/tt