Having some command-line utility that asks for input several times when you launch it, e.g. here I need to pass username and host to some command.
And I would like to make other command so it will pass these args
user:~$ run_something
username:
host:
How it is possible to pipe these parameters sequentially to command, e.g.:
user:~$ echo "my_user, my_host" | run_something
use xargs
eg.
echo "mydir" | xargs ls
Will list the files in the directory mydir
One way to do this is by using read from standard input:
run_something() {
read -r username host && declare -p username host;
}
then run it as:
echo "my_user my_host" | run_something
to get output:
declare -- username="my_user"
declare -- host="my_host"
Another way is to set variables directly before calling function:
run_something() {
declare -p username host
}
then run it as:
username='my_user' host='my_host' run_something
to get the same output:
declare -x username="my_user"
declare -x host="my_host"
You need to send the newlines that you would otherwise type after each line. run_something is expecting two lines of input:
printf '%s\n' "username" "host" | run_something
Related
I have the following aliases in .bash_aliases
ssh51ro="ssh -p12345 root#192.168.15.71"
ssh52ro="ssh -p12345 root#192.168.8.25"
ssh53ro="ssh -p12345 root#192.168.35.33"
...
ssh59ro="ssh -p12345 root#192.168.1.9"
I wonder if I can send the same command to multiple recipients in a for loop. I would normally increment an "i" variable and concatenate the name of the alias together.
for i in $( seq 1 9 ) ; do ssh5${i}ro date; done
But this doesn't seem to work. it says specifically that "ssh5iro command is not found".
How can I assemble and run the commands in this case
Do not store commands in variables. Use functions.
myssh() { ssh -p12345 root#"$#"; }
ssh51ro() { myssh 192.168.15.71 "$#"; }
ssh52ro() { myssh 192.168.8.25 "$#"; }
...
for i in $( seq 1 9 ) ; do ssh5${i}ro date; done
# maybe you want dynamic list of functions based on naming pattern:
for func in $(compgen -A function | grep ssh5.ro); do "$func" date; done
You may want to research Ansible and Puppet Bolt.
I am working on a bash script and I got a list of IP's that I wanted to add one by one in a CURL command.
For example given list on a file named list.txt
8.8.8.8
10.10.10.10
136.34.24.22
192.168.10.32
I wanted to add each value on curl command
curl -k -u $user:$password "https://logservice/jobs" --data-urlencode 'search=search index=test $ARRAYVALUE | head 1' > output.txt
Where $ARRAYVALUE is the IP address to be used on the command.
I will appreciate any hint.
Thanks
If I understood correctly, you want to:
map each line of a "list.txt" to an item of an array
loop over the newly created array inserting items one by one into your command invocation
Consider this, heavily commented, snippet. Look especially at mapfile and how variable is used in curl invocation, surrounded by double quotes.
#!/bin/bash
# declare a (non-associative) array
# each item is indexed numerically, starting from 0
declare -a ips
#put proper values here
user="userName"
password="password"
# put file into array, one line per array item
mapfile -t ips < list.txt
# counter used to access items with given index in an array
ii=0
# ${#ips[#]} returns array length
# -lt makes "less than" check
# while loops as long as condition is true
while [ ${ii} -lt ${#ips[#]} ] ; do
# ${ips[$ii]} accesses array item with the given (${ii}) index
# be sure to use __double__ quotes around variable, otherwise it will not be expanded (value will not be inserted) but treated as a string
curl -k -u $user:$password "https://logservice/jobs" --data-urlencode "search=search index=test ${ips[$ii]} | head -1" > output.txt
# increase counter to avoid infinite loop
# and access the next item in an array
((ii++))
done
You may read about mapfile in GNU Bash reference: Built-ins.
You may read about creating and accessing arrays in GNU Bash reference: Arrays
Check this great post about quotes in bash.
I hope you found this answer helpful.
I believe you need something like this :
#!/bin/bash
function FN()
{
filename=$1
declare -a IPs_ARRAY
i=0
user=$2
password=$3
while read ip
do
IPs_ARRAY[$i]=$ip
echo ${IPs_ARRAY[$i]}
# Uncomment for your actions ::
#curl -k -u $user:$password "https://logservice/jobs" --data-urlencode 'search=search index=test ${IPs_ARRAY[$i]} | head 1' > output.txt
(( i++ ))
done < $filename
}
#############
### MAIN ###
###########
read -p "Enter username: " username
read -p "Enter password: " password
# Call your function
filename="list.txt"
FN $filename $username $password
I've got the following code, which is supposed to take filenames as arguments, and send them to my email address, which is derived from a username:
#Email Script from linux
#Define username e.g. pp_roman
u="$USER"
#Remove the pp_ and store to variable e.g. roman
u2=${u#"pp_"}
#Define Email portion
em="#workemail.com"
#Combine the username e.g. roman#workemail.com
u3=$u2$em
#Arguments for script
for FILE1 in "$#"
do
filename="-a $FILE1"
done
##This returns the full string with $filename variables for arguments, and email from $u3
mailx $filename -s "Subject" $u3 < /dev/null
However when passing multiple arguments, only the last mentioned filename is sent as an attachment. How do I pass multiple arguments into the $filename variable all appended by "-a"?
Since you are using bash, the right thing to use is an array.
attachments=()
for f in "$#"
do
attachments+=(-a "$f")
done
mailx "${attachments[#]}" -s "Subject" "$u3" < /dev/null
The following bash script's goal is to read CSV file ( all_words.CSV ) and print parameters and values but I have very strange problem.
When I run the script all words parameters (word1-word8) was printed - until now every thing is fine!When I want to print as word1=$word1 outside of function then from some reason word1 not get the value?
Why all parameters (word1-word8) print the values in function, and when I want to print word1 outside the function then word1 is without value?
I tried with export command but it doesn’t help as; export word1=$word1
Please advice how it can be? What the problem here?
#!/bin/bash
read_csv ()
{
CSV_LINE=2
vars=()
c=1
while IFS=, read -ra arr; do
if ((c==1)); then
vars+=("${arr[#]}")
elif ((c==CSV_LINE)); then
for ((i=0; i<${#arr[#]}; i++)); do
declare ${vars[$i]}="${arr[$i]}"
done
fi
((c++))
done < all_words.CSV
echo CSV_LINE=$CSV_LINE
echo word1=$word1
echo word2=$word2
echo word3=$word3
echo word4=$word4
echo word5=$word5
echo word6=$word6
echo word7=$word7
echo word8=$word8
}
read_csv
echo word1=$word1
.
more all_words.CSV
word1,word2,word3,word4,word5,word6,word7,word8
&^#G TR /erfernfjer *&^NHY " "" ? / $#H,#Y^%" E "R$%*&*UJ,**U&^#%%#$^&// \\,^T%!#&^YG.+___KI*&HHTY,%%#$#!%^#&,P/\06E87*UHG11#
,edehu234##!&,~hum&T%6e4
example of script output:
./readWords_from_csv.bash
CSV_LINE=2
word1=&^#G TR / erfernfjer *&^NHY " "" ? / $#H
word2=#Y^%" E "R$%*&*UJ
word3=**U&^#%%#$^&//\\
word4=^T%!#&^YG.+___KI*&HHTY
word5=%%#$#!%^#&
word6=P/\06E87*UHG11#
word7=edehu234##!&
word8=~hum&T%6e4
word1=
man bash explains under declare:
When used in a function, declare makes NAMEs local, as with the local command.
declare -g ${vars[$i]}="${arr[$i]}"
# ^^
Use declare -g to declare a variable at global level in a function. From man bash:
declare [-aAfFgilrtux] [-p] [name[=value] ...]
[...] The -g option forces
variables to be created or modified at the global scope, even
when declare is executed in a shell function. It is ignored in
all other cases. [...]
Here is a simple demonstration of the -g flag (works as expected on GNU bash, version 4.2.37):
#!/bin/bash
function f() {
declare -g V
V="hello"
}
f
echo $V
Please advice ...
Better use printf:
printf -v "${vars[$i]}" "%s" "${arr[$i]}"
Although I'd suggest using an associative array instead. It's the more appropriate solution:
#!/bin/bash
declare -A CSV_VALUES
declare -a CSV_KEYS
function read_csv {
CSV_VALUES=() CSV_KEYS=()
local VALUES I
{
IFS=, read -ra CSV_KEYS
IFS=, read -ra VALUES
} < all_words.csv
for I in "${!CSV_KEYS[#]}"; do
CSV_VALUES[${CSV_KEYS[I]}]=${VALUES[I]}
done
}
read_csv ## Perhaps pass the filename to read_csv as an argument instead?
# We can do for KEY in "${!CVS_VALUES[#]}" but the order is uncertain.
for KEY in "${CSV_KEYS[#]}"; do
echo "CSV_VALUES[$KEY]=${CSV_VALUES[$KEY]}"
done
In Linux, say I have the following file (e.g. conf.properties):
HOST_URL=http://$HOSTNAME
STD_CONFIG=http://$HOSTNAME/config
USER_CONFIG=http://$HOSTNAME/config/$unconfigured
I want to create another file with all the environment variables replaced...e.g. say the environment variable $HOSTNAME is 'myhost' and $unconfigured is not set, a script should produce the following output:
HOST_URL=http://myhost
STD_CONFIG=http://myhost/config
USER_CONFIG=http://myhost/config/
I was thinking this could be done in a simple one-liner with some sort of sed/awk magic, but I'm no expert and my searches have been in vein, so appreciate any help.
Edit:
I should mention that the file can really be any format text file, for example xml. I just want to replace anything that looks like an env variable with whatever is currently set in the environment.
This is what envsubst is for.
echo 'Hello $USER'
Hello $USER
echo 'Hello $USER' | envsubst
Hello malvineous
You would probably use it more like this though:
envsubst < input.txt > output.txt
envsubst seems to be part of GNU gettext.
sed 's/$HOSTNAME/myhost/g;s/$unconfigured//g' yourfile.txt > another_file.txt
update:
Based on updates in your question, this won't be a good solution.
update2 :
This is based on an answer to a related question. I've hacked at it (I'm unfamiliar with perl) to remove undefined vars.
perl -p -e 's/\$\{([^}]+)\}/defined $ENV{$1} ? $ENV{$1} : $&/eg; s/\$\{([^}]+)\}//eg' yourfile.txt
Should work for any input text file, however you will need to define vars using the ${...} format which simplifies the string matching.
(rant regarding the evilness of eval moved to a separate post so as not to confuse readers)
"eval is evil"
This is not an answer, but a warning in response to using eval for this task. You really really really don't want to do that.
Exhibit 1: a malicious template file:
HOST_URL=http://$HOSTNAME
STD_CONFIG=http://$HOSTNAME/config
USER_CONFIG=http://$HOSTNAME/config/$unconfigured
&& cat /etc/redhat-release
An unsuspecting user:
[lsc#aphek]$ cat somefile | while read line; do echo $(eval echo `echo $line`); done
HOST_URL=http://xyz
STD_CONFIG=http://xyz/config
USER_CONFIG=http://xyz/config/
Red Hat Enterprise Linux WS release 4 (Nahant Update 9)
Note the last line!
Now, imagine the possibilities....
I'd do it like this:
# Set the $HOSTNAME and other variables
# Now evaluate the properties file as a shell script.
. config.properties
# Write the values
cat >somefile <<EOF
HOST_URL=$HOST_URL
STD_CONFIG=$STD_CONFIG
USER_CONFIG=$USER_CONFIG
EOF
Edit: Or this very nasty thing (I'm sure there's a better way)
for name in HOST_URL STD_CONFIG USER_CONFIG
echo "$name=$(eval echo `echo '$'$name`)" >>somefile
end
Thanks to #DarkDust I came up with this:
cat somefile | while read line; do echo $(eval echo `echo $line`); done > somefile.replaced
I used this oneliner to replace ${VARIABLE} style variables in a file:
TARGET_FILE=/etc/apache2/apache2.conf; for VARNAME in $(grep -P -o -e '\$\{\S+\}' ${TARGET_FILE} | sed -e 's|^\${||g' -e 's|}$||g' | sort -u); do sed -i "s|\${$(echo $VARNAME)}|${!VARNAME}|g" ${TARGET_FILE}; done
I'm pretty sure someone can do this in 1/3rd of the length using awk… feel challenged! ;)
Here is a snippet of Javascript that I like to have around for solving this exact problem:
// A Javascript version of envsubst for our builds
// Purpose: replace all ocurrences of ${VAR} with the equivalent var from the environment from stdin
var readline = require('readline');
var rl = readline.createInterface({
input: process.stdin,
output: process.stdout,
terminal: false
});
const environment = process.env;
rl.on('line', function(line) {
const newLine = line.replace(/\$\{([a-zA-Z0-9_]+)\}/g, function(_match, variable) {
const envVar = environment[variable];
return envVar ? envVar : '';
});
process.stdout.write(`${newLine}\n`);
});
Hopefully this helps somebody else.
Here's a short one-liner that uses python's curly brace formatting to safely do the magic:
contents=\"\"\"`cat $file`\"\"\"; python -c "import os;print $contents.format(**os.environ)"
avoids evil eval
allows outputting curly braces: use {{ instead of {
no need to specify vars explicitly when calling the script
For example, given properties file settings.properties:
# my properties file
someVar = {MY_ENV_VAR}
curlyBraceVar = has {{curly braces}}
Then, do the substitution with:
$ export MY_ENV_VAR="hello"
$ file=settings.properties
$ contents=\"\"\"`cat $file`\"\"\"; python -c "import os;print $contents.format(**os.environ)"
# my properties file
someVar = hello
curlyBraceVar = has {curly braces}
A script is here: https://raw.githubusercontent.com/aneilbaboo/machome/master/bin/substenv
if you have installed nodejs you can run
npx #utft/tt -e FOO=bar /path/to/input /path/to/output
or you can run it programmatically
https://github.com/utftufutukgyftryidytftuv/tt