Confused with shell script reading CSV data from file - linux

I'm fairly new to Linux/CentOS and shell scripting. Let's say I have a file named useraccounts.list. This file contains CSV data in this specific order:
first name, middle initial, last name, username, password
for example:
John,N,Snow,seords,cuai2Ohzdh
What I'm trying to do is traverse through this list creating user accounts from the provided data, and with the password at the end of each line. I would like to store it using the passwd command for each individual user.
Here is what I have so far:
#!/bin/sh
for i in 'more useraccounts.list'
do
echo "$1 $2 $3 $4 $5 $6"
done <file.txt
Can someone please help me out?

Before you get to how you will store the password, you need to sort out how you will read the values from the file correctly. In bash, the primary tool for determining how the shell will break a line into individual tokens is the Internal Field Separator (default: space tab newline). The process is called word-splitting The IFS variable allows you to set the characters that will control word-splitting.
You can use that to your advantage in reading lines such as yours by including a comma in IFS. This will allow you to specify individual variable to read each name, initial, last, user name and password into. A while loop is the normal way to accomplish this -- and it allows you to set the IFS for that block of code without effecting the remainder of the script.
An Example, in your case, would be:
#!/bin/bash
[ -z "$1" ] && { ## validate one argument given on command line
printf "error: insufficient input. usage: %s filename.\n" "${0##*/}"
exit 1
}
[ -r "$1" ] || { ## validate it is a readable filename
printf "error: file not found/readable '%s'.\n" "$1"
exit 1
}
## read each line in file separated by ','
# set Internal Field Separator to break on ',' and '\n'
# protect against lack of '\n' on last line with $pw test
while IFS=$',\n' read -r first mi last uname pw || [ -n "$pw" ]; do
printf "name: %-5s %s. %-6s user: %s pass: %s\n" \
"$first" "$mi" "$last" "$uname" "$pw"
## Create User Accounts/Store Password Here...
done <"$1"
exit 0
Input
$ cat dat/useracct.txt
John,N,Snow,seords,cuai2Ohzdh
Jill,O,Rain,reords,cuai3Ohzdh
Jane,P,Sleet,peords,cuai4Ohzdh
Output
$ bash readuserfile.sh dat/useracct.txt
name: John N. Snow user: seords pass: cuai2Ohzdh
name: Jill O. Rain user: reords pass: cuai3Ohzdh
name: Jane P. Sleet user: peords pass: cuai4Ohzdh
You can then create the user accounts with the desired options and store the password any way you like. Let me know if you have any questions.

Related

Inserting the date into a "MAIL" message body in a BASH script

I have a relatively simple BASH script to send mail from my Raspberry Pi. The first argument is the Subject line and the second is a string of data files to be attached.
It is basically working when I specify the message body as a file (line 6). But if I try to create a text sting containing the date as the message body it fails (line7). Here is my script:
#!/bin/bash
#echo $2
# To
TO="me#hotmail.com"
# Message
MESSAGE="output/MessageBody.txt"
MESSAGEx="Midnight `date '+%Y-%m-%d %H:%M:%S %Z'` Pi report"
echo $MESSAGE
echo $MESSAGEx
temp=$(echo $2 | tr ";" "\n")
declare -a attargs
for att in $temp; do
attargs+=( "-A" "$att" )
done
# Sending email using /bin/mail
/usr/bin/mail -s "$1" "$TO" ${attargs[#]} < $MESSAGEx
Here is the output from this command
/usr/pgms/sendtome.sh "test message" "/mnt/usbdrive/output/JSONstart.txt;/mnt/usbdrive/output/Outback_error.log;/mnt/usbdrive/output/OutbackReaderPrint.txt"
when I specify MESSAGEx as the message body:
/mnt/usbdrive/output/MessageBody.txt
Midnight 2019-08-14 07:40:31 MDT Pi report
/usr/pgms/sendtome.sh: line 22: $MESSAGEx: ambiguous redirect
If I use MESSAGE, ie the text file reference, it works.
How can it create a message body text paragraph which contains the date or some other item? Thanks....RDK
There's a number of issues here.
You should generally quote strings. Without quoting, the string after < is split (hence the error message) and the array you took so much care to collect will lose its purpose.
The thing after < needs to be the name of a file. In Bash you can use a here string <<<"$MESSAGEx" but the common and simple portable solution is to echo (or better printf) its value into a pipe.
You should prefer lower case for your private variable names, but this is mainly a stylistic recommendation. (There are reserved variables like PATH and SHELL which you really don't want to clobber; POSIX reserves upper case variable names for system use.)
Here's a refactoring which attempts to address these concerns.
#!/bin/bash
to="me#hotmail.com"
# Message
#msgfile="output/MessageBody.txt"
msgbody="Midnight `date '+%Y-%m-%d %H:%M:%S %Z'` Pi report"
#echo "$msgfile"
#echo "$msgbody"
declare -a attargs
for att in $(echo "$2" | tr ";" "\n"); do
attargs+=( "-A" "$att" )
done
/usr/bin/mail -s "$1" "${attargs[#]}" "$to"<<< "$msgbody"
Perhaps a better design would be to just shift the first argument and then use "$#" as the list of files to attach.

Expanding a string with a variable reference later, after the variable is assigned

I'm trying to combine two lists containing names (if available) and emails with a standard email text in bash (shell)
(I had to delete the irrelevant code as it contains some private info, so some of the code might look unusal.)
The first half of the code checks if there is a name list along with the email list.
The second half combines only the email address and text if no name is available, if the name list is available it also 'tries' to combine the name, email and text.
f1 = email list and f2 = name list.
As you can see in the first half of the code below, $f2 should show the names if the list is available but it does not show anything in the log file.
I been trying to sort this problem out for two days but nothing has worked. When names are available it always outputs as "Hello ..." when it should be "Hello John D..."
#FIRST HALF
if [ "$names" = "no" ]
then
text="Hello..."
elif [ "$names" = "yes" ]
then
text="Hello $f2..."
fi
#SECOND HALF
if [ "$names" = "no" ]
then
for i in $(cat $emaillist); do
echo "$text" >> /root/log
echo "$i" >> /root/log
done
elif [ "$names" = "yes" ]
then
paste $emaillist $namelist | while IFS="$(printf '\t')" read -r f1 f2
do
echo "$text" >> /root/log
echo "$f1" >> /root/log
done
fi
When you run text="Hello $f2", $f2 is looked up at the time of the assignment; an exact string is assigned to text, and only that exact string is used later, on echo "$text".
This is very desirable behavior: If shell variables' values could run arbitrary code, it would be impossible to write shell scripts that handled untrusted data safely... but it does mean that implementing your program requires some changes.
If you want to defer evaluation (looking up the value of $f2 at expansion time rather than assignment), don't use a shell variable at all: Use a function instead.
case $names in
yes) write_greeting() { echo "Hello $name..."; };;
*) write_greeting() { echo "Hello..."; };;
esac
while read -r name <&3 && read -r email <&4; do
write_greeting
echo "$email"
done 3<"$namelist" 4<"$emaillist" >>/root/log
Some enhancements in the code above:
You don't need paste to read from two streams in lockstep; you can simply open them on different file descriptors (above, FDs 3 and 4 are chosen; only 0, 1 and 2 are reserved, so larger numbers could have been selected as well) with a separate read command for each.
Opening your output sink only once for the entire loop (by putting the redirection after the done) is far more efficient than re-opening it every time you want to write a single line.
Expansions, such as "$namelist" and "$emaillist", are always quoted; this makes code more reliable if dealing with filenames with unusual characters (including spaces and glob expressions), or if IFS is at a non-default value.

How to use Read command in echo like fill in the blanks

Am writing a script.Eg:
echo "my name is 'read' and am from 'read' city" > outfile.txt
When it runs it's not printing the sentence first, i.e my name is. Rather it's asking first to enter 2 inputs for 2 read commands used, then its forming the complete sentence like "my name is sudhir and am from vizag city"
I want script to execute 1st "my name is read(ask for input) and am from read(ask for another input) city" and after giving inputs it should redirect to outfile.txt in one shot.
How to handle this? Is it feasible to achieve in single sentence?
Because I want to use same logic for 480 questions all in one file and persons how don't have any scripting knowledge should able to add more questions taking reference of previous questions present in same file.
There isn't a pretty way to do this, since read writes the newline character from the input to the terminal, but we can do this in two passes.
Get the input from the user
Write the results to the file
You could put the following in a script
#!/bin/bash
echo -n 'my name is '; read -r name
echo -n ' and I am from '; read -r city
echo ' city'
printf "my name is %s and I am from %s city\n" \
"$name" "$city" > output.txt
And to the user it would look like this
my name is sudhir
and I am from vizag
city
But it would be like this in the file
my name is sudhir and I am from vizag city
You can write a function that takes a string with placeholders and asks the user to input for each of them:
#!/bin/bash
fill() {
arg="$*"
result=""
while [[ "$arg" =~ ([^_]*)(_+)(.*) ]]
do
read -rp "${BASH_REMATCH[1]# }${BASH_REMATCH[2]}: " input
result+="${BASH_REMATCH[1]}${input}"
arg="${BASH_REMATCH[3]}"
done
result+="$arg"
printf '%s\n' "$result"
}
exec > outputfile
fill "My name is ____ and I am from ___."
fill "My new years resolution is ____."
Example:
$ ./myscript
My name is ____: Sudhir
and I am from ___: Vizag
My new years resolution is ____: learning Bash instead of asking SO to write my scripts
$ cat outputfile
My name is Sudhir and I am from Vizag.
My new years resolution is learning Bash instead of asking SO to write my scripts.

How to extract key value pairs from a file when values span multiple lines?

I'm a few weeks into bash scripting and I haven't advanced enough yet to get my head wrapped around this problem. Any help would be appreciated!
I have a "script.conf" file that contains the following:
key1=value1
key2=${HOME}/Folder
key3=( "k3v1" "k3 v2" "k3v3")
key4=( "k4v1"
"k4 v2"
"k4v3"
)
key5=value5
#key6="Do Not Include Me"
In a bash script, I want to read the contents of this script.conf file into an array. I've learned how to handle the scenarios for keys 1, 2, 3, and 5, but the key4 scenario throws a wrench into it with it spanning across multiple lines.
I've been exploring the use of sed -n '/=\s*[(]/,/[)]/{/' which does capture key4 and its value, but I can't figure out how to mix this so that the other keys are also captured in the matches. The range syntax is also new to me, so I haven't figured out how to separate the key/value. I feel like there is an easy regex that would accomplish what I want... in plain-text: "find and group the pattern ^(.*)= (for the key), then group everything after the '=' char until another ^(.*)= match is found, rinse and repeat". I guess if I do this, I need to change the while read line to not handle the key/value separation for me (I'll be looking into this while I'm waiting for a response). BTW, I think a solution where the value of key4 is flattened (new lines removed) would be acceptable; I know for key3 I have to store the value as a string and then convert it to an array later when I want to iterate over it since an array element apparently can't contain a list.
Am I on the right path with sed or is this a job for awk or some other tool? (I haven't ventured into awk yet). Is there an easier approach that I'm missing because I'm too deep into the forest (like changing the while read line in the LoadConfigFile function)?
Here is the code that I have so far in script.sh for processing and capturing the other pairs into the $config array:
__AppDir=$(dirname $0)
__AppName=${__ScriptName%.*}
typeset -A config #init config array
config=( #Setting Default Config values
[key1]="defaultValue1"
[key2]="${HOME}/defaultFolder"
[QuietMode]=0
[Verbose]=0 #Ex. Usage: [[ "${config[Verbose]}" -gt 0 ]] && echo ">>>Debug print"
)
function LoadConfigFile() {
local cfgFile="${1}"
shopt -s extglob #Needed to remove trailing spaces
if [ -f ${cfgFile} ]; then
while IFS='=' read -r key value; do
if [[ "${key:0:1}" == "#" ]]; then
#echo "Skipping Comment line: ${key}"
elif [ "${key:-EMPTY}" != "EMPTY" ]; then
value="${value%%\#*}" # Delete in-line, right comments
value="${value%%*( )}" # Delete trailing spaces
value="${value%%( )*}" # Delete leading spaces
#value="${value%\"*}" # Delete opening string quotes
#value="${value#\"*}" # Delete closing string quotes
#Manipulate any variables included in the value so that they can be expanded correctly
# - value must be stored in the format: "${var1}". `backticks`, "$var2", and "doubleQuotes" are left as is
value="${value//\"/\\\"}" # Escape double quotes for eval
value="${value//\`/\\\`}" # Escape backticks for eval
value="${value//\$/\\\$}" # Escape ALL '$' for eval
value="${value//\\\${/\${}" # Undo the protection of '$' if it was followed by a '{'
value=$(eval "printf '%s\n' \"${value}\"")
config[${key}]=${value} #Store the value into the config array at the specified key
echo " >>>DBG: Key = ${key}, Value = ${value}"
#else
# echo "Skipped Empty Key"
fi
done < "${cfgFile}"
fi
}
CONFIG_FILE=${__AppDir}/${__AppName}.conf
echo "Config File # ${CONFIG_FILE}"
LoadConfigFile ${CONFIG_FILE}
#Print elements of $config
echo "Script Config Values:"
echo "----------------------------"
for key in "${!config[#]}"; do #The '!' char gets an array of the keys, without it, we would get an array of the values
printf " %-20s = %s\n" "${key}" "${config[${key}]}"
done
echo "------ End Script Config ------"
#To convert to an array...
declare -a valAsArray=${config[RequiredAppPackages]} #Convert the value from a string to an array
echo "Count = ${#valAsArray[#]}"
for itemCfg in "${valAsArray[#]}"; do
echo " item = ${itemCfg}"
done
As I mentioned before, I'm just starting to learn bash and Linux scripting in general, so if you see that I'm doing some taboo things in other areas of my code too, please feel free to provide feedback in the comments... I don't want to start bad habits early on :-).
*If it matters, the OS is Ubuntu 14.04.
EDIT:
As requested, after reading the script.conf file, I would like for the elements in $config[#] to be equivalent to the following:
typeset -A config #init config array
config=(
[key1]="value1"
[key2]="${HOME}/Folder"
[key3]="( \"k3v1\" \"k3 v2\" \"k3v3\" )"
[key4]="( \"k4v1\" \"k4 v2\" \"k4v3\" )"
[key5]="value5"
)
I want to be able to convert the values of elements 'key4' and 'key3' into an array and iterated over them the same way in the following code:
declare -a keyValAsArray=${config[keyN]} #Convert the value from a string to an array
echo "Count = ${#keyValAsArray[#]}"
for item in "${keyValAsArray[#]}"; do
echo " item = ${item}"
done
I don't think it matters if \n is preserved for key4's value or not... that depends on if declare has a problem with it.
A shell is an environment from which to call tools with a language to sequence those calls. It is NOT a tool to manipulate text. The standard UNIX tool to manipulate text is awk. Trying to manipulate text in shell IS a bad habit, see why-is-using-a-shell-loop-to-process-text-considered-bad-pr‌​actice for SOME of the reasons why
You still didn't post the expected result of populating the config array so I'm not sure but I think this is what you wanted:
$ cat tst.sh
declare -A config="( $(awk '
{ gsub(/^[[:space:]]+|([[:space:]]+|#.*)$/,"") }
!NF { next }
/^[^="]+=/ {
name = gensub(/=.*/,"",1)
value = gensub(/^[^=]+=/,"",1)
n2v[name] = value
next
}
{ n2v[name] = n2v[name] OFS $0 }
END {
for (name in n2v) {
value = gensub(/"/,"\\\\&","g",n2v[name])
printf "[%s]=\"%s\"\n", name, value
}
}
' script.conf
) )"
declare -p config
$ ./tst.sh
declare -A config='([key5]="value5" [key4]="( \"k4v1\" \"k4 v2\" \"k4v3\" )" [key3]="( \"k3v1\" \"k3 v2\" \"k3v3\")" [key2]="/home/Ed/Folder" [key1]="value1" )'
The above uses GNU awk for gensub(), with other awks you'd use [g]sub() instead.

Bash script looping after end of file

My bash script asks user for their first name, last name, address and phone number and writes this information that is input by the user to a file of the format "firstname.lastname"; however I want to repeat this a number of times (I actually wanted to do something like a do..while loop where it runs atleast one time and asks user if they want to continue creating accounts or not but I see there is no do...while for bash it seems). So when I execute this bash script in the terminal it will ask for how many accounts to be made and I provide all the input but the it only runs one time. What am I doing wrong? Here is my script.
#!bin/bash
echo "How many accounts are you creating?"
read num
echo "Enter first name"
read fName
echo "Enter last name"
read lName
echo "Enter address"
read add
echo "Enter phone number"
read phn
echo "Enter gender m for male and f for female"
read gender
if [ "$gender" == "m" ]
then
sex="male"
elif [ "$gender" == "f" ]
then
sex="female"
else
echo"Invalid gender. Restart the script and enter a valid gender"
exit 0
fi
for (( i = 0; i<=num; i++))
do
cat > $fName.$lName <<-EOF
Full Name: $fName $lName
Address: $add
Phone number: $phn
Gender: $gender
EOF
done
zerobandwidth's answer is correct, but as an alternative answer, it is in fact quite easy to do what you initially wanted to do:
while true; do
# Your account creation code goes here
echo "Create another account (y/n)?"
read another
if [ "$another" != y ]; then
break
fi
done
Put your input code inside the loop, or wrap that code in a function and call the function inside the loop.

Resources