I have bash script that I use regularly in my job to automate a large job. I was making some changes today, but everything seemed fine. The script itself is about 1700 lines long. The first part of the script is all good and runs through all the user input and logic just fine. It then proceeds into the core of the script and stops working at exactly line 875 (tested the script with bash -x to find the break point). However, it breaks with the following error:
script.sh: line 1341: syntax error near unexpected token `;;'
script.sh: line 1341: ` ;;'
Line 1341 is in the middle of a case statement. The following code is the beginning of that block of code where it is breaking:
if [[ $VAR1 = "TRUE" && $VAR2 = "VAL2" ]]; then
VERSION=`XXXXXXXXXXXXXXXX`
## Set variables based on location $VAR3
case $VAR3 in
STR1 )
case $VERSION in
STR2 )
VAR4 = "STR5"
VAR5 = "STR6"
VAR6 = "STR7"
VAR7 = "STR8"
Line 1341 ---> ;;
STR3 )
VAR4="STR9"
VAR5="STR10"
VAR6="STR11"
VAR7="STR12"
;;
STR4 )
VAR4="STR13"
VAR5="STR14"
VAR6="STR15"
VAR7="STR16"
;;
esac
VAR8="STR17"
VAR9="STR18"
VAR10=1
VAR11="STR19"
;;
Because of the sensitive nature of what I do, I obviously had to remove quite a bit of information. I know this may make things more difficult to help me with. However, all VAR##="STR##" are standard variable declarations with string values, nothing special (no variable substitution, etc). All the variables are used later in the script. The code for VERSION returns a string value, which is used in the nested case.
The script was working fine up until my changes today, but I really didn't touch this section, with the exception of tweaking some of the STR values. I tried setting $VAR3 and $VERSION variables in quotes "", as well as the STR values used as the cases. I tried taking out this block entirely, only to have it fail on the next block (STR1 has a different value thus change the variable declarations). I have it output to the console what it is doing as well as checks for errors after most functions. There is nothing out of the ordinary on the console and nothing in the error log.
Any help would be appreciated, and I know I'm asking a lot.
By the way here is the code around line 875 where the script stops running (no errors generated based on the code here). Again, with bash -x I could see the VAR2 variable get set, but the script breaks before the next for loop starts.
## Create file ##
echo 'Creating files . . . '
j=0
p=1111
if [ $VAR1 = "TRUE" ]
then
VAR2=1
else
VAR2=2
fi
for i in `seq 1 $HOWMANY`; do <----Line 875
echo -n "Creating file . . . "
echo "XXXXXXXXXXX
Thanks again.
The problem is likely somewhere between line 875 (or a bit earlier) and line 1341. It maybe a misplaced quote or something less subtle. It will be essentially impossible for us to debug without all the original material between those lines.
Suggestion 1: run with 'bash -n -v' and see whether that gives you any insight into the problem.
Suggestion 2: split the script into smaller pieces that are more easily managed - and that can be separately debugged. The biggest scripts I have (out of 400 in my bin directory) are from the autoconf suite - they weigh in at just under 1100 lines; the next biggest is mine, and the 750 line script is too d..n big. The next biggest scripts are between 600 and 700 lines of Perl (including Perl documentation).
Having said 'missing quote', I see that your fragment close to line 875 has:
echo -n "Creating file . . . "
echo "XXXXXXXXXXX
with a missing close double quote from the second echo.
You also mentioned making changes, albeit not close to the point where the script breaks. Since you have the code under version control (you wouldn't dream of playing with a 1700 line script without backups, would you?), you should look at the actual changes again.
Or even back up to the previous working version, and make the changes again, one at a time, carefully, until you see why you broke something.
You have spaces around your equal signs in this section:
case $VERSION in
STR2 )
VAR4 = "STR5"
VAR5 = "STR6"
VAR6 = "STR7"
VAR7 = "STR8"
Take those out and you may be OK (unless that's a posting error).
Related
This question already has answers here:
Read user input inside a loop
(6 answers)
Closed 5 years ago.
First post here! I really need help on this one, I looked the issue on google, but can't manage to find an useful answer for me. So here's the problem.
I'm having fun coding some like of a framework in bash. Everyone can create their own module and add it to the framework. BUT. To know what arguments the script require, I created an "args.conf" file that must be in every module, that kinda looks like this:
LHOST;true;The IP the remote payload will connect to.
LPORT;true;The port the remote payload will connect to.
The first column is the argument name, the second defines if it's required or not, the third is the description. Anyway, long story short, the framework is supposed to read the args.conf file line by line to ask the user a value for every argument. Here's the piece of code:
info "Reading module $name argument list..."
while read line; do
echo $line > line.tmp
arg=`cut -d ";" -f 1 line.tmp`
requ=`cut -d ";" -f 2 line.tmp`
if [ $requ = "true" ]; then
echo "[This argument is required]"
else
echo "[This argument isn't required, leave a blank space if you don't wan't to use it]"
fi
read -p " $arg=" answer
echo $answer >> arglist.tmp
done < modules/$name/args.conf
tr '\n' ' ' < arglist.tmp > argline.tmp
argline=`cat argline.tmp`
info "Launching module $name..."
cd modules/$name
$interpreter $file $argline
cd ../..
rm arglist.tmp
rm argline.tmp
rm line.tmp
succes "Module $name execution completed."
As you can see, it's supposed to ask the user a value for every argument... But:
1) The read command seems to not be executing. It just skips it, and the argument has no value
2) Despite the fact that the args.conf file contains 3 lines, the loops seems to be executing just a single time. All I see on the screen is "[This argument is required]" just one time, and the module justs launch (and crashes because it has not the required arguments...).
Really don't know what to do, here... I hope someone here have an answer ^^'.
Thanks in advance!
(and sorry for eventual mistakes, I'm french)
Alpha.
As #that other guy pointed out in a comment, the problem is that all of the read commands in the loop are reading from the args.conf file, not the user. The way I'd handle this is by redirecting the conf file over a different file descriptor than stdin (fd #0); I like to use fd #3 for this:
while read -u3 line; do
...
done 3< modules/$name/args.conf
(Note: if your shell's read command doesn't understand the -u option, use read line <&3 instead.)
There are a number of other things in this script I'd recommend against:
Variable references without double-quotes around them, e.g. echo $line instead of echo "$line", and < modules/$name/args.conf instead of < "modules/$name/args.conf". Unquoted variable references get split into words (if they contain whitespace) and any wildcards that happen to match filenames will get replaced by a list of matching files. This can cause really weird and intermittent bugs. Unfortunately, your use of $argline depends on word splitting to separate multiple arguments; if you're using bash (not a generic POSIX shell) you can use arrays instead; I'll get to that.
You're using relative file paths everywhere, and cding in the script. This tends to be fragile and confusing, since file paths are different at different places in the script, and any relative paths passed in by the user will become invalid the first time the script cds somewhere else. Worse, you aren't checking for errors when you cd, so if any cd fails for any reason, then entire rest of the script will run in the wrong place and fail bizarrely. You'd be far better off figuring out where your system's root directory is (as an absolute path), then referencing everything from it (e.g. < "$module_root/modules/$name/args.conf").
Actually, you're not checking for errors anywhere. It's generally a good idea, when writing any sort of program, to try to think of what can go wrong and how your program should respond (and also to expect that things you didn't think of will also go wrong). Some people like to use set -e to make their scripts exit if any simple command fails, but this doesn't always do what you'd expect. I prefer to explicitly test the exit status of the commands in my script, with something like:
command1 || {
echo 'command1 failed!' >&2
exit 1
}
if command2; then
echo 'command2 succeeded!' >&2
else
echo 'command2 failed!' >&2
exit 1
fi
You're creating temp files in the current directory, which risks random conflicts (with other runs of the script at the same time, any files that happen to have names you're using, etc). It's better to create a temp directory at the beginning, then store everything in it (again, by absolute path):
module_tmp="$(mktemp -dt module-system)" || {
echo "Error creating temp directory" >&2
exit 1
}
...
echo "$answer" >> "$module_tmp/arglist.tmp"
(BTW, note that I'm using $() instead of backticks. They're easier to read, and don't have some subtle syntactic oddities that backticks have. I recommend switching.)
Speaking of which, you're overusing temp files; a lot of what you're doing with can be done just fine with shell variables and built-in shell features. For example, rather than reading line from the config file, then storing them in a temp file and using cut to split them into fields, you can simply echo to cut:
arg="$(echo "$line" | cut -d ";" -f 1)"
...or better yet, use read's built-in ability to split fields based on whatever IFS is set to:
while IFS=";" read -u3 arg requ description; do
(Note that since the assignment to IFS is a prefix to the read command, it only affects that one command; changing IFS globally can have weird effects, and should be avoided whenever possible.)
Similarly, storing the argument list in a file, converting newlines to spaces into another file, then reading that file... you can skip any or all of these steps. If you're using bash, store the arg list in an array:
arglist=()
while ...
arglist+=("$answer") # or ("#arg=$answer")? Not sure of your syntax.
done ...
"$module_root/modules/$name/$interpreter" "$file" "${arglist[#]}"
(That messy syntax, with the double-quotes, curly braces, square brackets, and at-sign, is the generally correct way to expand an array in bash).
If you can't count on bash extensions like arrays, you can at least do it the old messy way with a plain variable:
arglist=""
while ...
arglist="$arglist $answer" # or "$arglist $arg=$answer"? Not sure of your syntax.
done ...
"$module_root/modules/$name/$interpreter" "$file" $arglist
... but this runs the risk of arguments being word-split and/or expanded to lists of files.
I am reading a source code, and find these lines :
if [ -n "${INIT_NAMENODE+1}" ]
then
echo "Initializing namenode"
else
echo "Starting namenode"
fi
how should I interpred the 'if' condition : if [ -n "${INIT_NAMENODE+1}" ] ? ?
The nice thing about this code is that it is not written for a "Linux shell". It is written for the more general category of "UNIX shell". It will work in everything since V7 UNIX (1979) at least. People with lesser portability goals might write it without the -n.
The first item of interest is the ${foo+bar} syntax. This is a test for existence of the foo parameter. If $foo exists, then ${foo+bar} equals bar. If $foo doesn't exist, then ${foo+bar} equals the empty string.
If you look for this in your shell man page, it's usually documented as ${foo:+bar}, along with some other related forms like ${foo:-bar}, and somewhere nearby there's a note explaining that the colon can be omitted from all of them, resulting in slightly different behavior (with the colon, variables whose value is the empty string are treated the same as nonexistent variables).
Next we have the [ -n ... ] test. -n tests the following string for emptiness. It succeeds if the string is non-empty. From the previous paragraph we know that ${INIT_NAMENODE+1} is empty if and only if $INIT_NAMENODE doesn't exist. So the -n test succeeds if $INIT_NAMENODE exists. The value 1 doesn't really matter here - it would do the same thing if you changed the 1 to 2 or 0 or teapot. All that matters is that it's not an empty string, since -n doesn't care about the rest.
Try some examples from your shell prompt: echo ${PATH+hello} should say hello because you do have a $PATH variable. echo ${asdfghjkl+hello} should print a blank line.
So, in the context of the if statement, the purpose of the test is to do the first echo if the variable $INIT_NAMENODE exists, and the second echo if the variable $INIT_NAMENODE doesn't exist.
I am using a bash script to execute a program. The program must take the following argument. (The program is gnuplot.)
gnuplot -e "filename='output_0.csv'" 'plot.p'
I need to be able to assemble the following string: "filename='output_0.csv'"
My plan is to assemble the string STRING=filename='output_0.csv' and then do the following: gnuplot -r "$STRING" 'plot.p'. Note I left the words STRING without stackoverflow syntax style highlighting to emphasise the string I want to produce.
I'm not particularly proficient at bash, and so I have no idea how to do this.
I think that strings can be concatenated by using STRING="$STRING"stuff to append to string? I think that may be required?
As an extra layer of complication the value 0 is actually an integer which should increment by 1 each time the program is run. (Done by a for loop.) If I have n=1 in my program, how can I replace the 0 in the string by the "string value" or text version of the integer n?
A safest way to append something to an existing string would be to include squiggly brackets and quotes:
STRING="something"
STRING="${STRING}else"
You can create the "dynamic" portion of your command line with something like this:
somevalue=0
STRING="filename='output_${somevalue}.csv'"
There are other tools like printf which can handle more complex formatting.
somevalue=1
fmt="filename='output_%s.csv'"
STRING="$(printf "$fmt" "$somevalue")"
Regarding your "extra layer of complication", I gather that this increment has to happen in such a way as to store the value somewhere outside the program, or you'd be able to use a for loop to handle things. You can use temporary files for this:
#!/usr/bin/env bash
# Specify our counter file
counter=/tmp/my_counter
# If it doesn't exist, "prime" it with zero
if [ ! -f "$counter" ]; then
echo "0" > $counter
fi
# And if it STILL doesn't exist, fail.
if [ ! -f "$counter" ]; then
echo "ERROR: can't create counter." >&2
fi
# Read the last value...
read value < "$counter"
# and set up our string, per your question.
STRING="$(printf "filename='output_%d.csv'" "${value}")"
# Last, run your command, and if it succeeds, update the stored counter.
gnuplot -e "$STRING" 'plot.p' && echo "$((value + 1))" > $counter
As always, there's more than one way to solve this problem. With luck, this will give you a head start on your reading of the bash man page and other StackOverflow questions which will help you learn what you need!
An answer was posted, which I thought I had accepted already, but for some reason it has been deleted, possibly because it didn't quite answer the question.
I posted another similar question, and the answer to that helped me also answer this question. You can find said question and answer here: bash: Execute a string as a command
I'am new in Linux and I want to write a bash script that can read in a file name of a directory that starts with LED + some numbers.(Ex.: LED5.5.002)
In that directory there is only one file that will starts with LED. The problem is that this file will every time be updated, so the next time it will be for example LED6.5.012 and counting.
I searched and tried a little bit and came to this solution:
export fspec=/home/led/LED*
LedV=`basename $fspec`
echo $LedV
If I give in those commands one by one in my terminal it works fine, LedV= LED5.5.002 but if i run it in a bash scripts it gives the result: LedV = LED*
I search after another solution:
a=/home/led/LED*
LedV=$(basename $a)
echo $LedV
but here again the same, if i give it in one by one it's ok but in a script: LedV = LED*.
It's probably something small but because of my lack of knowledge over Linux I cannot find it. So can someone tell what is wrong?
Thanks! Jan
Shell expansions don't happen on scalar assignments, so in
varname=foo*
the expansion of "$varname" will literally be "foo*". It's more confusing when you consider that echo $varname (or in your case basename $varname; either way without the double quotes) will cause the expansion itself to be treated as a glob, so you may well think the variable contains all those filenames.
Array expansions are another story. You might just want
fspec=( /path/LED* )
echo "${fspec[0]##*/}" # A parameter expansion to strip off the dirname
That will work fine for bash. Since POSIX sh doesn't have arrays like this, I like to give an alternative approach:
for fspec in /path/LED*; do
break
done
echo "${fspec##*/}"
pwd
/usr/local/src
ls -1 /usr/local/src/mysql*
/usr/local/src/mysql-cluster-gpl-7.3.4-linux-glibc2.5-x86_64.tar.gz
/usr/local/src/mysql-dump_test_all_dbs.sql
if you only have 1 file, you will only get 1 result
MyFile=`ls -1 /home/led/LED*`
I wanted to run a simple bash timer and found this online (user brent7890)
#!/usr/bin/bash
timer=60
until [ "$timer" = 0 ]
do
clear
echo "$timer"
timer=`expr $timer - 1`
sleep 1
done
echo "-------Time to go home--------"
I couldn't copy and paste this code because the server is on another network. I typed it like this (below) and got an error on the line that starts with "until".
#!/usr/bin/bash
timer=60
#Note I forgot the space between [ and "
until ["$timer" = 0 ]
do
clear
echo "$timer"
timer=`expr $timer - 1`
sleep 1
done
echo "-------Time to go home--------"
Where is spacing like this documented? It seems strange that it matters. Bash scripts can be confusing, I want to understand why the space is important.
There are several rules, two basic of that are these:
You must separate all arguments of a command with spaces.
You must separate a command and the argument, that follows after, with a space.
[ here is a command (test).
If you write ["$timer" that means that you start command [60,
and that is, of course, incorrect. The name of the command is [.
The name of the command is always separated from the rest of the command line with a space. (you can have a command with a space in it, but in this case you must write the name of the command in "" or '').