I am a Unix/Linux shell script newbie and I have been asked to look at a script which contains the lines below. The following details in this question are vague but the person who wrote this code left no documentation and has since demised. Can anyone advise what they actually do?
There are two specific pieces of code. The first is simply line source polys.sh where polys.sh is a text file with contents:
failure="020o 040a"
success="002[a-d] 003[a-r] 004[a-s] 005[a-u]
Representing various parameters, I think, to do with the calculations the shell script performs. The nature of the calculations is, I am told, not important because the aim is to just get the script running.
The second piece of code is below and the relevant lines are delimited by Start and Stop comments. What I can tell you is that: $arg1 is blank, $opt1 is also blank, $poly is the path and name of a text file and ./search I believe to be a folder.
if [ $search == "yes" ]
then
# Search stage for squares containing zeros
#
# Start.
output="$outputs/search/"`basename $poly`
./search $opt1 $arg1 < $poly 2>&1 | tee $output
if tail -n1 $output | grep -v "success"
# End.
then
echo "SEARCH FAILURE" >> $output
continue
fi
# Save approximations
#
echo -n "SEARCH SUCCESS " >> $output
cat /tmp/iters >> $output
cp /tmp/zeros $inputs/search/`basename $poly`
else
echo "No search"
fi
EDIT Initial disclaimer as advised by Mr. Charles Duffy:
The below explanations assume you won't hit expansion-related bugs; please correct your code as advised by shellcheck.net to be assured that these explanations are correct
source polys.sh includes the code from the script polys.sh, which is a file in the same folder as the file sourcing it (hence just the filename, without its path).
Within that file:
failure="020o 040a"
success="002[a-d] 003[a-r] 004[a-s] 005[a-u]"
are two variable declarations; the variable $failure is set to "020o 040a" and $success to "002[a-d] 003[a-r] 004[a-s] 005[a-u]". As the file was sourced, these two variables are available in your script (do echo "$failure" and echo "$success" to see for yourself).
output="$outputs/search/`basename $poly`" has two parts to explain:
"$outputs/search/"
sets the variable $output to "$outputs/search/", i.e., to the value of the variable $outputs, appended by the string "/search/".,
`basename $poly`
anything in backticks is a command substitution, which interprets and runs the command returning its output, and the command basename $poly gets the base file or folder name from $poly, if it is a file path (e.g., basename $poly for poly="/dev/file.txt" yields file.txt); the output is appended as a string. to "$outputs/search/".
./search $opt1 $arg1 < $poly 2>&1 | tee $output is two commands, separated by a pipe |:
./search $opt1 $arg1 < $poly 2>&1
runs the executable file ./search (./ is shorthand for the current script's directory) with two arguments, $opt1 and $opt2 variables. $poly is the variable name which should represent a file path, of which the file path has its content redirected to the command (using <). The output of all errors (stderr, as 2) is redirected (>) to the standard output (stdout, or &2, the ampersand represents this is a file descriptor, not a file path, otherwise it would redirect output to a file named 2).
tee $output
tee pipes outputs stdin to stdout and to arguments as file paths. So tee "/home/nick/output" would save the stdin to a file at "/home/nick/output", as well as the stdout.
if tail -n1 $output | grep -v "success"
tail -n1 $output
gets the last line of the file at the "$output" variable's value.
grep -v "success"
searches for any non-match (-v inverts the match) in the last line from tail -n1 of "success" in a line (e.g., if the last line is "fail", it would pass the if statement as it does not contain "success")
Related
I have written a small script that will take the users input and then generate the md5sum values for it
count = 0
echo "Enter number of records"
read number
while [ $count -le $number ]
do
echo "Enter path"
read path
echo "file name"
read file_name
md5sum $path"/"$filename #it shows the md5sum value and path+filename
((count++))
done
How can I pass these values ( path,file name, and md5sums ) to CSV file. ( assuming the user chooses to enter more than 1 record)
The output should be like
/c/training,sample.txt,34234435345346549862123454651324 #placeholder values
/c/file,text.sh,4534534534534534345345435342342
Interactively prompting for the number of files to process is just obnoxious. Change the script so it accepts the files you want to process as command-line arguments.
#!/bin/sh
md5sum "$#" |
sed 's%^\([0-9a-f]*\) \(\(.*\)/\)?\([^/]*\)$%\3,\4,\1%'
There are no Bash-only constructs here, so I switched the shebang to /bin/sh; obviously, you are still free to use Bash if you like.
There is a reason md5sum prints the checksum before the path name. The reordered output will be ambiguous if you have file names which contain commas (or newlines, for that matter). Using CSV format is actually probably something you should avoid if you can; Unix tools generally work better with simpler formats like tab-delimited (which of course also breaks if you have file names with tabs in them).
Rather than prompting the user for both a path to a directory and the name of a file in that directory, you could prompt for a full path to the file. You can then extract what you need from that path using bash string manipulations.
#!/bin/bash
set -euo pipefail
function calc_md5() {
local path="${1}"
if [[ -f "${path}" ]] ; then
echo "${path%/*}, ${path##*/}, $(md5sum ${path} | awk '{ print $1 }')"
else
echo "
x - Script requires path to file.
Usage: $0 /path/to/file.txt
"
exit 1
fi
}
calc_md5 "$#"
Usage example:
$ ./script.sh /tmp/test/foo.txt
/tmp/test, foo.txt, b05403212c66bdc8ccc597fedf6cd5fe
I am currently learning linux bash scripting:
I have files in a folder with the following filename-pattern:
ABC01_-12ab_STRINGONE_logicMatches.txt
DEF02_-12ab_STRINGTWO_logicMatches.txt
JKL03_-12ab_STRINGTHREE_logicMatches.txt
I want to extract STRINGONE, STRINGTWO and STRINGTHREE as a list. To see, if my idea works, I wanted to echo my result to bash first.
Code of my bash-script (executed in the folder, where the files are located):
#!/bin/bash
for element in 'folder' do out='cut -d "_" -f2 $element | echo $out' done
Actual result:
error: unexpected end of file
Desired result:
STRINGONE
STRINGTWO
STRINGTHREE
(echoed in bash)
The idea you are doing is right. But the syntax of file globbing (looking for text files) and command substitution (running the cut command) is wrong. You need to do
for file in folder/*.txt;
# This condition handles the loop exit if no .txt files are found, and
# not throw errors
[ -f "$file" ] || continue
# The command-substitution syntax $(..) runs the command and returns the
# result out to the variable 'out'
out=$(cut -d "_" -f3 <<< "$file")
echo "$out"
done
I want to get a fifo's content and print it in a file, and I have this code:
path=$1 #path file get from script's input
if [ -p "$path" ];then #check if path is pipe
content = 'cat "$path"'
echo "$content" > output
exit 33
fi
My problem is that when I execute the cat "$path" line the script is stopped and the terminal displays the underscore.
I don't know how to solve this problem
P.S the fifo isn't empty and output is the file where I want to print fifo's content
If the FIFO is not empty, and there are no longer any file descriptors writing to that FIFO, you'll get EOF in the cat command. From man 7 pipe:
If all file descriptors referring to the write end of a pipe have been
closed, then an attempt to read(2) from the pipe will see end- of-file
(read(2) will return 0).
Source: man7.org/linux/man-pages/man7/pipe.7.html
Your assignment statement is incorrect.
Whitespace around = is not permitted.
You're confusing single quotes with backquotes. However, you should use $(...) for command substitution anyway.
The correct assignment is
content=$(cat "$path")
or more efficiently in bash,
content=$(< "$path")
I have an executable that takes a file and outputs a line.
I am running a loop over a directory:
for file in $DIRECTORY/*.png
do
./eval $file >> out.txt
done
The output of executable does not contain the name of the file.
I want to append the file name with each output.
EDIT1
Perhaps, I could not explain it correctly
I want the name of the file and the output of the program as well, which is processing the same file, Now I am doing following
for file in $DIRECTORY/*.png
do
echo -n $file >> out.txt
or
printf "%s" "$file" >> out.txt
./eval $file >> out.txt
done
For both new line is inserted
If I understood your question, what you want is:
get the name of the file,
...and the output or the program processing the file (in your case, eval),
...on the same line. And this last part is your problem.
Then I'd suggest composing a single line of text (using echo), comprising:
the name of the file, this is the $file part,
...followed by a separator, you may not need that but it may help further processing of the result. I used ":". You can skip this part if this is not interesting for you,
...followed by the output of the program processing the file: this is the $(...) construct
echo $file ":" $(./eval $file) >> out.txt
...and finally appending this line of text to a file, you got that part right.
please use like this
echo -n `echo ${file}|tr -d '\n'` >> out.txt
OR
newname=`echo ${file}|tr -d '\n'`
echo -n $newname >> out.txt
I want to run a program (when executed it produces logdata) out of a shell script and write the output into a text file. I failed to do so :/
$prog is the executed prog -> socat /dev/ttyUSB0,b9600 STDOUT
$log/$FILE is just path to a .txt file
I had a Perl script to do this:
open (S,$prog) ||die "Cannot open $prog ($!)\n";
open (R,">>","$log") ||die "Cannot open logfile $log!\n";
while (<S>) {
my $date = localtime->strftime('%d.%m.%Y;%H:%M:%S;');
print "$date$_";
}
I tried to do this in a shell script like this
#!/bin/sh
FILE=/var/log/mylogfile.log
SOCAT=/usr/bin/socat
DEV=/dev/ttyUSB0
BAUD=,b9600
PROG=$SOCAT $DEV$BAUD STDOUT
exec 3<&0
exec 0<$PROG
while read -r line
do
DATE=`date +%d.%m.%Y;%H:%M:%S;`
echo $DATE$line >> $FILE
done
exec 0<&3
Doesn't work at all...
How do I read the output of that prog and pipe it into my text file using a shell script? What did I do wrong (if I didn't do everything wrong)?
Final code:
#!/bin/sh
FILE=/var/log/mylogfile.log
SOCAT=/usr/bin/socat
DEV=/dev/ttyUSB0
BAUD=,b9600
CMD="$SOCAT $DEV$BAUD STDOUT"
$CMD |
while read -r line
do
echo "$(date +'%d.%m.%Y;%H:%M:%S;')$line" >> $FILE
done
To read from a process, use process substitution
exec 0< <( $PROG )
/bin/sh doesn't support it, so use /bin/bash instead.
To assign several words to a variable, quote or backslash whitespace:
PROG="$SOCAT $DEV$BAUD STDOUT"
Semicolon is special in shell, quote it or backslash it:
DATE=$(date '+%d.%m.%Y;%H:%M:%S;')
Moreover, no exec's are needed:
while ...
...
done < <( $PROG )
You might even add > $FILE after done instead of adding each line separately to the file.
Original answer
You haven't shown the error messages — which would have been helpful.
Your problem, though, is probably this line:
DATE=`date +%d.%m.%Y;%H:%M:%S;`
where the semicolons mark the end of a command, and there likely isn't a command %H that does anything useful, etc.
You need quotes around the format argument to date, and I'd use single quotes for this job:
DATE=$(date +'%d.%m.%Y;%H:%M:%S;')
or even replace the two lines in the body of the loop with:
echo "$(date +'%d.%m.%Y;%H:%M:%S;')$line" >> $FILE
The double quotes prevent a variety of problems.
That assumes you fix a bunch of other problems, such as the setting of the variables FILE and prog. Also, I'd probably use:
exec > $FILE
to initially zap the output file and then all subsequent standard output would go to that file, so the echo line becomes:
echo "$(date +'%d.%m.%Y;%H:%M:%S;')$line"
Amended answer
The question was originally missing lots of key information. It eventually got updated to include the complete code.
The problem I identified originally remains an issue, but you weren't running into it because the input redirection was not working. If you want the input to come from a process, use a pipe, or possibly process substitution. However, note that you have #!/bin/sh as your shebang line, and /bin/sh won't recognized process substitution; either change the shebang or use the pipe notation. Note that process substitution has advantages if the loop is setting variables that need to be accessed after the loop is complete.
$SOCAT $DEV$BAUD STDOUT |
while read -r line
do
…
done
or
while read -r line
do
…
done < <($SOCAT $DEV$BAUD STDOUT)
Note that your code contains the line:
PROG=$SOCAT $DEV$BAUD STDOUT
This runs the command identified by $DEV$BAUD with the argument STDOUT and the environment variable PROG set to the value of $SOCAT. That is not what you wanted.
You could use an array:
PROG=($SOCAT $DEV$BAUD STDOUT)
and then run:
"${PROG[#]}"
either in the pipe line:
"${PROG[#]}" |
while read -r line
do
…
done
or with process substitution:
while read -r line
do
…
done < <("${PROG[#]}")
Note that unless there is code after the final exec 0<&3, there was no particular virtue in the redirections involving file descriptor 3. You should also close 3 when you're done with it:
exec 0<&3 3>&-
The 'final' code includes the lines:
CMD="$SOCAT $DEV$BAUD STDOUT"
$CMD |
while read -r line
This works OK because there are no spaces in the arguments to the command. That's a common case, but beware of spaces in arguments and file paths.