Read and execute lines with spaces from file in bash - linux

I'm facing a problem in a bash shell script when I try to read some lines from a file and execute them one by one. The problem occurs when the line has an argument with spaces. Code:
while read i
do
$i
done < /usr/bin/tasks
tasks file:
mkdir Hello\ World
mkdir "Test Directory"
Both of the above instructions work perfectly when executed directly from the terminal, creating only two directories called "Hello World" and "Test Directory" respectively, but the same doesn't happen when the instructions are read and executed from the script, meaning that four directories are created.
Having said that, I would like to keep my code as simple as possible and, if possible, I'd prefer not to use the cat command. Thanks in advance for any help.

As simple as possible? You are re-implementing the . (or source, as bash allows you to spell it) command:
. /usr/bin/tasks
or
source /usr/bin/tasks
To execute one line at a time, use eval.
while IFS= read -r i; do
eval "$i"
done
This assumes that each line of the file contains one or more complete commands that can be executed individually.

Related

Bash script : Storing command with spaces and arguments in variable and then executing

Been banging my head against the wall for a couple hours so time to call in the experts. Writing a small script to run some reports on one of my office's systems and I was asked to take care of a Bash script for it. The program called "auto_rep" takes various options such as "-t" to run one task (to generate one type of report) and a "-1" to exit after one task. The options are separated by spaces when running the command from command-line. The command works directly from command line but I cannot get it to work from a script...
Below is the snippet of code causing me issues:
cmd=$(auto_rep -t createfin1report -1)
echo "running ${cmd} command..."
echo
eval $cmd
The problem is when I run the script, only the "auto_rep" part of the command (from $cmd variable) is run; basically running the program without any options. And it creates tons of reports without the "-t createfin1report -1" part of the command (yikes!). Glad I only tried it on our test system.
Anyone have any tips to help me out? Is my approach way off? BTW - had tried just storing the command in a non-array (cmd="auto_rep -t createfin1report -1") and that was causing me other headache with a "command not found" errors :)...
Thanks in advance!
Save output to an array, then executing this array.
declare -a cmd
cmd=( $(auto_rep -t createfin1report -1) )
echo Executing: "${cmd[#]}"
"${cmd[#]}"
Please make sure the output is a valid command, and spaces have been correctly placed in double-quotes.

What is the meaning of pwd|sed -e?

I found the below snippet at the .sh file of my project to define some path :
PGMPATH=`pwd|sed -e "s#/survey1##" `
What does the above line means ?
Reference of PGMPATH is used as below :
LIBS="${LIBS}:${PGMPATH}/edmz-par-api_1.4.jar"
LIBS="${LIBS}:${PGMPATH}/commons-logging.jar"
If it is telling the path where the jar file is located , please explain how it works .
So first you should know that this is two commands - pwd and sed -e "s#/survey1##" - and these two commands are being run together in a pipeline. That is, the output of the first command is being sent to the second command as input.
That is, in general, what | means in unix shell scripts.
So then, what do each of these commands do? pwd stands for "print working directory" and prints the current directory (where you ran the script from, unless the script itself had any cd commands in it).
sed is a command that's really a whole separate programming language that people do many simple text-processing commands with. The simple sed program you have here - s#/survey1## - strips the string /survey1 out of its input, and prints the result.
So the end result is that the variable PGMPATH becomes the current directory with /survey1 stripped out of it.

Running a script inside a loop - echo file names

I have a basic script that runs inside another script. I call mass_split.sh which then invokes split_file.sh. The splite_file.sh takes two arguments -s file_name.txt and -c 1 (how many slices to cut the file). However I trying to run a loop to find all text file names in directory ./ and then input the results to the cut_file.sh . I am getting no results back and then text files are not being split.
mass_split.sh
#!/bin/bash
for f in ./*.txt
do
sudo bash split_file.sh -s echo "file '$f'"; -c 10
done
Maybe this has something to do with that errant semicolon after the string literal, which is almost certainly not doing what you want (unless you have another executable that you're intentionally running called -c).

What does this shell script line of code mean

I need some help understanding following shell script line,
apphome = "`cd \`dirname $0\` && pwd && cd - >/dev/null`"
All I understand is, this is creating a variable called apphome.
This is not a valid shell code.
The shell don't allow spaces around =
For the rest, while this seems broken, it try to cd to the dir of the script itself, display the current dir & finally cd back to the latest cd place redirecting his standard output STDOUT to the /dev/null trash-bin (that's makes not any sense, cd display only on standard error STDERR when it fails, never on STDOUT)
If you want to do this in a proper a simple way :
apphome="$(dirname $0)"
That's all you need.
NOTE
The backquote
`
is used in the old-style command substitution, e.g.
foo=`command`
The
foo=$(command)
syntax is recommended instead. Backslash handling inside $() is less surprising, and $() is easier to nest. See http://mywiki.wooledge.org/BashFAQ/082
It seems to assign a command to the "apphome" variable. This command can be executed later.
dirname returns a directory portion of a file name. $0 is the name of the script this line contains (if I am not mistaken).
Now, executing dirname <name> will return a directory, and cd will use the value.
So, what it would do is execute three command in the row assuming that each one of them succeeds. The commands are:
cd `dirname [name of the script]`
pwd
cd -
First command will change directory to the directory containing your script; second will print current directory; third will take yo back to the original directory. Output of the third command will not be printed out.
In summary, it will print out a name of a directory containing the script that contains the line in question.
At least, this is how I understand it.

Edit shell script while it's running

Can you edit a shell script while it's running and have the changes affect the running script?
I'm curious about the specific case of a csh script I have that batch runs a bunch of different build flavors and runs all night. If something occurs to me mid operation, I'd like to go in and add additional commands, or comment out un-executed ones.
If not possible, is there any shell or batch-mechanism that would allow me to do this?
Of course I've tried it, but it will be hours before I see if it worked or not, and I'm curious about what's happening or not happening behind the scenes.
It does affect, at least bash in my environment, but in very unpleasant way. See these codes. First a.sh:
#!/bin/sh
echo "First echo"
read y
echo "$y"
echo "That's all."
b.sh:
#!/bin/sh
echo "First echo"
read y
echo "Inserted"
echo "$y"
# echo "That's all."
Do
$ cp a.sh run.sh
$ ./run.sh
$ # open another terminal
$ cp b.sh run.sh # while 'read' is in effect
$ # Then type "hello."
In my case, the output is always:
hello
hello
That's all.
That's all.
(Of course it's far better to automate it, but the above example is readable.)
[edit] This is unpredictable, thus dangerous. The best workaround is , as described here put all in a brace, and before the closing brace, put "exit". Read the linked answer well to avoid pitfalls.
[added] The exact behavior depends on one extra newline, and perhaps also on your Unix flavor, filesystem, etc. If you simply want to see some influences, simply add "echo foo/bar" to b.sh before and/or after the "read" line.
Try this... create a file called bash-is-odd.sh:
#!/bin/bash
echo "echo yes i do odd things" >> bash-is-odd.sh
That demonstrates that bash is, indeed, interpreting the script "as you go". Indeed, editing a long-running script has unpredictable results, inserting random characters etc. Why? Because bash reads from the last byte position, so editing shifts the location of the current character being read.
Bash is, in a word, very, very unsafe because of this "feature". svn and rsync when used with bash scripts are particularly troubling, because by default they "merge" the results... editing in place. rsync has a mode that fixes this. svn and git do not.
I present a solution. Create a file called /bin/bashx:
#!/bin/bash
source "$1"
Now use #!/bin/bashx on your scripts and always run them with bashx instead of bash. This fixes the issue - you can safely rsync your scripts.
Alternative (in-line) solution proposed/tested by #AF7:
{
# your script
exit $?
}
Curly braces protect against edits, and exit protects against appends. Of course, we'd all be much better off if bash came with an option, like -w (whole file), or something that did this.
Break your script into functions, and each time a function is called you source it from a separate file. Then you could edit the files at any time and your running script will pick up the changes next time it gets sourced.
foo() {
source foo.sh
}
foo
Good question!
Hope this simple script helps
#!/bin/sh
echo "Waiting..."
echo "echo \"Success! Edits to a .sh while it executes do affect the executing script! I added this line to myself during execution\" " >> ${0}
sleep 5
echo "When I was run, this was the last line"
It does seem under linux that changes made to an executing .sh are enacted by the executing script, if you can type fast enough!
An interesting side note - if you are running a Python script it does not change. (This is probably blatantly obvious to anyone who understands how shell runs Python scripts, but thought it might be a useful reminder for someone looking for this functionality.)
I created:
#!/usr/bin/env python3
import time
print('Starts')
time.sleep(10)
print('Finishes unchanged')
Then in another shell, while this is sleeping, edit the last line. When this completes it displays the unaltered line, presumably because it is running a .pyc? Same happens on Ubuntu and macOS.
I don't have csh installed, but
#!/bin/sh
echo Waiting...
sleep 60
echo Change didn't happen
Run that, quickly edit the last line to read
echo Change happened
Output is
Waiting...
/home/dave/tmp/change.sh: 4: Syntax error: Unterminated quoted string
Hrmph.
I guess edits to the shell scripts don't take effect until they're rerun.
If this is all in a single script, then no it will not work. However, if you set it up as a driver script calling sub-scripts, then you might be able to change a sub-script before it's called, or before it's called again if you're looping, and in that case I believe those changes would be reflected in the execution.
I'm hearing no... but what about with some indirection:
BatchRunner.sh
Command1.sh
Command2.sh
Command1.sh
runSomething
Command2.sh
runSomethingElse
Then you should be able to edit the contents of each command file before BatchRunner gets to it right?
OR
A cleaner version would have BatchRunner look to a single file where it would consecutively run one line at a time. Then you should be able to edit this second file while the first is running right?
Use Zsh instead for your scripting.
AFAICT, Zsh does not exhibit this frustrating behavior.
usually, it uncommon to edit your script while its running. All you have to do is to put in control check for your operations. Use if/else statements to check for conditions. If something fail, then do this, else do that. That's the way to go.
Scripts don't work that way; the executing copy is independent from the source file that you are editing. Next time the script is run, it will be based on the most recently saved version of the source file.
It might be wise to break out this script into multiple files, and run them individually. This will reduce the execution time to failure. (ie, split the batch into one build flavor scripts, running each one individually to see which one is causing the trouble).

Resources