I can't get my bash script to run - linux

This is the script that I used to that will not run, but I am hoping someone can help me figure out what the issue is. I am new to unix
#!/bin/bash
# cat copyit
# copies files
numofargs=$#
listoffiles=
listofcopy=
# Capture all of the arguments passed to the command, store all of the arguments, except
# for the last (the destination)
while [ "$#" -gt 1 ]
do
listoffiles="$listoffiles $1"
shift
done
destination="$1"
# If there are less than two arguments that are entered, or if there are more than two
# arguments, and the last argument is not a valid directory, then display an
# error message
if [ "$numofargs" -lt 2 -o "$numofargs" -gt 2 -a ! -d "$destination" ]
then
echo "Usage: copyit sourcefile destinationfile"
echo" copyit sourcefile(s) directory"
exit 1
fi
# look at each sourcefile
for fromfile in $listoffiles
do
# see if destination file is a directory
if [ -d "$destination" ]
then
destfile="$destination/`basename $fromfile`"
else
destfile="$destination"
fi
# Add the file to the copy list if the file does not already exist, or it
# the user
# says that the file can be overwritten
if [ -f "$destfile" ]
then
echo "$destfile already exist; overwrite it? (yes/no)? \c"
read ans
if [ "$ans" = yes ]
then
listofcopy="$listofcopy $fromfile"
fi
else
listofcopy="$listofcopy $fromfile"
fi
done
# If there is something to copy - copy it
if [ -n "$listofcopy" ]
then
mv $listofcopy $destination
fi
This is what I got and it seems that the script didn't execute all though I did invoke it. I am hoping that someone can help me
[taniamack#localhost ~]$ chmod 555 tryto.txt
[taniamack#localhost ~]$ tryto.txt
bash: tryto.txt: command not found...
[taniamack#localhost ~]$ ./tryto.txt
./tryto.txt: line 7: $'\r': command not found
./tryto.txt: line 11: $'\r': command not found
./tryto.txt: line 16: $'\r': command not found
./tryto.txt: line 43: syntax error near unexpected token `$'do\r''
'/tryto.txt: line 43: `do

Looks like your file contains Windows new line formatting: "\r\n". On Unix, a new line is just "\n". You can use dos2unix (apt-get install dos2unix), to convert your files.
Also have a look at the chmod manual (man chmod).
Most of the time i just use chmod +x ./my_file to give execution rights

I see a few issues. First of all, a mode of 555 means that no one can write to the file. You probably want chmod 755. Second of all, you need to add the current directory to your $PATH variable. In Windows, you also have a %PATH%, but by default the current directory . is always in %PATH%, but in Unix, adding the current directory is highly discouraged because of security concerns. The standard is to put your scripts under the $HOME/bin directory and make that directory the last entry in your $PATH.
First of all: Indent correctly. When you enter a loop or an if statement, indent the lines by four characters (that's the standard). It makes it much easier to read your program.
Another issue is your line endings. It looks like some of the lines have a Windows line ending on them while most others have a Unix/Linux/Mac line ending. Windows ends each line with two characters - Carriage Return and Linefeed while Unix/Linux/Mac end each line with just a Linefeed. The \r is used to represent the Carriage Return character. Use a program editor like vim or gedit. A good program editor will make sure that your line endings are consistent and correct.

Related

Test if a file exists

I struggle with the examples given if a file exists.
I want to check if multiple files exists in order to perform further operations.
ls -al:
-rwxrwxrwx 1 root tomcat 6 Dec 16 11:25 documents_2019-12-12.tar.gz
echo < [ -e ./documents_2019-12-12.tar.gz ]:
bash: [: No such file or directory
Can somebody tell me what i'm doing wrong?
Edit:
I have a backup direcory with two files:
database_date.sql
documents_date.tar.gz
I need to check if both files for a given date are available. The directory shall contain these file-pairs for several dates.
What you have here is a misunderstanding of where specific syntax is used. The [ -e ./documents_2019-12-12.tar.gz ] part of your command is syntax specific to the if clause in bash. Here's an example
if [ -e ./documents_2019-12-12.tar.gz ]
then
echo "File Exists!"
fi
The square brackets [] are used to surround the check being performed by the if statement and the -e flag is specific to these if checks. More info here http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_07_01.html
To explain the error you're seeing, the < operator takes a file to the right and feeds the contents to the command on the left. In your case the < sees the [ as the thing on the right so we try and read it as a file. Such a file doesn't exist so bash helpfully tells you that there's an error (the bash: [: No such file or directory bit) and then quits out.

bash for loops - how to use the ${file} variable?

I would like to run a command on a list of paired files with the following format
SAMPLE_1.1.fq.gz SAMPLE_1.2.fq.gz
SAMPLE_2.1.fq.gz SAMPLE_2.2.fq.gz
etc...
etc. That are in a directory called ../cleaned-trimmed
I have the list of samples in a txt file (samples_final.txt - one sample per line) in a directory called info.
SAMPLE_1
SAMPLE_2
SAMPLE_3
I would like to run the following command on all the samples:
gsnap <args> --output-file=./alignments.gsnap/SAMPLE_1.mapped.sam --failed-input=./alignments.gsnap/SAMPLE_1.unmapped.fa ../cleaned-trimmed/SAMPLE_1.1.fq.gz ../cleaned-trimmed/SAMPLE_1.2.fq.gz
Where the args are the database used, command flags, etc.
I modified the script from a previous answer in stackoverflow to build a loop as follows:
for file in $(<../info/samples_final.txt)
do
gsnap <args> --output-file=./alignments.gsnap/${file}.mapped.sam --failed-input=./alignments.gsnap/${file}.unmapped.fa ../cleaned-trimmed/${file}.1.fq.gz ../cleaned-trimmed/${file}.2.fq.gz
done
but it does not pass the variables on correctly.
How do I pass on the values from samples_final.txt to the command?
At the moment the script garbles the file names when I run the loop. So, for example, if I run a test on the file "for_test2.txt":
SAMPLE_1
SAMPLE_2
Using the echo command:
for file in $(<../info/for_test2.txt)
do
echo ../cleaned-trimmed/${file}.1.fq.gz
done
I get the following output:
.1.fq.gzed-trimmed/SAMPLE_1
.1.fq.gzed-trimmed/SAMPLE_2
.1.fq.gzed-trimmed/
So it seems to have replaced ../clean with the .1.fq.gz
I genuinely do not understand the logic of this.
You have dos line endings in your file. The "carrier return" character makes the cursor jump to the beginning of the current line, that's why the .1.fq.gz part in your last code snipped is printed on the beginning of the line. You can first convert your file to normal line endings:
dos2unix ../info/for_test2.txt
Then read the file line by line and execute your command. Remember to quote your variables:
while IFS= read -r file; do
# protect against empty lines in input file
if [ -z "$file" ]; then continue; fi
gsnap <args> --output-file=./alignments.gsnap/"$file".mapped.sam --failed-input=./alignments.gsnap/"$file".unmapped.fa ../cleaned-trimmed/"$file".1.fq.gz ../cleaned-trimmed/"$file".2.fq.gz
done <../info/for_test2.txt
or like a pro with xargs:
<../info/for_test2.txt xargs -I{} gsnap <args> --output-file=./alignments.gsnap/{}.mapped.sam --failed-input=./alignments.gsnap/{}.unmapped.fa ../cleaned-trimmed/{}.1.fq.gz ../cleaned-trimmed/{}.2.fq.gz

Assign full text file path to a variable and use variable as file path in sh file

I am trying to create a shell script for logs and trying to append data into a text file. I have write this sample "test.sh" code for testing:
#!/bin/sh -e
touch /home/sample.txt
SPTH = '/home/sample'.txt
echo "MY LOG FILE" >> "$SPTH"
echo "DUMP started at $(date +'%d-%m-%Y %H:%M:%S')" >> /home/sample.txt
echo "DUMP finished at $(date +'%d-%m-%Y %H:%M:%S')" >> /home/sample.txt
but in above code all lines are working correct except one line of code i.e.
echo "MY LOG FILE" >> "$SPTH"
It is giving error:
test.sh: line 6: : No such file or directory
I want to replace this full path of file "/home/sample.txt" to variable "$SPATH".
I am executing my shell script using
sh test.sh
What I am doing wrong.
Variable assignments in bash shell does not allow you to have spaces within. It will be actually interpreted as command with = and the subsequent keywords as arguments to the first word, which is wrong.
Change your code to
SPTH="/home/sample.txt"
That is the reason why SPTH was not assigned to the actual path you intended it to have. And you have no reason to have single-quote here and excluding the extension part. Using it fully within double-quotes is absolutely fine.
The syntax for the command line is that the first token is a command, tokens are separated by whitespace. So:
SPTH = '/home/sample'.txt
Has the command as SPTH, the second token is =, and so on. You might think this is daft, but most shells behave like this for historical reasons.
So you need to remove the whitespace:
SPTH='/home/sample'.txt

source command not detecting the file in Linux bash

Here is an excerpt from my Bash script
#!/bin/sh
# Check SPEED parameter validity
if [ "$4" = "SPEED" ]
then
source ${SIM_DIR}/setEnv.sh speed
elif [ "$4" = "NORMAL" ]
then
pushd ${SIM_DIR}/scripts
source setEnv
else
ERROR "Invalid SPEED|NORMAL parameter ($4)."
exit 1
fi
In command line, I am giving the option as NORMAL when I run the script. There is no file called setEnv.sh in the ${SIM_DIR}/scripts location. There is however a file called setEnv and its first line is #!/bin/bash -x. I get the following error:
line 176: source: setEnv: file not found
Could anybody kindly point out what is wrong with my script?
source uses PATH lookups to find names that do not contain slashes, and your PATH (correctly) does not contain ., so the current directory is not searched for setEnv. Use source ./setEnv.
The shebang line is ignored by source.

Unable to activate virtualenv via bash script

I'm running a project inside a virtualenv in python. Here's the path to the virtualenv.
~/iss/issp/bin
The problem is when I try to run the activate script with:
source activate
it throws the following error.
:~/iss/issp/bin$ source activate
: command not found
bash: activate: line 4: syntax error near unexpected token `$'{\r''
'ash: activate: line 4: `deactivate () {
Here's the code inside the script:
# This file must be used with "source bin/activate" *from bash*
# you cannot run it directly
deactivate () {
unset pydoc
# reset old environment variables
if [ -n "$_OLD_VIRTUAL_PATH" ] ; then
PATH="$_OLD_VIRTUAL_PATH"
export PATH
unset _OLD_VIRTUAL_PATH
fi
if [ -n "$_OLD_VIRTUAL_PYTHONHOME" ] ; then
PYTHONHOME="$_OLD_VIRTUAL_PYTHONHOME"
export PYTHONHOME
unset _OLD_VIRTUAL_PYTHONHOME
fi
# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands. Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "$BASH" -o -n "$ZSH_VERSION" ] ; then
hash -r 2>/dev/null
fi
if [ -n "$_OLD_VIRTUAL_PS1" ] ; then
PS1="$_OLD_VIRTUAL_PS1"
export PS1
unset _OLD_VIRTUAL_PS1
fi
unset VIRTUAL_ENV
if [ ! "$1" = "nondestructive" ] ; then
# Self destruct!
unset -f deactivate
fi
}
# unset irrelevant variables
deactivate nondestructive
VIRTUAL_ENV="/home/pablo/issp"
export VIRTUAL_ENV
_OLD_VIRTUAL_PATH="$PATH"
PATH="$VIRTUAL_ENV/bin:$PATH"
export PATH
# unset PYTHONHOME if set
# this will fail if PYTHONHOME is set to the empty string (which is bad anyway)
# could use `if (set -u; : $PYTHONHOME) ;` in bash
if [ -n "$PYTHONHOME" ] ; then
_OLD_VIRTUAL_PYTHONHOME="$PYTHONHOME"
unset PYTHONHOME
fi
if [ -z "$VIRTUAL_ENV_DISABLE_PROMPT" ] ; then
_OLD_VIRTUAL_PS1="$PS1"
if [ "x" != x ] ; then
PS1="$PS1"
else
if [ "`basename \"$VIRTUAL_ENV\"`" = "__" ] ; then
# special case for Aspen magic directories
# see http://www.zetadev.com/software/aspen/
PS1="[`basename \`dirname \"$VIRTUAL_ENV\"\``] $PS1"
else
PS1="(`basename \"$VIRTUAL_ENV\"`)$PS1"
fi
fi
export PS1
fi
alias pydoc="python -m pydoc"
# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands. Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "$BASH" -o -n "$ZSH_VERSION" ] ; then
hash -r 2>/dev/null
fi
The issue is that the activate script has Windows line endings. We can confirm this by running the below command on the command line.
$ file activate
which returns
activate: ASCII text, with CRLF line terminators
CRLF line terminators means windows line endings.
Therefore we need to convert them to Unix line endings so that we can source the file.
We have a few of options
dos2unix activate - (this edits the file in place)
sed -i 's/\r$//' activate (also edits the file in place)
mv activate activate_with_windows_line_endings && (tr -d '\r' < activate_with_windows_line_endings ) > activate here we we just remove any occurrence of \r and the original file is preserved.
Open the file in your favourite text editor. Most will have a way to both show the current line endings of the file (bottom right of image below) and a way to convert them to Unix line endings (In Notepad++ we go to Edit → EOL Conversion → Unix (LF) and then save. Here is a screenshot of how to do it in Notepad++
Finally
Now source activate should work.
Just had the same problem, and decided to do hexdump -C bin/activate to figure out. Turns out my bin/activate file had CR/LF line endings instead of just CR, changing them with (tr -d '\r' < bin/activate) > bin/activatefixed my problem.
I ran into this problem using VS Code and the venv package from python's standard library, since I use bash for my default terminal.
Bash scripts should always use LF instead of CRLF line endings. This is now fixed for the python virtualenv package, but it's still an issue with venv.
As described here, https://bugs.python.org/issue43437, venv copies its activate script template in binary mode when creating new virtual environments. Therefore we only need to convert the original to Unix line endings, using any of Mark's methods above.
The activate script template is located here: <python3_root>/Lib/venv/scripts/common/activate
To fix it in place: sed -i 's/\r$//' activate
As long as that file keeps its LF line endings, new virtual environments created with venv should have viable activation scripts.
Maybe you have an alias in your .bashrc file, that's why deactivate takes like the command, not like a function
instead
deactivate() {
use this
function deactivate() {

Resources