unix sftp issue - linux

I am not posting entire code here but part of it.Below code giving errors.I am trying to store all sftp commands and then performing actual sftp.
export SFTP_BATCH_FILE='/var/tmp/SFTP_BATCH_FILE'
#------------------------------------------------------------------------
# Create sftp script
#------------------------------------------------------------------------
rm -f $SFTP_BATCH_FILE
echo "lcd $SOURCE_FILE_DIRECTORY " > $SFTP_BATCH_FILE
echo "cd $DESTINATION_FILE_DIRECTORY " >> $SFTP_BATCH_FILE
if [ -z $FILE_TO_UPLOAD_TESTD ] then
echo "put $FILE_TO_UPLOAD_TESTD " >> $SFTP_BATCH_FILE
fi
if [ -z $FILE_TO_UPLOAD_TESTDF ] then
echo "put $FILE_TO_UPLOAD_TESTDF " >> $SFTP_BATCH_FILE
fi
echo "bye" >> $SFTP_BATCH_FILE
#------------------------------------------------------------------------
# Do sftp
#------------------------------------------------------------------------
echo " Before SFTP " >> $LOG_FILE
if [[ -z $ FILE_TO_UPLOAD && -z $ FILE_TO_UPLOAD1 ]] then
echo “No files to transfer” >> $LOG_FILE
mv $LOG_FILE $LOG_DIRECTORY
exit 1
else
echo “Attempting to connect to Remote Server $REMOTE_SERVER_PROD” >> $LOG_FILE
/usr/bin/sftp –v -oPort=$SFTP_PORT -b $SFTP_BATCH_FILE $SOURCE_FUNCTIONAL_ID#$REMOTE_SERVER_PROD >> $LOG_FILE 2 >> $LOG_FILE
fi
Errors i am getting:
rm: /var/tmp/SFTP_BATCH_FILE is a directory
test.ksh[89]: /var/tmp/SFTP_BATCH_FILE: cannot create
test.ksh[90]: /var/tmp/SFTP_BATCH_FILE: cannot create
Regards,
Chai

The clue is in the error message
rm: /var/tmp/SFTP_BATCH_FILE is a directory
As the directory is still there your subsequent commands cannot create the SFTP_BATCH_FILE file.
rm -f cannot remove directories. Use rm -rf instead.
Edit:
Just to clarify, -r is recursive meaning that directories are delete as well, -f is force which means that nonexistent files/directories don't cause an error and the command doesn't prompt.

Related

Shell Script failing to run a command

I have a debian server set up with mounted drives from google drive - these sometimes get unmounted, so I have made a script to check for, and remount, unmounted drives.
The script will check against a file in the mounted directory and if it's missing it will attempt to remount the drive.
It will succesfully check all mounts, but is only able to remount "plexdrive" and "plexlib" it fails to mount "decrypt".
There is nothing wrong with the mounting command as it works fine outside of the script.
The script in question:
#!/bin/bash
## Small hack that makes this script work in cronjob
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home22/krealle/check.rclone.mount
## Logfile
LOGFILE=~/log/check-mounts.log
## Variables
Errors=0
## create $LOGFILE
if [ ! -f $LOGFILE ]; then
touch $LOGFILE
echo "INFO: Logfile created." >> $LOGFILE
fi
echo "" >> $LOGFILE
echo "$(date "+%d/%m/%Y %T")" >> $LOGFILE
## Ultimate Check - Smaller log file!
if [[ -f /home22/krealle/mounts/plexdrive/plexdrive_mounted ]] && [[ -f /home22/krealle/mounts/plexdrive-decrypt/mounted ]] && [[ -f /home22/krealle/PlexMedia/PlexLib/mounted ]]; then
echo "INFO: Errors found: $Errors - skipping individual checks." >> $LOGFILE
exit
fi
## check plexdrive mount
if [[ -f /home22/krealle/mounts/plexdrive/plexdrive_mounted ]]; then
echo "INFO: *Plexdrive* Mountcheck successful." >> $LOGFILE
else
echo "ERROR: *Plexdrive* Drive not mounted remount in progress." >> $LOGFILE
## first unmount broken/busy remotes.
fusermount -zu /home22/krealle/mounts/plexdrive
## Mount Plexdrive.
screen -S plexdrive.s: -d -m ./plexdrive -o allow_other -v 4 /home22/krealle/mounts/plexdrive
if [[ -f /home22/krealle/mounts/plexdrive/plexdrive_mounted ]]; then
echo "INFO: *Plexdrive* Remount successful." >> $LOGFILE
else
((Errors++))
echo "CRITICAL: *Plexdrive* Remount failed." >> $LOGFILE
fi
fi
## check decrypt mount
if [[ -f /home22/krealle/mounts/plexdrive-decrypt/mounted ]]; then
echo "INFO: *Plexdrive-Decrypt* Mountcheck successful." >> $LOGFILE
else
echo "ERROR: *Plexdrive-Decrypt* Drive not mounted remount in progress." >> $LOGFILE
## first unmount broken/busy remotes.
fusermount -zu /home22/krealle/mounts/plexdrive-decrypt
## Mount Plexdrive-decrypt.
screen -S rclone.decrypt: -d -m rclone mount --read-only --allow-non-empty --allow-other --buffer-size 128M -v plexdrive2: /home22/krealle/mounts/plexdrive-decrypt
if [[ -f /home22/krealle/mounts/plexdrive-decrypt/mounted ]]; then
echo "INFO: *Plexdrive-Decrypt* Remount successful." >> $LOGFILE
else
((Errors++))
echo "CRITICAL: *Plexdrive-Decrypt* Remount failed." >> $LOGFILE
fi
fi
## check plexlib mount
if [[ -f /home22/krealle/PlexMedia/PlexLib/mounted ]]; then
echo "INFO: *PlexLib* Mountcheck successful." >> $LOGFILE
else
echo "ERROR: *PlexLib* Drive not mounted remount in progress." >> $LOGFILE
## first unmount broken/busy remotes.
fusermount -zu /home22/krealle/PlexMedia/PlexLib
## Mount PlexLib.
unionfs-fuse -o cow,allow_other,direct_io,nonempty,auto_cache,sync_read /home22/krealle/PlexMedia/Local=RW:/home22/krealle/mounts/plexdrive-decrypt=RO /home22/krealle/PlexMedia/PlexLib
if [[ -f /home22/krealle/PlexMedia/PlexLib/mounted ]]; then
echo "INFO: *PlexLib* Remount successful." >> $LOGFILE
else
((Errors++))
echo "CRITICAL: *PlexLib* Remount failed." >> $LOGFILE
fi
fi
echo "Total errors: $Errors " >> $LOGFILE
My script is based on the one found here.

Bash silent error processing

I'm trying to run programs (for example mv file1.txt file2.txt) in my .sh script and I need to hide errors, but handle it with my script.
Currently I'm trying to do something like
EXECUTE="mv -v $VOL $BACKUP_YESTERDAY_CRYPT"
{
EXEC_ERROR=$($EXECUTE)
} &2>> $LOG_FILE
if [[ -n $EXEC_ERROR ]]; then
echo "There is an error!"
fi
But it doesn't work at all - it shows an error (for example mv: cannot stat 'file1.txt': No such file or directory) and $EXEC_ERROR variable is empty.
Is there any way to get output to variable + to log file?
How about something like:
mv -v $VOL $BACKUP_YESTERDAY_CRYPT 2>> $LOG_FILE
if [[ ! ( $? -eq 0 ) ]] ; then
echo "There is an error\!"
fi
Though $? is good for saving and processing exit codes, the if statement is designed to take any command, not just [ or [[:
if ! mv -v "$VOL" "$BACKUP_YESTERDAY_CRYPT" 2>> $LOG_FILE; then
echo "There is an error!"
fi
This includes saving variables:
if OUTPUT=$(mv -v "$VOL" "$BACKUP_YESTERDAY_CRYPT" 2>> $LOG_FILE); then
echo ">>> $OUTPUT <<<"
fi
In fact, if can take more than one command, as its man page describes. Documentation on boolean operators such as !, &&, and || is hidden within the description of shell commands, where they form pipelines (!) and lists (&&, ||).
Try this:
mv sourcefile destfile 2> /dev/null 1>logfile
returnstatus=`echo $?`
if [[ $returnstatus -ne 0 ]]; then
echo "There was an error!"
fi

Symlink check - Linux Bash Script

I'm trying to create a script that searches through a directory to find symlinks that point to non-existing objects.
I have a file in a directory with a deleted symlink, but for some reason when i run the below script It says file exists.
#!/bin/bash
ls -l $1 |
if [ -d $1 ]
then
while read file
do
if test -e $1
then
echo "file exists"
else
echo "file does not exist"
fi
done
else
echo "No directory given"
fi
Thanks
Check this page. It has a test for broken links. It uses the -h operator to identify a symlink and the -e operator to check existance.
From that page:
linkchk () {
for element in $1/*; do
[ -h "$element" -a ! -e "$element" ] && echo \"$element\"
[ -d "$element" ] && linkchk $element
# Of course, '-h' tests for symbolic link, '-d' for directory.
done
}
# Send each arg that was passed to the script to the linkchk() function
#+ if it is a valid directoy. If not, then print the error message
#+ and usage info.
##################
for directory in $directorys; do
if [ -d $directory ]
then linkchk $directory
else
echo "$directory is not a directory"
echo "Usage: $0 dir1 dir2 ..."
fi
done
exit $?
You can test whether link is valid or not using:
[[ -f "$link" ]] && echo "points to a valid file"
To check if it is indeed a link use -L:
[[ -L "$link" ]] && echo "it's a link"
There seems to be a program named symlinks that does, among other things, what you're looking for.

Md5 Hash to identify and archive images

This is my first ever bash script and I am trying to iron out all of the creases and make the script run nicely. The script is to archive all of the specified .jpg files that it finds in multiple directories on a HDD/Flash drive. There are files with the same name but different content so I have used an Md5 sum to hash them.
I am getting the directory does not exist error in Geany but it runs fine from command bar missing out two of the images. I have tried everything I can think of to fix it. Is it messy code that is doing this?
#!/bin/sh
if [ ! -d "$1" ]; then
echo Directory "$1" cannot be found. Please try again.
exit
fi
if [ $# -eq 1 ]; then
echo "usage: Phar image_path archive_path"
exit
fi
if [ -d "$2" ]; then
echo "archive exists"
else
echo "the directory 'archive' does't exist. Creating directory 'archive'."
mkdir -p ~/archive
fi
find $1 -iname "IMG_[0-9][0-9][0-9][0-9].JPG" | cat > list.txt
[ -f ~/my-documents/md5.txt ] && rm md5.txt || break
while read line;
do md5sum $line | xargs >> md5.txt
done < list.txt
sort -k 1,1 -u md5.txt | cat > uniquemd5.txt
cut -d " " -f 2- uniquemd5.txt > uniquelist.txt
sort uniquelist.txt -r -o uniquelist.txt
for line in $(cat uniquelist.txt)
do
file=$(basename $line) path="$2/file"
if [ ! -f $path ];
then
cp $line $2
else
cp $line $path.JPG
fi
done
You haven't guarded against spaces in the folder and file names everywhere.
For instance:
cp $line $2
should be:
cp "$line" "$2"
You should start by eliminating these spaces as a source to your error by evaluating each variable you are referencing and adding ""'s.
If you still get the error please provide us with the arguments used and which directory that does not exist.

check if file exists on remote host with ssh

I would like to check if a certain file exists on the remote host.
I tried this:
$ if [ ssh user#localhost -p 19999 -e /home/user/Dropbox/path/Research_and_Development/Puffer_and_Traps/Repeaters_Network/UBC_LOGS/log1349544129.tar.bz2 ] then echo "okidoke"; else "not okay!" fi
-sh: syntax error: unexpected "else" (expecting "then")
In addition to the answers above, there's the shorthand way to do it:
ssh -q $HOST [[ -f $FILE_PATH ]] && echo "File exists" || echo "File does not exist";
-q is quiet mode, it will suppress warnings and messages.
As #Mat mentioned, one advantage of testing like this is that you can easily swap out the -f for any test operator you like: -nt, -d, -s etc...
Test Operators: http://tldp.org/LDP/abs/html/fto.html
Here is a simple approach:
#!/bin/bash
USE_IP='-o StrictHostKeyChecking=no username#192.168.1.2'
FILE_NAME=/home/user/file.txt
SSH_PASS='sshpass -p password-for-remote-machine'
if $SSH_PASS ssh $USE_IP stat $FILE_NAME \> /dev/null 2\>\&1
then
echo "File exists"
else
echo "File does not exist"
fi
You need to install sshpass on your machine to work it.
Can't get much simpler than this :)
ssh host "test -e /path/to/file"
if [ $? -eq 0 ]; then
# your file exists
fi
As suggested by dimo414, this can be collapsed to:
if ssh host "test -e /path/to/file"; then
# your file exists
fi
one line, proper quoting
ssh remote_host test -f "/path/to/file" && echo found || echo not found
You're missing ;s. The general syntax if you put it all in one line would be:
if thing ; then ... ; else ... ; fi
The thing can be pretty much anything that returns an exit code. The then branch is taken if that thing returns 0, the else branch otherwise.
[ isn't syntax, it's the test program (check out ls /bin/[, it actually exists, man test for the docs – although can also have a built-in version with different/additional features.) which is used to test various common conditions on files and variables. (Note that [[ on the other hand is syntax and is handled by your shell, if it supports it).
For your case, you don't want to use test directly, you want to test something on the remote host. So try something like:
if ssh user#host test -e "$file" ; then ... ; else ... ; fi
Test if a file exists:
HOST="example.com"
FILE="/path/to/file"
if ssh $HOST "test -e $FILE"; then
echo "File exists."
else
echo "File does not exist."
fi
And the opposite, test if a file does not exist:
HOST="example.com"
FILE="/path/to/file"
if ! ssh $HOST "test -e $FILE"; then
echo "File does not exist."
else
echo "File exists."
fi
ssh -q $HOST [[ -f $FILE_PATH ]] && echo "File exists"
The above will run the echo command on the machine you're running the ssh command from. To get the remote server to run the command:
ssh -q $HOST "[[ ! -f $FILE_PATH ]] && touch $FILE_PATH"
Silent check if file exist and perform if not
if ! ssh $USER#$HOST "test -e file.txt" 2> /dev/null; then
echo "File not exist"
fi
You can specify the shell to be used by the remote host locally.
echo 'echo "Bash version: ${BASH_VERSION}"' | ssh -q localhost bash
And be careful to (single-)quote the variables you wish to be expanded by the remote host; otherwise variable expansion will be done by your local shell!
# example for local / remote variable expansion
{
echo "[[ $- == *i* ]] && echo 'Interactive' || echo 'Not interactive'" |
ssh -q localhost bash
echo '[[ $- == *i* ]] && echo "Interactive" || echo "Not interactive"' |
ssh -q localhost bash
}
So, to check if a certain file exists on the remote host you can do the following:
host='localhost' # localhost as test case
file='~/.bash_history'
if `echo 'test -f '"${file}"' && exit 0 || exit 1' | ssh -q "${host}" sh`; then
#if `echo '[[ -f '"${file}"' ]] && exit 0 || exit 1' | ssh -q "${host}" bash`; then
echo exists
else
echo does not exist
fi
I wanted also to check if a remote file exist but with RSH. I have tried the previous solutions but they didn't work with RSH.
Finally, I did I short function which works fine:
function existRemoteFile ()
{
REMOTE=$1
FILE=$2
RESULT=$(rsh -l user $REMOTE "test -e $FILE && echo \"0\" || echo \"1\"")
if [ $RESULT -eq 0 ]
then
return 0
else
return 1
fi
}
On CentOS machine, the oneliner bash that worked for me was:
if ssh <servername> "stat <filename> > /dev/null 2>&1"; then echo "file exists"; else echo "file doesnt exits"; fi
It needed I/O redirection (as the top answer) as well as quotes around the command to be run on remote.
This also works :
if ssh user#ip "[ -s /path/file_name ]" ;then
status=RECEIVED ;
else
status=MISSING ;
fi
#its simple
if [[ "`ssh -q user#hostname ls /dir/filename.abc 2>dev/null`" == "/dir/filename.abc" ]]
then
echo "file exists"
else
echo "file not exists"
fi

Resources