The script appears to be correct. However, after FTP'ing all the files in the directory, it gives me the error that it is trying to move a directory into a directory of itself.
Any ideas on why this is occurring?
mysql -u ????? -p????? -h ????? db < $SCRIPT_FOLDER/script.sql > script.xls
echo "###############################################################################"
echo "FTP the files"
#for FILE in `ls $SOURCE_FOLDER/`
for FILE in $SOURCE_FOLDER/*.xls
do
echo "# Uploading $SOURCE_FOLDER/$FILE" >> /tmp/CasesReport.copy.out
sshpass -p ???? sftp -oBatchMode=no -b - user#ftp << END
cd /source/directory/
put $SOURCE_FOLDER/$FILE
bye
END
echo "Moving $FILE to $SOURCE_FOLDER/history/"
mv $SOURCE_FOLDER/$FILE $SOURCE_FOLDER/history/$FILE
$FILE already contains $SOURCE_FOLDER, so you put command is doubling the path.
Example
$ cd /tmp
$ touch foo.txt bar.txt
$ cd
$ SOURCE_FOLDER=/tmp
$ for FILE in $SOURCE_FOLDER/*.txt; do echo "put $SOURCE_FOLDER/$FILE"; done
put /tmp//tmp/bar.txt
put /tmp//tmp/foo.txt
Inside the for loop, just use "$FILE"
Related
The following commands work on my terminal but not in my shell script. I later found out that my terminal was /bin/tcsh. Can somebody tell me what changes I need to do for /bin/sh. Here are the commands I need to change:
cp source_dir/*/dir1/*.xml destination_dir/
Error in sh-> cp: cannot stat `source_dir/*/dir1/*.xml': No such file or directory
sed -i "s+${initial_name}+${final_name}+" $file_name
This one does not complain but does not work as well.
I am adding an example for testing. The code tends to rename the names of xml files and also the contents of xml files. For example-
The file name crr.ya.na.aa.xml should be changed to aa.xml
The same name inside crr.ya.na.aa.xml should also be changed from crr.ya.na.aa to aa
Here is the code:
#!/bin/sh
# Create dir structure for testing
rm -rf audience
mkdir audience
mkdir audience/dir1 audience/dir2 audience/dir3
mkdir audience/dir1/ipxact audience/dir2/ipxact audience/dir3/ipxact
touch audience/dir1/ipxact/crr.ya.na.aa.xml
echo "<spirit:name>crr.ya.na.aa</spirit:name>" > audience/dir1/ipxact/crr.ya.na.aa.xml
touch audience/dir2/ipxact/crr.ya.na.bb.xml
echo "<spirit:name>crr.ya.na.bb</spirit:name>" > audience/dir2/ipxact/crr.ya.na.bb.xml
touch audience/dir3/ipxact/crr.ya.na.cc.xml
echo "<spirit:name>crr.ya.na.cc</spirit:name>" > audience/dir3/ipxact/crr.ya.na.cc.xml
# Create a dir for ipxact_drop files if it does not exist
mkdir -p ipxact_drop
rm -rf ipxact_drop/*
cp audience/*/ipxact/*.xml ipxact_drop/
ls ipxact_drop/ > ipxact_drop_files.log
cat ipxact_drop_files.log | \
awk '{ split($0,a,"."); print a[length(a)-1] "." a[length(a)] }' ipxact_drop_files.log > file_names.log
cat ipxact_drop_files.log | \
awk '{ split($0,a,"."); print "mv ipxact_drop/" $0 " ipxact_drop/" a[length(a)-1] "." a[length(a)] }' ipxact_drop_files.log > command.log
chmod +x command.log
./command.log
while read line
do
echo ipxact_drop/$line
initial_name=`grep -m 1 crr ipxact_drop/$line | sed -e 's/<spirit:name>//' | sed -e 's/<\/spirit:name>//' `
final_name="${line%.*}"
echo $initial_name
echo $final_name
sed -i "s+${initial_name}+${final_name}+" ipxact_drop/$line
done < file_names.log
echo " ***** SCRIPT RUN FINISHED *****"
Only the sed command at the end is not working
I was reading some other posts and understood that xml files can have problems with scripts. Here is what that worked for me upto now.
To remove cp error: replace #!/bin/sh -f with #!/bin/sh
To remove sed error for the test input: replace sed -i ...... with sed -i.back ....
I am trying to run a command
mv /var/www/my_folder/reports.html /tmp/
it is running properly. But I want to put a condition like if that file exists then only run the command. Is there anything like that?
I can put a shell file instead.
for shell a tried below thing
if [ -e /var/www/my_folder/reports.html ]
then
mv /var/www/my_folder/reports.html /tmp/
fi
But I need a command. Can some one help me with this?
Moving the file /var/www/my_folder/reports.html only if it exists and regular file:
[ -f "/var/www/my_folder/reports.html" ] && mv "/var/www/my_folder/reports.html" /tmp/
-f - returns true value if file exists and regular file
if exist file and then move or echo messages through standard error output
test -e /var/www/my_folder/reports.html && mv /var/www/my_folder/reports.html /tmp/ || echo "not existing the file" >&2
You can do it simply in a shell script
#!/bin/bash
# Check for the file
ls /var/www/my_folder/ | grep reports.html > /dev/null
# check output of the previous command
if [ $? -eq 0 ]
then
# echo -e "Found file"
mv /var/www/my_folder/reports.html /tmp/
else
# echo -e "File is not in there"
fi
Hope it helps
Maybe your use case is "Create if not exist, then copy always".
then: touch myfile && cp myfile mydest/
I am trying to ls the directories and print them out but nothing is being displayed. I am able to SSH and execute the first pwd. However, anything within the the for loop has no output. I know for sure there are directories called event-test- because I've done so manually. I've manually entered the directory (/data/kafka/tmp/kafka-logs/) and ran this piece of code and the correct output appeared so I'm not sure why
manually entered has correct output:
for i in `ls | grep "event-test"`; do echo $i; done;
script:
for h in ${hosts[*]}; do
ssh -i trinity-prod-keypair.pem bc2-user#$h << EOF
sudo bash
cd /data/kafka/tmp/kafka-logs/
pwd
for i in `ls | grep "event-test-"`; do
pwd
echo $i;
done;
exit;
exit;
EOF
done
It is because
`ls | grep "event-test-"`
is executing on your localhost not on remote host. Besides parsing ls is error prone and not even needed. You can do:
for h in "${hosts[#]}"; do
ssh -t -t trinity-prod-keypair.pem bc2-user#$h <<'EOF'
sudo bash
cd /data/kafka/tmp/kafka-logs/
pwd
for i in *event-test-*; do
pwd
echo "$i"
done
exit
exit
EOF
done
When parsing ls it is good practice to do ls -1 to get a prettier list to parse. Additionally, when trying to find files named "event-test-" I would recommend the find command. Since I am not completely sure what you are attempting to do other than list the locations of these "event-test" files I'd recommend something more similar to the following:
for h in "${hosts[#]}"; do ssh trinity-prod-keypair.pem bc2-user#$h -t -t "find /data/kafka/tmp/kafka-logs/ -type f -name *event-test-*;" ; done
This will give you a pretty output of the full path to the file and the file name.
I hope this helps.
I have server A test-lx, and server B test2-lx, I want to transfer files from server A to server B.
While transfering the files i'll need to create a driectory only if it's not exist, how can i check if a directory exist during the lftp conenction? How can i out several files in one command instead of doing this in 2 lines.
Is there an option to use find -maxdepth 1 -name DirName
Here is my code:
lftp -u drop-up,1Q2w3e4R ftp://ta1bbn01:21 << EOF
cd $desFolder
mkdir test
cd test
put $srcFil
put $srcFile
bye
EOF
Simple way with ftp:
#!/bin/bash
ftp -inv ip << EOF
user username password
cd /home/xxx/xxx/what/you/want/
put what_you_want_to_upload
bye
EOF
With lftp:
#!/bin/bash
lftp -u username,password ip << EOF
cd /home/xxx/xxx/what/you/want/
put what_you_want_to_upload
bye
EOF
From lftp manual:
-u <user>[,<pass>] use the user/password for authentication
You can use mkdir for create a directory. And you can use put command several time like this:
put what_you_want_to_upload
put what_you_want_to_upload2
put what_you_want_to_upload3
And you can close connection with bye
You can check folder is exist or not like this:
#!/bin/bash
checkfolder=$(lftp -c "open -u user,pass ip; ls /home/test1/test1231")
if [ "$checkfolder" == "" ];
then
echo "folder does not exist"
else
echo "folder exist"
fi
From lftp manual:
-c <cmd> execute the commands and exit
And you can open another connection for put some files.
I don't know how to check folder is exist or not with one connection, but I can do that like this. Maybe you can find better solution:
#!/bin/bash
checkfolder=$(lftp -c "open -u user,pass ip; ls /home/test1/test2")
if [ "$checkfolder" == "" ];
then
lftp -u user,pass ip << EOF
mkdir test2
cd test2
put testfile.txt
bye
EOF
else
echo "The directory already exists - exiting"
fi
I used the same basic coding outline as phe however I found that using ls /foldername will output "folder does not exist" if the folder is empty. To solve this I use
#!/bin/bash
checkfolder=$(lftp -c "open -u user,pass ip; ls | grep /test1231")
if [ "$checkfolder" == "" ];
then
echo "folder does not exist"
else
echo "folder exists"
fi
Please note this only works if the folder is in the root directory. For sub directories in a folder the following should work.
#!/bin/bash
checkfolder=$(lftp -c "open -u user,pass ip; find | grep home/test1/test1231")
if [ "$checkfolder" == "" ];
then
echo "folder does not exist"
else
echo "folder exists"
fi
First prepare credentials record into ~/.netrc file as:
machine site-url-here
login user-login-here
password user-password-here
so you don't have to expose your password on the command line to use this in script.
Then call:
lftp -e "lftp-command-here" ftps://user-login-here#site-url-here/initial-folder-here/`
In my case I run mget -c * lftp command for getting all logs from java spring boot application running linux app instance at azure infrastructure.
Of course you can put your commands separated by semicolon there.
I have a basic bash script that I'm using to copy a file and upload it to my FTP:
cp -i /var/mobile/file.db /var
cd /var
HOST=MYFTPHOST
USER=USERNAME
PASS=PASSWORD
ftp -inv $HOST << EOF
user $USER $PASS
cd websitefolder
put sms.db
bye
EOF
rm -f file.db
When I run the script, it saves the file to my FTP perfectly. But I'm running the script from different computer's so somehow, I'd like the script to upload the file.db to my FTP like this everytime it uploads it:
file1.db
file2.db
file3.db
file4.db
Your question is a little unclear, but if I understand correctly, you're trying to name the database files in sequential order without overwriting any old files. You'll have to get the list of files from the FTP server in order to find out what files have already been uploaded.
This code will get the list of files from the server that begin with "file" and end with ".db", count them, then change the name of your "file.db" to "fileXX.db", where "XX" is the next number in the naming sequence (i.e. file1.db, file2.db, file3.db, etc).
I'm not sure where "sms.db" came from. I've changed it to "file.db" in the script.
cp -i /var/mobile/file.db /var
cd /var
HOST=MYFTPHOST
USER=USERNAME
PASS=PASSWORD
ftp -inv $HOST << EOF
user $USER $PASS
cd websitefolder
LIST=$(ls | grep file*.db)
bye
EOF
FILECOUNT=0
for FILE in $LIST
do
if [ -f $FILE ];
then
FILECOUNT+=1
done
FILECOUNT+=1
NEXTDB="file$FILECOUNT.db"
mv file.db $NEXTDB
ftp -inv $HOST << EOF
put $NEXTDB
bye
EOF