scp user#server:/home/loghost??/logfiles.log .
i'm using above scp command in my unix script to download all the logs from loghost folder.
there are mutliple loghost are avaible in my server(i.e. loghost01,loghost02,loghost03)
The log name is same in all the loghost folder. So while scping, the logs are getting override. Is there a way to change the logname while copying?
for server in loghost01 loghost02 loghost03; do
mkdir -p $server;
scp user#$server:/home/$server/logfiles.log $server/;
done
I think something like that might help.
It takes a list of your servers, scps files over to a folder named loghost##/logfiles.log.
If you have a list of servers in a text file, replace the top line with:
for server in `cat file_containing_servers`; do
Put logs from different servers into different directories:
for server in loghost{01,02,03}
do
mkdir -p $server
scp user#$server:/home/$server/logfiles.log ./$server/
done
Put logs from different servers into the same directory with different names:
for server in loghost{01,02,03}
do
scp user#$server:/home/$server/logfiles.log ./$server.log
done
Related
I log into server_a and run .sh file, which has the following script:
scp user#server_b:/my_folder/my_file.xml user#server_b:/my_new_folder/
to copy files from my_folder to my_new_folder at server_b. It doesn't throw an error, but no files are copied.
Notes:
server_b is accessed by the pre-set rsa_keys.
server_a: unix
server_b: ubuntu
can SCP files from/to these servers without any issues
The end goal is to move or copy/remove files.
There are two possibilities:
Connect from server_a to server_b and do local copy:
ssh user#server_b "cp /my_folder/my_file.xml /my_new_folder/"
Do copy over the server_a. Your method would require the server_b to be able to authenticate to itself, which is probably not the case:
scp -3 user#server_b:/my_folder/my_file.xml user#server_b:/my_new_folder/
Also note that your code copies only one file and not files as you write in the title.
If you are logged on to the server, why are you authenticating again:
scp user#server_b:/my_folder/my_file.xml user#server_b:/my_new_folder/
You should be in the directory of file or simply use scp and use -v parameter to see the debug information.
Run as follows:
scp -v /my_folder/my_file.xml user#server_b:/my_new_folder/
It is not a directory nor it is recursive, so you do not need to -r parameter.
I am trying to find a way to create and update a tar archive of files on a remote system where we don't have write permissions (the remote file system is read only) over ssh. I've figured out that the way to create a archive is,
ssh user#remoteServer "tar cvpjf - /" > backup.tgz
However, I would like to know if there is some way of performing only incremental backups from this point on (of only files that have actually changed?). Any help with this is much appreciated.
You can try using the --listed-incremental option of tar:
http://www.gnu.org/software/tar/manual/html_node/Incremental-Dumps.html
The main problem is that you have no option to pipe the snar file through the stdout because you are already piping backup.tgz so the best option to store it would be to create the file in the /tmp directory where you should have write permissions and then download it at the end of the backup session.
For example:
ssh user#remoteServer "tar --listed-incremental=/tmp/backup-1.snar -cvpjf - /" > backup-1.tgz
scp user#remoteServer:/tmp/backup-1.snar
And in the following session you will use that .snar file to avoid copying the same files:
scp backup-1.snar user#remoteServer:/tmp/backup-1.snar
ssh user#remoteServer "tar --listed-incremental=/tmp/backup-1.snar -cvpjf - /" > backup-2.tgz
Sorry if it's too much simple question. But I am Java developer, no idea of shell scripting.
I googled, but couldn't find exactly what I am looking for.
My requirement
Connect to remote server using Sftp [authentication based on pub/pri
keys]. A variable to point to private key file
Transfer files with
specific extension [.log] to local server folder. Variable to set
remote server path and local folder
Rename the transferred file in
remote server
Log all the transferred files in a .txt file
Can any one give me shell script for this?
This is so far I framed from suggestions.
Still some questions left on my side ;)
export PRIVKEY=${private_key_path}
export RMTHOST=user#remotehost
export RMTDIR=/logs/*.log
export LOCDIR=/downloaded/logs/
export LOG=sucess.txt
scp -i $PRIVKEY $RMTHOST:$RMTDIR $LOCDIR
for i in 'ls -1 $LOCDIR/*.log'
do
echo $i >> $LOG
done
ssh $RMTHOST -c "for i in `ls -1 $RMTDIR; do mv /logs/$i /logs/$i.transferred; done"
What about this approach?
Connect to remote server using Sftp [authentication based on pub/pri keys]. A variable to point to private key file
Transfer files with specific extension [.log] to local server folder. Variable to set remote server path and local folder
scp your_user#server:/dir/of/file/*.log /your/local/dir
Log all the transferred files in a .txt file
for file in /your/local/dir/*.log
do
echo "$file" >> $your_txt
done
Rename the transferred file in remote server
ssh your_user#server -c "for file in /dir/of/file/*.log; do mv /dir/of/file/"$file" /dir/of/file/new_name_based_on"$file"; done"
Use scp(secure copy) command to transfer file. You may also want to add the -C switch, which compresses the file. That can speed things up a bit. i.e. copy file1 from server1 to server2,
On server1:
#!/bin/sh
scp -C /home/user/file1 root#server2.com:/home/user
Edit:
#!/bin/sh
scp -i {path/to/pub/pri/key/file} /home/user/file1 root#server2.com:/home/user
Can we ftp specific files from a directory. And these specific files that needs to be transferred will be specified in config file.
Can we use a for loop once logged into ftp (in a script) for this purpose.
Will a normal ftp work when transferring files from Unix to win ftp server.
Thanks,
Ravi
You can use straight shell. This assumes your login directory is /home/ravi
Try this one time only:
echo "machine serverB user ravi password ravipasswd" > /home/ravi/.netrc
chmod 600 /home/ravi/.netrc
test that .netrc works - ftp serverB should log you straight in.
Shell script that reads config.file, which is just a list of files to send
while read fname
do
ftp serverB <<EOF
get $fname
bye
EOF # leave the EOF in column #1 of the script file
done < config.file
This gets file from serverB. Change get $fname to put $fname to send files from serverA to serverB
That certainly is possible. You can transfeer files listed in some file by implementing a script using an ftp client (buildin or via calling a cli client). The protocol is system independant, therefore it is possible to transfer files between systems running different operating systems. There is only one catch: remember that MS-Windows uses a case insensitive file system, other systems differ in that.
I'm using lftp to mirror files from external server but now what I need is to after sucessful download rename source directory (on remote server). Basicaly what I need is to open connection on remote server list directories, download all dirs that name starts from "todo" i.e. todo.20121019 after sucess I must rename downloaded directory to "done.20121019". There might be more than one dir on the server.
Remote FTP server works only with active connection.
#!/bin/bash
directories=`lftp -f lftp_script_file.lf |grep done|awk '{print $NF}'`
for i in $directories
do
echo $i //here I get list of directories that should be downloaded and renamed
done
lftp_script_file.lf just list directires:
set ftp:passive-mode false;
open ftp://user:pass$#10.10.10.123
ls my_sub_dir/
Is there a way to:
open connection to ftp server
find directories that I want to download
add those dirs to queue and download
rename directories on remote server
in batch file?
What I was trying to achive was to list dirs find interesing ones, download and rename but I cant find a way to post list of dirs to lftp via bash script and "set ftp:passive-mode false".
To be able to substitute variables into lftp commands use something like this:
lftp -e "cmd1;cmd2"