I need to transfer a bunch of files from a production host to my local machine. I'm already in the directory that I need to transfer the files from. I know the names of log files that I need to transfer to my local machine. They are log.timestamp.hostnames and these tend to be long. How can I transfer in bulk using scp ? Is there an easier way than just typing the long file names ? Can I get it out from a filename ?
Use wildcards:
scp log.* user#host:/target/directory
If you didn't want to copy over all of your files in the current directory (which would just be using ./*), what you could do is parse all of the files in your current directory and run a regular expression on it to match up log.timestamp.hostname then pipe that into scp. For the regex I found this example regex with find. To send big files here is an example: scp syntax. Something along the lines of:
scp $(find . -regextype sed -regex ".*/log\.[a-z0-9\-]\.[a-z0-9\-]") user#remote:~/
You will probably want to modify the regex to make it work.
This command line execution option helped to solve my issue of transfering a subset of files. As the AIX unix does not provide the -regextype option with find I used the grep command instead in order to retrieve files tab1.msg to tab9 msg
scp $(find . -name "*" | grep tab.\.msg) user#host:/tmp
Related
I'm trying to download multiple files trough SFTP on a linux server using
sftp -o IdentityFile=key <user>#<server><<END
get -r folder
exit
END
which will download all contents on a folder. It appears that find and grep are invalid commands, so are for loops.
I need to download files having a name containing a string e.g.
test_0.txt
test_1.txt
but no file.txt
Do you really need the -r switch? Are there really any subdirectories in the folder? You do not mention that.
If there are no subdirectories, you can use a simple get with a file mask:
cd folder
get *test*
Are you required to use sftp? A tool like rsync that operates over ssh has flexible include/exclude options. For example:
rsync -a <user>#<server>:folder/ folder/ \
--include='test_*.txt' --exclude='*.txt'
This requires rsync to be installed on the remote system, but that's very common these days. If rsync isn't available, you could do something similar using tar:
ssh <user>#<server> tar -cf- folder/ | tar -xvf- --wildcards '*/test_*.txt'
This tars up all the files remotely, but then only extracts files matching your target pattern on the receiving side.
I need to download all 492 files from a Linux directory which contain a given String within the file. I can't quite manage to find a command which can accomplish this from my searching so far. Could anybody help me out?
Cheers.
Use grep to filter the files with a given string and loop over them to scp like this
for file in $(grep <some-pattern> <directory>); do scp $file <remote>; done;
Just in case if you need to filter out also the files in subdirectories of directory add the -R option to grep
I'm struggling to only retrieve subfolders from a remote windows share with the following directory structure using smbclient. Is there a way of issuing a command to only get folders? The command I have so far is:
smbclient //$host/$share -U"$USER%$PASSWORD" -c 'cd RootFolder; prompt; recurse; mget Test*\'
RootFolder/
Test001/
Revisions.txt
Test002/
Revisions.txt
Test003/
Revisions.txt
Test001=2012_12_05.log
Test001=2012_12_06.log
Test001=2012_12_07.log
Test001=2012_12_08.log
... more log files here
You could pipe the output of your command through grep, looking for lines that end with /.
smbclient ... | egrep '/$'
Instead, you could mount the remote windows file system and then use the find command to search for folders. The find command will search recursively for all directories only. This would be my recommended approach. Assuming you mount the windows filesystem as /mnt/win_host...
find /mnt/win_host -type d
Two questions to ask:
1. I am using scp command to copy a file from a remote server, I am wondering how can I specify the place where to paste the copied file on my local computer?
for example, if I wanna copy a test.txt file from a remote server and paste it on my local computer under /home/myname/tmp/ what is the proper command?
is it
scp SERVER_ADDRESS /home/myname/tmp/
2. If I want to search a file whose name contain text of "test" , what is the command I should use? I mean search for any file with name test , ('_' is a wildcard)
--------------------------- update ------------------------
what is the difference between "find" and "grep"?
1:
scp SERVER_ADDRESS:/path/to/remote/file.txt /path/to/local/file.txt
2:
find . -name "*test*"
This will search for files/directories containing "test" anywhere in the filename. The search will start from the current directory . To search in another path, use find /path/ -name "*test*". If you only want to search in files, that is, exclude directories, then add -type f before the -name option.
First man scp is your friend (as are all man pages in general).
Yes: in full, that'd be like scp server:/path/to/file.txt /local/path/.
Your main options here are:
locate test (if you have locate installed and its database is up to date)
-or-
find /path/name -name '*test*' to find any named files inside the /path/name directory and all its children.
I'm fairly new to Linux (CentOS in this case). I have a folder with about 2000 files in it. I'd like to ideally execute a command at the command prompt that would write out the name of all the files into a single txt file.
If I have to, I could write an actual program to do it too, I was just thinking there might be a way to simply do it from the command prompt.
you can just use
ls > filenames.txt
(usually, start a shell by using "Terminal", or "shell", or "Bash".) You may need to use cd to go to that folder first, or you can ls ~/docs > filenames.txt
If only names of regular files immediately contained within a directory (assume it's ~/dirs) are needed, you can do
find ~/docs -type f -maxdepth 1 > filenames.txt