Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
I'm trying to copy all the files from one directory to another, removing all file extensions at the same time.
From directory 0001:
0001/a/1.jpg
0001/b/2.txt
To directory 0002:
0002/a/1
0002/b/2
I've tried several find ... | xargs c...p with no luck.
Recursive copies are really easy to to with tar. In your case:
tar -C 0001 -cf - --transform 's/\(.\+\)\.[^.]\+$/\1/' . |
tar -C 0002 -xf -
If you haven't tar with --transform this can works:
TRG=/target/some/where
SRC=/my/source/dir
cd "$SRC"
find . -type f -name \*.\* -printf "mkdir -p '$TRG/%h' && cp '%p' '$TRG/%p'\n" |\
sed 's:/\.::;s:/./:/:' |\
xargs -I% sh -c "%"
No spaces after the \, need simple end of line, or you can join it to one line like:
find . -type f -name \*.\* -printf "mkdir -p '$TRG/%h' && cp '%p' '$TRG/%p'\n" | sed 's:/\.::;s:/./:/:' | xargs -I% sh -c "%"
Explanation:
the find will find all plain files what have extensions in you SRC (source) directory
the find's printf will prepare the needed shell commands:
command for create the needed directory tree at the TRG (target dir)
command for copying
the sed doing some cosmetic path cleaning, (like correcting /some/path/./other/dir)
the xargs will take the whole line
and execute the prepared commands with shell
But, it will be much better:
simply make an exact copy in 1st step
rename files in 2nd step
easier, cleaner and FASTER (don't need checking/creating the target subdirs)!
Here's some find + bash + install that will do the trick:
for src in `find 0001 -type f` # for all files in 0001...
do
dst=${src/#0001/0002} # match and change beginning of string
dst=${dst%.*} # strip extension
install -D $src $dst # copy to dst, creating directories as necessary
done
This will change the permission mode of all copied files to rwxr-xr-x by default, changeable with install's --mode option.
I came up with this ugly duckling:
find 0001 -type d | sed 's/^0001/0002/g' | xargs mkdir
find 0001 -type f | sed 's/^0001//g' | awk -F '.' '{printf "cp -p 0001%s 0002%s\n", $0, $1}' | sh
The first line creates the directory tree, and the second line copies the files. Problems with this are:
There is only handling for directories and regular files (no
symbolic links etc.)
If there are any periods (besides the
extension) or special characters (spaces, etc.) in the filenames
then the second command won't work.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I'm trying to create a file called 1 in all subdirectories.
e.g the main directory is abc and sub-directories are abc/xyz, abc/ghi/123
It must contain a full path and file name.
You can do something along the lines of
for d in $(find abc -type d); do
touch "$d"/1
done
Use:
find abc -type d -print0 | while IFS= read -r -d '' dir; do
readlink -f "$dir"/1 > "$dir"/1
done
It finds all directories recursive in directory named abc
For each directory found, it read it to "dir" variable
readlink -f shows full path to a path "$dir"/1
> "$dir"/1 writes to the file "$dir"/1, truncates it before writing, creates if it does not exists
The -print0 and -d '' are for handling spaces and newlines in directory names.
And a version using xargs:
find abc -type d -print0 | xargs -0 -n1 -- sh -c 'readlink -f "$1"/1 > "$1"/1` --
But we can also use find's -exec, which should probably be the fastest:
find abc -type d -exec sh -c 'readlink -f "$1"/1 > "$1"/1' -- {} \;
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
I need to copy all *.jar files from a directory maintaining the folder structure of the parent directories. How can I do it in UNIX/Linux terminal?
The command cp -r *.jar /destination_dir is not what I am looking for.
rsync is useful for local file copying as well as between machines. This will do what you want:
rsync -avm --include='*.jar' -f 'hide,! */' . /destination_dir
The entire directory structure from . is copied to /destination_dir, but only the .jar files are copied. The -a ensures all permissions and times on files are unchanged. The -m will omit empty directories. -v is for verbose output.
For a dry run add a -n, it will tell you what it would do but not actually copy anything.
If you don't need the directory structure only the jar files, you can use:
shopt -s globstar
cp **/*.jar destination_dir
If you want the directory structure you can check cp's --parents option.
If your find has an -exec switch, and cp an -t option:
find . -name "*.jar" -exec cp -t /destination_dir {} +
If you find doesn't provide the "+" for parallel invocation, you can use ";" but then you can omit the -t:
find . -name "*.jar" -exec cp {} /destination_dir ";"
cp --parents `find -name \*.jar` destination/
from man cp:
--parents
use full source file name under DIRECTORY
tar -cf - `find . -name "*.jar" -print` | ( cd /destination_dir && tar xBf - )
If you want to maintain the same directory hierarchy under the destination, you could use
(cd SOURCE && find . -type f -name \*.jar -exec tar cf - {} +) \
| (cd DESTINATION && tar xf -)
This way of doing it, instead of expanding the output of find within back-ticks, has the advantage of being able to handle any number of files.
find . -name \*.jar | xargs cp -t /destination_dir
Assuming your jar filenames do not contain spaces, and your cp has the "-t" option. If cp can't do "-t"
find . -name \*.jar | xargs -I FILE cp FILE /destination_dir
I want to recursively search a directory tree and get the 10 most recently modified files.
For each one of these files, i want to create a symlink in my /home/mostrecent/ directory.
I know i could solve this with a scripting language, but I'm a bit miffed that I can't do it with a linux command!
So far i have this:
find /home/myfiles -type f -printf '%TY-%Tm-%Td %TT %p\n' | sort | tail -n 10 | cut -c 32-
How do i create a symlink in /home/mostrecent for each one of these files, without using a scripting language?
Actually, bash is a scripting language, more than capable of doing that sort of stuff even from the command line :-)
Assuming that the command you posted works (and it seems to, based on my cursory testing), you can just do:
i=0
for f in $(CMD) ; do
ln -s $f $HOME/recent$i
((i++))
done
Or, as a one-liner:
i=0;for f in $(CMD);do ln -s $f $HOME/recent$i;((i++));done
This will create the files recent0 through recent9 in your home directory, which are symlinks to the most recent files.
Obviously, you should put your actual command where I've put the marker text CMD above. I've used the marker just so it formats nicely here on SO.
As Jan Hudec points out in a comment, that will only work for files without spaces, evil things in my opinion :-)
But, since people seem to use them, you can use the safer:
i=0
CMD | while read f; do
ln -s $f $HOME/recent$i
((i++))
done
And, again, the one-liner version:
i=0;CMD|while read f;do ln -s $f $HOME/recent$i;((i++));done
I solved this with sed.
All hail sed!
find /home/myfiles/ -type f -printf '%TY-%Tm-%Td %TT %p\n' | sort | tail -n 10 | cut -c 21- | sed -e "s/^/ln -s \"/" -e "s/$/\"/" -e "s/$/ \"\/home\recent\/\"/" | sh
If i pipe sed to cat instead of sh, this is the output:
ln -s "/home/myfiles/1.simplest" "/home/recent/"
ln -s "/home/myfiles/2.with space" "/home/recent/"
ln -s "/home/myfiles/3.with'apostraphe" "/home/recent/"
ln -s "/home/myfiles/4.with'apostrophe space" "/home/recent/"
Thanks for your help.
Create symlinks to several file types modified in the last 24 hrs, with the same filenames (but a different path of course)
Thanks to Pax, Jan and Jon, with a little modification...
Make a 'recent' directory
mkdir ~/recent
Create 'getrecentfiles.sh' and add...
#!/usr/bin/bash
find $HOME -mtime 0 -name \*.txt -print -o \
-mtime 0 -name \*.pdf -print -o \
-mtime 0 -name \*.extensionname -print -o | while read f; do
ln -s $f $HOME/recent/
done
filters:
-mtime (n*24hrs) is time since last modified (n=1 shows only files modified bw 24-48hrs ago)
-o is the OR operator for multiple files (default is AND)
Change it to executable, add it to your startup scripts and make a shortcut to ~/recent on your desktop, to have the latest files you want on hand!
I untarred something into a directory that already contained a lot of things. I wanted to untar into a separate directory instead. Now there are too many files to distinguish between. However the files that I have untarred have been created just now (right ?) and the original files haven’t been modified for long (at least a day). Is there a way to delete just these untarred files based on their creation information ?
Tar usually restores file timestamps, so filtering by time is not likely to work.
If you still have the tar file, you can use it to delete what you unpacked with something like:
tar tf file.tar --quoting-style=shell-always |xargs rm -i
The above will work in most cases, but not all (filenames that have a carriage return in them will break it), so be careful.
You could remove the directories by adding -r to that, but it's probably safer to just remove the toplevel directories manually.
find . -mtime -1 -type f | xargs rm
but test first with
find . -mtime -1 -type f | xargs echo
There are several different answers to this question in order of increasing complexity.
First, if this is a one off, and in this particular instance you are absolutely sure that there are no weird characters in your filenames (spaces are OK, but not tabs, newlines or other control characters, nor unicode characters) this will work:
tar -tf file.tar | egrep '^(\./)?[^/]+(/)?$' | egrep -v '^\./$' | tr '\n' '\0' | xargs -0 rm -r
All that egrepping is to skip out on all the subdirectories of the subdirectories.
Another way to do this that works with funky filenames is this:
mkdir foodir
cd foodir
tar -xf ../file.tar
for file in *; do rm -rf ../"$file"; done
That will create a directory in which your archive has been expanded, but it sounds like you wanted that already anyway. It also will not handle any files who's names start with ..
To make that method work with files that start with ., do this:
mkdir foodir
cd foodir
tar -xf ../file.tar
find . -mindepth 1 -maxdepth 1 -print0 | xargs -0 sh -c 'for file in "$#"; do rm -rf ../"$file"; done' junk
Lastly, taking from Mat's answer, you can do this and it will work for any filename and not require you to untar the directory again:
tar -tf file.tar | egrep '^(\./)?[^/]+(/)?$' | grep -v '^\./$' | tr '\n' '\0' | xargs -0 bash -c 'for fname in "$#"; do fname="$(echo -ne "$fname")"; echo -n "$fname"; echo -ne "\0"; done' junk | xargs -0 rm -r
You can handle files and directories in one pass with:
tar -tf ../test/bob.tar --quoting-style=shell-always | sed -e "s/^\(.*\/\)'$/rmdir \1'/; t; s/^\(.*\)$/rm \1/;" | sort | bash
You can see what is going to happen leave off the pipe to 'bash'
tar -tf ../test/bob.tar --quoting-style=shell-always | sed -e "s/^\(.*\/\)'$/rmdir \1'/; t; s/^\(.*\)$/rm \1/;" | sort
to handle filenames with linefeeds you need more processing.
Is it possible to copy a single file to multiple directories using the cp command ?
I tried the following , which did not work:
cp file1 /foo/ /bar/
cp file1 {/foo/,/bar}
I know it's possible using a for loop, or find. But is it possible using the gnu cp command?
You can't do this with cp alone but you can combine cp with xargs:
echo dir1 dir2 dir3 | xargs -n 1 cp file1
Will copy file1 to dir1, dir2, and dir3. xargs will call cp 3 times to do this, see the man page for xargs for details.
No, cp can copy multiple sources but will only copy to a single destination. You need to arrange to invoke cp multiple times - once per destination - for what you want to do; using, as you say, a loop or some other tool.
Wildcards also work with Roberts code
echo ./fs*/* | xargs -n 1 cp test
I would use cat and tee based on the answers I saw at https://superuser.com/questions/32630/parallel-file-copy-from-single-source-to-multiple-targets instead of cp.
For example:
cat inputfile | tee outfile1 outfile2 > /dev/null
As far as I can see it you can use the following:
ls | xargs -n 1 cp -i file.dat
The -i option of cp command means that you will be asked whether to overwrite a file in the current directory with the file.dat. Though it is not a completely automatic solution it worked out for me.
These answers all seem more complicated than the obvious:
for i in /foo /bar; do cp "$file1" "$i"; done
ls -db di*/subdir | xargs -n 1 cp File
-b in case there is a space in directory name otherwise it will be broken as a different item by xargs, had this problem with the echo version
Not using cp per se, but...
This came up for me in the context of copying lots of Gopro footage off of a (slow) SD card to three (slow) USB drives. I wanted to read the data only once, because it took forever. And I wanted it recursive.
$ tar cf - src | tee >( cd dest1 ; tar xf - ) >( cd dest2 ; tar xf - ) | ( cd dest3 ; tar xf - )
(And you can add more of those >() sections if you want more outputs.)
I haven't benchmarked that, but it's definitely a lot faster than cp-in-a-loop (or a bunch of parallel cp invocations).
If you want to do it without a forked command:
tee <inputfile file2 file3 file4 ... >/dev/null
To use copying with xargs to directories using wildcards on Mac OS, the only solution that worked for me with spaces in the directory name is:
find ./fs*/* -type d -print0 | xargs -0 -n 1 cp test
Where test is the file to copy
And ./fs*/* the directories to copy to
The problem is that xargs sees spaces as a new argument, the solutions to change the delimiter character using -d or -E is unfortunately not properly working on Mac OS.
Essentially equivalent to the xargs answer, but in case you want parallel execution:
parallel -q cp file1 ::: /foo/ /bar/
So, for example, to copy file1 into all subdirectories of current folder (including recursion):
parallel -q cp file1 ::: `find -mindepth 1 -type d`
N.B.: This probably only conveys any noticeable speed gains for very specific use cases, e.g. if each target directory is a distinct disk.
It is also functionally similar to the '-P' argument for xargs.
No - you cannot.
I've found on multiple occasions that I could use this functionality so I've made my own tool to do this for me.
http://github.com/ddavison/branch
pretty simple -
branch myfile dir1 dir2 dir3
ls -d */ | xargs -iA cp file.txt A
Suppose you want to copy fileName.txt to all sub-directories within present working directory.
Get all sub-directories names through ls and save them to some temporary file say, allFolders.txt
ls > allFolders.txt
Print the list and pass it to command xargs.
cat allFolders.txt | xargs -n 1 cp fileName.txt
Another way is to use cat and tee as follows:
cat <source file> | tee <destination file 1> | tee <destination file 2> [...] > <last destination file>
I think this would be pretty inefficient though, since the job would be split among several processes (one per destination) and the hard drive would be writing several files at once over different parts of the platter. However if you wanted to write a file out to several different drives, this method would probably be pretty efficient (as all copies could happen concurrently).
Using a bash script
DESTINATIONPATH[0]="xxx/yyy"
DESTINATIONPATH[1]="aaa/bbb"
..
DESTINATIONPATH[5]="MainLine/USER"
NumberOfDestinations=6
for (( i=0; i<NumberOfDestinations; i++))
do
cp SourcePath/fileName.ext ${DESTINATIONPATH[$i]}
done
exit
if you want to copy multiple folders to multiple folders one can do something like this:
echo dir1 dir2 dir3 | xargs -n 1 cp -r /path/toyourdir/{subdir1,subdir2,subdir3}
If all your target directories match a path expression — like they're all subdirectories of path/to — then just use find in combination with cp like this:
find ./path/to/* -type d -exec cp [file name] {} \;
That's it.
If you need to be specific on into which folders to copy the file you can combine find with one or more greps. For example to replace any occurences of favicon.ico in any subfolder you can use:
find . | grep favicon\.ico | xargs -n 1 cp -f /root/favicon.ico
This will copy to the immediate sub-directories, if you want to go deeper, adjust the -maxdepth parameter.
find . -mindepth 1 -maxdepth 1 -type d| xargs -n 1 cp -i index.html
If you don't want to copy to all directories, hopefully you can filter the directories you are not interested in. Example copying to all folders starting with a
find . -mindepth 1 -maxdepth 1 -type d| grep \/a |xargs -n 1 cp -i index.html
If copying to a arbitrary/disjoint set of directories you'll need Robert Gamble's suggestion.
I like to copy a file into multiple directories as such:
cp file1 /foo/; cp file1 /bar/; cp file1 /foo2/; cp file1 /bar2/
And copying a directory into other directories:
cp -r dir1/ /foo/; cp -r dir1/ /bar/; cp -r dir1/ /foo2/; cp -r dir1/ /bar2/
I know it's like issuing several commands, but it works well for me when I want to type 1 line and walk away for a while.
For example if you are in the parent directory of you destination folders you can do:
for i in $(ls); do cp sourcefile $i; done