I have a directory with the following structure:
file_1
file_2
dir_1
dir_2
# etc.
new_subdir
I'd like to make a copy of all the existing files and directories located in this directory in new_subdir. How can I accomplish this via the linux terminal?
This is an old question, but none of the answers seem to work (they cause the destination folder to be copied recursively into itself), so I figured I'd offer up some working examples:
Copy via find -exec:
find . ! -regex '.*/new_subdir' ! -regex '.' -exec cp -r '{}' new_subdir \;
This code uses regex to find all files and directories (in the current directory) which are not new_subdir and copies them into new_subdir. The ! -regex '.' bit is in there to keep the current directory itself from being included. Using find is the most powerful technique I know, but it's long-winded and a bit confusing at times.
Copy with extglob:
cp -r !(new_subdir) new_subdir
If you have extglob enabled for your bash terminal (which is probably the case), then you can use ! to copy all things in the current directory which are not new_subdir into new_subdir.
Copy without extglob:
mv * new_subdir ; cp -r new_subdir/* .
If you don't have extglob and find doesn't appeal to you and you really want to do something hacky, you can move all of the files into the subdirectory, then recursively copy them back to the original directory. Unlike cp which copies the destination folder into itself, mv just throws an error when it tries to move the destination folder inside of itself. (But it successfully moves every other file and folder.)
You mean like
cp -R * new_subdir
?
cp take -R as argument which means recursive (so, copy also directories), * means all files (and directories).
Although * includes new_subdir itself, but cp detects this case and ignores new_subdir (so it doesn't copy it into itself!)
Try something like:
cp -R * /path_to_new_dir/
Related
I have a script called summarize.sh which produces a summary of the file/dirs inside of a directory. I would like to have it run recursively down the whole tree from the top. Whats a good way to do this?
I have tried to loop it with a for loop with
for dir in */; do
cd $dir
./summarize.sh
cd ..
however it returns ./summarize.sh: no file or directory
Is it because I am not moving the script as I run it? I am not very familiar with unix directories.
You can recursively list files using find . -type f and make your script take the interested file as a first argument, so you can do find . -type f -exec myScript.sh {} \;
If you want directories only, use find . -type d instead, or if you want both use just find . without restriction.
Additional option by name, e.g. find . -name '*.py'
Finally, if you do not want to recurse down the directory structure, i.e. only summarize the top level, you can use -maxdepth 1 option, so something like find . -type d -maxdepth 1 -exec myScript.sh {} \;.
The issue is that you are changing to a different directory with the cd command while your summarize.sh script is not located in these directories. One possible solution is to use an absolute path instead of a relative one. For example, change:
./summarize.sh
to something like:
/path/to/file/summarize.sh
Alternatively, under the given example code, you can also use a relative path pointing to the previous directory like this:
../summarize.sh
Try this code if you are running Bash 4.0 or later:
#! /bin/bash -p
shopt -s nullglob # Globs expand to nothing when they match nothing
shopt -s globstar # Enable ** to expand over the directory hierarchy
summarizer_path=$PWD/summarize.sh
for dir in **/ ; do
cd -- "$dir"
"$summarizer_path"
cd - >/dev/null
done
shopt -s nullglob avoids an error in case there are no directories under the current one.
The summarizer_path variable is set to an absolute path for the summarize.sh program. That is necessary to allow it to be run in directories other than the current one. (./summarize.sh only works in the current directory, ..)
Use cd -- ... to avoid problems if any directory name begins with '-'.
cd - >/dev/null to cd to the previous directory, and throw away its path when it is output by cd -.
Shellcheck issues several warnings about the code above, all to do with the use of cd. I'd fix them for "real" code.
When I try to copy files from one directory to another on Mac, I use the following command:
sudo cp -r dir1/* dir2/
At first glance, it seems that it works well. However, it always skips the files like .git, .gitignore and .htaccess. Why is that? Do those files have some kind of privileged status?
If it helps, the dir1 is the result of
git clone
command that I run prior to that.
In the command:
cp -r dir1/* dir2/
the shell expands dir1/* and passes the result to cp. The * pattern does not match a leading ., that is it matches files which are usually visible. You can use the pattern dir1/.* to match those entries beginning with a ., but for cp there is a better way. From man cp:
If source_file designates a directory, cp copies the directory and the entire subtree connected at that point. If the source_file ends in a /, the contents of the directory are copied rather than the directory itself.
So what you are after is:
cp -r dir1/ dir2/
You can include the hidden files and folders if you replace the * (star) with a . (dot) in your command.
I want to copy a file named TEST to a bunch of folders named 1/ 2/ ... 9/
I was trying to use
cp -v TEST ./[1-9]/
# which gives the result:
TEST '->' ./9/fractionofanions
cp: omitting directory './1'
.
.
cp: omitting directory './8'
Can anyone explain why it only copied to folder 9 in the first place, and also any workaround to do what I need? Thanks in advance.
cp can copy multiple files to a directory, but not files to multiple directories. In this instance, you are attempting to copy TEST and directories 1-8 to directory 9/ - see man cp
for more information.
However, you can use the following to copy a file into multiple directories, using find as a helper:
find [1-9] -exec cp file.txt {} \;
As you can verify in man cp, there can only be one target directory specified for cp. You can use a loop, though:
for target in ./[1-9]/ ; do
cp -v TEST "$target"
done
Let's assume I have following source directory
source/
subdir1/file1
subdir1/file2
subdir2/file3
subdir3/file4
and target directory
target
subdir1/file5
subdir2/file6
subdir4/file7
I would like to move content of source subdirectories to right target subdirectories so result look like this
target
subdir1/file1
subdir1/file2
subdir1/file5
subdir2/file6
subdir2/file3
subdir3/file4
subdir4/file7
Is there some Linux command to do this or must I write a script myself?
To suimmarize, it is important to move, not copy. That rules out cp and rsync but allows mv. mv, however, has the issue that it is not good at merging the old directory into the new.
In the examples that you gave, the target directory had the complete directory tree but lacked files. If that is the case, try:
cd /source ; find . -type f -exec sh -c 'mv "$1" "/target/$1"' _ {} \;
The above starts by selecting the source as the current directory with cd /source. Next, we use find which is the usual *nix utility for finding files/directories. In this case, give find the -type f option to tell it to look only for files. With the -exec option, we tell it to move any such files found to the target directory.
You have choices for how to deal with conflicts between the two directory trees. You can give mv the -f option and it will overwrite files in the target without asking, or you can give it the -n option and it will never overwrite a target file, or your can give it the -i option and it will ask you each time.
In case the target directory tree is incomplete
If the target directory tree is missing some directories that are in the source, the we have to create them on the fly. This adds just minor complication:
cd /source ; find . -type f -exec sh -c 'mkdir -p "/target/${1%/*}"; mv "$1" "/target/$1"' _ {} \;
The mkdir -p command assures that the directory we want exists before we try to move the file there.
Additional notes
The form ${1%/*} is an example of one of the shells powerful features called "parameter expansion". This particular feature is suffix removal. In general, it looks like ${parameter%word} which tells bash to expand word and remove it from the end of parameter. In our case, the name of the parameter is 1, meaning the first argument to the script. We want to remove the file name and just leave behind the directory that the file is in. So, the word /* tells the shell to remove the last slash and any characters which follow.
The commands above use both single and double quotes. They have to be copied exactly for the command to work properly.
To sync dorectory maybe used rsync
Example:
rsync -avzh source/ target/
More info man rsync
Move (no copy)
rsync --remove-source-files -avzh source/ target/
Its must be a popular question but I could not find an answer.
How to move all files via * including hidden files as well to parent directory like this:
mv /path/subfolder/* /path/
This will move all files to parent directory like expected but will not move hidden files. How to do that?
You can find a comprehensive set of solutions on this in UNIX & Linux's answer to How do you move all files (including hidden) from one directory to another?. It shows solutions in Bash, zsh, ksh93, standard (POSIX) sh, etc.
You can use these two commands together:
mv /path/subfolder/* /path/ # your current approach
mv /path/subfolder/.* /path/ # this one for hidden files
Or all together (thanks pfnuesel):
mv /path/subfolder/{.,}* /path/
Which expands to:
mv /path/subfolder/* /path/subfolder/.* /path/
(example: echo a{.,}b expands to a.b ab)
Note this will show a couple of warnings:
mv: cannot move ‘/path/subfolder/.’ to /path/.’: Device or resource busy
mv: cannot remove /path/subfolder/..’: Is a directory
Just ignore them: this happens because /path/subfolder/{.,}* also expands to /path/subfolder/. and /path/subfolder/.., which are the directory and the parent directory (See What do “.” and “..” mean when in a folder?).
If you want to just copy, you can use a mere:
cp -r /path/subfolder/. /path/
# ^
# note the dot!
This will copy all files, both normal and hidden ones, since /path/subfolder/. expands to "everything from this directory" (Source: How to copy with cp to include hidden files and hidden directories and their contents?)
I think this is the most elegant, as it also does not try to move ..:
mv /source/path/{.[!.],}* /destination/path
This will move all files to parent directory like expected but will
not move hidden files. How to do that?
You could turn on dotglob:
shopt -s dotglob # This would cause mv below to match hidden files
mv /path/subfolder/* /path/
In order to turn off dotglob, you'd need to say:
shopt -u dotglob
Alternative simpler solution is to use rsync utility:
sudo rsync -vuar --delete-after --dry-run path/subfolder/ path/
Note: Above command will show what is going to be changed. To execute the actual changes, remove --dry-run.
The advantage is that the original folder (subfolder) would be removed as well as part of the command, and when using mv examples here you still need to clean up your folders, not to mention additional headache to cover hidden and non-hidden files in one single pattern.
In addition rsync provides support of copying/moving files between remotes and it would make sure that files are copied exactly as they originally were (-a).
The used -u parameter would skip existing newer files, -r recurse into directories and -v would increase verbosity.
By using the find command in conjunction with the mv command, you can prevent the mv command from trying to move directories (e.g. .. and .) and subdirectories. Here's one option:
find /path/subfolder -maxdepth 1 -type f -name '*' -exec mv -n {} /path \;
There are problems with some of the other answers provided. For example, each of the following will try to move subdirectories from the source path:
1) mv /path/subfolder/* /path/ ; mv /path/subfolder/.* /path/
2) mv /path/subfolder/{.,}* /path/
3) mv /source/path/{.[!.],}* /destination/path
Also, 2) includes the . and .. files and 3) misses files like ..foobar, ...barfoo, etc.
You could use, mv /source/path/{.[!.],..?,}* /destination/path, which would include the files missed by 3), but it would still try to move subdirectories. Using the find command with the mv command as I describe above eliminates all these problems.
Let me introduce you to my friend "dotglob". It turns on and off whether or not "*" includes hidden files.
$ mkdir test
$ cd test
$ touch a b c .hidden .hi .den
$ ls -a
. .. .den .hi .hidden a b c
$ shopt -u dotglob
$ ls *
a b c
$ for i in * ; do echo I found: $i ; done
I found: a
I found: b
I found: c
$ shopt -s dotglob
$ ls *
.den .hi .hidden a b c
$ for i in * ; do echo I found: $i ; done
I found: .den
I found: .hi
I found: .hidden
I found: a
I found: b
I found: c
It defaults to "off".
$ shopt dotglob
dotglob off
It is best to turn it back on when you are done otherwise you will confuse things that assume it will be off.
My solution for this problem when I have to copy all the files (including . files) to a target directory retaining the permissions is: (overwrite if already exists)
yes | cp -rvp /source/directory /destination/directory/
yes is for automatically overwriting destination files,
r recursive,
v verbose,
p retain permissions.
Notice that the source path is not ending with a / (so all the files/directory and . files are copied)
Destination directory ends with / as we are placing contents of the source folder to destination as a whole.
Just do
for I in $(ls -A dir)
do
mv dir/$I newDir
done
Assuming you are in the subfolder
run find . -maxdepth 1 -exec mv {} .. \;