Its must be a popular question but I could not find an answer.
How to move all files via * including hidden files as well to parent directory like this:
mv /path/subfolder/* /path/
This will move all files to parent directory like expected but will not move hidden files. How to do that?
You can find a comprehensive set of solutions on this in UNIX & Linux's answer to How do you move all files (including hidden) from one directory to another?. It shows solutions in Bash, zsh, ksh93, standard (POSIX) sh, etc.
You can use these two commands together:
mv /path/subfolder/* /path/ # your current approach
mv /path/subfolder/.* /path/ # this one for hidden files
Or all together (thanks pfnuesel):
mv /path/subfolder/{.,}* /path/
Which expands to:
mv /path/subfolder/* /path/subfolder/.* /path/
(example: echo a{.,}b expands to a.b ab)
Note this will show a couple of warnings:
mv: cannot move ‘/path/subfolder/.’ to /path/.’: Device or resource busy
mv: cannot remove /path/subfolder/..’: Is a directory
Just ignore them: this happens because /path/subfolder/{.,}* also expands to /path/subfolder/. and /path/subfolder/.., which are the directory and the parent directory (See What do “.” and “..” mean when in a folder?).
If you want to just copy, you can use a mere:
cp -r /path/subfolder/. /path/
# ^
# note the dot!
This will copy all files, both normal and hidden ones, since /path/subfolder/. expands to "everything from this directory" (Source: How to copy with cp to include hidden files and hidden directories and their contents?)
I think this is the most elegant, as it also does not try to move ..:
mv /source/path/{.[!.],}* /destination/path
This will move all files to parent directory like expected but will
not move hidden files. How to do that?
You could turn on dotglob:
shopt -s dotglob # This would cause mv below to match hidden files
mv /path/subfolder/* /path/
In order to turn off dotglob, you'd need to say:
shopt -u dotglob
Alternative simpler solution is to use rsync utility:
sudo rsync -vuar --delete-after --dry-run path/subfolder/ path/
Note: Above command will show what is going to be changed. To execute the actual changes, remove --dry-run.
The advantage is that the original folder (subfolder) would be removed as well as part of the command, and when using mv examples here you still need to clean up your folders, not to mention additional headache to cover hidden and non-hidden files in one single pattern.
In addition rsync provides support of copying/moving files between remotes and it would make sure that files are copied exactly as they originally were (-a).
The used -u parameter would skip existing newer files, -r recurse into directories and -v would increase verbosity.
By using the find command in conjunction with the mv command, you can prevent the mv command from trying to move directories (e.g. .. and .) and subdirectories. Here's one option:
find /path/subfolder -maxdepth 1 -type f -name '*' -exec mv -n {} /path \;
There are problems with some of the other answers provided. For example, each of the following will try to move subdirectories from the source path:
1) mv /path/subfolder/* /path/ ; mv /path/subfolder/.* /path/
2) mv /path/subfolder/{.,}* /path/
3) mv /source/path/{.[!.],}* /destination/path
Also, 2) includes the . and .. files and 3) misses files like ..foobar, ...barfoo, etc.
You could use, mv /source/path/{.[!.],..?,}* /destination/path, which would include the files missed by 3), but it would still try to move subdirectories. Using the find command with the mv command as I describe above eliminates all these problems.
Let me introduce you to my friend "dotglob". It turns on and off whether or not "*" includes hidden files.
$ mkdir test
$ cd test
$ touch a b c .hidden .hi .den
$ ls -a
. .. .den .hi .hidden a b c
$ shopt -u dotglob
$ ls *
a b c
$ for i in * ; do echo I found: $i ; done
I found: a
I found: b
I found: c
$ shopt -s dotglob
$ ls *
.den .hi .hidden a b c
$ for i in * ; do echo I found: $i ; done
I found: .den
I found: .hi
I found: .hidden
I found: a
I found: b
I found: c
It defaults to "off".
$ shopt dotglob
dotglob off
It is best to turn it back on when you are done otherwise you will confuse things that assume it will be off.
My solution for this problem when I have to copy all the files (including . files) to a target directory retaining the permissions is: (overwrite if already exists)
yes | cp -rvp /source/directory /destination/directory/
yes is for automatically overwriting destination files,
r recursive,
v verbose,
p retain permissions.
Notice that the source path is not ending with a / (so all the files/directory and . files are copied)
Destination directory ends with / as we are placing contents of the source folder to destination as a whole.
Just do
for I in $(ls -A dir)
do
mv dir/$I newDir
done
Assuming you are in the subfolder
run find . -maxdepth 1 -exec mv {} .. \;
Related
I have the following structure:
FolderA
Sub1
Sub2
filexx.csv
filexx.doc
FolderB
Sub1
Sub2
fileyy.csv
fileyy.doc
I want to write a script that will move the .csv files into the folder sub1 for each parent directory (Folder A, Folder B and so on) giving me the following structure:
FolderA
Sub1
filexx.csv
Sub2
filexx.doc
FolderB
Sub1
fileyy.csv
Sub2
fileyy.doc
This is what I have till now but I get the error mv: cannot stat *.csv: No such file or directory
for f in */*/*.csv; do
mv -v "$f" */*/Sub1;
done
for f in */*/*.doc; do
mv -v "$f" */*/Sub2;
done
I am new to bash scripting so please forgive me if I have made a very obvious mistake. I know I can do this in Python as well but it will be lengthier which is why I would like a solution using linux commands.
find . -name "*.csv" -type f -execdir mv '{}' Sub1/ \;
Using find, search for all files with the extension .csv and then when we find them, execute a move command from within the directory containing the files, moving the files to directory Sub1
find . -name "*.doc" -type f -execdir mv '{}' Sub2/ \;
Follow the same principle for files with the extension .doc but this time, move the files to Sub2.
I believe you are getting this error because no file matched your wildcard. When it happens, the for loop will give $f the value of the wildcard itself. You are basically trying to move the file *.csv which does not exist.
To prevent this behavior, you can add shopt -s nullglob at the top of your script. When using this, if no file is found, your script won't enter the loop.
My advise is, make sure you run your script from the correct location when using wildcards like this. But maybe what you meant to do by writing */*/*.csv is to recursively match all the csv files. If that's what you intended to do, this is not the right way to do it.
To recursively match all csv/doc/etc files using native bash you can add shopt -s globstar to the top of your script and use **/*.csv as wildcard
#!/bin/bash
shopt -s globstar nullglob
for f in **/*.csv; do
mv "$f" Destination/ # Note that $f is surrounded by "" to handle whitespaces in filenames
done
You could also use the find (1) utility to achieve that. But if you're planning to do more processing on the files than just moving them, a for loop might be cleaner as you won't have to inline everything in the same command.
Side note : "Linux commands" as you say are actually not Linux commands, they are part of the GNU utilities (https://www.gnu.org/gnu/linux-and-gnu.en.html)
If csv files you want to move are in the top directories (from the point of view of the current directory), but not in the subdirectories of them, then simply:
#!/bin/bash
for dir in */; do
mv -v "$dir"*.csv "${dir}Sub1/"
mv -v "$dir"*.doc "${dir}Sub2/"
done
If the files in all subdirectories are wanted to be moved similarly, then:
shopt -s globstar
for file in **/*.csv; do
mv -v "$file" "${file%/*}/Sub1/"
done
for file in **/*.doc; do
mv -v "$file" "${file%/*}/Sub2/"
done
Note that, the directories Sub1 and Sub2 are relative to the directory where csv and doc files reside.
I have a script called summarize.sh which produces a summary of the file/dirs inside of a directory. I would like to have it run recursively down the whole tree from the top. Whats a good way to do this?
I have tried to loop it with a for loop with
for dir in */; do
cd $dir
./summarize.sh
cd ..
however it returns ./summarize.sh: no file or directory
Is it because I am not moving the script as I run it? I am not very familiar with unix directories.
You can recursively list files using find . -type f and make your script take the interested file as a first argument, so you can do find . -type f -exec myScript.sh {} \;
If you want directories only, use find . -type d instead, or if you want both use just find . without restriction.
Additional option by name, e.g. find . -name '*.py'
Finally, if you do not want to recurse down the directory structure, i.e. only summarize the top level, you can use -maxdepth 1 option, so something like find . -type d -maxdepth 1 -exec myScript.sh {} \;.
The issue is that you are changing to a different directory with the cd command while your summarize.sh script is not located in these directories. One possible solution is to use an absolute path instead of a relative one. For example, change:
./summarize.sh
to something like:
/path/to/file/summarize.sh
Alternatively, under the given example code, you can also use a relative path pointing to the previous directory like this:
../summarize.sh
Try this code if you are running Bash 4.0 or later:
#! /bin/bash -p
shopt -s nullglob # Globs expand to nothing when they match nothing
shopt -s globstar # Enable ** to expand over the directory hierarchy
summarizer_path=$PWD/summarize.sh
for dir in **/ ; do
cd -- "$dir"
"$summarizer_path"
cd - >/dev/null
done
shopt -s nullglob avoids an error in case there are no directories under the current one.
The summarizer_path variable is set to an absolute path for the summarize.sh program. That is necessary to allow it to be run in directories other than the current one. (./summarize.sh only works in the current directory, ..)
Use cd -- ... to avoid problems if any directory name begins with '-'.
cd - >/dev/null to cd to the previous directory, and throw away its path when it is output by cd -.
Shellcheck issues several warnings about the code above, all to do with the use of cd. I'd fix them for "real" code.
Let's assume I have following source directory
source/
subdir1/file1
subdir1/file2
subdir2/file3
subdir3/file4
and target directory
target
subdir1/file5
subdir2/file6
subdir4/file7
I would like to move content of source subdirectories to right target subdirectories so result look like this
target
subdir1/file1
subdir1/file2
subdir1/file5
subdir2/file6
subdir2/file3
subdir3/file4
subdir4/file7
Is there some Linux command to do this or must I write a script myself?
To suimmarize, it is important to move, not copy. That rules out cp and rsync but allows mv. mv, however, has the issue that it is not good at merging the old directory into the new.
In the examples that you gave, the target directory had the complete directory tree but lacked files. If that is the case, try:
cd /source ; find . -type f -exec sh -c 'mv "$1" "/target/$1"' _ {} \;
The above starts by selecting the source as the current directory with cd /source. Next, we use find which is the usual *nix utility for finding files/directories. In this case, give find the -type f option to tell it to look only for files. With the -exec option, we tell it to move any such files found to the target directory.
You have choices for how to deal with conflicts between the two directory trees. You can give mv the -f option and it will overwrite files in the target without asking, or you can give it the -n option and it will never overwrite a target file, or your can give it the -i option and it will ask you each time.
In case the target directory tree is incomplete
If the target directory tree is missing some directories that are in the source, the we have to create them on the fly. This adds just minor complication:
cd /source ; find . -type f -exec sh -c 'mkdir -p "/target/${1%/*}"; mv "$1" "/target/$1"' _ {} \;
The mkdir -p command assures that the directory we want exists before we try to move the file there.
Additional notes
The form ${1%/*} is an example of one of the shells powerful features called "parameter expansion". This particular feature is suffix removal. In general, it looks like ${parameter%word} which tells bash to expand word and remove it from the end of parameter. In our case, the name of the parameter is 1, meaning the first argument to the script. We want to remove the file name and just leave behind the directory that the file is in. So, the word /* tells the shell to remove the last slash and any characters which follow.
The commands above use both single and double quotes. They have to be copied exactly for the command to work properly.
To sync dorectory maybe used rsync
Example:
rsync -avzh source/ target/
More info man rsync
Move (no copy)
rsync --remove-source-files -avzh source/ target/
I would like to know whether rm can remove all files within a directory (but not the subfolders or files within the subfolders)?
I know some people use:
rm -f /direcname/*.*
but this assumes the filename has an extension which not all do (I want all files - with or without an extension to be removed).
Although find allows you to delete files using -exec rm {} \; you can use
find /direcname -maxdepth 1 -type f -delete
and it is faster. Using -delete implies the -depth option, which means process directory contents before directory.
find /direcname -maxdepth 1 -type f -exec rm {} \;
Explanation:
find searches for files and directories within /direcname
-maxdepth restricts it to looking for files and directories that are direct children of /direcname
-type f restricts the search to files
-exec rm {} \; runs the command rm {} for each file (after substituting the file's path in place of {}).
I would like to know whether rm can remove all files within a directory (but not the subfolders or files within the subfolders)?
That's easy:
$ rm folder/*
Without the -r, the rm command won't touch sub-directories or the files they contain. This will only remove the files in folder and not the sub-directories or their files.
You will see errors telling you that folder/foo is a directory can cannot be removed, but that's actually okay with you. If you want to eliminate these messages, just redirect STDERR:
$ rm folder/* 2> /dev/null
By the way, the exit status of the rm command may not be zero, so you can't check rm for errors. If that's important, you'll have to loop:
$ for file in *
> do
> [[ -f $file ]] && rm $file
> [[ $? -ne 0 ]] && echo "Error in removing file '$file'"
> done
This should work in BASH even if the file names have spaces in them.
You can use
find /direcname -maxdepth 1 -type f -exec rm -f {} \;
A shell solution (without the non-standard find -maxdepth) would be
for file in .* *; do
test -f "$file" && rm "$file"
done
Some shells, notably zsh and perhaps bash version 4 (but not version 3), have a syntax to do that.
With zsh you might just type
rm /dir/path/*(.)
and if you would want to remove any file whose name starts with foo, recursively in subdirectories, you could do
rm /dir/path/**/foo*(.)
the double star feature is (with IMHO better interactive completion) in my opinion enough to switch to zsh for interactive shells. YMMV
The dot in parenthesis suffix indicates that you want only files (not symlinks or directories) to be expanded by the zsh shell.
Unix isn't DOS. There is no special "extension" field in a file name. Any characters after a dot are just part of the name and are called the suffix. There can be more than one suffix, for example.tar.gz. The shell glob character * matches across the . character; it is oblivious to suffixes. So the MS-DOS *.* is just * In Unix.
Almost. * does not match files which start with a .. Objects named with a leading dot are, by convention, "hidden". They do not show up in ls either unless you specify -a.
(This means that the . and .. directory entries for "self" and "parent" are considered hidden.)
To match hidden entries also, use .*
The rm command does not remove directories (when not operated recursively with -r).
Try rm <directory> and see. Even if the directory is empty, it will refuse.
So, the way you remove all (non-hidden) files, pipes, devices, sockets and symbolic links from a directory (but leave the subdirectories alone) is in fact:
rm /path/to/directory/*
to also remove the hidden ones which start with .:
rm /path/to/directory/{*,.*}
This syntax is brace expansion. Brace expansion is not pattern matching; it is just a short-hand for generating multiple arguments, in this case:
rm /path/to/directory/* /path/to/directory/.*
this expansion takes place first first and then globbing takes place to generate the names to be deleted.
Note that various solutions posted here have various issues:
find /path/to/directory -type f -delete
# -delete is not Unix standard; GNU find extension
# without -maxdepth 1 this will recurse over all files
# -maxdepth is also a GNU extension
# -type f finds only files; so this neglects to delete symlinks, fifos, etc.
The GNU find solutions have the benefit that they work even if the number of directory entries to be deleted is huge: too large to pass in a single call to rm. Another benefit is that the built-in -delete does not have issues with passing funny path names to an external command.
The portable workaround for the problem of too many directory entries is to list the entries with ls and pipe to xargs:
( cd /path/to/directory ; ls -a | xargs rm -- )
The parentheses mean "do these commands in a sub-process"; this way the effect of the cd is forgotten, which is useful in scripting. ls -a includes the hidden files.
We now include a -- after rm which means "this is the last option; everything else is a non-option argument". This guards us against directory entries whose names are indistinguishable from options. What if a file is called -rf and ends up the first argument? Then you have rm -rf ... which will blow off subdirectories.
The easiest way to do this is to use:
rm *
In order to remove directories, you must specify the option -r
rm -r
so your directories and anything contained in them will not be removed by using
rm *
per the man page for rm, its purpose is to remove files, which is why this works
If we want to delete all files and directories we use, rm -rf *.
But what if i want all files and directories be deleted at a shot, except one particular file?
Is there any command for that? rm -rf * gives the ease of deletion at one shot, but deletes even my favourite file/directory.
Thanks in advance
find can be a very good friend:
$ ls
a/ b/ c/
$ find * -maxdepth 0 -name 'b' -prune -o -exec rm -rf '{}' ';'
$ ls
b/
$
Explanation:
find * -maxdepth 0: select everything selected by * without descending into any directories
-name 'b' -prune: do not bother (-prune) with anything that matches the condition -name 'b'
-o -exec rm -rf '{}' ';': call rm -rf for everything else
By the way, another, possibly simpler, way would be to move or rename your favourite directory so that it is not in the way:
$ ls
a/ b/ c/
$ mv b .b
$ ls
a/ c/
$ rm -rf *
$ mv .b b
$ ls
b/
Short answer
ls | grep -v "z.txt" | xargs rm
Details:
The thought process for the above command is :
List all files (ls)
Ignore one file named "z.txt" (grep -v "z.txt")
Delete the listed files other than z.txt (xargs rm)
Example
Create 5 files as shown below:
echo "a.txt b.txt c.txt d.txt z.txt" | xargs touch
List all files except z.txt
ls|grep -v "z.txt"
a.txt
b.txt
c.txt
d.txt
We can now delete(rm) the listed files by using the xargs utility :
ls|grep -v "z.txt"|xargs rm
You can type it right in the command-line or use this keystroke in the script
files=`ls -l | grep -v "my_favorite_dir"`; for file in $files; do rm -rvf $file; done
P.S. I suggest -i switch for rm to prevent delition of important data.
P.P.S You can write the small script based on this solution and place it to the /usr/bin (e.g. /usr/bin/rmf). Now you can use it as and ordinary app:
rmf my_favorite_dir
The script looks like (just a sketch):
#!/bin/sh
if [[ -z $1 ]]; then
files=`ls -l`
else
files=`ls -l | grep -v $1`
fi;
for file in $files; do
rm -rvi $file
done;
At least in zsh
rm -rf ^filename
could be an option, if you only want to preserve one single file.
If it's just one file, one simple way is to move that file to /tmp or something, rm -Rf the directory and then move it back. You could alias this as a simple command.
The other option is to do a find and then grep out what you don't want (using -v or directly using one of finds predicates) and then rming the remaining files.
For a single file, I'd do the former. For anything more, I'd write something custom similar to what thkala said.
In bash you have the !() glob operator, which inverts the matched pattern. So to delete everything except the file my_file_name.txt, try this:
shopt -s extglob
rm -f !(my_file_name.txt)
See this article for more details:
http://karper.wordpress.com/2010/11/17/deleting-all-files-in-a-directory-with-exceptions/
I don't know of such a program, but I have wanted it in the past for some times. The basic syntax would be:
IFS='
' for f in $(except "*.c" "*.h" -- *); do
printf '%s\n' "$f"
done
The program I have in mind has three modes:
exact matching (with the option -e)
glob matching (default, like shown in the above example)
regex matching (with the option -r)
It takes the patterns to be excluded from the command line, followed by the separator --, followed by the file names. Alternatively, the file names might be read from stdin (if the option -s is given), each on a line.
Such a program should not be hard to write, in either C or the Shell Command Language. And it makes a good excercise for learning the Unix basics. When you do it as a shell program, you have to watch for filenames containing whitespace and other special characters, of course.
I see a lot of longwinded means here, that work, but with
a/ b/ c/ d/ e/
rm -rf *.* !(b*)
this removes everything except directory b/ and its contents (assuming your file is in b/.
Then just cd b/ and
rm -rf *.* !(filename)
to remove everything else, but the file (named "filename") that you want to keep.
mv subdir/preciousfile ./
rm -rf subdir
mkdir subdir
mv preciousfile subdir/
This looks tedious, but it is rather safe
avoids complex logic
never use rm -rf *, its results depend on your current directory (which could be / ;-)
never use a globbing *: its expansion is limited by ARGV_MAX.
allows you to check the error after each command, and maybe avoid the disaster caused by the next command.
avoids nasty problems caused by space or NL in the filenames.
cd ..
ln trash/useful.file ./
rm -rf trash/*
mv useful.file trash/
you need to use regular expression for this. Write a regular expression which selects all other files except the one you need.