How can I remove everything else in a folder except FileA, even hidden files?
I use Ubuntu.
I tried the following unsuccessfully
rm [^fileA]
find . -not -name fileA -exec rm {} \;
Note that this will only delete files, not folders. Believe me, you don't want to delete folders like that.
Use extglob. Assuming that FileA is a regular file (i.e. does not begin with a .), then you can do:
shopt -s extglob # Enable extglob
rm !(FileA) .* # Remove all regular files not named FileA and all hidden files
If instead FileA is a hidden file, this won't work, since the !(pattern) construct only creates a list of all regular files not matching pattern.
You also could do it interactively,
rm -i * .*
The * is for all files (except hidden files).
The .* is for all hidden files
gene#vmware:/tmp/test$ ls -al
total 8
drwxr-xr-x 2 gene gene 4096 2009-03-11 12:51 .
drwxrwxrwt 12 root root 4096 2009-03-11 12:51 ..
-rw-r--r-- 1 gene gene 0 2009-03-11 12:51 fileA
-rw-r--r-- 1 gene gene 0 2009-03-11 12:51 .fileB
gene#vmware:/tmp/test$ rm -i * .*
rm: remove regular empty file `fileA'? n
rm: cannot remove directory `.'
rm: cannot remove directory `..'
rm: remove regular empty file `.fileB'? y
gene#vmware:/tmp/test$ ls -al
total 8
drwxr-xr-x 2 gene gene 4096 2009-03-11 12:51 .
drwxrwxrwt 12 root root 4096 2009-03-11 12:51 ..
-rw-r--r-- 1 gene gene 0 2009-03-11 12:51 fileA
For multiple files, the following will remove all files apart from those that have FileA or FileB in the name.
for file in *
do
if [ x`echo $file | grep -ve "\(FileA\|FileB\)"` == x ]; then
rm $file
fi
done
It's more useful in a long list of files. If it's only a short list, I'd go with CoverosGene's response.
Most ways of doing this based on parsing the directory list are likely to be error prone.
If you have write access to the parent directory, and your necessary file is in sub-directory foo, how about:
% mkdir bar
% mv foo/fileA bar
% rm -rf foo
% mv bar foo
i.e. get your essential file(s) the hell out of the way first!
Related
Today I experienced something unbelievable. My goal was to mv files newer than 7 days to another directory. The directory exist.
I used command:
find ./* -newermt $(date +%Y-%m-%d -d '7 day ago') -type f -print | xargs -I '{}' mv {} ../update_error_handled
Then, unbelievably the files were gone, I went to the folder used ls -lA and didn't found any files I moved. What happened? CentOS 7.0, there were no directory mount, original files missing, tried to grep -r "content" / - found nothing... .
So why it did behave that way?
Beforehand I launched
find ./* -newermt $(date +%Y-%m-%d -d '7 day ago') -type f -print it returned:
./file66.xml
./file67.xml
...etc.
It really do sucks to lose data in such a way.
To clarify: Directory existed before moving files. Directory does not contain my files I tried to move today, only older ones.
Did you create in advance a directory at ../update_error_handled?
If not, then all that you have left from the files will be the last one, which will be called ../update_error_handled.
In order to avoid such mistakes, I always make sure that the destination directory exists, by adding /. at the end of the destination directory name.
Unsafe approach:
$ rm -rf file dest_dir
$ touch file
$ ls -ld file dest_dir
ls: cannot access 'dest_dir': No such file or directory
-rw-rw-r--. 1 u u 0 Sep 28 11:53 file
$ mv file dest_dir
$ ls -ld file dest_dir
ls: cannot access 'file': No such file or directory
-rw-rw-r--. 1 u u 0 Sep 28 11:53 dest_dir
Using the unsafe approach, file was renamed to a file named dest_dir.
Safe approach:
$ rm -rf file dest_dir
$ touch file
$ ls -ld file dest_dir
ls: cannot access 'dest_dir': No such file or directory
-rw-rw-r--. 1 u u 0 Sep 28 11:54 file
$ mv file dest_dir/.
mv: cannot move 'file' to 'dest_dir/.': No such file or directory
$ ls -ld file dest_dir
ls: cannot access 'dest_dir': No such file or directory
-rw-rw-r--. 1 u u 0 Sep 28 11:54 file
Using the safe approach, the mv command failed and the file file remained intact.
I need to search for XML files inside a directory tree and create links for them on another directory (staging_ojs_pootle), naming these links with the file path (replacing slashes per dots).
the bash command is not working, I got stuck on the replacement part. Seems like the variable from xargs, named 'file', is not accessible inside the replacement code (${file/\//.})
find directory/ -name '*.xml' | xargs -I 'file' echo "ln" file staging_ojs_pootle/${file/\//.}
The replacement inside ${} result gives me an empty string.
Tried using sed but regular expressions were replacing all or just the last slash :/
find directory/ -name '*.xml' | xargs -I 'file' echo "ln" file staging_ojs_pootle/file |sed -e '/^ln/s/\(staging_ojs_pootle.*\)[\/]\(.*\)/\1.\2/g'
regards
Try this:
$ find directory/ -name '*.xml' |sed -r 'h;s|/|.|g;G;s|([^\n]+)\n(.+)|ln \2 staging_ojs_pootle/\1|e'
For example:
$ mkdir -p /tmp/test
$ touch {1,2,3,4}.xml
# use /tmp/test as staging_ojs_pootle
$ find /tmp/test -name '*.xml' |sed -r 'h;s|/|.|g;G;s|([^\n]+)\n(.+)|ln \2 /tmp/test/\1|e'
$ ls -al /tmp/test
total 8
drwxr-xr-x. 2 root root 4096 Jun 15 13:09 .
drwxrwxrwt. 9 root root 4096 Jun 15 11:45 ..
-rw-r--r--. 2 root root 0 Jun 15 11:45 1.xml
-rw-r--r--. 2 root root 0 Jun 15 11:45 2.xml
-rw-r--r--. 2 root root 0 Jun 15 11:45 3.xml
-rw-r--r--. 2 root root 0 Jun 15 11:45 4.xml
-rw-r--r--. 2 root root 0 Jun 15 11:45 .tmp.test.1.xml
-rw-r--r--. 2 root root 0 Jun 15 11:45 .tmp.test.2.xml
-rw-r--r--. 2 root root 0 Jun 15 11:45 .tmp.test.3.xml
-rw-r--r--. 2 root root 0 Jun 15 11:45 .tmp.test.4.xml
# if don NOT use the e modifier of s command, we can get the final command
$ find /tmp/test -name '*.xml' |sed -r 'h;s|/|.|g;G;s|([^\n]+)\n(.+)|ln \2 /tmp/test/\1|'
ln /tmp/test/1.xml /tmp/test/.tmp.test.1.xml
ln /tmp/test/2.xml /tmp/test/.tmp.test.2.xml
ln /tmp/test/3.xml /tmp/test/.tmp.test.3.xml
ln /tmp/test/4.xml /tmp/test/.tmp.test.4.xml
Explains:
for each xml file, use h to keep the origin filename in hold space.
the use s|/|.|g to substitute all / to . for xml filename.
use G to append the hold space to pattern space, then pattern space is CHANGED_FILENAME\nORIGIN_FILENAME.
use s|([^\n]+)\n(.+)|ln \2 staging_ojs_pootle/\1|e' to merge the command with CHANGED_FILENAME and ORIGIN_FILENAME, then use e modifier of s command to execute the command assembled above, which will do the actual works.
Hope this helps!
If you can be sure that the names of your XML files do not contain any word-splitting characters, you can use something like:
find directory -name "*.xml" | sed 'p;s/\//./' | xargs -n2 echo ln
Suppose I've got a directory that looks like:
-rw-r--r-- 1 some-user wheel 0 file1
-rw-r--r-- 1 some-user wheel 257 file2
-rw-r--r-- 1 some-user wheel 0 file3
-rwxr-xr-x 1 some-user wheel 212 file4
-rw-r--r-- 1 some-user wheel 2012 file5
.... more files here.
If it's relevant, assume that the names of the files are more random than just file#.
How do I remove only the files that are empty (meaning that the file has 0 bytes in it) in a directory, using rm and grep or sed in some form?
The easiest way is to run find with -empty test and -delete action, e.g.:
find -type f -empty -delete
The command finds all files (-type f) in the current directory and its subdirectories, tests if the matched files are empty, and applies -delete action, if -empty returns true.
If you want to restrict the operation to specific levels of depth, use -mindepth and -maxdepth global options.
The command is:
cd DirectoryWithTheFiles
rm -f $(find . -size 0)
This question already has answers here:
How to loop over files in directory and change path and add suffix to filename
(6 answers)
Closed 5 years ago.
I need to store the name of every file contained in a directory with a bash script and processes it in some way:
drwxrwxr-x 5 matteorr matteorr 4096 Jan 10 17:37 Cluster
drwxr-xr-x 2 matteorr matteorr 4096 Jan 19 10:43 Desktop
drwxrwxr-x 9 matteorr matteorr 4096 Jan 20 10:01 Developer
drwxr-xr-x 11 matteorr matteorr 4096 Dec 20 13:55 Documents
drwxr-xr-x 2 matteorr matteorr 12288 Jan 20 13:44 Downloads
drwx------ 11 matteorr matteorr 4096 Jan 20 14:01 Dropbox
drwxr-xr-x 2 matteorr matteorr 4096 Oct 18 18:43 Music
drwxr-xr-x 2 matteorr matteorr 4096 Jan 19 22:12 Pictures
drwxr-xr-x 2 matteorr matteorr 4096 Oct 18 18:43 Public
drwxr-xr-x 2 matteorr matteorr 4096 Oct 18 18:43 Templates
drwxr-xr-x 2 matteorr matteorr 4096 Oct 18 18:43 Videos
with the following command I'm able to split the result of ls -l in between all the spaces and then access the last element, which contains the name:
ls -l | awk '{split($0,array," ")} END{print array[9]}'
However it returns only the last line (i.e. Videos) so I need to iterate it over all the lines returned by the ls -l command.
how can I do this?
Is there a better way to approach this whole problem?
ADDED PART
To be a little more specific on what I need to do:
For all the files contained in a directory if it is a file I won't do anything, if it is a directory I should append the name of the directory to all the files it contains.
So supposing the directory Videos has the files:
-rwxr-xr-x 2 matteorr matteorr 4096 Oct 18 18:43 video1.mpeg
-rwxr-xr-x 2 matteorr matteorr 4096 Oct 18 18:43 Video2.wmv
I need to rename them as follows:
-rwxr-xr-x 2 matteorr matteorr 4096 Oct 18 18:43 video1_Videos.mpeg
-rwxr-xr-x 2 matteorr matteorr 4096 Oct 18 18:43 Video2_Videos.wmv
A better way would be to use bash globbing
Just listing all files
echo *
Or doing something with them
for file in *; do
echo "$file" # or do something else
done
Or recursively with bash 4+
shopt -s globstar
for file in **/*; do
echo "$file" # or do something else
done
Update to get directory name and append it to all files within it
Replace mv with an echo to test what it does. Also note ${file##*.} assumes the extension is everything after the last period, so if you had a file like file.tar.gz in directory on, below would turn it into file.tar_on.gz. As far as I know there is no easy way to handle this problem, though you could skip files with multiple . if you want)
#!/bin/bash
d="/some/dir/to/do/this/on"
name=${d##*/} #name=on
for file in "$d"/*; do
extension=${file##*.}
filename=${file%.*}
filename=${filename##*/}
[[ -f $file ]] && mv "$file" "$d/${filename}_${name}.$extension"
done
e.g.
> ls /some/dir/to/do/this/on
video1.mpeg Video2.wmv
> ./abovescript
> ls /some/dir/to/do/this/on
video1_on.mpeg Video2_on.wmv
Explanation
In bash you can do this
${parameter#word} Removes shortest matching prefix
${parameter##word} Removes longest matching prefix
${parameter%word} Removes shortest matching suffix
${parameter%%word} Removes longest matching suffix
To remove everything anything (*) before and including the last period, I did below
extension=${file##*.}
To remove everything including and from the last period, I did below (think about shortest match here as going from right to left, e.g. * looks for any non-period text right to left, then when it finds a period it removes that whole section)
filename=${file%.*}
To remove everything up to and including the last /, I did below.
filename=${filename##*/}
Some other notes:
"$d/${filename}_${name}.$extension" Variables can have _ so I switched syntax for a couple of variables here for it to work
"$d"/* Expands to every file of any type (regular, dir, symlink etc...) directly in "$d"
What is wrong with
ls > myfile.txt
This will only list the file names (nothing else) and send them to myfile.txt
If you want to go the awk route, just do
ls -l | awk '{print $9}'
The default action of awk is to split fields on space - and this prints the 9th field for every line…
If you want to do other things with the file names, you can just extend your awk script. For example, an array with these file names could be created with
ls -l | awk '{a[NR]=$9}'
and you can use this array (called a) in further processing. If the processing requires something other than awk (from the comments I think it does), you would be better off with something that looks like
#!/bin/bash
for f in $1"/"*
do
if [ -d "$f" ] ; then
./listdir $f
else
echo $f
fi
done
Save this as listdir in your current directory, and you're good to go.
./listdir .
Will list the entire directory, recursing down (with full relative path appended) as needed.
If you want this to be available "from anywhere" (it is a pretty useful command after all) you would put it somewhere in your path (and do a "rehash" command so it will be "known"); then you don't need the ./ at the start of the command.
Good question! Glad you asked. Parsing ls's output is rarely the right thing to do. There are myriad ways to process a list of files. It depends what you want to do with them.
Here are some examples of things you can do. I've used touch as an example command. Replace that with whatever command or commands you want to do.
To run a command over multiple files, often you can simply pass all the files on the command-line.
touch /var/myapp/*
To loop over the files in the current directory:
for file in *; do
touch "$file"
done
To loop over files in another directory:
for file in /some/dir/*; do
touch "$file"
done
To rename files named *.txt to '*.bak', both here and in sub-directories:
find . -name '*.txt' -exec mv {} {}.bak \;
To delete JPEGs in Bob's home directory (damn you Bob and your wandering eyes):
find ~bob/ -name '*.jpg' -delete
To loop over files recursively and do complicated things to them:
find /dir/to/search -print0 | while read -d $'\0' file; do
echo "$file"
touch "$file"
if [[ -L $file ]]; then
# $file is a symlink, do something special
fi
done
ls -l | awk '{split($0,array," ")} {print array[9]}'
or
ls -l | awk '{print $9}'
but why not just ls?
I have a folder that contains versions of my application, each time I upload a new version a new sub-folder is created for it, the sub-folder name is the current timestamp, here is a printout of the main folder used (ls -l |grep ^d):
drwxrwxr-x 7 root root 4096 2011-03-31 16:18 20110331161649
drwxrwxr-x 7 root root 4096 2011-03-31 16:21 20110331161914
drwxrwxr-x 7 root root 4096 2011-03-31 16:53 20110331165035
drwxrwxr-x 7 root root 4096 2011-03-31 16:59 20110331165712
drwxrwxr-x 7 root root 4096 2011-04-03 20:18 20110403201607
drwxrwxr-x 7 root root 4096 2011-04-03 20:38 20110403203613
drwxrwxr-x 7 root root 4096 2011-04-04 14:39 20110405143725
drwxrwxr-x 7 root root 4096 2011-04-06 15:24 20110406151805
drwxrwxr-x 7 root root 4096 2011-04-06 15:36 20110406153157
drwxrwxr-x 7 root root 4096 2011-04-06 16:02 20110406155913
drwxrwxr-x 7 root root 4096 2011-04-10 21:10 20110410210928
drwxrwxr-x 7 root root 4096 2011-04-10 21:50 20110410214939
drwxrwxr-x 7 root root 4096 2011-04-10 22:15 20110410221414
drwxrwxr-x 7 root root 4096 2011-04-11 22:19 20110411221810
drwxrwxr-x 7 root root 4096 2011-05-01 21:30 20110501212953
drwxrwxr-x 7 root root 4096 2011-05-01 23:02 20110501230121
drwxrwxr-x 7 root root 4096 2011-05-03 21:57 20110503215252
drwxrwxr-x 7 root root 4096 2011-05-06 16:17 20110506161546
drwxrwxr-x 7 root root 4096 2011-05-11 10:00 20110511095709
drwxrwxr-x 7 root root 4096 2011-05-11 10:13 20110511100938
drwxrwxr-x 7 root root 4096 2011-05-12 14:34 20110512143143
drwxrwxr-x 7 root root 4096 2011-05-13 22:13 20110513220824
drwxrwxr-x 7 root root 4096 2011-05-14 22:26 20110514222548
drwxrwxr-x 7 root root 4096 2011-05-14 23:03 20110514230258
I'm looking for a command that will leave the last 10 versions (sub-folders) and deletes the rest.
Any thoughts?
There you go. (edited)
ls -dt */ | tail -n +11 | xargs rm -rf
First list directories recently modified then take all of them except first 10, then send them to rm -rf.
ls -dt1 /path/to/folder/*/ | sed '11,$p' | rm -r
this assumes those are the only directories and no others are present in the working directory.
ls -dt1 will normally only print the newest directory however the /*/ will
only match directories and print their full paths the 1 ensures one
line per match/listing t sorts time with newest at the top.
sed takes the 11th line on down to the bottom and prints only those lines, which are then passed to rm.
You can use xargs, but for testing you may wish to remove | rm -r to see if the directories are listed properly first.
If the directories' names contain the date one can delete all but the last 10 directories with the default alphabetical sort
ls -d */ | head -n -10 | xargs rm -rf
ls -lt | grep ^d | sed -e '1,10d' | awk '{sub(/.* /, ""); print }' | xargs rm -rf
Explanation:
list all contents of current directory in chronological order (most recent files first)
filter out all the directories
ignore the 10 first lines / directories
use awk to extract the file names from the remaining 'ls -l' output
remove the files
EDIT:
find . -maxdepth 1 -type d ! -name \\.| sort | tac | sed -e '1,10d' | xargs rm -rf
I suggest the following sequence. I use a similar approach on my Synology NAS to delete old backups. It doesn't rely on the folder names, instead it uses the last modified time to decide which folders to delete. It also uses zero-termination in order to correctly handle quotes, spaces and newline characters in the folder names:
find /path/to/folder -maxdepth 1 -mindepth 1 -type d -printf '%Ts\t' -print0 \
| sort -rnz \
| tail -n +11 -z \
| cut -f2- -z \
| xargs -0 -r rm -rf
IMPORTANT: This will delete any matching folders! I strongly recommend doing a test run first by replacing the last command xargs -0 -r rm -rf with xargs -0 which will echo the matching folders instead of deleting them.
A short explanation of each step:
find /path/to/folder -maxdepth 1 -mindepth 1 -type d -printf '%Ts\t' -print0
Find all directories (-type d) directly inside the backup folder (-maxdepth 1) except the backup folder itself (-mindepth 1), print (-printf) the Unix time (%Ts) of the last modification followed by a tab character (\t, used in step 4) and the full file name followed by a null character (-print0).
sort -rnz
Sort the zero-terminated items (-z) from the previous step using a numerical comparison (-n) and reverse the order (-r). The result is a list of all folders sorted by their last modification time in descending order.
tail -n +11 -z
Print the last lines (tail) from the previous step starting from line 11 (-n +11) considering each line as zero-terminated (-z). This excludes the newest 10 folders (by modification time) from the remaining steps.
cut -f2- -z
Cut each line from the second field until the end (-f2-) treating each line as zero-terminaded (-z) to obtain a list containing the full path to each folder older than 10 days.
xargs -r -0 rm -rf
Take the zero-terminated (-0) items from the previous step (xargs), and, if there are any (-r avoids running the command passed to xargs if there are no nonblank characters), force delete (rm -rf) them.
Your directory names are sorted in chronological order, which makes this easy. The list of directories in chronological order is just *, or [0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9] to be more precise. So you want to delete all but the last 10 of them.
set [0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]/
while [ $# -gt 10 ]; do
rm -rf "$1"
shift
fi
(While there are more than 10 directories left, delete the oldest one.)