If you run rm -r /var/some/dir/* and the dir is empty an error is returned
How can you supress this error so its not returned?
I use it in a linux bash script
Apart from redirecting to stderr, you can also use -f:
rm -rf /var/some/dir/*
From man rm:
-f, --force
ignore nonexistent files and arguments, never prompt
Redirect stderr:
rm -r /var/some/dir/* 2>/dev/null
Since you question mentions [if] dir is empty an error is returned (so assuming the directory exists), that's because the glob /var/some/dir/* matches nothing, hence bash (with the default unset nullglob and failglob) will pass the string /var/some/dir/* verbatim. And there are no such files, so rm complains.
Since you're using this in a bash script, I would advise you to proceed as follows:
shopt -s nullglob
shopt -u failglob
declare -a files=( /var/some/dir/* )
if (( ${#files[#]} )); then
rm -r -- "${files[#]}" # || error handling
fi
The shopt -s nullglob will make globs expand to nothing if no matching files exist,
You put the matching files in an array files (and because nullglob is set, the array files is empty if there are no matches),
If there are matching files, then rm them.
Proceeding like this will make your script more robust, as you'll be able to catch a genuine error occuring in rm (with the || error handling part) and not confuse this with an error occurring from a non-matched glob.
Doing so, you will not launch a random command with uncontrolled arguments.
Remark. It is not quite true that the error you obtain occurs only when the directory is empty. It occurs when the directory contains no files, except possibly hidden files. If you want to really purge the directory from also potential hidden files, you must shopt -s dotglob so that the glob will also match hidden files.
Related
Im trying to move all the contents of the current directory to a new folder in the current directory using a script
mv !\(.svn\|$line\|.\|..\) $line
error is
mv: cannot stat '!(.svn|RSSIFXServicesCommon|.|..)': No such file or directory
I echoed the command to output and if i copy and execute the command myself, it works.
I tried enabling extglob
With extended regex (shopt -s extglob), there's no need to quote your regex:
mkdir -p /tmp/t/4
touch /tmp/t/{1,2,3}
shopt -s extglob
cd /tmp/t
mv !(4|.|..) 4
This also works if I put this in a shell script.
I have made a script that takes all of the files in the current directory, checks if it is a regular file or a folder and sets permissions to them.
My problem is when I encounter hidden files\folders and files\folders that have spaces in the name.
Here is my script:
#!/bin/bash
FILES=$(pwd)/*
for f in $FILES
do
if [ -f $f ]; then chmod u+x $f; fi
if [ -d $f ]; then chmod u=w,g+r,o-rwx $f; fi
done
Here is an example of an error I get from the testing computer:
'test/.bwhajtbzmu xswxcgqsvz' has incorrect permissions: expected 250, got 414.
The other errors are basically the same.
I am not actually sure what is the problem here, if it is the fact that it is a hidden file or that it has a space in the name. I guess both things are the problem.
How can I modify the script so that it can work with hidden files and files that have space in the name ?
Thank you
PS. Please don't question the usefulness of the script, it is a school homework.
Handling whitespace in file names is tricky. First rule is: doublequotes around all usages of variables. Otherwise the shell interprets the spaces as separators. Unfortunately you cannot simply hold a list in a variable. You have to use array variables for this.
#!/bin/bash
FILES=( "$(pwd)"/* )
for f in "${FILES[#]}"
do
if [ -f "$f" ]; then chmod u+x "$f"; fi
if [ -d "$f" ]; then chmod u=w,g+r,o-rwx "$f"; fi
done
For handling hidden files and folders (the ones starting with a dot .) you should best set the shell option dotglob which makes * also match these (which it otherwise doesn't). (Using .* is not good as it matches also . and .. which normally aren't wanted and things like .??* will not match stuff like .a which is normally wanted.):
shopt -s dotglob
FILES=( "$(pwd)"/* )
shopt -u dotglob
I would not recommend leaving it on, so I switch it off after using it.
Situation:
I need a bash script that deletes all files in the current folder, except all the files mentioned in a file called ".rmignore". This file may contain addresses relative to the current folder, that might also contain asterisks(*). For example:
1.php
2/1.php
1/*.php
What I've tried:
I tried to use GLOBIGNORE but that didn't work well.
I also tried to use find with grep, like follows:
find . | grep -Fxv $(echo $(cat .rmignore) | tr ' ' "\n")
It is considered bad practice to pipe the exit of find to another command. You can use -exec, -execdir followed by the command and '{}' as a placeholder for the file, and ';' to indicate the end of your command. You can also use '+' to pipe commands together IIRC.
In your case, you want to list all the contend of a directory, and remove files one by one.
#!/usr/bin/env bash
set -o nounset
set -o errexit
shopt -s nullglob # allows glob to expand to nothing if no match
shopt -s globstar # process recursively current directory
my:rm_all() {
local ignore_file=".rmignore"
local ignore_array=()
while read -r glob; # Generate files list
do
ignore_array+=(${glob});
done < "${ignore_file}"
echo "${ignore_array[#]}"
for file in **; # iterate over all the content of the current directory
do
if [ -f "${file}" ]; # file exist and is file
then
local do_rmfile=true;
# Remove only if matches regex
for ignore in "${ignore_array[#]}"; # Iterate over files to keep
do
[[ "${file}" == "${ignore}" ]] && do_rmfile=false; #rm ${file};
done
${do_rmfile} && echo "Removing ${file}"
fi
done
}
my:rm_all;
If we assume that none of the files in .rmignore contain newlines in their name, the following might suffice:
# Gather our exclusions...
mapfile -t excl < .rmignore
# Reverse the array (put data in indexes)
declare -A arr=()
for file in "${excl[#]}"; do arr[$file]=1; done
# Walk through files, deleting anything that's not in the associative array.
shopt -s globstar
for file in **; do
[ -n "${arr[$file]}" ] && continue
echo rm -fv "$file"
done
Note: untested. :-) Also, associative arrays were introduced with Bash 4.
An alternate method might be to populate an array with the whole file list, then remove the exclusions. This might be impractical if you're dealing with hundreds of thousands of files.
shopt -s globstar
declare -A filelist=()
# Build a list of all files...
for file in **; do filelist[$file]=1; done
# Remove files to be ignored.
while read -r file; do unset filelist[$file]; done < .rmignore
# Annd .. delete.
echo rm -v "${!filelist[#]}"
Also untested.
Warning: rm at your own risk. May contain nuts. Keep backups.
I note that neither of these solutions will handle wildcards in your .rmignore file. For that, you might need some extra processing...
shopt -s globstar
declare -A filelist=()
# Build a list...
for file in **; do filelist[$file]=1; done
# Remove PATTERNS...
while read -r glob; do
for file in $glob; do
unset filelist[$file]
done
done < .rmignore
# And remove whatever's left.
echo rm -v "${!filelist[#]}"
And .. you guessed it. Untested. This depends on $f expanding as a glob.
Lastly, if you want a heavier-weight solution, you can use find and grep:
find . -type f -not -exec grep -q -f '{}' .rmignore \; -delete
This runs a grep for EACH file being considered. And it's not a bash solution, it only relies on find which is pretty universal.
Note that ALL of these solutions are at risk of errors if you have files that contain newlines.
This line do perfectly the job
find . -type f | grep -vFf .rmignore
If you have rsync, you might be able to copy an empty directory to the target one, with suitable rsync ignore files. Try it first with -n, to see what it will attempt, before running it for real!
This is another bash solution that seems to work ok in my tests:
while read -r line;do
exclude+=$(find . -type f -path "./$line")$'\n'
done <.rmignore
echo "ignored files:"
printf '%s\n' "$exclude"
echo "files to be deleted"
echo rm $(LC_ALL=C sort <(find . -type f) <(printf '%s\n' "$exclude") |uniq -u ) #intentionally non quoted to remove new lines
Test it online here
Alternatively, you may want to look at the simplest format:
rm $(ls -1 | grep -v .rmignore)
This is basically a follow-up to Linux: Move 1 million files into prefix-based created Folders
The original question:
I want to write a shell command to rename all of those images into the
following format:
original: filename.jpg new: /f/i/l/filename.jpg
Now, I want to take all of those files and add an additional level to the directory structure, e.g:
original: /f/i/l/filename.jpg new: /f/i/l/e/filename.jpg
Is this possible to do with command line or bash?
One way to do it is to simply loop over all the directories you already have, and in each bottom-level subdirectory create the new subdirectory and move the files:
for d in ?/?/?/; do (
cd "$d" &&
printf '%.4s\0' * | uniq -z |
xargs -0 bash -c 'for prefix do
s=${prefix:3:1}
mkdir -p "$s" && mv "$prefix"* "$s"
done' _
) done
That probably needs a bit of explanation.
The glob ?/?/?/ matches all directory paths made up of three single-character subdirectories. Because it ends with a /, everything it matches is a directory so there is no need to test.
( cd "$d" && ...; )
executes ... after cd'ing to the appropriate subdirectory. Putting that block inside ( ) causes it to be executed in a subshell, which means the scope of the cd will be restricted to the parenthesized block. That's easier and safer than putting cd .. at the end.
We then collecting the subdirectories first, by finding the unique initial strings of the files:
printf '%.4s\0' * | uniq -z | xargs -0 ...
That extracts the first four letters of each filename, nul-terminating each one, then passes this list to uniq to eliminate duplicates, providing the -z option because the input is nul-terminated, and then passes the list of unique prefixes to xargs, again using -0 to indicate that the list is nul-terminated. xargs executes a command with a list of arguments, issuing the command several times only if necessary to avoid exceeding the command-line limit. (We probably could have avoided the use of xargs but it doesn't cost that much and it's a lot safer.)
The command called with xargs is bash itself; we use the -c option to pass it a command to be executed. That command iterates over its arguments by using the for arg in syntax. Each argument is a unique prefix; we extract the fourth character from the prefix to construct the new subdirectory and then mv all files whose names start with the prefix into the newly created directory.
The _ at the end of the xargs invocation will be passed to bash (as with all the rest of the arguments); bash -c uses the first argument following the command as the $0 argument to the script, which is not part of the command line arguments iterated over by the for arg in syntax. So putting the _ there means that the argument list constructed by xargs will be precisely $1, $2, ... in the execution of the bash command.
Okay, so I've created a very crude solution:
#!/bin/bash
for file1 in *; do
if [[ -d "$file1" ]]; then
cd "$file1"
for file2 in *; do
if [[ -d "$file2" ]]; then
cd "$file2"
for file3 in *; do
if [[ -d "$file3" ]]; then
cd "$file3"
for file4 in *; do
if [[ -f "$file4" ]]; then
echo "mkdir -p ${file4:3:1}/; mv $file4 ${file4:3:1}/;"
mkdir -p ${file4:3:1}/; mv $file4 ${file4:3:1}/;
fi
done
cd ..
fi
done
cd ..
fi
done
cd ..
fi
done
I should warn that this is untested, as my actual structure varies slightly, but I wanted to keep the question/answer consistent with the original question for clarity.
That being said, I'm sure a much more elegant solution exists than this one.
I started using "sudo rm -r" to delete files/directories. I even put it as an alias of rm.
I normally know what I am doing and I am quite experience linux user.
However, I would like that when I press the "ENTER", before the execution of rm, a list of files will show up on the screen and a prompt at the end to OK the deletion of files.
Options -i -I -v does not do what I want. I want only one prompt for all the printed files on screen.
Thank you.
##
# Double-check files to delete.
delcheck() {
printf 'Here are the %d files you said you wanted to delete:\n' "$#"
printf '"%s"\n' "$#"
read -p 'Do you want to delete them? [y/N] ' doit
case "$doit" in
[yY]) rm "$#";;
*) printf 'No files deleted\n';;
esac
}
This is a shell function that (when used properly) will do what you want. However, if you load the function in your current shell then try to use it with sudo, it won't do what you expect because sudo creates a separate shell. So you'd need to make this a shell script…
#!/bin/bash
… same code as above …
# All this script does is create the function and then execute it.
# It's lazy, but functions are nice.
delcheck "$#"
…then make sure sudo can access it. Put it in some place that is in the sudo execution PATH (Depending on sudo configuration.) Then if you really want to execute it precisely as sudo rm -r * you will still need to name the script rm, (which in my opinion is dangerous) and make sure its PATH is before /bin in your PATH. (Also dangerous). But there you go.
Here's a nice option
Alias rm to echo | xargs -p rm
The -p option means "interactive" - it will display the entire command (including any expanded file lists) and ask you to confirm
It will NOT ask about the recursively removed files. But it will expand rm * .o to:
rm -rf * .o
rm -rf program.cc program.cc~ program program.o backup?... # NO NO NO NO NO!
Which is much nicer than receiving the error
rm: .o file not found
Edit: corrected the solution based on chepner comment. My previous solutions had a bug :(
This simple script prompts for a y response before deleting the files specified.
rmc script file:
read -p "ok to delete? " ans
case $ans in
[yY]*) sudo rm "$#" ;;
*) echo "Nothing deleted";;
esac
Invoke thus
./rmc *.tmp
I created a script to do this. The solution is similar to #kojiro's.
Save the script with the filename del. Run the command sudo chmod a=r+w+x del to make the script an executable. In the directory in which you want to save the script, export the path by entering export PATH=$PATH:/path/to/the/del/executable in your '~/.bashrc' file and run source ~/.bashrc.
Here, the syntax of rm is preserved, except instead of typing rm ..., type del ... where del is the name of the bash script below.
#! /bin/bash
# Safely delete files
args=("$#") # store all arguments passed to shell
N=$# # number of arguments passed to shell
#echo $#
#echo $#
#echo ${args[#]:0}
echo "Files to delete:"
echo
n=`expr $N - 1`
for i in `seq 0 $n`
do
str=${args[i]}
if [ ${str:0:1} != "-" ]; then
echo $str
fi
done
echo
read -r -p "Delete these files? [y/n] " response
case $response in
[yY][eE][sS]|[yY])
rm ${args[#]:0}
esac