I have huge file archives that I host on my old-skool BBS. The [Mystic] software isn't as forgiving or capable as Linux with long-filenames OR extended characters.
Filenames should be less than 80 characters long.
Filenames should only have chars A-Z & 1-9. No "! # # $ % ^ &", etc - nor letters with tildes or carets over them.
Here is a sample of what one collections directories looks like:
pi#bbs:/mnt/Beers4TB/opendirs/TDC19 $ ls -all
total 28
drwxrwxr-x 6 pi pi 4096 Sep 16 08:08 .
drwxrwxr-x 11 pi pi 4096 Oct 6 15:04 ..
drwxrwxr-x 2 pi pi 4096 Sep 13 20:13 ANSi
drwxrwxr-x 2 pi pi 4096 Oct 6 21:16 Drivers
drwxrwxr-x 10 pi pi 4096 Sep 16 08:12 Games
-rw-rw-r-- 1 pi pi 1056 Sep 13 20:12 INTRO.TXT
drwxrwxr-x 2 pi pi 4096 Sep 16 08:08 ListsNotes
And within /subdirectories they may go 2, 3 or more deep.
Here is a sample of what some files are currently named:
pi#bbs:/mnt/Beers4TB/opendirs/TDC19/Games/Applications $ ls M*
'Mean 18 - Golf Menu [SW] (1988)(Robert J. Butler) [Sports, Golf, Utility].zip'
'Mean 18 - M18 (1988)(Ken Hopkins) [Sports, Golf, Utility].zip'
'Metaltech- Battledrome Game Editor (1994)(Sierra On-Line, Inc.) [Utility].zip'
'Might and Magic III Character Editor (1991)(Blackbeard'\''s Ghost) [Utility].zip'
'Might Magic 3 Character viewer-editor v1.1 (1991)(Mark Betz and Chris Lampton) [Editor].zip'
I have worked on some things that show promise... this echo/sed command removes SOME high chars:
echo "Might and Magic III Character Editor (1991)(Blackbeard'''s Ghost) [Utility].zip" | sed -r -e 's/\x27+//g' -e 's/[][")(]//g' -e 's/[ ]+//g'
(It renames the file:)
Might_and_Magic_III_Character_Editor_1991_Blackbeards_Ghost_Utility.zip
Then, I have a command that will rename ONE entire /subdirectory, but it DOESN'T remove any characters:
for f in *.zip; do mv "${f}" "${f//[][\")( ]/_}"; done
That's good... but I have to get rid of the high characters... and, this method adds multiple spaces in filenames sometimes - which adds to that max 80 filename limit - and theres no safegaurds built in...
I worked on adding in support for going thru multiple /subdirectories, but I KNOW that my syntax is still wrong... you can, however, see what I was attempting to do:
P=$(pwd); for D in $(find . -maxdepth 1 -type d); do cd $D; for f in *.zip; do mv "${f}" "${f//[][\")( ]/_}"; cd $P; done
So, in closing - I'm open to any Linux commands that will:
Remove any characters that are NOT A-Z or 1-9.
Remove any extra spaces in filenames.
Make sure filenames are only 80 characters long max, simply removing the last bit before the .zip (or .anything) extension.
Begin in a main /directory and rename all files in each /subdirectory within the main.
Very last; I always try to put things together first... I get help from associates second - and I come to the interwebs last... but I want to UNDERSTAND how to code this exact sort of thing myself. If you have any suggestions of where to learn, that would be received well too. I tried to post this question CORRECTLY this time, pls forgive if I haven't gotten every rule correct.
pAULIE42o
. . . . .
/s
The command tr -cd deletes all characters which are not in the given list.
for f in *.zip; do
mv "$f" "$(tr -cd 'A-Za-z0-9. \n' <<< "$f")"
done
You can use sed to add a space between adjacent parentheses:
for f in *.zip; do
mv "$f" "$(sed 's/)(/ /g' <<< "$f" | tr -cd 'A-Za-z0-9. \n'))"
done
And you can use sed to merge multiple spaces.
for f in *.zip; do
mv "$f" "$(sed 's/)(/ /g' <<< "$f" | tr -cd 'A-Za-z0-9. \n' | sed 's/ \+/ /g'))"
done
I recommend you to punycode the names, but I have no proper way (adequate answer) to reduce the lengths of the files to fit in 80 characters long (the punycode process is completely reversible and maintains the ascii codes in their places, giving you a readable file name, and it can be modified to consider the character case of the name characters)
For the extra length encoding, I'd use some kind of fixed length hash function to avoid name clashes, but this process is not reversible at all, you'll be losing part of the name. You need to think a bit on your possibilities to be able to help you in this.
Edit: convert sequences of unwanted characters to one single underscore.
I assume that when you write "Filenames should only have chars A-Z & 1-9" you include lower case letters, plus the underscore to replace any sequence of unwanted characters. I also assume that you don't want leading or trailing underscores in the basenames after substitution.
Let's first write a small bash script file that takes the path of a zip file as first an only parameter ($1), separates the directory ($d) and file ($f) parts with dirname and basename, computes the new file name with tr, sed and cut, and renames the file:
$ cat /mnt/Beers4TB/opendirs/TDC19/renamer.sh
#!/usr/bin/env bash
d="$(dirname "$1")"
f="$(basename -s .zip "$1" | tr -c a-zA-Z1-9 _ | sed 's/__*/_/g' |
cut -c 1-76 | sed 's/^_//;s/_$//')"
mv "$1" "$d/$f.zip"
Next, let's make the script executable (chmod) and use find to walk the hierarchy and call the script on each found zip file (first backup your files, just in case something goes wrong):
$ cd /mnt/Beers4TB/opendirs/TDC19
$ chmod +x renamer.sh
$ find . -type f -name '*.zip' -exec ./renamer.sh '{}' \;
(in the exec action of find {} is replaced by the found file path).
Explanations:
tr is used to replace all unwanted characters by underscores (_). Option -c takes the complement of the specified character set:
$ f='!!!Mean 18 - Golf Menu [SW] ('
$ printf '%s' "$f" | tr -c a-zA-Z1-9 _
___Mean_18___Golf_Menu__SW___
sed is used to replace sequences of underscores by only one underscore (s/__*/_/g), delete a leading underscore (s/^_//) and delete a trailing underscore (s/_$//):
$ f="___Mean_18___Golf_Menu__SW___"
$ printf '%s' "$f" | sed 's/__*/_/g'
_Mean_18_Golf_Menu_SW_
$ f="_Mean_18_Golf_Menu_SW_"
$ printf '%s' "$f" | sed 's/^_//;s/_$//'
Mean_18_Golf_Menu_SW
cut is used to clip the modified base name to 80-4=76 characters. After restoring the .zip suffix it will have 80 characters at most. The -c X-Y option of cut selects characters number X to Y:
$ f='abcdefghi'
$ printf '%s' "$f" | cut -c 1-4
abcd
Using a while + read loop, Process Substitution and find plus mv to rename the files.
The script.
#!/usr/bin/env bash
shopt -s extglob nullglob
while IFS= read -rd '' directory; do
if [[ -e $directory && -x $directory ]] ; then
(
printf 'Entering directory %s\n' "$directory"
cd "$directory" || exit
files=(*.zip)
(( ${#files[*]} )) || {
printf 'There are no files ending in *.zip here!, moving on...\n'
continue
}
for file_name_with_extension in *.zip; do
extension=${file_name_with_extension##*.}
file_name_without_extension=${file_name_with_extension%."$extension"}
change_spaces_to_underscore="${file_name_without_extension//+([[:space:]])/_}"
remove_everything_that_is_not_alnum_and_under_score="${change_spaces_to_underscore//[![:alnum:]_]}"
change_every_underscore_with_a_single_under_score="${remove_everything_that_is_not_alnum_and_under_score//+(_)/_}"
new_file_name="$change_every_underscore_with_a_single_under_score.$extension"
mv -v "$file_name_with_extension" "${new_file_name::80}"
done
)
fi
done < <(find . ! -name . -type d -print0)
The script for creating dummy directories and files.
#!/usr/bin/env bash
mkdir -p foo/bar/baz/more/qux/sux
cd foo/ && touch 'Mean 18 - Golf Menu [SW] (1988)(Robert J. Butler) [Sports, Golf, Utility].zip'
cd bar/ && touch 'Mean 18 - M18 (1988)(Ken Hopkins) [Sports, Golf, Utility].zip'
cd baz/ && touch 'Metaltech- Battledrome Game Editor (1994)(Sierra On-Line, Inc.) [Utility].mp4'
cd more/ && touch 'Might and Magic III Character Editor (1991)(Blackbeard'\''s Ghost) [Utility].zip'
cd qux/ && touch 'Might Magic 3 Character viewer-editor v1.1 (1991)(Mark Betz and Chris Lampton) [Editor].zip'
cd sux/ && touch 'Might Magic 3 Character viewer-editor v1.1 (1991)(Mark Betz and Chris Lampton) [Editor].jpg'
Checking the directory tree with tree
tree foo/
foo/
├── bar
│ ├── baz
│ │ ├── Metaltech- Battledrome Game Editor (1994)(Sierra On-Line, Inc.) [Utility].mp4
│ │ └── more
│ │ ├── Might and Magic III Character Editor (1991)(Blackbeard's Ghost) [Utility].zip
│ │ └── qux
│ │ ├── Might Magic 3 Character viewer-editor v1.1 (1991)(Mark Betz and Chris Lampton) [Editor].zip
│ │ └── sux
│ │ └── Might Magic 3 Character viewer-editor v1.1 (1991)(Mark Betz and Chris Lampton) [Editor].jpg
│ └── Mean 18 - M18 (1988)(Ken Hopkins) [Sports, Golf, Utility].zip
└── Mean 18 - Golf Menu [SW] (1988)(Robert J. Butler) [Sports, Golf, Utility].zip
5 directories, 6 files
Using find to print the files.
find foo/ ! -name . -type f
The output is
foo/Mean 18 - Golf Menu [SW] (1988)(Robert J. Butler) [Sports, Golf, Utility].zip
foo/bar/Mean 18 - M18 (1988)(Ken Hopkins) [Sports, Golf, Utility].zip
foo/bar/baz/more/Might and Magic III Character Editor (1991)(Blackbeard's Ghost) [Utility].zip
foo/bar/baz/more/qux/sux/Might Magic 3 Character viewer-editor v1.1 (1991)(Mark Betz and Chris Lampton) [Editor].jpg
foo/bar/baz/more/qux/Might Magic 3 Character viewer-editor v1.1 (1991)(Mark Betz and Chris Lampton) [Editor].zip
foo/bar/baz/Metaltech- Battledrome Game Editor (1994)(Sierra On-Line, Inc.) [Utility].mp4
Running the script inside the top level directory print something like:
Entering directory ./foo
mv -v Mean 18 - Golf Menu [SW] (1988)(Robert J. Butler) [Sports, Golf, Utility].zip Mean_18_Golf_Menu_SW_1988Robert_J_Butler_Sports_Golf_Utility.zip
Entering directory ./foo/bar
mv -v Mean 18 - M18 (1988)(Ken Hopkins) [Sports, Golf, Utility].zip Mean_18_M18_1988Ken_Hopkins_Sports_Golf_Utility.zip
Entering directory ./foo/bar/baz
There are no files ending in *.zip here!, moving on...
Entering directory ./foo/bar/baz/more
mv -v Might and Magic III Character Editor (1991)(Blackbeard's Ghost) [Utility].zip Might_and_Magic_III_Character_Editor_1991Blackbeards_Ghost_Utility.zip
Entering directory ./foo/bar/baz/more/qux
mv -v Might Magic 3 Character viewer-editor v1.1 (1991)(Mark Betz and Chris Lampton) [Editor].zip Might_Magic_3_Character_viewereditor_v11_1991Mark_Betz_and_Chris_Lampton_Editor.
Entering directory ./foo/bar/baz/more/qux/sux
There are no files ending in *.zip here!, moving on...
Remove the echo if you're satisfied with the output in order for mv to rename the files.
Without the echo the output is something like:
Entering directory ./foo
renamed 'Mean 18 - Golf Menu [SW] (1988)(Robert J. Butler) [Sports, Golf, Utility].zip' -> 'Mean_18_Golf_Menu_SW_1988Robert_J_Butler_Sports_Golf_Utility.zip'
Entering directory ./foo/bar
renamed 'Mean 18 - M18 (1988)(Ken Hopkins) [Sports, Golf, Utility].zip' -> 'Mean_18_M18_1988Ken_Hopkins_Sports_Golf_Utility.zip'
Entering directory ./foo/bar/baz
There are no files ending in *.zip here!, moving on...
Entering directory ./foo/bar/baz/more
renamed 'Might and Magic III Character Editor (1991)(Blackbeard'\''s Ghost) [Utility].zip' -> 'Might_and_Magic_III_Character_Editor_1991Blackbeards_Ghost_Utility.zip'
Entering directory ./foo/bar/baz/more/qux
renamed 'Might Magic 3 Character viewer-editor v1.1 (1991)(Mark Betz and Chris Lampton) [Editor].zip' -> 'Might_Magic_3_Character_viewereditor_v11_1991Mark_Betz_and_Chris_Lampton_Editor.'
Entering directory ./foo/bar/baz/more/qux/sux
There are no files ending in *.zip here!, moving on...
This would be much better if we could convert sequences of unwanted character to one single underscore. Such as, instead of: XArchRogueTool(1984)(Unknown)[Utility].zip Could the output be:
X_Arch_Rogue_Tool_(1984)_(Unknown)_[Utility].zip?
Change the value of remove_everything_that_is_not_alnum_and_under_score
from:
remove_everything_that_is_not_alnum_and_under_score="${change_spaces_to_underscore//[![:alnum:]_]}"
to
remove_everything_that_is_not_alnum_and_under_score="${change_spaces_to_underscore//[![:alnum:]_()\[\]]}"
To exclude parenthesis ( ) and brackets [ ]
Add the code below the line where change_every_underscore_with_a_single_under_score is at.
insert_underscore_in_between_parens="${change_every_underscore_with_a_single_under_score//')('/')_('}"
Change the value of new_file_name= to "$insert_underscore_in_between_parens.$extension"
new_file_name="$insert_underscore_in_between_parens.$extension"
Pointing the directory to the script requires a bit of modification.
Add the code below after the shebang
directory_to_process="$1"
if [[ ! -e "$directory_to_process" ]]; then
printf >&2 '%s no such file or directory!\n' "$directory_to_process"
exit 1
elif [[ ! -d "$directory_to_process" ]]; then
printf >&2 '%s does not appear to be a directory!\n' "$directory_to_process"
exit 1
fi
Then change the . from find
find "$directory_to_process" ! -name . -type d -print0
The new script.
#!/usr/bin/env bash
directory_to_process="$1"
if [[ ! -e "$directory_to_process" ]]; then
printf >&2 '[%s] no such file or directory!\n' "$directory_to_process"
exit 1
elif [[ ! -d "$directory_to_process" ]]; then
printf >&2 '[%s] does not appear to be a directory!\n' "$directory_to_process"
exit 1
fi
shopt -s extglob nullglob
while IFS= read -rd '' directory; do
if [[ -e $directory && -x $directory ]] ; then
(
printf 'Entering directory %s\n' "$directory"
cd "$directory" || exit
files=(*.zip)
(( ${#files[*]} )) || {
printf 'There are no files ending in *.zip here!, moving on...\n'
continue
}
for file_name_with_extension in *.zip; do
extension=${file_name_with_extension##*.}
file_name_without_extension=${file_name_with_extension%."$extension"}
change_spaces_to_underscore="${file_name_without_extension//+([[:space:]])/_}"
remove_everything_that_is_not_alnum_and_under_score="${change_spaces_to_underscore//[![:alnum:]_()\[\]]}"
change_every_underscore_with_a_single_under_score="${remove_everything_that_is_not_alnum_and_under_score//+(_)/_}"
insert_underscore_in_between_parens="${change_every_underscore_with_a_single_under_score//')('/')_('}"
new_file_name="$insert_underscore_in_between_parens.$extension"
echo mv -v "$file_name_with_extension" "${new_file_name:0:80}"
done
)
fi
done < <(find "$directory_to_process" ! -name . -type d -print0)
Now you give the directory as an argument to the script. e.g.
./script.sh foo/
Or an absolute path.
./script.sh /path/to/foo
If you add the script to your PATH and make it executable then you can.
script.sh /path/to/foo
Assuming your script name is script.sh and the directory you want to process is named foo
Change the value of 80 to a lower value if needed.
See help continue and help test
See Parameter Expansion
The -print0 from find(1) is a GNU and *BSD feature.
See How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?
See How can I check whether a directory is empty or not? How do I check for any *.mpg files, or count how many there are?
If your mv(1) supports the -n flag that would be nice to use.
Related
The issue: I have bunch of files split across multiple directories, which all have the same name (input.txt).
What I am after: I want to firstly copy all of these to a new directory, while adding the containing directory as a suffux to avoid confusion between them and prevent overwriting. This is the basis of what I am trying to do:
cp -nr /foo/bar/*/input.txt /new/path/
Where do I go from here?
To respond to the comments below, if my file structure in /old/directory contains folders:
/old/directory/1/input.txt
/old/directory/2/input.txt
/old/directory/3/input.txt
This is an example of my desired output:
/new/directory/ should contain:
1input.txt
2input.txt
3input.txt
Thanks
This will do the trick and also handle any directories that might have spaces in their names (or any other odd characters).
#!/bin/bash
tld=./old/directory
newpath=./new/directory
while IFS= read -r -d $'\0' file; do
tmp="${file#*${tld}/}"
echo cp "$file" "$newpath/${tmp//\//}"
done < <(find "$tld" -type f -name "input.txt" -print0)
Proof of Concept
$ tree ./old/directory/
./old/directory/
├── 1
│ └── input.txt
├── 2
│ └── input.txt
└── 3
├── 3a
│ └── input.txt
└── input.txt
4 directories, 4 files
$ ./mvinput.sh
cp ./old/directory/3/input.txt ./new/directory/3input.txt
cp ./old/directory/3/3a/input.txt ./new/directory/33ainput.txt
cp ./old/directory/1/input.txt ./new/directory/1input.txt
cp ./old/directory/2/input.txt ./new/directory/2input.txt
Well, the tough news is that there's not an obvious manner of doing this in one line - not one that isn't nonsensically difficult to understand anyway. There may be a way to do it with rsync, and I'm sure that someone smarter than I could do it in awk, but in my opinion, you're better off making a script, or even writing a custom binary that does this for you.
find . -name input.txt | while read line
do
cp "$line" /new/path/`echo $line | cut -d '/' -f3- | sed 's/\//_/'`
done
Note that you'll probably have to change the -f3- part of the cut command in order to select which directory name you want to start your suffix as.
One approach is to use an array to save the files, also since / is not allowed on file names an alternative is to change it to something else, like say an underscore.
#!/usr/bin/env bash
##: Just in case there are no files/directories the * glob will not expand by itself.
shopt -s nullglob
files=(foo/bar/*/input.txt)
for file in "${files[#]}"; do
new_file=${file//\//_} ##: Replace all /'s with an _ by Parameter Expansion
echo cp -v "$file" new/path/"$new_file"
done
As per OP's request here is the new answer.
#!/usr/bin/env bash
shopt -s nullglob
##: Although files=(/old/directory/{1..3}/input.txt)
##: can be a replacement, no need for nullglob
files=(/old/directory/*/input.txt)
for file in "${files[#]}"; do
tmp=${file%[0-9]*}
new_file=${file#*$tmp}
echo cp -v "$file" new/path/"${new_file//\//}"
done
Another option is to split the fields using / as the delimiter.
#!/usr/bin/env bash
##: Do not expand a literal glob * if there are no files/directories
shopt -s nullglob
##: Save the whole path and files in an array.
files=(/old/directory/*/input.txt)
for file in "${files[#]}"; do
IFS='/' read -ra path <<< "$file" ##: split via / and save in an array
tmp=${path[#]:(-2)} ##: Remain only the last 2, e.g. 1 input.txt
new_file=${tmp// } ##: Remove the space, becomes 1input.txt
echo cp -v "$file" new/path/"$new_file"
done
Remove the echo if you think that the output is correct.
It is easy to understand which directory the file came from just replace the under scores with a /
There are multiple directories which contain a file with the same name:
direct_afaap/file.txt
direct_fgrdw/file.txt
direct_sardf/file.txt
...
Now I want to extract them to another directory, direct_new and with a different file name such as:
[mylinux~ ]$ ls direct_new/
file_1.txt file_2.txt file_3.txt
How can I do this?
BTW, if I want to put part of the name in original directory into the file name such as:
[mylinux~ ]$ ls direct_new/
file_afaap.txt file_fgrdw.txt file_sardf.txt
What can I do?
This little BaSH script will do it both ways:
#!/bin/sh
#
# counter
i=0
# put your new directory here
# can't be similar to dir_*, otherwise bash will
# expand it too
mkdir newdir
for file in `ls dir_*/*`; do
# gets only the name of the file, without directory
fname=`basename $file`
# gets just the file name, without extension
name=${fname%.*}
# gets just the extention
ext=${fname#*.}
# get the directory name
dir=`dirname $file`
# get the directory suffix
suffix=${dir#*_}
# rename the file using counter
fname_counter="${name}_$((i=$i+1)).$ext"
# rename the file using dir suffic
fname_suffix="${name}_$suffix.$ext"
# copy files using both methods, you pick yours
cp $file "newdir/$fname_counter"
cp $file "newdir/$fname_suffix"
done
And the output:
$ ls -R
cp.sh*
dir_asdf/
dir_ljklj/
dir_qwvas/
newdir/
out
./dir_asdf:
file.txt
./dir_ljklj:
file.txt
./dir_qwvas:
file.txt
./newdir:
file_1.txt
file_2.txt
file_3.txt
file_asdf.txt
file_ljklj.txt
file_qwvas.txt
while read -r line; do
suffix=$(sed 's/^.*_\(.*\)\/.*$/\1/' <<<$line)
newfile=$(sed 's/\.txt/$suffix\.txt/' <<<$line)
cp "$line" "~/direct_new/$newfile"
done <file_list.txt
where file_list is a list of your files.
You can achieve this with Bash parameter expansion:
dest_dir=direct_new
# dir based naming
for file in direct_*/file.txt; do
[[ -f "$file" ]] || continue # skip if not a regular file
dir="${file%/*}" # get the dir name from path
cp "$file" "$dest_dir/file_${dir#*direct_}.txt"
done
# count based naming
counter=0
for file in direct_*/file.txt; do
[[ -f "$file" ]] || continue # skip if not a regular file
cp "$file" "$dest_dir/file_$((++counter)).txt"
done
dir="${file%/*}" removes all characters starting from /, basically, giving us the dirname
${dir#*direct_} removes the direct_ prefix from dirname
((++counter)) uses Bash arithmetic expression to pre-increment the counter
See also:
Why you shouldn't parse the output of ls(1)
Get file directory path from file path
How to use double or single brackets, parentheses, curly braces
It may not be quite what you want, but it will do the job. Use cp --backup=numbered <source_file> <destination_directory:
$ find . -name test.sh
./ansible/test/integration/roles/test_command_shell/files/test.sh
./ansible/test/integration/roles/test_script/files/test.sh
./Documents/CGI/Code/ec-scripts/work/bin/test.sh
./Documents/CGI/Code/ec-scripts/trunk/bin/test.sh
./Test/test.sh
./bin/test.sh
./test.sh
$ mkdir BACKUPS
$ find . -name test.sh -exec cp --backup=numbered {} BACKUPS \;
cp: './BACKUPS/test.sh' and 'BACKUPS/test.sh' are the same file
$ ls -l BACKUPS
total 28
-rwxrwxr-x. 1 jack jack 121 Jun 9 10:29 test.sh
-rwxrwxr-x. 1 jack jack 34 Jun 9 10:29 test.sh.~1~
-rwxrwxr-x. 1 jack jack 34 Jun 9 10:29 test.sh.~2~
-rwxrwxr-x. 1 jack jack 388 Jun 9 10:29 test.sh.~3~
-rwxrwxr-x. 1 jack jack 388 Jun 9 10:29 test.sh.~4~
-rwxrwxr-x. 1 jack jack 20 Jun 9 10:29 test.sh.~5~
-rwxrwxr-x. 1 jack jack 157 Jun 9 10:29 test.sh.~6~
If you really want to put part of the folder name in, you have to decide exactly what part you want. You could, of course, just replace the directory separator character with some other character, and put the whole path into the filename.
I have duplicated files in directory on Linux machine which are listed like that:
ltulikowski#lukasz-pc:~$ ls -1
abcd
abcd.1
abcd.2
abdc
abdc.1
acbd
I want to remove all files witch aren't single so as a result I should have:
ltulikowski#lukasz-pc:~$ ls -1
acbd
The function uses extglob, so before execution, set extglob: shopt -s extglob
rm_if_dup_exist(){
arr=()
for file in *.+([0-9]);do
base=${file%.*};
if [[ -e $base ]]; then
arr+=("$base" "$file")
fi
done
rm -f -- "${arr[#]}"
}
This will also support file names with several digits after the . e.g. abcd.250 is also acceptable.
Usage example with your input:
$ touch abcd abcd.1 abcd.2 abdc abdc.1 acbd
$ rm_if_dup_exist
$ ls
acbd
Please notice that if, for example, abcd.1 exist but abcd does not exist, it won't delete abcd.1.
here is one way to do it
for f in *.[0-9]; do rm ${f%.*}*; done
may get exceptions since some files will be deleted more than once (abcd in your example). If versions always start with .1 you can restrict to match to that.
You can use:
while read -r f; do
rm "$f"*
done < <(printf "%s\n" * | cut -d. -f1 | uniq -d)
printf, cut and uniq are used to get duplicate entries (part before dot) in current directory.
The command
rm *.*
Should do the trick if I understand you correctly
Use ls to confirm first
I have n files, named f1, f2, ..., fn. For each of these files, I have to execute a sed command, and name the new files as file1, file2, ..., filen.
I need the new files to keep the same number as their original ones. Can anyone help?
Here's what I've tried so far:
#!/bin/sh
for element in *
do
echo "$element" sed -n '/Col3/p' $element > Quest $element
done
If we assume that all the your files are in the form in your question...
$ ls -l
total 2
-rw-r--r-- 1 ghoti wheel 0 Jan 3 13:20 f1
-rw-r--r-- 1 ghoti wheel 0 Jan 3 13:20 f2
-rw-r--r-- 1 ghoti wheel 0 Jan 3 13:20 f3
-rw-r--r-- 1 ghoti wheel 0 Jan 3 13:20 f4
then you're on the right track with a for loop. But you probably want to narrow your search to only the files that are important to you.
In bash, you can use extglob to control this sort of thing. For example:
#!/usr/bin/env bash
shopt -s extglob
for file in +([a-z])+([0-9]); do
echo "Old: $file / New: file${file##[a-z]}"
done
This matches any files whose names consist of letters followed by numbers.
If, on the other hand, you want to make this portable, so that it will work in a POSIX shell (since in your question you've specified /bin/sh), you might put the detection into the loop itself:
#!/bin/sh
for file in *; do
if ! expr "$file" : '[a-z][a-z]*[0-9][0-9]*$' >/dev/null; then
continue
fi
echo "Old: $file / New: file${file##[a-z]}"
done
In both of these examples, we use POSIX "Parameter Expansion" to strip off the letters at the beginning of the filename.
#!/bin/env bash
for FILE in *
do
[[ "$FILE" =~ [0-9]+$ ]] && mv "$FILE" file${BASH_REMATCH[0]}
done
The expression within [[ ]] is a test, which tests for a match against a regular expression, which looks for a string ending in a number. If the match is successful, the matched number can be found in the bash array variable BASH_REMATCH at index 0. The part after && is executed if the test succeeds, and renames the file to fileNN,
I'm writing a simple Bash script that needs to find out how many files are in a directory. Basically, if the directory contains no files, the script will exit. If it has any, it will do other things.
How can I find out if there are files in a directory using Bash?
Thanks!
List almost (no . or ..) all files and directories and count the lines: ls -A1 /mydir | wc -l
If you'd like only files I think you can use find instead: find /mydir -maxdepth 1 -type f | wc -l
shopt -s nullglob
cd /path/to/dir/
arr=( * )
echo "${#arr[#]}"
for i in "${!arr[#]}"; do echo "${arr[i]}"; done
The other answers have shown you ways to get the number of files. Here is how to use the number in your script.
(This is presented in the context of a function, with $1 being the directory name specified when you run it. To use it on the current directory, just omit that, or you can hardwire a directory name in that location.)
checkdir(){
numfiles=$(ls -A "$1" | wc -l)
if [ $numfiles -gt 0 ]
then
echo YES;
else
echo NO;
fi
}
Note: this will count directories as files. If you would like only files, then replace the ls...wc portion with this line:
ls -AF "$1" | grep -v "/$" | wc -l
I needed the same thing yesterday, and found this (it's near the bottom of the page):
# From http://www.etalabs.net/sh_tricks.html
is_empty () (
cd "$1"
set -- .[!.]* ; test -f "$1" && return 1
set -- ..?* ; test -f "$1" && return 1
set -- * ; test -f "$1" && return 1
return 0 )
It's POSIX-compatible, and uses three patterns to match any file name other than . and .. which are guaranteed to exist in an otherwise empty directory.
The first two lines match all files starting with a single . (of length at least 2) or one or more . (of length at least 3), which covers all hidden files that aren't . or ... The third pattern matches all non-hidden files.
Notice that the body of the function is a (...) expression, not {...}. This forces a subshell, in which it is safe to change the working directory to simplify the patterns.