I have files with the names:
Ff6_01.png
Ff6_02.png
Ff6_03.png
...
...
FF1_01.png
FF1_02.png
FF1_03.png
I want to remove the first two letters of every file name, because then I would have a correct order of the files. Does anyone know the command in the linux shell?
You can use the syntax ${file:2} to refer to the name starting from the 3rd char.
Hence, you may do:
for file in F*png
do
mv "$file" "${file:2}"
done
In case ${file:2} did not work to you (neither rename), you can also use sed or cut:
for file in F*png
do
new_file=$(sed 's/^..//' <<< "$file") <---- cuts first two chars
new_file=$(cut -c3- <<< "$file") <---- the same
mv "$file" "$new_file"
done
Test
$ file="Ff6_01.png"
$ touch $file
$ ls
Ff6_01.png
$ mv "$file" "${file:2}"
$ ls
6_01.png
Related
I am new to bash loops and trying to rename all files in a directory to their appropriate md5 values.
There are 5 sample files in the directory.
For testing purpose, I am trying to first just print md5 hashes of all files in the directory using below command and it is working fine.
for i in `ls`; do md5sum $i; done
Output:
edc47be8af3a7d4d55402ebae9f04f0a file1
72cf1321d5f3d2e9e1be8abd971f42f5 file2
4b7b590d6d522f6da7e3a9d12d622a07 file3
357af1e7f8141581361ac5d39efa4d89 file4
1445c4c1fb27abd9061ada3b30a18b44 file5
Now I am trying to rename each file with its appropriate md5 hashes by following command:
for i in `ls`; do mv $i md5sum $i; done
Failed Output:
mv: target 'file1' is not a directory
mv: target 'file2' is not a directory
mv: target 'file3' is not a directory
mv: target 'file4' is not a directory
mv: target 'file5' is not a directory
What am I missing here?
Your command is expanded to
mv file1 edc47be8af3a7d4d55402ebae9f04f0a file1
When mv has more than two non-option arguments, it understands the last argument to be the target directory to which all the preceding files should be moved. But there's no directory file1.
You can use parameter expansion to remove the filename from the string.
Parameter expansion is usually faster then running an external command like cut or sed, but if you aren't renaming thousands of files, it probably doesn't matter.
for f in *; do
m=$(md5sum "$f")
mv "$f" ${m%% *} # Remove everything after the first space
done
Also note that I don't parse the output of ls, but let the shell expand the glob. It's safer and works (with proper quoting) for filenames containing whitespace.
Syntax. Yes I was giving wrong syntax.
With some trial and errors with the command, I finally came up with the correct syntax.
I noticed that md5sum $i was giving me 2 column-ed output.
edc47be8af3a7d4d55402ebae9f04f0a file1
72cf1321d5f3d2e9e1be8abd971f42f5 file2
4b7b590d6d522f6da7e3a9d12d622a07 file3
357af1e7f8141581361ac5d39efa4d89 file4
1445c4c1fb27abd9061ada3b30a18b44 file5
By firing second command for i in ls; do mv $i md5sum $i; done, I was basically telling terminal to do something like :
mv $i md5sum $i
which, upto my knowledge, turns out to be
mv file1 <md5 value> file1 <-- this was the issue.
How I resolved the issue?
I used cut command to filter out required value and made new one-liner as below:
for i in `ls`; do mv $i "$(md5sum $i | cut -d " " -f 1)"; done
[Edit]
According to another answer and comment by #stark, #choroba and #tripleee, it is better to use * instead of ls.
for i in *; do mv $i "$(md5sum $i | cut -d " " -f 1)"; done
#choroba's answer is also a good addition here. Turning it into one-liner requirement, below is his solution:
for i in *; do m=$(md5sum $i); mv "$i" ${m%% *};done
When creating playlists I often came across files that would break the playing process. Such would be files with spaces or apostrophes. I would fix it with the following command
for file in *.; do mv "$file" `echo $file | tr " " '_'` ; done **(for spaces)**
Now I more often come across files with commas, apostrophes, brackets and other characters. How would I modify the command to remove such characters?
Also tried rename 's/[^a-zA-Z0-9_-]//' *.mp4 but it doesnt seem to remove spaces or commas
for file in *; do mv "$file" $(echo "$file" | sed -e 's/[^A-Za-z0-9._-]/_/g'); done &
Your rename would work if you add the g modifier to it, this performs all substitutions instead of only the first one:
$ echo "$file"
foo bar,spam.egg
$ rename -n 's/[^a-zA-Z0-9_-]//' "$file"
foo bar,spam.egg renamed as foobar,spam.egg
$ rename -n 's/[^a-zA-Z0-9_-]//g' "$file"
foo bar,spam.egg renamed as foobarspamegg
You can do this will bash alone, with parameter expansion:
For removing everything except a-zA-Z0-9_- from file names, assuming variable file contains the filename, using character class [:alnum:] to match all alphabetic characters and digits from current locale:
"${file//[^[:alnum:]_-]/}"
or explicitly, change the LC_COLLATE to C:
"${file//[^a-zA-Z0-9_-]/}"
Example:
$ file='foo bar,spam.egg'
$ echo "${file//[^[:alnum:]_-]/}"
foobarspamegg
I have duplicated files in directory on Linux machine which are listed like that:
ltulikowski#lukasz-pc:~$ ls -1
abcd
abcd.1
abcd.2
abdc
abdc.1
acbd
I want to remove all files witch aren't single so as a result I should have:
ltulikowski#lukasz-pc:~$ ls -1
acbd
The function uses extglob, so before execution, set extglob: shopt -s extglob
rm_if_dup_exist(){
arr=()
for file in *.+([0-9]);do
base=${file%.*};
if [[ -e $base ]]; then
arr+=("$base" "$file")
fi
done
rm -f -- "${arr[#]}"
}
This will also support file names with several digits after the . e.g. abcd.250 is also acceptable.
Usage example with your input:
$ touch abcd abcd.1 abcd.2 abdc abdc.1 acbd
$ rm_if_dup_exist
$ ls
acbd
Please notice that if, for example, abcd.1 exist but abcd does not exist, it won't delete abcd.1.
here is one way to do it
for f in *.[0-9]; do rm ${f%.*}*; done
may get exceptions since some files will be deleted more than once (abcd in your example). If versions always start with .1 you can restrict to match to that.
You can use:
while read -r f; do
rm "$f"*
done < <(printf "%s\n" * | cut -d. -f1 | uniq -d)
printf, cut and uniq are used to get duplicate entries (part before dot) in current directory.
The command
rm *.*
Should do the trick if I understand you correctly
Use ls to confirm first
I have a long text file with list of file masks I want to delete
Example:
/tmp/aaa.jpg
/var/www1/*
/var/www/qwerty.php
I need delete them. Tried rm `cat 1.txt` and it says the list is too long.
Found this command, but when I check folders from the list, some of them still have files
xargs rm <1.txt Manual rm call removes files from such folders, so no issue with permissions.
This is not very efficient, but will work if you need glob patterns (as in /var/www/*)
for f in $(cat 1.txt) ; do
rm "$f"
done
If you don't have any patterns and are sure your paths in the file do not contain whitespaces or other weird things, you can use xargs like so:
xargs rm < 1.txt
Assuming that the list of files is in the file 1.txt, then do:
xargs rm -r <1.txt
The -r option causes recursion into any directories named in 1.txt.
If any files are read-only, use the -f option to force the deletion:
xargs rm -rf <1.txt
Be cautious with input to any tool that does programmatic deletions. Make certain that the files named in the input file are really to be deleted. Be especially careful about seemingly simple typos. For example, if you enter a space between a file and its suffix, it will appear to be two separate file names:
file .txt
is actually two separate files: file and .txt.
This may not seem so dangerous, but if the typo is something like this:
myoldfiles *
Then instead of deleting all files that begin with myoldfiles, you'll end up deleting myoldfiles and all non-dot-files and directories in the current directory. Probably not what you wanted.
Use this:
while IFS= read -r file ; do rm -- "$file" ; done < delete.list
If you need glob expansion you can omit quoting $file:
IFS=""
while read -r file ; do rm -- $file ; done < delete.list
But be warned that file names can contain "problematic" content and I would use the unquoted version. Imagine this pattern in the file
*
*/*
*/*/*
This would delete quite a lot from the current directory! I would encourage you to prepare the delete list in a way that glob patterns aren't required anymore, and then use quoting like in my first example.
You could use '\n' for define the new line character as delimiter.
xargs -d '\n' rm < 1.txt
Be careful with the -rf because it can delete what you don't want to if the 1.txt contains paths with spaces. That's why the new line delimiter a bit safer.
On BSD systems, you could use -0 option to use new line characters as delimiter like this:
xargs -0 rm < 1.txt
xargs -I{} sh -c 'rm "{}"' < 1.txt should do what you want. Be careful with this command as one incorrect entry in that file could cause a lot of trouble.
This answer was edited after #tdavies pointed out that the original did not do shell expansion.
You can use this one-liner:
cat 1.txt | xargs echo rm | sh
Which does shell expansion but executes rm the minimum number of times.
Just to provide an another way, you can also simply use the following command
$ cat to_remove
/tmp/file1
/tmp/file2
/tmp/file3
$ rm $( cat to_remove )
In this particular case, due to the dangers cited in other answers, I would
Edit in e.g. Vim and :%s/\s/\\\0/g, escaping all space characters with a backslash.
Then :%s/^/rm -rf /, prepending the command. With -r you don't have to worry to have directories listed after the files contained therein, and with -f it won't complain due to missing files or duplicate entries.
Run all the commands: $ source 1.txt
cat 1.txt | xargs rm -f | bash Run the command will do the following for files only.
cat 1.txt | xargs rm -rf | bash Run the command will do the following recursive behaviour.
Here's another looping example. This one also contains an 'if-statement' as an example of checking to see if the entry is a 'file' (or a 'directory' for example):
for f in $(cat 1.txt); do if [ -f $f ]; then rm $f; fi; done
Here you can use set of folders from deletelist.txt while avoiding some patterns as well
foreach f (cat deletelist.txt)
rm -rf ls | egrep -v "needthisfile|*.cpp|*.h"
end
This will allow file names to have spaces (reproducible example).
# Select files of interest, here, only text files for ex.
find -type f -exec file {} \; > findresult.txt
grep ": ASCII text$" findresult.txt > textfiles.txt
# leave only the path to the file removing suffix and prefix
sed -i -e 's/:.*$//' textfiles.txt
sed -i -e 's/\.\///' textfiles.txt
#write a script that deletes the files in textfiles.txt
IFS_backup=$IFS
IFS=$(echo "\n\b")
for f in $(cat textfiles.txt);
do
rm "$f";
done
IFS=$IFS_backup
# save script as "some.sh" and run: sh some.sh
In case somebody prefers sed and removing without wildcard expansion:
sed -e "s/^\(.*\)$/rm -f -- \'\1\'/" deletelist.txt | /bin/sh
Reminder: use absolute pathnames in the file or make sure you are in the right directory.
And for completeness the same with awk:
awk '{printf "rm -f -- '\''%s'\''\n",$1}' deletelist.txt | /bin/sh
Wildcard expansion will work if the single quotes are remove, but this is dangerous in case the filename contains spaces. This would need to add quotes around the wildcards.
I have a folder with about 1,700 files. They are all named like 1.txt or 1497.txt, etc. I would like to rename all the files so that all the filenames are four digits long.
I.e., 23.txt becomes 0023.txt.
What is a shell script that will do this? Or a related question: How do I use grep to only match lines that contain \d.txt (i.e., one digit, then a period, then the letters txt)?
Here's what I have so far:
for a in [command i need help with]
do
mv $a 000$a
done
Basically, run that three times, with commands there to find one digit, two digits, and three digit filenames (with the number of initial zeros changed).
Try:
for a in [0-9]*.txt; do
mv $a `printf %04d.%s ${a%.*} ${a##*.}`
done
Change the filename pattern ([0-9]*.txt) as necessary.
A general-purpose enumerated rename that makes no assumptions about the initial set of filenames:
X=1;
for i in *.txt; do
mv $i $(printf %04d.%s ${X%.*} ${i##*.})
let X="$X+1"
done
On the same topic:
Bash script to pad file names
Extract filename and extension in bash
Using the rename (prename in some cases) script that is sometimes installed with Perl, you can use Perl expressions to do the renaming. The script skips renaming if there's a name collision.
The command below renames only files that have four or fewer digits followed by a ".txt" extension. It does not rename files that do not strictly conform to that pattern. It does not truncate names that consist of more than four digits.
rename 'unless (/0+[0-9]{4}.txt/) {s/^([0-9]{1,3}\.txt)$/000$1/g;s/0*([0-9]{4}\..*)/$1/}' *
A few examples:
Original Becomes
1.txt 0001.txt
02.txt 0002.txt
123.txt 0123.txt
00000.txt 00000.txt
1.23.txt 1.23.txt
Other answers given so far will attempt to rename files that don't conform to the pattern, produce errors for filenames that contain non-digit characters, perform renames that produce name collisions, try and fail to rename files that have spaces in their names and possibly other problems.
for a in *.txt; do
b=$(printf %04d.txt ${a%.txt})
if [ $a != $b ]; then
mv $a $b
fi
done
One-liner:
ls | awk '/^([0-9]+)\.txt$/ { printf("%s %04d.txt\n", $0, $1) }' | xargs -n2 mv
How do I use grep to only match lines that contain \d.txt (IE 1 digit, then a period, then the letters txt)?
grep -E '^[0-9]\.txt$'
Let's assume you have files with datatype .dat in your folder. Just copy this code to a file named run.sh, make it executable by running chmode +x run.sh and then execute using ./run.sh:
#!/bin/bash
num=0
for i in *.dat
do
a=`printf "%05d" $num`
mv "$i" "filename_$a.dat"
let "num = $(($num + 1))"
done
This will convert all files in your folder to filename_00000.dat, filename_00001.dat, etc.
This version also supports handling strings before(after) the number. But basically you can do any regex matching+printf as long as your awk supports it. And it supports whitespace characters (except newlines) in filenames too.
for f in *.txt ;do
mv "$f" "$(
awk -v f="$f" '{
if ( match(f, /^([a-zA-Z_-]*)([0-9]+)(\..+)/, a)) {
printf("%s%04d%s", a[1], a[2], a[3])
} else {
print(f)
}
}' <<<''
)"
done
To only match single digit text files, you can do...
$ ls | grep '[0-9]\.txt'
One-liner hint:
while [ -f ./result/result`printf "%03d" $a`.txt ]; do a=$((a+1));done
RESULT=result/result`printf "%03d" $a`.txt
To provide a solution that's cautiously written to be correct even in the presence of filenames with spaces:
#!/usr/bin/env bash
pattern='%04d' # pad with four digits: change this to taste
# enable extglob syntax: +([[:digit:]]) means "one or more digits"
# enable the nullglob flag: If no matches exist, a glob returns nothing (not itself).
shopt -s extglob nullglob
for f in [[:digit:]]*; do # iterate over filenames that start with digits
suffix=${f##+([[:digit:]])} # find the suffix (everything after the last digit)
number=${f%"$suffix"} # find the number (everything before the suffix)
printf -v new "$pattern" "$number" "$suffix" # pad the number, then append the suffix
if [[ $f != "$new" ]]; then # if the result differs from the old name
mv -- "$f" "$new" # ...then rename the file.
fi
done
There is a rename.ul command installed from util-linux package (at least in Ubuntu) by default installed.
It's use is (do a man rename.ul):
rename [options] expression replacement file...
The command will replace the first occurrence of expression with the given replacement for the provided files.
While forming the command you can use:
rename.ul -nv replace-me with-this in-all?-these-files*
for not doing any changes but reading what changes that command would make. When sure just reexecute the command without the -v (verbose) and -n (no-act) options
for your case the commands are:
rename.ul "" 000 ?.txt
rename.ul "" 00 ??.txt
rename.ul "" 0 ???.txt