I have file names (from image tiles) consisting of two numbers separated by an underscore, e.g.
142_27.jpg
7_39.jpg
1_120.jpg
How can I (in linux) add leading zeros to both of these numbers? What I want is the file names as
142_027.jpg
007_039.jpg
001_120.jpg
You can use a single awk command to format filenames with leading zeroes using a printf:
for f in *.jpg; do
echo mv "$f" $(awk -F '[_.]' '{printf "%03d_%03d.%s", $1, $2, $3}' <<< "$f")
done
This will output:
mv 142_27.jpg 142_027.jpg
mv 1_120.jpg 001_120.jpg
mv 7_39.jpg 007_039.jpg
Once you're satisfied with the output, remove echo before mv command.
With perl based rename command
$ touch 142_27.jpg 7_39.jpg 1_120.jpg
$ rename -n 's/\d+/sprintf "%03d", $&/ge' *.jpg
rename(1_120.jpg, 001_120.jpg)
rename(142_27.jpg, 142_027.jpg)
rename(7_39.jpg, 007_039.jpg)
The -n option is for dry run, remove it for actual renaming
If perl based rename command is not available:
$ for f in *.jpg; do echo mv "$f" "$(echo "$f" | perl -pe 's/\d+/sprintf "%03d", $&/ge')"; done
mv 1_120.jpg 001_120.jpg
mv 142_27.jpg 142_027.jpg
mv 7_39.jpg 007_039.jpg
Change echo mv to just mv once dry run seems okay
You can do it with a little shell-script using sed:
for i in *.jpg;
do
new=`echo "$i" | sed -n -e '{ s/^\([0-9]\{0,3\}\)_\([0-9]\{0,3\}\).jpg/000\1_000\2.jpg/ };
{s/\([0-9]*\)\([0-9]\{3\}\)_\([0-9]*\)\([0-9]\{3\}\).jpg/\2_\4.jpg/p };'`;
mv "$i" "$new";
done;
I first append three leading zeros at the said places by default and afterwards cut off as many digits as necessary brginning at the start at said places so that only 3 digits are left
with bash substitution(a,b)
windows(bash)
for f in *.jpg;do a=${f%_*};b=${f#*_};mv $f $(printf "%03d_%07s" $a $b);done
linux
for f in *.jpg;do a=${f%_*};b=${f#*_};b=${b%.*};mv $f $(printf "%03d_%03d".jpg $a $b);done
Related
I wrote a script which removes spaces in a single folder/file name. I want to make it work so that it removes all spaces in folder/files name in the directory the script exists.
MY Script:
#!/bin/bash
var=$(ls | grep " ")
test=$(echo $var | sed 's/ //')
mv "$var" $test
How it worked
Thank you for helping!
Try this
ls | grep " " | while read file_name
do
mv "$file_name" "$(echo $file_name | sed -E 's/ +//g')"
done
sed -E is so that you can use some simple regex, and / +/ so it can work in case of multiple consecutive spaces such as . And /g so it replaces every occurrences such as foo baa .txt .
Something like this might work:
for f in * ; do
if [[ "$f" =~ \ ]] ; then
mv "$f" "${f// /_}"
fi
done
Explanattion:
for f in * ; do
loops over all file names in the directory. It doesn't have the quirks of ls that make parsing the output of ls a bad idea.
if [[ "$f" =~ \ ]] ; then
This is the bash way of pattern matching. The \ is the pattern. You need to escape the space with a backslash, otherwise the shell will not recognize it as a pattern.
mv "$f" "${f// /_}"
${f// /_} is the bash way of pattern-substitution. The // means replace all occurrences. The syntax is ${variable//pattern/replacement} to replace all patterns in the variable with the replacement.
I am currently using the following command:
grep -l -Z -E '.*?FindMyRegex' /home/user/folder/*.csv | xargs -0 -I{} mv {} /home/destination/folder
This works fine. The problem is it uses grep on the entire file.
I would like to use the grep command on the FIRST line of the file only.
I have tried to use head -1 file | at the beginning, but it did not work.
A change I would add to your script is -
for file in *.csv; do
head -1 "$file" | grep -l -Z -E '.*?FindMyRegex' | xargs -0 -I{} mv {} /home/destination/folder;
done
you can maybe try sed '1q' file.csv | grep ... to search the regexp only in the first line.
You don't need grep or find, as long as your files don't have embedded newlines.
I don't know an easy way off the top of my head to get sed to delimit with nulls.
mv $( for f in /home/user/folder/*.csv;
do sed -ns '1 { /yourPattern/F; q; }' $f;
done ) /home/destination/folder/
EDIT
Rewrote with a loop. This will run a separate instance of sed to check each file, but at least it shouldn't read beyond the first line. It will fail syntactically if there are no hits.
You might need -E depending on your regex.
-n says don't print records from the files.
-s says treat each file as a distinct input - this is so the filenames aren't always the first one.
This does require GNU sed for the F.
gawk 'FNR==1{if($0~/PATTERN/)
printf "mv %s %s\n",FILENAME, "/target";nextfile}' /path/*.csv
First of all, in your regex: .*?FindMyRegex the .*? doesn't make any sense, they could be removed.
The above awk (gawk) one-liner will build up mv file target command lines for you. You can check them, if you are satisfied with them, pipe the output to |sh , the commands are gonna be executed.
replace PATTERN by your regex pattern, and /target by the real target dir.
The one-liner is assuming that the filenames don't contain special chars (space i.e.), if it is the case, add "s to the mv cmd.
using GNU awk to find the filenames, pipe the filenames into xargs
gawk -v pattern="myRegex" '
FNR == 1 {if ($0 ~ pattern) printf "%s\0", FILENAME; nextfile}
' *.csv | xargs -0 echo mv -t destination
If it looks OK, remove "echo"
Try this Shellcheck-clean Bash code:
#! /bin/bash
shopt -s nullglob # Globs that match nothing expand to nothing
shopt -s dotglob # Globs match files whose names start with '.'
dest=/home/destination/folder
for file in *.csv ; do
head -n 1 -- "$file" | grep -qE '.*?FindMyRegex' && mv -- "$file" "$dest"
done
shopt -s nullglob prevents an error if there are no .csv files in the directory.
shopt -s dotglob ensures that files whose name starts with '.' are handled.
The -- in the options for head and mv ensures that files whose names begin with - are handled correctly.
The quotes in "$file" and "$dest" ensure that names that contain whitespace (actually $IFS) characters (including newlines) or glob metacharacters are handled correctly.
Note that the .*? in the reqular expression is probably redundant, and may not do what you think it does (grep -E doesn't do non-greedy matching).
$ ls /tmp/foo/
file1.txt file2.txt
$ ls /tmp/bar/
file20.txt
$ ls /tmp/foo/file*.txt | grep -o -E '[0-9]+' | sort -n | paste -s -d,
1,2
How to fetch the number in the filename from both the directories? in the above example, I need to get 1,2,20, its in bash shell.
UPDATE:
$ ls /tmp/foo/file*.txt /tmp/bar/file*.txt /tmp/jaz99/file*.txt /tmp/nah/file*.txt | grep -o -E '[0-9]+' | sort -n | paste -s -d,
ls: cannot access /tmp/nah/file*.txt: No such file or directory
1,2,20,30,99
in this case, it should not print 99 (as its not matched by *), and should not print the error if file not found.
You can get this done using a loop with output of find:
s=
# run a loop using find command in a process substitution
while IFS= read -d '' -r file; do
file="${file##*/}" # strip down all directory paths
s+="${file//[!0-9]/}," # remove all non-numeric characters and append comma
done < <(find /tmp/{foo,bar,nah,jaz99} -name '*.txt' -print0 2>/dev/null)
echo "${s%,}" # remove last comma from string
Output
1,2,20,30
Here's my take on this. Use arrays. No need to use external tools like sed or awk or find.
#!/usr/bin/env bash
declare -a a=()
for f in /tmp/{foo,bar,nah}/file*.txt; do
[[ $f =~ .*file([0-9]+).* ]]
a+=( ${BASH_REMATCH[1]} )
done
IFS=,
echo "${a[*]}"
The [[...]] expression populates the $BASH_REMATCH array with regex components. You can use that to extract the numbers and place them in a new temporary array, which you can express with comma separators using $IFS.
Results:
$ mkdir /tmp/foo /tmp/bar
$ touch /tmp/foo/file{1,2}.txt /tmp/bar/file20.txt
$ ./doit
1,2,20
I have a list of almost 500 pdf files with the following filename structure:
XXXX-YYYY-MM-DD.pdf
where XXXX is a variable lenght numeric code (1 to 4 digits) always delimitated by "-", for example:
51-2016-08-22.pdf
776-2016-08-22.pdf
3881-2016-08-22.pdf
4-2016-08-22.pdf
2860-2016-08-22.pdf
The goal is to copy each file into its own directory, naming the directories like the pattern (ie: file 776-2016-08-22.pdf goes to directory 776). How can I use awk or sed to delimitate the variable lenght field?
Here's my code:
for f in *.pdf
do
FOLDERNAME=`echo $f| awk (awk or sed missing code here)`
mkdir /my/dir/structure/$FOLDERNAME
cp $f /my/dir/structure/$FOLDERNAME/
done
Thanks for your support.
You can use:
for f in *.pdf; do
d="${f%%-*}"
mkdir -p "$d" && cp "$f" "$d"
done
As rightly pointed out by ed-morton, This is NOT recommended solution as it fails in many cases. Please follow https://stackoverflow.com/a/39089589/3834860
Keeping this answer for reference.
awk -F '-' to specify delimiter and '{print $1}' for first element before delimiter.
for f in *.pdf
do
FOLDERNAME=`echo $f| awk -F '-' '{print $1}'`
mkdir /my/dir/structure/$FOLDERNAME
cp $f /my/dir/structure/$FOLDERNAME/
done
Say I have a directory with a bunch of site names in it.
i.e.
dev.domain.com
dev.domain2.com
dev.domain3.com
How can I rename those to <domain>.com on the linux cli using piping and/or redirection bash?
I get to a point than am stuck.
find . -maxdepth 1 | grep -v "all" | cut --delimiter="." -f3 | awk '{ print $0 }'
Gives me the domain part, but I can't get past that. Not sure awk is the answer either. Any advice is appreciated.
To strip the leading 'dev.' from names it should be like this:
for i in $(find * -maxdepth 1 -type d); do mv $i $(echo $i | sed 's/dev.\(.*\)/\1/'); done
for i in *; do mv $i $( echo $i | sed 's/\([^\.]*\).\([^\.]*\).\([^\.]*\)/\2.\1/' ); done
Explained:
for i in *; do ....; done
do it for every file
echo $i | sed 's/\([^\.]*\).\([^\.]*\).\([^\.]*\)/\2.\1/'
takes three groups of "every character except ." and changes their order
\2.\1 means: print second group, a dot, first group
the $( ... ) takes output of sed and "pastes" it after mv $i and is called "command substitution" http://www.gnu.org/software/bash/manual/bashref.html#Command-Substitution
Try the rename command. It can take a regular expression like this:
rename 's/\.domain.*/.com/' *.com
under the directory you want to work with, try :
ls -dF *|grep "/$"|awk 'BEGIN{FS=OFS="."} {print "mv "$0" "$2,$3}'
will print mv command. if you want to do the rename, add "|sh" and the end:
ls -dF *|grep "/$"|awk 'BEGIN{FS=OFS="."} {print "mv "$0" "$2,$3}'|sh