So I have this idea of moving folders to different places depending on what the name of the source folder is.
I have came up with this little script, which does kind of work. It mismatches some names, and also it sometimes just straight up doesn't work.
script:
#!/bin/bash
for result in $(ls -d /path/to/folders/*/);
do
Size=${#result}
StripFrom=$(expr index "$result" 'S\b[0-9]\b')
Strip=4
Stripped=$(($StripFrom-$Strip))
EndStrip=$(($Size-$Stripped))
EndStrip=-$EndStrip
Serie=${result:23:$EndStrip}
mv $result /path/to/TV/$Serie/
done
What I'm trying to do:
Get a list of all folders in "/path/to/folders/"
For each of them, search for the text S0, S1, S2 e.t.c.
Get the name of the tv-show which is the part infront of "S[0-0]"
Then move the folder to "/path/to/TV/Name-of-show"
I don't know if I'm going at this the wrong way all together.
23 is the number of characters in the path /path/to/folders/, by the way.
You can extract the series name with a regular expression:
regex='/([^/]*).S[[:digit:]]{2}[^/]*/$'
for dir in /path/to/folders/*/; do
if [[ $dir =~ $regex ]]; then
mv "$dir" /path/to/TV/"${BASH_REMATCH[1]}"
fi
done
The expression looks for S followed by two digits between the last two / of the path and captures everything between the second last / and the S, minus the character before the S.
For an example content of /path/to/folders/ of
folders
├── show1.S01E14.blahblah
├── show2.S11E01.text
└── unrelateddir
the commands issued are
mv /path/to/folders/show1.S01E14.blahblah/ /path/to/TV/show1
mv /path/to/folders/show2.S11E01.text/ /path/to/TV/show2
Remarks for your script:
You don't have to loop over the output of ls as in $(ls -d /path/to/folders/*/) (see here why that isn't a good idea in general), you can use the glob directly.
Not entirely sure about this one, but S\b[0-9]\b would fail to match and S followed by two digits, as there is no word boundary between the digits.
In arithmetic expansion, you don't need the $ signs: Stripped=$(($StripFrom-$Strip)) can be written Stripped=$(( StripFrom - Strip )).
Maybe to make things easier:
find /path/to/folders -name \*S[0-9]\* | while read $p
do
result=$(basename "$p")
SEASON=$(expr "$result" : '[[:print:]]\+\(S[[:digit:]]*\)')
SHOW=$(expr "$result" : '\([[:print:]]\+\)S[[:digit:]]*') # with the trailing "."
echo "show: $SHOW / season: $SEASON"
done
Related
Task: concatinate array of string with delimiter, dilimeter is "/".
Metatask: i've a folder with many files. Need to copy them into another folder.
So i need to get "name of file" and "path to folder".
What's wrong: delimiter "/" works incorrectly. It doesn't concatinate with my strings. If i try to use "\/" - string disappeare at all.
What's going on?
loc_path='./test/*'
delim='\/'
for itt in $loc_path; do
IFS=$delim
read -ra res <<< "$itt"
str=''
for ((i = 1; i \<= ${#res[#]}; i++)); do
#str=($str${res[$i]}$delim)
str="$str${res[$i]}$delim"
done
echo $str
done
Please, give to two answers:
how to solve task-problem
better way to solve metatask
There is an issue in delim='\/'. Firstly, you need not to protect slash. Secondly all characters are already protected between simple quotes.
There is a syntax issue with your concatenation. You must not use parenthesis here! They can be used to open a sub shell. We need not that.
To solve your 'meta-task', you should avoid to use IFS, or read. They are complex to use (for example by modifying IFS globally as you do, you change how echo display the res array. It can mislead you while you troubleshoot...) I suggest you use more simple tool like: basename, etc.
Here few scripts to solve your meta (scholar?) task:
# one line :-)
cp src/* dst/
# to illustrate basename etc
for file in "$SRC/"*; do
dest="$DST/$(basename $file)"
cp "$file" "$dest"
done
# with a change of directory
cd "$SRC"
for file in *; do cp "$file" "$DST/$file"; done
cd -
# Change of directory and a sub shell
(cd "$SRC" ; for file in *; do cp "$file" "$DST/$file"; done)
Task solution:
arr=( string1 string2 string3 ) # array of strings
str=$( IFS='/'; printf '%s' "${arr[*]}" ) # concatenated with / as delimiter
$str will be the single string string1/string2/string3.
Meta task solution:
Few files:
cp path/to/source/folder/* path/to/dest/folder
Note that * matches any type of file and that it does not match hidden names. For hidden names, use shopt -s dotglob in bash. This will fail if there are thousands of files (argument list too long).
Few or many files files, only non-directories:
for pathaname in path/to/source/folder/*; do
[ ! -type d "$pathame" ] && cp "$pathname" path/to/dest/folder
done
or, with find,
find path/to/source/folder -maxdepth 1 ! -type d -exec cp {} path/to/dest/folder \;
The difference between these two is that the shell loop will refuse to copy symbolic links that resolve to directories, while the find command will copy them.
If any of this isn't particularly clear, please let me know and I'll do my best to clarify.
I basically need to sort a set of files with various extensions and similar patterns to the filename, into directories and subdirectories that match the pattern and type of extension.
To elaborate a bit:
All files, regardless of extension, begin with the pattern "zz####" where #### is a number from 1 to 900; "zz1.zip through zz950.zip, zz1.mov through zz950.mov, zz1.mp4 through zz950.mp4"
Some files contain additional characters; "zz360_hello_world.zip"
Some files contain spaces; "zz370_hello world.zip"
I need these files to be sorted and moved into directories and subdirectories following a particular format: "/home/hello/zz1/zip, /home/hello/zz1/vid"
If the directories and/or subdirectories don't exist, I need them created.
Example:
zz400_testing.zip ----> /home/hello/zz400/zip
zz400 testing video.mov ----> /home/hello/zz400/vid
zz500.zip ----> /home/hello/zz500/zip
zz500_testing another video.mp4 ----> /home/hello/zz500/vid
I found a few answers around here for simpler use-cases, but wasn't able to get anything working for my particular needs.
Any help at all would be much appreciated.
Thank you!
EDIT: Adding the code I've been messing with
for f in *.zip; do
set=`echo "$f"|sed 's/[0-9].*//'`
dir="/home/demo/$set/photos"
mkdir -p "$dir"
mv "$f" "$dir"
done
I think I'm just having trouble wrapping my head around how to match with regex. I've got this far with it:
[demo#alpha grep]$ echo zz433.zip|sed 's/[0-9].*//'
zz
The script will run the mkdir, and even move the zip files into their proper place. I just can't get it to create the proper top-level directory (zz433).
The sed command here doesn't do what you're trying to achieve:
set=`echo "$f"|sed 's/[0-9].*//'`
The meaning of the regular expression [0-9].* is "a digit followed by anything".
The s/// command of sed performs a replacement.
The result is effectively removing everything from the input starting from the first digit.
So for "zz360_hello_world.zip" it removes everything starting from "3",
leaving only "zz".
Note also that to match the files, the pattern *.zip doesn't match your description. You're looking for files starting with "zz" and a number from 1 up to 900. If you don't mind including numbers > 900 then you can write the loop expression like this:
for f in zz[0-9][^0-9]* zz[0-9][0-9][^0-9]* zz[0-9][0-9][0-9][^0-9]*; do
Or the same thing more compactly:
for f in zz{[0-9],[0-9][0-9],[0-9][0-9][0-9]}[^0-9]*; do
These are glob patterns.
zz[0-9][^0-9]* means "start with 'zz', followed by a digit, followed by a non-digit, followed by anything".
In the above example I use three patterns to cover the cases of "zz" followed by 1, 2 or 3 digits, followed by a non-digit.
The second example is a more compact form of the first,
the idea is that a{b,c}d expands to abd and acd.
Next, to get the appropriate prefix, you could use pattern matching with a case statement and extract substrings.
The syntax of these patterns is the same glob syntax as in the previous example in the for statement.
case "$f" in
zz[0-9][0-9][0-9]*) prefix=${f:0:5} ;;
zz[0-9][0-9]*) prefix=${f:0:4} ;;
zz[0-9]*) prefix=${f:0:3} ;;
esac
It seems you also want to create grouping by file type. You could get the file extension by chopping off the beginning of the name until the dot with ext=${f##*.}, and then use a case statement as in the earlier example to map extensions to the desired directory names.
Putting the above together:
for f in zz{[0-9],[0-9][0-9],[0-9][0-9][0-9]}[^0-9]*; do
case "$f" in
zz[0-9][0-9][0-9]*) prefix=${f:0:5} ;;
zz[0-9][0-9]*) prefix=${f:0:4} ;;
zz[0-9]*) prefix=${f:0:3} ;;
esac
ext=${f##*.}
case "$ext" in
mov|mp4) group=vid ;;
*) group=$ext ;;
esac
dir="/home/demo/$prefix/$group"
mkdir -p "$dir"
mv "$f" "$dir"
done
I've answered part of my own question!
for f in *.zip; do
set=`echo "$f"|grep -o -P 'zz[0-9]+.{0,0}'`
dir="/home/demo/$set/photos"
mkdir -p "$dir"
mv "$f" "$dir"
done
Basically, the following script will grab files like:
zz232.zip
zz233test.zip
zz234 test.zip
Then it will create the top-level directory (zz####), the photos sub-directory, and move the file into place:
/home/demo/zz232/photos/zz232.zip
/home/demo/zz233/photos/zz233test.zip
/home/demo/zz234/photos/zz234 test.zip
Moving on to expanding the script for additional functionality.
Thanks all!
How about:
#!/bin/bash
IFS=$'\n'
for file in *; do
if [[ $file =~ ^(zz[0-9]+).*\.(zip|mov|mp4)$ ]]; then
ext=${BASH_REMATCH[2]}
if [ $ext = "mov" -o $ext = "mp4" ]; then
ext="vid"
fi
dir="/home/hello/${BASH_REMATCH[1]}/$ext"
mkdir -p $dir
mv "$file" $dir
fi
done
Hope this helps.
I am looking for a way to make a simple loop in bash over everything my directory contains, i.e. files, directories and links including hidden ones.
I will prefer if it could be specifically in bash but it has to be the most general. Of course, file names (and directory names) can have white space, break line, symbols. Everything but "/" and ASCII NULL (0×0), even at the first character. Also, the result should exclude the '.' and '..' directories.
Here is a generator of files on which the loop has to deal with :
#!/bin/bash
mkdir -p test
cd test
touch A 1 ! "hello world" \$\"sym.dat .hidden " start with space" $'\n start with a newline'
mkdir -p ". hidden with space" $'My Personal\nDirectory'
So my loop should look like (but has to deal with the tricky stuff above):
for i in * ;
echo ">$i<"
done
My closest try was the use of ls and bash array, but it is not working with, is:
IFS=$(echo -en "\n\b")
l=( $(ls -A .) )
for i in ${l[#]} ; do
echo ">$i<"
done
unset IFS
Or using bash arrays but the ".." directory is not exclude:
IFS=$(echo -en "\n\b")
l=( [[:print:]]* .[[:print:]]* )
for i in ${l[#]} ; do
echo ">$i<"
done
unset IFS
* doesn't match files beginning with ., so you just need to be explicit:
for i in * .[^.]*; do
echo ">$i<"
done
.[^.]* will match all files and directories starting with ., followed by a non-. character, followed by zero or more characters. In other words, it's like the simpler .*, but excludes . and ... If you need to match something like ..foo, then you might add ..?* to the list of patterns.
As chepner noted in the comments below, this solution assumes you're running GNU bash along with GNU find GNU sort...
GNU find can be prevented from recursing into subdirectories with the -maxdepth option. Then use -print0 to end every filename with a 0x00 byte instead of the newline you'd usually get from -print.
The sort -z sorts the filenames between the 0x00 bytes.
Then, you can use sed to get rid of the dot and dot-dot directory entries (although GNU find seems to exclude the .. already).
I also used sed to get read of the ./ in front of every filename. basename could do that too, but older systems didn't have basename, and you might not trust it to handle the funky characters right.
(These sed commands each required two cases: one for a pattern at the start of the string, and one for the pattern between 0x00 bytes. These were so ugly I split them out into separate functions.)
The read command doesn't have a -z or -0 option like some commands, but you can fake it with -d "" and blanking the IFS environment variable.
The additional -r option prevents a backslash-newline combo from being interpreted as a line continuation. (A file called backslash\\nnewline would otherwise be mangled to backslashnewline.) It might be worth seeing if other backslash-combos get interpreted as escape sequences.
remove_dot_and_dotdot_dirs()
{
sed \
-e 's/^[.]\{1,2\}\x00//' \
-e 's/\x00[.]\{1,2\}\x00/\x00/g'
}
remove_leading_dotslash()
{
sed \
-e 's/^[.]\///' \
-e 's/\x00[.]\//\x00/g'
}
IFS=""
find . -maxdepth 1 -print0 |
sort -z |
remove_dot_and_dotdot_dirs |
remove_leading_dotslash |
while read -r -d "" filename
do
echo "Doing something with file '${filename}'..."
done
It may not be the most favorable way but I tried bellow thing
while read line ; do echo $line; done <<< $(ls -a | grep -v -w ".")
check the below trail which I did
Try the find command, something like:
find .
That will list all the files in all recursive directories.
To output only files excluding the leading . or .. try:
find . -type f -printf %P\\n
I have a folder that is full of .bak files and some other files also. I need to remove the extension of all .bak files in that folder. How do I make a command which will accept a folder name and then remove the extension of all .bak files in that folder ?
Thanks.
To remove a string from the end of a BASH variable, use the ${var%ending} syntax. It's one of a number of string manipulations available to you in BASH.
Use it like this:
# Run in the same directory as the files
for FILENAME in *.bak; do mv "$FILENAME" "${FILENAME%.bak}"; done
That works nicely as a one-liner, but you could also wrap it as a script to work in an arbitrary directory:
# If we're passed a parameter, cd into that directory. Otherwise, do nothing.
if [ -n "$1" ]; then
cd "$1"
fi
for FILENAME in *.bak; do mv "$FILENAME" "${FILENAME%.bak}"; done
Note that while quoting your variables is almost always a good practice, the for FILENAME in *.bak is still dangerous if any of your filenames might contain spaces. Read David W.'s answer for a more-robust solution, and this document for alternative solutions.
There are several ways to remove file suffixes:
In BASH and Kornshell, you can use the environment variable filtering. Search for ${parameter%word} in the BASH manpage for complete information. Basically, # is a left filter and % is a right filter. You can remember this because # is to the left of %.
If you use a double filter (i.e. ## or %%, you are trying to filter on the biggest match. If you have a single filter (i.e. # or %, you are trying to filter on the smallest match.
What matches is filtered out and you get the rest of the string:
file="this/is/my/file/name.txt"
echo ${file#*/} #Matches is "this/` and will print out "is/my/file/name.txt"
echo ${file##*/} #Matches "this/is/my/file/" and will print out "name.txt"
echo ${file%/*} #Matches "/name.txt" and will print out "/this/is/my/file"
echo ${file%%/*} #Matches "/is/my/file/name.txt" and will print out "this"
Notice this is a glob match and not a regular expression match!. If you want to remove a file suffix:
file_sans_ext=${file%.*}
The .* will match on the period and all characters after it. Since it is a single %, it will match on the smallest glob on the right side of the string. If the filter can't match anything, it the same as your original string.
You can verify a file suffix with something like this:
if [ "${file}" != "${file%.bak}" ]
then
echo "$file is a type '.bak' file"
else
echo "$file is not a type '.bak' file"
fi
Or you could do this:
file_suffix=$(file##*.}
echo "My file is a file '.$file_suffix'"
Note that this will remove the period of the file extension.
Next, we will loop:
find . -name "*.bak" -print0 | while read -d $'\0' file
do
echo "mv '$file' '${file%.bak}'"
done | tee find.out
The find command finds the files you specify. The -print0 separates out the names of the files with a NUL symbol -- which is one of the few characters not allowed in a file name. The -d $\0means that your input separators are NUL symbols. See how nicely thefind -print0andread -d $'\0'` together?
You should almost never use the for file in $(*.bak) method. This will fail if the files have any white space in the name.
Notice that this command doesn't actually move any files. Instead, it produces a find.out file with a list of all the file renames. You should always do something like this when you do commands that operate on massive amounts of files just to be sure everything is fine.
Once you've determined that all the commands in find.out are correct, you can run it like a shell script:
$ bash find.out
rename .bak '' *.bak
(rename is in the util-linux package)
Caveat: there is no error checking:
#!/bin/bash
cd "$1"
for i in *.bak ; do mv -f "$i" "${i%%.bak}" ; done
You can always use the find command to get all the subdirectories
for FILENAME in `find . -name "*.bak"`; do mv --force "$FILENAME" "${FILENAME%.bak}"; done
Let's say I have the following files in my current directory:
1.jpg
1original.jpg
2.jpg
2original.jpg
3.jpg
4.jpg
Is there a terminal/bash/linux command that can do something like
if the file [an integer]original.jpg exists,
then move [an integer].jpg and [an integer]original.jpg to another directory.
Executing such a command will cause 1.jpg, 1original.jpg, 2.jpg and 2original.jpg to be in their own directory.
NOTE
This doesn't have to be one command. I can be a combination of simple commands. Maybe something like copy original files to a new directory. Then do some regular expression filter on files in the newdir to get a list of file names from old directory that still need to be copied over etc..
Turning on extended glob support will allow you to write a regular-expression-like pattern. This can handle files with multi-digit integers, such as '87.jpg' and '87original.jpg'. Bash parameter expansion can then be used to strip "original" from the name of a found file to allow you to move the two related files together.
shopt -s extglob
for f in +([[:digit:]])original.jpg; do
mv $f ${f/original/} otherDirectory
done
In an extended pattern, +( x ) matches one or more of the things inside the parentheses, analogous to the regular expression x+. Here, x is any digit. Therefore, we match all files in the current directory whose name consists of 1 or more digits followed by "original.jpg".
${f/original/} is an example of bash's pattern substitution. It removes the first occurrence of the string "original" from the value of f. So if f is the string "1original.jpg", then ${f/original/} is the string "1.jpg".
well, not directly, but it's an oneliner (edit: not anymore):
for i in [0-9].jpg; do
orig=${i%.*}original.jpg
[ -f $orig ] && mv $i $orig another_dir/
done
edit: probably I should point out my solution:
for i in [0-9].jpg: execute the loop body for each jpg file with one number as filename. store whole filename in $i
orig={i%.*}original.jpg: save in $orig the possible filename for the "original file"
[ -f $orig ]: check via test(1) (the [ ... ] stuff) if the original file for $i exists. if yes, move both files to another_dir. this is done via &&: the part after it will be only executed if the test was successful.
This should work for any strictly numeric prefix, i.e. 234.jpg
for f in *original.jpg; do
pre=${f%original.jpg}
if [[ -e "$pre.jpg" && "$pre" -eq "$pre" ]] 2>/dev/null; then
mv "$f" "$pre.jpg" targetDir
fi
done
"$pre" -eq "$pre" gives an error if not integer
EDIT:
this fails if there exist original.jpg and .jpg both.
$pre is then nullstring and "$pre" -eq "$pre" is true.
The following would work and is easy to understand (replace out with the output directory, and {1..9} with the actual range of your numbers.
for x in {1..9}
do
if [ -e ${x}original.jpg ]
then
mv $x.jpg out
mv ${x}original.jpg out
fi
done
You can obviously also enter it as a single line.
You can use Regex statements to find "matches" in the files names that you are looking through. Then perform your actions on the "matches" you find.
integer=0; while [ $integer -le 9 ] ; do if [ -e ${integer}original.jpg ] ; then mv -vi ${integer}.jpg ${integer}original.jpg lol/ ; fi ; integer=$[ $integer + 1 ] ; done
Note that here, "lol" is the destination directory. You can change it to anything you like. Also, you can change the 9 in while [ $integer -le 9 ] to check integers larger than 9. Right now it starts at 0* and stops after checking 9*.
Edit: If you want to, you can replace the semicolons in my code with carriage returns and it may be easier to read. Also, you can paste the whole block into the terminal this way, even if that might not immediately be obvious.