I created a script for moving the files based on the below reference. I am trying to capture all the files activity that are picked up and moved from source to destination along with any unsuccessful files.
I tried pipping the output to a log file, but after operation the log file size is 0. Any recommendations please?
Reference Doc#
https://unix.stackexchange.com/questions/59112/preserve-directory-structure-when-moving-files-using-find
Below is the code block
destination=$(cd -- "$destination" && pwd)
cd -- "$source" &&
find . -type f -newermt $startdays -not -newermt $enddays -exec sh -c '
for x do
mkdir -p "$0/${x%/*}"
mv "$x" "$0/$x"
done
' "$destination" {} + >> output.log
By default, mv does not produce any output. If you want it to produce output, try mv -v.
Related
I want to create symlinks to all files in 'myfiles' which are not already linked to and specify the destination folder for the just-created symlinks.
I am using the following cmd, successfully, to generate the list of existing links, which point to 'myfolder' :
find ~/my-existing-links/ -lname '*/myfiles/*' -printf "%f\n" > results.txt
And I'm using the following cmd to reverse match i.e. to list the files in myfiles which are not linked to:
ls ~/myfiles | grep -vf results.txt > results2.txt
So, results2.txt has a list of the files, each of which I now want to create a new symlink to.... in a folder called ~/newlinks .
I know it is possible to feed 'ln -s' a file list using the find / exec combination i.e.
find ~/myfiles/ -exec ln -s {} -t ~/newlinks \; -print
.... but that would be the unfiltered file list in myfiles. I want to use the filtered list.
Any ideas how I can do this? I'm going to be adding files to myfiles regularly and so will periodically visit the folder for the purpose of generating symlinks for all the new files so I can divi the links up logically(rather than change the original filename).
Try with xargs:
cat results2.txt | xargs -I{} ln -s {} ~/newlinks
You can use xargs to apply the links, so that your composite command might look like this:
find ~/myfiles/ | grep -vf results.txt | xargs make-my-links
and make-my-links would be a script something like this:
#!/bin/sh
for source in "$#"
do
ln -s "$source" -t ~/newlinks
done
The separate script and loop are used with xargs because it does not accept a command-template, but will (default) send as many of the inputs as it thinks will fit on a command-line.
So, you have 3 entities of type directory:
~/myfiles/: contains your files.
~/my-existing-links/: contains links to files from ~/myfiles/.
~/newlinks/: contains links to new files from ~/myfiles/
To me, the third entity is rather unnecessary. Why the new links aren't created directly in ~/my-existing-links/?
I would only use a script to update the list of links in ~/my-existing-links/, whenever new files are added in ~/myfiles/:
update_v1.sh
#!/bin/bash
for f in $(find ~/myfiles -type f); do
ln -sf "$f" "~/my-existing-links/$(basename $f)"
done
update_v2.sh
find ~/myfiles -type f -exec sh -c \
'for f; do ln -sf "$f" "~/my-existing-links/${f#*/}"; done' sh {} +
update_print.sh
#!/bin/bash
for f in $(find ~/myfiles -type f); do
if [[ ! -L "~/my-existing-links/${f#*/}" ]]; then
echo "Link not existing for $f"
fi
done
Thanks, Thomas and pasaba... I found a way to do it:
So I did the following from ~/newlinks :
while read line; do ln -s "$line" "${line##*/}" ; done < ~/myfiles/results2.txt
Thanks again for your time.
I'm trying to move media and other files which are in a specified directory to another directory and create another one if it does not exits (where the files will go), and create a directory the remaining files with different extensions will go. My first problem is that my script is not making a new directory and it is not moving the files to other directories and what code can I use to move files with different extensions to one directory?
This is what i have had so far, correct me where I'm wrong and help modify my script:
#!/bin/bash
From=/home/katy/doc
To=/home/katy/mo #directory where the media files will go
WA=/home/katy/do # directory where the other files will go
if [ ! -d "$To" ]; then
mkdir -p "$To"
fi
cd $From
find path -type f -name"*.mp4" -exec mv {} $To \;
I'd solve it somewhat like this:
#!/bin/bash
From=/home/katy/doc
To=/home/katy/mo # directory where the media files will go
WA=/home/katy/do # directory where the other files will go
cd "$From"
find . -type f \
| while read file; do
dir="$(dirname "$file")"
base="$(basename "$file")"
if [[ "$file" =~ \.mp4$ ]]; then
target="$To"
else
target="$WA"
fi
mkdir -p "$target/$dir"
mv -i "$file" "$target/$dir/$base"
done
Notes:
mkdir -p will not complain if the directory already exists, so there's no need to check for that.
Put double quotes around all filenames in case they contain spaces.
By piping the output of find into a while loop, you also avoid getting bitten by spaces, because read will read until a newline.
You can modify the regex according to taste, e.g. \.(mp3|mp4|wma|ogg)$.
In case you didn't know, $(...) will run the given command and stick its output back in the place of the $(...) (called command substitution). It is almost the same as `...` but slightly better (details).
In order to test it, put echo in front of mv. (Note that quotes will disappear in the output.)
cd $From
find . -type f -name "*.mp4" -exec mv {} $To \;
^^^
or
find $From -type f -name "*.mp4" -exec mv {} $To \;
^^^^^
cd $From
mv *.mp4 $To;
mv * $WA;
In a shell script how would I find a file by a particular name and then navigate to that directory to do further operations on it?
From here I am going to copy the file across to another directory (but I can do that already just adding it in for context.)
You can use something like:
cd -- "$(dirname "$(find / -type f -name ls | head -1)")"
This will locate the first ls regular file then change to that directory.
In terms of what each bit does:
The find will start at / and search down, listing out all regular files (-type f) called ls (-name ls). There are other things you can add to find to further restrict the files you get.
The | head -1 will filter out all but the first line.
$() is a way to take the output of a command and put it on the command line for another command.
dirname can take a full file specification and give you the path bit.
cd just changes to that directory, the -- is used to prevent treating a directory name beginning with a hyphen from being treated as an option to cd.
If you execute each bit in sequence, you can see what happens:
pax[/home/pax]> find / -type f -name ls
/usr/bin/ls
pax[/home/pax]> find / -type f -name ls | head -1
/usr/bin/ls
pax[/home/pax]> dirname "$(find / -type f -name ls | head -1)"
/usr/bin
pax[/home/pax]> cd -- "$(dirname "$(find / -type f -name ls | head -1)")"
pax[/usr/bin]> _
The following should be more safe:
cd -- "$(find / -name ls -type f -printf '%h' -quit)"
Advantages:
The double dash prevents the interpretation of a directory name starting with a hyphen as an option (find doesn't produce such file names, but it's not harmful and might be required for similar constructs)
-name check before -type check because the latter sometimes requires a stat
No dirname required because the %h specifier already prints the directory name
-quit to stop the search after the first file found, thus no head required which would cause the script to fail on directory names containing newlines
no one suggesting locate (which is much quicker for huge trees) ?
zsh:
cd $(locate zoo.txt|head -1)(:h)
cd ${$(locate zoo.txt)[1]:h}
cd ${$(locate -r "/zoo.txt$")[1]:h}
or could be slow
cd **/zoo.txt(:h)
bash:
cd $(dirname $(locate -l1 -r "/zoo.txt$"))
Based on this answer to a similar question, other useful choice could be having 2 commands, 1st to find the file and 2nd to navigate to its directory:
find ./ -name "champions.txt"
cd "$(dirname "$(!!)")"
Where !! is history expansion meaning 'the previous command'.
Expanding on answers already given, if you'd like to navigate iteratively to every file that find locates and perform operations in each directory:
for i in $(find /path/to/search/root -name filename -type f)
do (
cd $(dirname $(realpath $i));
your_commands;
)
done
if you are just finding the file and then moving it elsewhere, just use find and -exec
find /path -type f -iname "mytext.txt" -exec mv "{}" /destination +;
function fReturnFilepathOfContainingDirectory {
#fReturnFilepathOfContainingDirectory_2012.0709.18:19
#$1=File
local vlFl
local vlGwkdvlFl
local vlItrtn
local vlPrdct
vlFl=$1
vlGwkdvlFl=`echo $vlFl | gawk -F/ '{ $NF="" ; print $0 }'`
for vlItrtn in `echo $vlGwkdvlFl` ;do
vlPrdct=`echo $vlPrdct'/'$vlItrtn`
done
echo $vlPrdct
}
Simply this way, isn't this elegant?
cdf yourfile.py
Of course you need to set it up first, but you need to do this only once:
Add following line into your .bashrc or .zshrc, whatever you use as your shell initialization script.
source ~/bin/cdf.sh
And add this code into ~/bin/cdf.sh file that you need to create from scratch.
#!/bin/bash
function cdf() {
THEFILE=$1
echo "cd into directory of ${THEFILE}"
# For Mac, replace find with mdfind to get it a lot faster. And it does not need args ". -name" part.
THEDIR=$(find . -name ${THEFILE} |head -1 |grep -Eo "/[ /._A-Za-z0-9\-]+/")
cd ${THEDIR}
}
If it's a program in your PATH, you can do:
cd "$(dirname "$(which ls)")"
or in Bash:
cd "$(dirname "$(type -P ls)")"
which uses one less external executable.
This uses no externals:
dest=$(type -P ls); cd "${dest%/*}"
If your file is only in one location you could try the following:
cd "$(find ~/ -name [filename] -exec dirname {} \;)" && ...
You can use -exec to invoke dirname with the path that find returns (which goes where the {} placeholder is). That will change directories. You can also add double ampersands ( && ) to execute the next command after the shell has changed directory.
For example:
cd "$(find ~/ -name need_to_find_this.rb -exec dirname {} \;)" && ruby need_to_find_this.rb
It will look for that ruby file, change to the directory, then run it from within that folder. This example assumes the filename is unique and that for some reason the ruby script has to run from within its directory. If the filename is not unique you'll get many locations passed to cd, it will return an error then it won't change directories.
try this. i created this for my own use.
cd ~
touch mycd
sudo chmod +x mycd
nano mycd
cd $( ./mycd search_directory target_directory )"
if [ $1 == '--help' ]
then
echo -e "usage: cd \$( ./mycd \$1 \$2 )"
echo -e "usage: cd \$( ./mycd search_directory target_directory )"
else
find "$1"/ -name "$2" -type d -exec echo {} \; -quit
fi
cd -- "$(sudo find / -type d -iname "dir name goes here" 2>/dev/null)"
keep all quotes (all this does is just send you to the directory you want, after that you can just put commands after that)
I have found a useful shell script that shows all files in a directory recursively.
Where it prints the file name echo "$i"; #Display File name.
I would instead like to run an ffmpeg command on non MP3 files, how can I do this? I have very limited knowledge of shell scripts so I appreciate if I was spoon fed! :)
//if file is NOT MP3
ffmpeg -i [the_file] -sameq [same_file_name_with_mp3_extension]
//delete old file
Here is the shell script for reference.
DIR="."
function list_files()
{
if !(test -d "$1")
then echo $1; return;
fi
cd "$1"
echo; echo `pwd`:; #Display Directory name
for i in *
do
if test -d "$i" #if dictionary
then
list_files "$i" #recursively list files
cd ..
else
echo "$i"; #Display File name
fi
done
}
if [ $# -eq 0 ]
then list_files .
exit 0
fi
for i in $*
do
DIR="$1"
list_files "$DIR"
shift 1 #To read next directory/file name
done
You can do the same with a find one-liner. Assuming the files you want to process are all wav:
find /path/ -type f -name "*wav" -exec ffmpeg -i {} -sameq {}.mp3 \;
If you want to find "rm" files, and delete them after conversion:
find /path/ -type f -name "*.rm" -exec ffmpeg -i {} -sameq {}.mp3 && rm {} \;
That said, if you want to do it with the shell script you showed, take the line that says
echo "$i";
replace it with this:
ffmpeg -i "$i" -sameq "$i".mp3
$i is a variable. A few lines up, you have:
for i in *
this basically means "for every element in * (which in turn stands for all files in the current directory, it's what's called a "shell expansion"), put the name of the element/file in the variable i, and then execute all the code between "do" and "done" ". So for each iteration, i will contain the name of one of the files in this directory.
There's also a section that tests whether i is a directory and if so, it recursively lists its contents.
A quick final note: the \; at the end of the find command IS significant and it NEEDS to have a space before the backslash, otherwise it won't work.
Your shell script seems to be essentially ls -1R, so it's probably easier to just use that. As for running ffmpeg on non-MP3 files, it's probably easier to use find instead of writing a whole shell script to do it. Assuming you're identifying MP3 files by their extension:
find your-path -not -name "*.mp3" -exec ffmpeg -i '{}' -sameq '{}.mp3' \;
I have the following directory structure:
+-archive
+-a
+-data.txt
+-b
+-data.txt
+-incoming
+-a
+-data.txt
+-c
+-data.txt
How do I do the equivalent of mv incoming/* archive/ but have the contents of the files in incoming appended to those in archive rather than overwrite them?
# move to incoming/ so that we don't
# need to strip a path prefix
cd incoming
# create directories that are missing in archive
for d in `find . -type d`; do
if [ ! -d "../archive/$d" ]; then
mkdir -p "../archive/$d"
fi
done
# concatenate all files to already existing
# ones (or automatically create them)
for f in `find . -type f`; do
cat "$f" >> "../archive/$f"
done
This should find any file in incoming and concatenate it to an existing file in archive.
The important part is to be inside incoming, because else we'd had to strip the path prefix (which is possible, but in the above case unnecessary). In the above case, a value of $f typically looks like ./a/data.txt, and hence the redirection goes to ../archive/./a/data.txt.
run it on the current directory.
find ./incoming -type f | while read -r FILE
do
dest=${FILE/incoming/archive}
cat "$FILE" >> "$dest"
done
the one in incoming/c would not be appended though
Here's a version with proper quoting:
#!/bin/sh
if [ -z "$1" ]; then
# acting as parent script
find incoming -type f -exec "$0" {} \;
else
# acting as child script
for in_file; do
if [ -f "$in_file" ]; then
destfile="${in_file/incoming/archive}"
test -d "$(dirname "$destfile")" || mkdir -p "$_"
cat "$in_file" >> "$destfile" &&
rm -f "$in_file"
fi
done
fi