Say for example you had a shell script that utilises inotifywait in the following form
inotifywait -m -e create /home/user1/*/*
Essentially you are asking it to monitor all folders two levels under user1. I've noticed with inotify that it can not dynamically index new folders.
For example you have a file system like
/home
/user1
/a
/b
Then you run the inotifywait script, it will monitor actions on the 'b' folder.
If you were to then add a new folder 'c' under 'a' (same level as 'b'), inotify will not monitor actions on this folder. Anyone got a way of remedying this?
Go up a level and monitor that with the recursive switch.
inotifywait -r -m -e create /home/user1/*
I don't know what you are using this for, but for indefinite watching you might want to check out iwatch.
Related
I am writing a bash script that looks at each file in a directory and does some sort of action to it. It's supposed to look something like this (maybe?).
for file in "$dir"* ; do
something
done
Cool, right? The problem is, this directory is being updated frequently (with new files). There is no guarantee that, at some point, I will technically be done with all the files in the dir (therefore exiting the for-loop), but not actually done feeding the directory with extra files. There is no guarantee that I will never be done feeding the directory (well... take that with a grain of salt).
I do NOT want to process the same file more than once.
I was thinking of making a while loop that runs forever and keeps updating some file-list A, while making another file-list B that keeps track of all the files I already processed, and the first file in file-list A that is not in file-list B gets processed.
Is there a better method? Does this method even work? Thanks
Edit: Mandatory "I am bash newb"
#Barmar has a good suggestion. One way to handle this is using inotify to watch for new files. After installing the inotify-tools on your system, you can use the inotifywait command to feed new-file events into a loop.
You may start with something like:
inotifywait -m -e MOVED_TO,CLOSED_WRITE myfolder |
while read dir events file, do
echo "Processing file $file"
...do something with $dir/$file...
mv $dir/$file /some/place/for/processed/files
done
This inotifywait command will generate events for (a) files that are moved into the directory and (b) files that are closed after being opened for writing. This will generally get you what you want, but there are always corner cases that depend on your particular application.
The output of inotifywait looks something like:
tmp/work/ CLOSE_WRITE,CLOSE file1
tmp/work/ MOVED_TO file2
I am looking for help with a script that will allow me to copy the contents of a directory, or the whole directory to another directory on a schedule or when new files arrive in the source.
For example:
/stuff/folder1/file.txt
Copy to:
/stuff/folder2/file.txt
either when new files arrive or on a recurring schedule.
I do use a Centos machine.
You can use the following program to update folder2 whenever you save new files or update folder1:
while inotifywait -r -e modify -e move -e create -e delete; do
cp -r /stuff/folder1/. /stuff/folder2/
done
For the schedule thing I would add cp -r /stuff/folder1/. /stuff/folder2/ into a cron job. Instead of cp you can also use rsync. Please also have a look on the manpage of inotifywait.
Note: The above script will start the copy after the first file was altered inside the directory folder1. If you modify many files in folder1 in the same time, you might want to put a sleep command inside the while loop. But in this case it is better to add the copy command at the end of the program which alters the files of folder1.
This is first question in Stack overflow. need an inotify compatible script writing that will monitor a certain directory, and if any new files/folders are created in in, copy those files to another folder. I need the script to monitor constantly for changes rather than run periodically.
Thanx in advance.
You can use inotifywait, from the inotify-tools page, to build something like this. A typical use:
inotifywait -m /tmp | while read path events name; do
echo "Now I am going to do something with $name in directory $path."
done
There are oodles of options for controlling how inotifywait operates; consult the man page for details.
I need to write a shell script to run as a cron task, or preferably on creation of a file in a certain folder.
I have an incoming and an outgoing folder (they will be used to log mail). There will be files created with codes as follows...
bmo-001-012-dfd-11 for outgoing and 012-dfd-003-11 for incoming. I need to filter the project/client code (012-dfd) and then place it in a folder in the specific project folder.
Project folders are located in /projects and follow the format 012-dfd. I need to create symbolic links inside the incoming or outgoing folders of the projects, that leads to the correct file in the general incoming and outgoing folders.
/incoming/012-dfd-003-11.pdf -> /projects/012-dfd/incoming/012-dfd-003-11.pdf
/outgoing/bmo-001-012-dfd-11.pdf -> /projects/012-dfd/outgoing/bmo-001-012-dfd-11.pdf
So my questions
How would I make my script run when a file is added to either incoming or outgoing folder
Additionally, is there any associated disadvantages with running upon file modification compared with running as cron task every 5 mins
How would I get the filename of recent (since script last run) files
How would I extract the code from the filename
How would I use the code to create a symlink in the desired folder
EDIT: What I ended up doing...
while inotifywait outgoing; do find -L . -type l -delete; ls outgoing | php -R '
if(
preg_match("/^\w{3}-\d{3}-(\d{3}-\w{3})-\d{2}(.+)$/", $argn, $m)
&& $m[1] && (file_exists("projects/$m[1]/outgoing/$argn") != TRUE)
){
`ln -s $(pwd)/outgoing/$argn projects/$m[1]/outgoing/$argn;`;
}
'; done;
This works quite well - cleaning up deleted symlinks also (with find -L . -type l -delete) but I would prefer to do it without the overhead of calling php. I just don't know bash well enough yet.
Some near-answers for your task breakdown:
On linux, use inotify, possibly through one of its command-line tools, or script language bindings.
See above
Assuming the project name can be extracted thinking positionally from your examples (meaning not only does the project name follows a strict 7-character format, but what precedes it in the outgoing file also does):
echo `basename /incoming/012-dfd-003-11.pdf` | cut -c 1-7
012-dfd
echo `basename /outgoing/bmo-001-012-dfd-11.pdf`| cut -c 9-15
012-dfd
mkdir -p /projects/$i/incoming/ creates directory /projects/012-dfd/incoming/ if i = 012-dfd,
ln -s /incoming/foo /projects/$i/incoming/foo creates a symbolic link from the latter argument, to the preexisting, former file /incoming/foo.
How would I make my script run when a file is added to either incoming or outgoing folder
Additionally, is there any associated disadvantages with running upon file modification compared with running as cron task
every 5 mins
If a 5 minutes delay isn't an issue, I would go for the cron job (it's easier and -IMHO- more flexible)
How would I get the filename of recent (since script last run) files
If your script runs every 5 minutes, then you can tell that all the files created in between now (and now - 5 minutes) are newso, using the command ls or find you can list those files.
How would I extract the code from the filename
You can use the sed command
How would I use the code to create a symlink in the desired folder
Once you have the desired file names, you can usen ln -s command to create the symbolic link
Im trying to use inotifywait to detect every time a file or folder gets moved into a folder in realtime (eg. /root in the case)
I tried this, which does detect both folders and files, but this is for a created file, I want it for a moved file/folder.
inotifywait --monitor --format %f --event create /root
So then I use this, but using this only sees when a folder is moved in, when I move in a file nothing is shown... :(
inotifywait --monitor --format %f --event moved_to /root
Any idea what's going on?
PS, Im using Linux, Debian 5 (Lenny).
You can specify many events with inotify. In your case it seems you need something like :
inotifywait --monitor --format %f --event move --event create /root
It should works. If you need more, read carefully the man page :
-e <event>, --event <event>
Listen for specific event(s) only.
The events which can be listened for are listed in the EVENTS section.
This option can be specified more than once.
If omitted, all events are listened for.
[...]
EVENTS
The following events are valid for use with the -e option:
[...]
move A file or directory was moved from or to a watched directory.
Note that this is actually implemented simply by listening for both moved_to
and moved_from, hence all close events received will be output as one or both
of these, not MOVE.
create A file or directory was created within a watched directory.
It works for me with move / touch. Hope it helps ...