I write a Python app that supports Linux and Windows, and I have a file object (that has a fileno() method returning a file descriptor), and I need to wait for IO completion on it.
In this particular case it's database connection (Postgresql), and I need to process notifications without a load on CPU (like it is described here).
On Linux it's quite simple:
select.select([fd], [], [])
But I would like to be able to do something like that on Windows too, and I'm struggling to find the correct way to achieve it.
It looks like WaitForSingleObject function could be handy, but as far as I understand it does not accept file descriptors.
How could this possibly be resolved?
Related
In Linux one is able to asynchronously read from a file by calling aio_read(3) from C. A sigevent structure is one of the parameters and there are different options one can specify to be notified when the operation is complete. Let me summarize:
SIGEV_NONE no notification.
The status can be checked with aio_error(3). The operation is async, but completion must be busily awaited in some loop which is not what I want.
SIGEV_SIGNAL a signal is raised to the process.
In theory, this can be caught in Haskell by installing a signal handler via System.Posix.Signals. There is a problem though: the API of SignalInfo doesn't include the crucial si_value that let's one communicate some specifics about the read request, like a StaticPtr. This is unfortunate.
SIGEV_THREAD this would start a new thread, according to the documentation.
I don't know how to represent this in Haskell. My best guess would be an IO () action. I'm not sure how to write the accompanying native code.
How can I use aio_read or something of that sort in Haskell? I will probably not get around using FFI on this (or a library).
Ηi,
Say I have a file called something.txt. I would like to find the most recent program to modify it, specifically the full path to said program (eg. /usr/bin/nano). I only need to worry about files modified while my program is running, so I can add an event listener at program startup and find out what program modified it when my program was running.
Thanks!
auditd in Linux could perform actions regarding file modifications
See the following URI xmodulo.com/how-to-monitor-file-access-on-linux.html
Something like this generally isn't going to be possible for arbitrary processes. If these aren't arbitrary processes, then you could use some sort of network bus (e.g. redis) to publish "write" messages. Otherwise your only other bet would be to implement your own filesystem using FUSE. Even with FUSE though, you may not always have access to the pid depending on who/what is writing to the file and the security setup of your OS.
I'd like to be able to intercept filenames with a certain prefix from any of my child processes that I launch. This would be names like "pipe://pipe_name". I think wrapping the open() system call would be a good way to do this for my application, but I'd like to do it without having to compile a separate shared library and hooking it with the LD_PRELOAD trick (or using FUSE and having to have a mounted directory)
I'll be forking the processes myself, is there a way to redirect open() to my own function before forking and have it persist in the child after an exec()?
Edit: The thought behind this is that I want to implement multi-reader pipes by having an intermediate process tee() the data from one pipe into all the others. I'd like this to be transparent to my child processes, so that they can take a filename and open() it, and, if it's a pipe, I'll return the file descriptor for it, while if it's a normal file, I'll just pass that to the regular open() function. Any alternative way to do this that makes it transparent to the child processes would interesting to hear. I'd like to not have to compile a separate library that has to be pre-linked though.
I believe the answer here is no, it's not possible. To my knowledge, there's only three ways to achieve this:
LD_PRELOAD trick, compile a .so that's pre-loaded to override the system call
Implement a FUSE filesystem and pass a path into it to the client program, intercept calls.
Use PTRACE to intercept system calls, and fix them up as needed.
2 and 3 will be very slow as they'll have to intercept every I/O call and swap back to user space to handle it. Probably 500% slower than normal. 1 requires building and maintaining an external shared library to link in.
For my application, where I'm just wanting to be able to pass in a path to a pipe, I realized I can open both ends in my process (via pipe()), then go grab the path to the read end in /proc//fd and pass that path to the client program, which gives me everything I need I think.
I am trying to communicate with an arduino using the code from,
http://todbot.com/blog/2006/12/06/arduino-serial-c-code-to-talk-to-arduino/
which uses termios to open and talk to an arduino in a non blocking way which works fine. What I was wondering is when on Java using rxtx I can register a callback function that is called when ever there is data on the line so that I don't have to manually check. I googled but could not find any info on how to register a callback function?
I am on Linux/OS X and I am trying to avoid launching a new thread just to watch/read data on the line.
Posix itself does not define a way to assign a callback function to be called when serial data arrives. The standard way to do this type of processing if you don't want to use a seperate thread is to use the select library function. This allows you program to define a set of file descriptors your program is interested in and then go to sleep. Select will automatically wake up your process if something interesting happens to one of the file descriptors you've declared interest in (such as new data becoming available for reading). This avoids having to busy-wait and poll multiple descriptors for activity.
Another option would be to use a library like libevent which sits on top of the Posix layer and provides the callback infrastructure. http://monkey.org/~provos/libevent/
Boost.Asio can provide callback functionality when using serial ports. It runs on Linux and Mac OS X.
I am new to VC++ and programming. I have a task in which I am supposed to design a file watcher in VC++.
The problem goes this way:
I have to monitor some log files continously; whenever a particular log file gets deleted(this deletion is done by some other program), I have to open a TextFile and write some data and the timestamp into it.
How do I go about it? Please help!!
First, you need to setup a system to monitor for file events from that folder.
To get started, take a look at FindFirstChangeNotification().
You'll basically get a waitable handle from that.
Then, were it me, I'd have a thread that waited on that event. Each time the event triggers, the thread resumes, queries for the change details (what file), then perform the needed actions, and resume sleeping on that handle again.
You'll need some additional semaphore or something to use to interrupt this worker-thread and wake it so that you can tell it to quit. Simple to do: have your thread's main loop do a WaitForMultipleObjects - the "wake up semaphore" and the FindFirstChangeNotification handle. When you wake up, check which even notified you, then either process the file change or quit.
MFC has a slightly different way of handling it (slightly) but to do this using the Win32 API what you'd typcially do is use the Directory Management Functions to set up a change notification handle for the directory the file goes in. Then you can wait on the handle and when something happens inside that directory your wait completes, and you can check to see if it was a change to the file that you care about.
Look at the docs for FindFirstChangeNotification and ReadDirectoryChangesW for more information.
Try the Windows Management Instrumentation (WMI) if you have enough privileges. AFAIK it is also the most efficient way to handle the filesystem events.
Handle or query the __InstanceDeletionEvent, __InstanceModificationEvent or __InstanceCreationEvent for the deletion, modification or creation events respectively and filter the files and target path that you want.
Take a look at the WMI Reference/C++ invocation.
For a full-scale example take a look at codeproject querying example.
I strongly recommmend you consider using the implementation here. This API is not 100% reliable, but this code does a good job of wrapping it. If your filesystem traffic is local and not too frequent, it should work well for you.