Monitoring file changes using select() within a loop - linux

I am trying to write a program that will constantly keep track of the changes in a file and do several actions accordingly. I am using inotify and select within a loop to track file modifications in a non-blocking manner. The basic structure of the file tracking portion of my program is as follows.
#include <cstdio>
#include <signal.h>
#include <limits.h>
#include <sys/inotify.h>
#include <fcntl.h>
#include <iostream>
#include <fstream>
#include <string>
int main( int argc, char **argv )
{
const char *filename = "input.txt";
int inotfd = inotify_init();
char buffer[1];
int watch_desc = inotify_add_watch(inotfd, filename, IN_MODIFY);
size_t bufsiz = sizeof(struct inotify_event) + 1;
struct inotify_event* event = ( struct inotify_event * ) &buffer[0];
fd_set rfds;
FD_ZERO (&rfds);
struct timeval timeout;
while(1)
{
/*select() intitialisation.*/
FD_SET(inotfd,&rfds); //keyboard to be listened
timeout.tv_sec = 10;
timeout.tv_usec = 0;
int res=select(FD_SETSIZE,&rfds,NULL,NULL,&timeout);
FD_ZERO(&rfds);
printf("File Changed\n");
}
}
I checked the select manual page and reset the fd_set descriptor each time select() returns. However, whenever I modify the file (input.txt), this code just loops infinitely. I not very experienced using inotify and select, so, I am sure if the problem is with the way I use inotify or select. I would appreciate any hints and recommentations.

you have to read the contents of the buffer after the select returns. if the select() finds data in the buffer, it returns. so, perform read() on that file descriptor (inotfd). read call reads the data and returns amount of bytes it read. now, the buffer is empty and in the next iteration, the select() call waits until any data is available in the buffer.
while(1)
{
// ...
char pBuf[1024];
res=select(FD_SETSIZE,&rfds,NULL,NULL,&timeout);
read(inotfd,&pBuf, BUF_SIZE);
// ...
}

Related

What does lseek() mean for a directory file descriptor?

According to strace, lseek(fd, 0, SEEK_END) = 9223372036854775807 when fd refers to a directory. Why is this syscall succeeding at all? What does lseek() mean for a dir fd?
On my test system, if you use opendir(), and readdir() through all the entries in the directory, telldir() then returns the same value:
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <dirent.h>
int main(int argc, char *argv[]) {
int fd = open(".", O_RDONLY);
if (fd < 0) {
perror("open");
return 1;
}
off_t o = lseek(fd, 0, SEEK_END);
if (o == (off_t)-1) {
perror("lseek");
return 1;
}
printf("Via lseek: %ld\n", (long)o);
close(fd);
DIR *d = opendir(".");
if (!d) {
perror("opendir");
return 1;
}
while (readdir(d)) {
}
printf("via telldir: %ld\n", telldir(d));
closedir(d);
return 0;
}
outputs
Via lseek: 9223372036854775807
via telldir: 9223372036854775807
Quoting from the telldir(3) man page:
In early filesystems, the value returned by telldir() was a simple file offset within a directory. Modern filesystems use tree or hash structures, rather than flat tables, to represent directories. On such filesystems, the value returned by telldir() (and used internally by readdir(3)) is a "cookie" that is used by the implementation to derive a position within a directory. Application programs should treat this strictly as an opaque value, making no assumptions about its contents.
It's a magic number that indicates that the index into the directory's contents is at the end. Don't count on the number always being the same, or being portable. It's a black box. And stick with the dirent API for traversing directory contents unless you really know exactly what you're doing (Under the hood on Linux + glibc, opendir(3) calls openat(2) on the directory, readdir(3) fetches information about its contents with getdents(2), and seekdir(3) calls lseek(2), but that's just implementation details)

bus error with mmap

I get bus error (core dumped) when trying to write to memory. I want to write to a binary file using mmap() and open() functions in Linux. I want to write integers from 1 to 100 in the binary file by mapping it to memory instead of writing to the file directly.
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/mman.h>
#include <string.h>
#include <stdlib.h>
#define FILE_SIZE 0x100
int main(int argc,char *argv[])
{
int fd;
void *pmap;
printf("im here");
//fd=open(argv[1],O_RDWR|O_CREAT,S_IRUSR|S_IWUSR);
fd=open("numbers.raw",O_RDWR);
if(fd == -1)
{
perror("open");
exit(1);
}
lseek(fd,FILE_SIZE+1,SEEK_SET); //checking the file length
lseek(fd,0,SEEK_SET);//points to start of the file
//create the memory mapping
pmap = mmap(0,FILE_SIZE,PROT_WRITE,MAP_SHARED,fd,0);
if(pmap == MAP_FAILED)
{
perror("mmap") ;
close(fd);
exit(1);
}
close(fd);
for(int i=1;i<=100;i++)
sprintf(pmap,"%d",i);
return 0;
}
Your comment says you are "checking the file length" but you never check the return value of that call. I'd bet it is failing since your file is not large enough, hence the bus error later.
There are multiple other unrelated mistakes in your file as well, by
the way:
Your file size assumes 0x100 bytes are enough to store 100 integers in binary. This is not the case for 64 bit systems.
You aren't actually storing binary numbers - you are storing strings of the numbers.
You aren't advancing where you write, so you write all the numbers at the start of the file, one on top of the other.

Ptrace reset a breakpoint

I am having trouble resetting a process after I have hit a breakpoint with Ptrace. I am essentially wrapping this code in python.
I am running this on 64 bit Ubuntu.
I understand the concept of resetting the data at the location and decrementing the instruction pointer, but after I get the trap signal and do that, my process is not finishing.
Code snippet:
# Continue to bp
res = libc.ptrace(PTRACE_CONT,pid,0,0)
libc.wait(byref(wait_status))
if _wifstopped(wait_status):
print('Breakpoint hit. Signal: %s' % (strsignal(_wstopsig(wait_status))))
else:
print('Error process failed to stop')
exit(1)
# Reset Instruction pointer
data = get_registers(pid)
print_rip(data)
data.rip -= 1
res = set_registers(pid,data)
# Verify rip
print_rip(get_registers(pid))
# Reset Instruction
out = set_text(pid,c_ulonglong(addr),c_ulonglong(initial_data))
if out != 0:
print_errno()
print_text(c_ulonglong(addr),c_ulonglong(get_text(c_void_p(addr))))
And I run a PTRACE_DETACH right after returning from this code.
When I run this, it hits the breakpoint the parent process returns successfully, but the child does not resume and finish its code.
If I comment out the call to the breakpoint function it just attaches ptrace to the process and then detaches it, and the program runs fine.
The program itself is just a small c program that prints 10 times to a file.
Full code is in this paste
Is there an error anyone sees with my breakpoint code?
I ended up writing a C program that was as exact a duplicate of the python code as possible:
#include <stdio.h>
#include <stdarg.h>
#include <stdlib.h>
#include <string.h>
#include <signal.h>
#include <syscall.h>
#include <sys/ptrace.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <sys/reg.h>
#include <sys/user.h>
#include <unistd.h>
#include <errno.h>
#include <time.h>
void set_unset_bp(pid){
int wait_status;
struct user_regs_struct regs;
unsigned long long addr = 0x0000000000400710;
unsigned long long data = ptrace(PTRACE_PEEKTEXT,pid,(void *)addr,0);
printf("Orig data: 0x%016x\n",data);
unsigned long long trap = (data & 0xFFFFFFFFFFFFFF00) | 0xCC;
ptrace(PTRACE_POKETEXT,pid,(void *)addr,(void *)trap);
ptrace(PTRACE_CONT,pid,0,0);
wait(&wait_status);
if(WIFSTOPPED(wait_status)){
printf("Signal recieved: %s\n",strsignal(WSTOPSIG(wait_status)));
}else{
perror("wait");
}
ptrace(PTRACE_POKETEXT,pid,(void *)addr,(void *)data);
ptrace(PTRACE_GETREGS,pid,0,&regs);
regs.rip -=1;
ptrace(PTRACE_SETREGS,pid,0,&regs);
data = ptrace(PTRACE_PEEKTEXT,pid,(void *)addr,0);
printf("Data after resetting bp data: 0x%016x\n",data);
ptrace(PTRACE_CONT,pid,0,0);
}
int main(void){
//Fork child process
extern int errno;
int pid = fork();
if(pid ==0){//Child
ptrace(PTRACE_TRACEME,0,0,0);
int out = execl("/home/chris/workspace/eliben-debugger/print","/home/chris/workspace/eliben-debugger/print",0);
if(out != 0){
printf("Error Value is: %s\n", strerror(errno));
}
}else{ //Parent
wait(0);
printf("Got stop signal, we just execv'd\n");
set_unset_bp(pid);
printf("Finished setting and unsetting\n");
wait(0);
printf("Got signal, detaching\n");
ptrace(PTRACE_DETACH,pid,0,0);
wait(0);
printf("Parent exiting after waiting for child to finish\n");
}
exit(0);
}
After comparing the output to my Python output I noticed that according to python my original data was 0xfffffffffffe4be8 and 0x00000000fffe4be8.
This lead me to believe that my return data was getting truncated to a 32 bit value.
I changed my get and set methods to something like this, setting the return type to a void pointer:
def get_text(addr):
restype = libc.ptrace.restype
libc.ptrace.restype = c_void_p
out = libc.ptrace(PTRACE_PEEKTEXT,pid,addr, 0)
libc.ptrace.restype = restype
return out
def set_text(pid,addr,data):
return libc.ptrace(PTRACE_POKETEXT,pid,addr,data)
Can't tell you how it works yet, but I was able to get the child process executing successfully after the trap.

Bus error opening and mmap'ing a file

I want to create a file and map it into memory. I think that my code will work but when I run it I'm getting a "bus error". I searched google but I'm not sure how to fix the problem. Here is my code:
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <errno.h>
#include <sys/types.h>
#include <unistd.h>
#include <sys/mman.h>
#include <string.h>
int main(void)
{
int file_fd,page_size;
char buffer[10]="perfect";
char *map;
file_fd=open("/tmp/test.txt",O_RDWR | O_CREAT | O_TRUNC ,(mode_t)0600);
if(file_fd == -1)
{
perror("open");
return 2;
}
page_size = getpagesize();
map = mmap(0,page_size,PROT_READ | PROT_WRITE,MAP_SHARED,file_fd,page_size);
if(map == MAP_FAILED)
{
perror("mmap");
return 3;
}
strcpy(map, buffer);
munmap(map, page_size);
close(file_fd);
return 0;
}
You are creating a new zero sized file, you can't extend the file size with mmap. You'll get a bus error when you try to write outside the content of the file.
Use e.g. fallocate() on the file descriptor to allocate room in the file.
Note that you're also passing the page_size as the offset to mmap, which doesn't seem to make much sense in your example, you'll have to first extend the file to pagesize + strlen(buffer) + 1 if you want to write buf at that location. More likely you want to start at the beginning of the file, so pass 0 as the last argument to mmap.

Ordering file location on linux partition

I have a process which processes a lot of files (~96,000 files, ~12 TB data). Several runs of the process has left the files scattered about the drive. Each iteration in the process, uses several files. This leads to a lot of whipsawing around the disk collecting the files.
Ideally, I would like the process to write the files it uses in order, so that the next run will read them in order (file sizes change). Is there a way to hint at a physical ordering/grouping, short of writing to the raw partition?
Any other suggestions would be helpful.
Thanks
There are two system calls you might lookup: fadvise64, fallocate tell the kernel how you intend to read or write a given file.
Another tip is the "Orlov block allocator" (Wikipedia, LWN) affects the way the kernel will allocate new directories and file-entries.
In the end I decided not to worry about writing the files in any particular ordering. Instead, before I started a run, I would figure out where the first block of each file was located, and then sort the file processing order by first block location. Not perfect, but it did make a big difference in processing times.
Here's the C code I used to get the first block of supplied file list I adapted it from example code I found online (can't seem to find the original source).
#include <stdio.h>
#include <sys/ioctl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <assert.h>
#include <unistd.h>
#include <string.h>
#include <errno.h>
#include <linux/fs.h>
//
// Get the first block for each file passed to stdin,
// write filename & first block for each file to stdout
//
int main(int argc, char **argv) {
int fd;
int block;
char fname[512];
while(fgets(fname, 511, stdin) != NULL) {
fname[strlen(fname) - 1] = '\0';
assert(fd=open(fname, O_RDONLY));
block = 0;
if (ioctl(fd, FIBMAP, &block)) {
printf("FIBMAP ioctl failed - errno: %s\n", strerror(errno));
}
printf("%010d, %s\n", block, fname);
close(fd);
}
return 0;
}

Resources