is popen really thread safe? - multithreading

Recently, I am running into trouble of calling popen, it seems to be not thread safe.
The following is the code snippet from the source code link: http://androidxref.com/9.0.0_r3/xref/bionic/libc/upstream-netbsd/lib/libc/gen/popen.c
static struct pid {
struct pid *next;
FILE *fp;
int fd;
pid_t pid;
} *pidlist;
static rwlock_t pidlist_lock = RWLOCK_INITIALIZER;
FILE *
popen(const char *command, const char *type)
{
struct pid *cur, *old;
...
pipe2(pdes, flags) // A
...
(void)rwlock_rdlock(&pidlist_lock); // C
...
switch (pid = vfork()) { // C.1
case 0: /* Child. */
...
_exit(127);
/* NOTREACHED */
}
/* Parent; */
...
/* Link into list of file descriptors. */
cur->fp = iop;
cur->pid = pid;
cur->next = pidlist; // D
pidlist = cur; // E
(void)rwlock_unlock(&pidlist_lock); // F
...
}
Observing the above code, it acquired a read-lock at C, but, inside the scope, it did some write operations at E. So, there will be possibly multiple read threads who are writing to the variable "pidlist" in the same time.
Does anyone know whether it is a true issue or not?

Looking at the code, it is not how you outline it.
Here is a paste from the given link http://androidxref.com/9.0.0_r3/xref/bionic/libc/upstream-netbsd/lib/libc/gen/popen.c
80FILE *
81popen(const char *command, const char *type)
82{
83 struct pid *cur, *old;
94 if (strchr(xtype, '+')) {
...
100 } else {
103 if (pipe2(pdes, flags) == -1)
104 return NULL;
105 }
106
...
113
114 (void)rwlock_rdlock(&pidlist_lock);
115 (void)__readlockenv();
116 switch (pid = vfork()) {
117 case -1: /* Error. */
...
127 case 0: /* Child. */
...
154 execl(_PATH_BSHELL, "sh", "-c", command, NULL);
155 _exit(127);
156 /* NOTREACHED */
157 }
158 (void)__unlockenv();
159
160 /* Parent; assume fdopen can't fail. */
161 if (*xtype == 'r') {
...
167 } else {
...
173 }
...
179 pidlist = cur;
180 (void)rwlock_unlock(&pidlist_lock);
181
182 return (iop);
183}
It is
do the file stuff (pipe2() or socketpair())
acquire the lock (this will avoid the child doing silly things)
vfork()
on the child, reorganize filedes and execl
on parent do some filedes stuff unlock and return
I do not see what would be the problem.

Related

Linux kernel timer init_timer. What happens if called many times?

I am trying to hunt a bug that cause intermittent crash at PC around get_next_timer_interrupt() code and sometimes at run_timer_softirq()
I found a driver that potentially calls init_timer() often with the same static argument passed to it. (timer_list)
Will this cause issue?
What exactly does init_timer do and is there a function that does the reverse to destroy it?
Thanks
Calling init_timer() a lot should not cause any problem. The code which is eventually invoked is:
621 static void do_init_timer(struct timer_list *timer, unsigned int flags,
622 const char *name, struct lock_class_key *key)
623 {
624 struct tvec_base *base = __raw_get_cpu_var(tvec_bases);
625
626 timer->entry.next = NULL;
627 timer->base = (void *)((unsigned long)base | flags);
628 timer->slack = -1;
629 #ifdef CONFIG_TIMER_STATS
630 timer->start_site = NULL;
631 timer->start_pid = -1;
632 memset(timer->start_comm, 0, TASK_COMM_LEN);
633 #endif
634 lockdep_init_map(&timer->lockdep_map, name, key, 0);
635 }
That gets invoked by a few macros calling down to it. Start here to follow the flow.

std::condition_variable::wait Access Violation

I'm currently programming a concurrent queue while learning how to use the multithreading features of C++11.
When the consumer calls the dequeue() function and the queue has no entries, the function should wait until another thread calls enqueue(). I'm using a condition_variable for that.
My tests worked fine with a few entries and threads, but when I'm using more (up to 100000 elements, 20 producers, only 1 consumer), I get an Access Violation inside the condition_variable::wait function:
Unbehandelte Ausnahme bei 0x5A2C7EEC (msvcr110d.dll) in Tests.exe:
0xC0000005: Zugriffsverletzung beim Lesen an Position 0xFEEEFEF6
I've been stuck with this for hours. I hope you can help me. Thank you.
The Code:
// --------------------------------------------------------------------------------------
// Concurrent Queue
// --------------------------------------------------------------------------------------
#pragma once
#include <atomic> // Atomic operations for lock-free operations
#include <mutex>
#include <condition_variable>
using namespace std;
// --------------------------------------------------------------------------------------
// Declarations
// --------------------------------------------------------------------------------------
template<typename T>
class ConcurrentQueue;
template<typename T>
class ConcurrentQueueEntry;
// --------------------------------------------------------------------------------------
// Queue
// --------------------------------------------------------------------------------------
template<typename T>
class ConcurrentQueue {
public:
ConcurrentQueue();
~ConcurrentQueue();
void enqueue(const T value);
T try_dequeue();
T dequeue();
unsigned long count() const;
private:
atomic<ConcurrentQueueEntry<T>*> front;
atomic<ConcurrentQueueEntry<T>*> rear;
atomic_ulong i_count;
mutex dequeueWaitMutex;
condition_variable dequeueWaitCV;
};
// --------------------------------------------------------------------------------------
// Entry
// --------------------------------------------------------------------------------------
template<typename T>
class ConcurrentQueueEntry {
public:
ConcurrentQueueEntry(T _value);
T value;
atomic<ConcurrentQueueEntry<T>*> next;
};
// --------------------------------------------------------------------------------------
// Exception: Queue is empty
// --------------------------------------------------------------------------------------
class EmptyQueueException {};
// --------------------------------------------------------------------------------------
// Constructors and Destructor
// --------------------------------------------------------------------------------------
// Create Queue
template<typename T>
ConcurrentQueue<T>::ConcurrentQueue()
: front(), rear(), i_count(), dequeueWaitMutex(), dequeueWaitCV()
{
i_count.store(0);
}
// Delete Queue
template<typename T>
ConcurrentQueue<T>::~ConcurrentQueue()
{
ConcurrentQueueEntry<T>* previous = this->front.load();
while(previous != NULL) {
ConcurrentQueueEntry<T>* next = previous->next.load();
delete previous;
previous = next;
}
}
// Create Entry
template<typename T>
ConcurrentQueueEntry<T>::ConcurrentQueueEntry
(T _value)
: value(_value), next(NULL)
{ }
// --------------------------------------------------------------------------------------
// Public Methods
// --------------------------------------------------------------------------------------
// Enqueue
template<typename T>
void ConcurrentQueue<T>::enqueue
(const T value)
{
// create, append
ConcurrentQueueEntry<T>* entry = new ConcurrentQueueEntry<T>(value);
ConcurrentQueueEntry<T>* former_rear = this->rear.exchange(entry);
// connect
if(former_rear == NULL) {
this->front.store(entry);
}
else {
former_rear->next.store(entry);
}
// Add
++i_count;
dequeueWaitCV.notify_one();
}
// Dequeue (aborts if queue is empty)
template<typename T>
T ConcurrentQueue<T>::try_dequeue()
{
ConcurrentQueueEntry<T>* front = this->front.load();
while(front != NULL &&
!this->front.compare_exchange_weak(front, front->next.load()));
if(front == NULL)
throw EmptyQueueException();
--i_count;
T value = front->value;
delete front;
return value;
}
// Dequeue (waits if queue is empty)
template<typename T>
T ConcurrentQueue<T>::dequeue() {
while(true) {
try {
return this->try_dequeue();
}
catch(EmptyQueueException) {
unique_lock<mutex> lock(dequeueWaitMutex);
dequeueWaitCV.wait(lock, [&] { return this->count() == 0; });
}
}
}
// Count entries
template<typename T>
unsigned long ConcurrentQueue<T>::count() const {
return this->i_count.load();
}
The Call Stack:
msvcr110d.dll!Concurrency::details::LockQueueNode::IsTicketValid() Zeile 924 C++
msvcr110d.dll!Concurrency::details::LockQueueNode::UpdateQueuePosition(Concurrency::details::LockQueueNode * pPreviousNode) Zeile 811 C++
msvcr110d.dll!Concurrency::critical_section::_Acquire_lock(void * _PLockingNode, bool _FHasExternalNode) Zeile 1193 C++
msvcr110d.dll!Concurrency::critical_section::lock() Zeile 1028 C++
msvcr110d.dll!Concurrency::details::_Condition_variable::wait(Concurrency::critical_section & _Lck) Zeile 576 C++
msvcp110d.dll!do_wait(_Cnd_internal_imp_t * * cond, _Mtx_internal_imp_t * * mtx, const xtime * target) Zeile 47 C++
msvcp110d.dll!_Cnd_wait(_Cnd_internal_imp_t * * cond, _Mtx_internal_imp_t * * mtx) Zeile 73 C++
Tests.exe!std::_Cnd_waitX(_Cnd_internal_imp_t * * _Cnd, _Mtx_internal_imp_t * * _Mtx) Zeile 93 C++
Tests.exe!std::condition_variable::wait(std::unique_lock<std::mutex> & _Lck) Zeile 60 C++
Tests.exe!std::condition_variable::wait<<lambda_61c2d1dffb87d02ed418fe62879bb063> >(std::unique_lock<std::mutex> & _Lck, ConcurrentQueue<long>::dequeue::__l7::<lambda_61c2d1dffb87d02ed418fe62879bb063> _Pred) Zeile 67 C++
Tests.exe!ConcurrentQueue<long>::dequeue() Zeile 156 C++
Tests.exe!<lambda_c8c79a4136723f6fef9d0a0557ed768b>::operator()() Zeile 38 C++
Tests.exe!std::_Bind<0,void,<lambda_c8c79a4136723f6fef9d0a0557ed768b>,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil>::operator()() Zeile 1152 C++
Tests.exe!std::_LaunchPad<std::_Bind<0,void,<lambda_c8c79a4136723f6fef9d0a0557ed768b>,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil> >::_Run(std::_LaunchPad<std::_Bind<0,void,<lambda_c8c79a4136723f6fef9d0a0557ed768b>,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil> > * _Ln) Zeile 196 C++
Tests.exe!std::_LaunchPad<std::_Bind<0,void,<lambda_c8c79a4136723f6fef9d0a0557ed768b>,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil,std::_Nil> >::_Go() Zeile 187 C++
msvcp110d.dll!_Call_func(void * _Data) Zeile 52 C++
msvcr110d.dll!_callthreadstartex() Zeile 354 C
msvcr110d.dll!_threadstartex(void * ptd) Zeile 337 C
kernel32.dll!747f850d() Unbekannt
[Unten angegebene Rahmen sind möglicherweise nicht korrekt und/oder fehlen, keine Symbole geladen für kernel32.dll]
ntdll.dll!7719bf39() Unbekannt
ntdll.dll!7719bf0c() Unbekannt
Two notes up front for debugging:
0xFEEEFEF6 is 0xFEEEFEEE + 8. The 0xFEEEFEEE is a signal value which the debug runtime stores in some variables. I think that this value indicates that the owning object's destructor was already called. In contrast, before initialization, the value is 0xCDCDCDCD I think.
You can configure VS to turn off internationalization. That way you get English error messages, which are more suitable for the forum here.
Now, concerning your code, the first thing is that this is not "the code" but rather just part thereof. Distil a minimal example, please. However, in the meantime, you could make your queue class uncopyable and unassignable (using ".. = delete;"). If I'm guessing right, you will then get compile errors which come from copying the queue, probably during thread startup.

QThread crashes the program?

I implement QThread like this, but get program crashed when it runs.
I've searched and seen posts saying it is not the correct way to use QThread.
But I cannot find any reason for the crashes of my program, what I do is only
triggering 'on_Create_triggered()' and I guarantee the mutex is locked and unlocked properly.
I have tested the program for two days(testing only by 'std::cerr << ...;' prints results), but still cannot find reason. What I guess is that the thread may wait for the lock too long and cause program to crash. (not sounds reasonable...) :)
My codes:
Background.h
class Background : public QThread
{
Q_OBJECT
public:
Background(int& val,DEVMAP& map, QQueue<LogInfoItem*>& queue, QList<DEV*>& devlist, QList<IconLabel*>& icllist,QMutex& m)
:val_i(val),DevMap(map), LogInfoQueue(queue), DevInfoList(devlist), IconLabelList(icllist),mutex(m)
{}
~Background();
protected:
void run(void);
private:
DEVMAP& DevMap;
QQueue<LogInfoItem*>&LogInfoQueue;
QList<DEV*>& DevInfoList;
QList<IconLabel*>& IconLabelList;
int& val_i;
QMutex& mutex;
void rcv();
};
Background.cpp
#include "background.h"
Background::~Background()
{
LogFile->close();
}
void Background::run(void)
{
initFile();
while(1)
{
msleep(5);
rcv();
}
}
void Background::rcv()
{
mutex.lock();
...
...//access DevMap, LogInfoQueue, DevInfoList, IconLabelList and val_i;
...
mutex.unlock();
}
MainWindow:(MainWindow has Background* back as property)
void MainWindow::initThread()
{
back = new Background(val_i, dev_map, logDisplayQueue, devInfoList, iconLabelList, mutex);
back->start();
}
void MainWindow::on_Create_triggered()
{
mutex.lock();
...
...//access DevMap, LogInfoQueue, DevInfoList, IconLabelList and val_i;
...
mutex.unlock();
}
I have found the reason, which is more subtle.
(I use some codes written by others but believe it is not broken, what I got totally wrong! :) )
The broken codes:
#define DATABUFLEN 96
typedef struct Para//totally 100bytes
{
UINT8 type;
UINT8 len;
UINT8 inType;
UINT8 inLen;
UINT8 value[DATABUFLEN];//96 bytes here
}ERRORTLV;
class BitState
{
public:
UINT8 dataBuf[DATABUFLEN];
......
};
And the function using it:
bool BitState::rcvData() //the function crosses bound of array
{
UINT8 data[12] =
{
0x72, 0x0A, 0x97, 0x08,
0x06, 0x0A, 0x0C, 0x0F,
0x1E, 0x2A, 0x50, 0x5F,
}; //only 12 bytes
UINT32 dataLen = 110;
memcpy(this->dataBuf, data, dataLen); //copy 110 bytes to dataBuf //but no error or warning from compiler, and no runtime error indicates the cross
}
bool BitState::parseData(BitLog* bitLog)//pass pointer of dataBuf to para_tmp, but only use 0x08 + 4 = 12 bytes of dataBuf
{
Para* para_tmp;
if(*(this->dataBuf) == 0x77)
{
para_tmp = (ERRORTLV*)this->dataBuf;
}
if(para_tmp->type != 0x72 || para_tmp->inType != 0x97 || (para_tmp->len - para_tmp->inLen) != 2) // inLen == 0x08
{
return false;
}
else
{
//parse dataBuf according to Para's structure
this->bitState.reset();
for(int i = 0; i < para_tmp->inLen; i++) // inLen == 0x08 only !!!
{
this->bitState[para_tmp->value[i]-6] = 1;
}
if(this->bitState.none())
this->setState(NORMAL);
else
this->setState(FAULT);
QString currentTime = (QDateTime::currentDateTime()).toString("yyyy.MM.dd hh:mm:ss.zzz");
string sysTime = string((const char *)currentTime.toLocal8Bit());
this->setCurTime(sysTime);
this->addLog(sysTime, bitLog);
}
return true;
}
bool BitState::addLog(std::string sysTime, BitLog* bitLog)// this function is right
{
bitLog->basicInfo = this->basicInfo;//not in data Buf, already allocated and initialized, (right)
bitLog->bitState = this->bitState; //state is set by setState(..)
bitLog->rcvTime = sysTime; //time
return true;
}
Generally speaking, the program allocates 96 bytes to a byte array, but use 'memcpy(...)' to copy 110 bytes to the array, later uses only 12 bytes of the array.
All kinds of crashes appear, which are confusing and frustrating...:( :( :(

multi-thread opening file hangs when fanotify is on

I use this fanotify sample to monitor open/access perms on the whole file system(/): http://git.infradead.org/users/eparis/fanotify-example.git.
Then I have a test program with multiple threads, each thread iterate the sample foder and open/close the files in it, sometimes my program hangs at open().
OS: Ubuntu 2.6.38-11 x86_64.
Is it a bug of fanotify that it does not support multiple-thread opening?
The code of my test program:
#include <stdlib.h>
#include <stdio.h>
#include <pthread.h>
#include <ctype.h>
#include <unistd.h>
#include <dirent.h>
#include <string.h>
#include <sys/stat.h>
#include <errno.h>
#include <fcntl.h>
#include <dirent.h>
//open file function
void open_file( char* file )
{
int fd = -1;
fd = open( file, O_WRONLY, 0x666 );
if( fd >= 0 )
{
printf("open:%s\n", file );
close( fd );
}
}
//iterate directory function
void printdir(char *dir, int depth)
{
DIR *dp;
struct stat statbuf;
char pathbuf[2048] = {0};
struct dirent entry;
struct dirent *entryPtr = NULL;
//printf("opendir %s\n", dir );
usleep( 300 );
if((dp = opendir(dir)) == NULL) {
if( errno != ENOTDIR )
{
fprintf(stderr,"cannot open directory: %s\n", dir);
perror("open fial");
}
return;
}
readdir_r( dp, &entry, &entryPtr );
while( entryPtr != NULL)
{
snprintf(pathbuf,2000, "%s/%s\0", dir, entry.d_name );
printf("iteraotr:%s\n", pathbuf );
lstat( pathbuf, &statbuf );
if(S_ISDIR( statbuf.st_mode ))
{
/* Found a directory, but ignore . and .. */
if(strcmp(".",entry.d_name) == 0 ||
strcmp("..",entry.d_name) == 0)
{
}
else
{
//printf("%d,%s\n",depth, entry->d_name);
printdir( pathbuf, depth+1);
}
}
else
{
//printf("%*s%s\n",depth,"",entry->d_name);
open_file( pathbuf );
}
readdir_r( dp, &entry, &entryPtr );
}
closedir(dp);
}
//thread function
void* iterator_dir( void* data )
{
char* path = (char*)data;
printf("In iterator_dir(): %s\n", path );
printdir( path, 0 );
return NULL;
}
pthread_t threadID[10] = {0};
//main function
int main( int argc, char** argv )
{
if( argc < 3 )
{
printf("Usage: %s <thread_num> <file>\n", argv[0] );
exit(0);
}
if( isdigit( (char)*argv[1] ) == 0 )
{
printf(" Thread num is 0 - 9\n");
exit(0);
}
int thread_num = atoi( argv[1] );
char* res;
pthread_attr_t attr;
pthread_attr_init(&attr);
int i = 0;
for( i = 0; i < thread_num; ++i )
{
pthread_create( &threadID[i], &attr, &iterator_dir, argv[2]);
}
for( i = 0; i < thread_num; ++i )
{
pthread_join( threadID[i] , &res );
}
}
2011-09-28 Edit:
I comment the open file operation, only keep the iterate directory part. The application still hangs.
This is the output of strace:
enter code here
pid 10692] open("/home/byang//.config/menus", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC <unfinished ...>
[pid 10691] write(1, "1213966080 opendir /home/byang//"..., 56) = 56
.........
[pid 10689] madvise(0x7f3c48dbc000, 8368128, MADV_DONTNEED) = 0
[pid 10689] _exit(0) = ?
Process 10689 detached
[pid 10688] <... futex resumed> ) = 0
[pid 10688] futex(0x7f3c47db99d0, FUTEX_WAIT, 10692, NULL <unfinished ...>
It hangs here, when I close the fanotify, it continues...
[pid 10692] <... open resumed> ) = 11
[pid 10692] getdents(11, /* 4 entries */, 32768) = 128
[pid 10692] lstat("/home/byang//.config/menus/applications.menu", {st_mode=S_IFREG|0644, st_size=233, ...}) = 0
10688 is the parent thread; 10689,10691,10692 are the child threads that iterating the directories.
It seems that 10692 are waiting the reply from fanotify?
It's a bug of Kernel's fanotify.
I posted a patch to Linux-Kernel:
When multiple threadsiterate the same direcotry, some thread will hang.
This patch let fanotify differentiate access events from different
threads, prevent fanotify from merging access events from different
threads.
http://marc.info/?l=linux-kernel&m=131822913806350&w=2
-------------------------------
diff -r -u linux-3.1-rc4_orig/fs/notify/fanotify/fanotify.c
linux-3.1-rc4/fs/notify/fanotify/fanotify.c
--- linux-3.1-rc4_orig/fs/notify/fanotify/fanotify.c 2011-08-29
12:16:01.000000000 +0800
+++ linux-3.1-rc4/fs/notify/fanotify/fanotify.c 2011-10-10
12:28:23.276847000 +0800
## -15,7 +15,8 ##
if (old->to_tell == new->to_tell &&
old->data_type == new->data_type &&
- old->tgid == new->tgid) {
+ old->tgid == new->tgid &&
+ old->pid == new->pid) {
switch (old->data_type) {
case (FSNOTIFY_EVENT_PATH):
if ((old->path.mnt == new->path.mnt) &&
## -144,11 +145,19 ##
return PTR_ERR(notify_event);
#ifdef CONFIG_FANOTIFY_ACCESS_PERMISSIONS
- if (event->mask & FAN_ALL_PERM_EVENTS) {
- /* if we merged we need to wait on the new event */
- if (notify_event)
- event = notify_event;
- ret = fanotify_get_response_from_access(group, event);
+ //if overflow, do not wait for response
+ if(fsnotify_isoverflow(event))
+ {
+ pr_debug("fanotify overflow!\n");
+ }
+ else
+ {
+ if (event->mask & FAN_ALL_PERM_EVENTS) {
+ /* if we merged we need to wait on the new event */
+ if (notify_event)
+ event = notify_event;
+ ret = fanotify_get_response_from_access(group, event);
+ }
}
#endif
diff -r -u linux-3.1-rc4_orig/fs/notify/notification.c
linux-3.1-rc4/fs/notify/notification.c
--- linux-3.1-rc4_orig/fs/notify/notification.c 2011-08-29
12:16:01.000000000 +0800
+++ linux-3.1-rc4/fs/notify/notification.c 2011-10-10 12:27:09.331787000 +0800
## -95,6 +95,7 ##
BUG_ON(!list_empty(&event->private_data_list));
kfree(event->file_name);
+ put_pid(event->pid);
put_pid(event->tgid);
kmem_cache_free(fsnotify_event_cachep, event);
}
## -132,6 +133,14 ##
return priv;
}
+bool fsnotify_isoverflow(struct fsnotify_event *event)
+{
+ if(event==q_overflow_event)
+ {
+ return true;
+ }
+ return false;
+}
/*
* Add an event to the group notification queue. The group can later pull this
* event off the queue to deal with. If the event is successfully added to the
## -374,6 +383,7 ##
return NULL;
}
}
+ event->pid = get_pid(old_event->pid);
event->tgid = get_pid(old_event->tgid);
if (event->data_type == FSNOTIFY_EVENT_PATH)
path_get(&event->path);
## -417,6 +427,7 ##
event->name_len = strlen(event->file_name);
}
+ event->pid = get_pid(task_pid(current));
event->tgid = get_pid(task_tgid(current));
event->sync_cookie = cookie;
event->to_tell = to_tell;
diff -r -u linux-3.1-rc4_orig/include/linux/fsnotify_backend.h
linux-3.1-rc4/include/linux/fsnotify_backend.h
--- linux-3.1-rc4_orig/include/linux/fsnotify_backend.h 2011-08-29
12:16:01.000000000 +0800
+++ linux-3.1-rc4/include/linux/fsnotify_backend.h 2011-10-10
12:27:48.587369000 +0800
## -238,6 +238,7 ##
u32 sync_cookie; /* used to corrolate events, namely inotify mv events */
const unsigned char *file_name;
size_t name_len;
+ struct pid *pid;
struct pid *tgid;
#ifdef CONFIG_FANOTIFY_ACCESS_PERMISSIONS
## -378,6 +379,8 ##
struct fsnotify_event_private_data *priv,
struct fsnotify_event *(*merge)(struct list_head *,
struct fsnotify_event *));
+/*true if the event is an overflow event*/
+extern bool fsnotify_isoverflow(struct fsnotify_event *event);
/* true if the group notification queue is empty */
extern bool fsnotify_notify_queue_is_empty(struct fsnotify_group *group);
/* return, but do not dequeue the first event on the notification queue */

Reading a child process's /proc/pid/mem file from the parent

In the program below, I am trying to cause the following to happen:
Process A assigns a value to a stack variable a.
Process A (parent) creates process B (child) with PID child_pid.
Process B calls function func1, passing a pointer to a.
Process B changes the value of variable a through the pointer.
Process B opens its /proc/self/mem file, seeks to the page containing a, and prints the new value of a.
Process A (at the same time) opens /proc/child_pid/mem, seeks to the right page, and prints the new value of a.
The problem is that, in step 6, the parent only sees the old value of a in /proc/child_pid/mem, while the child can indeed see the new value in its /proc/self/mem. Why is this the case? Is there any way that I can get the parent to to see the child's changes to its address space through the /proc filesystem?
#include <fcntl.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/wait.h>
#include <unistd.h>
#define PAGE_SIZE 0x1000
#define LOG_PAGE_SIZE 0xc
#define PAGE_ROUND_DOWN(v) ((v) & (~(PAGE_SIZE - 1)))
#define PAGE_ROUND_UP(v) (((v) + PAGE_SIZE - 1) & (~(PAGE_SIZE - 1)))
#define OFFSET_IN_PAGE(v) ((v) & (PAGE_SIZE - 1))
# if defined ARCH && ARCH == 32
#define BP "ebp"
#define SP "esp"
#else
#define BP "rbp"
#define SP "rsp"
#endif
typedef struct arg_t {
int a;
} arg_t;
void func1(void * data) {
arg_t * arg_ptr = (arg_t *)data;
printf("func1: old value: %d\n", arg_ptr->a);
arg_ptr->a = 53;
printf("func1: address: %p\n", &arg_ptr->a);
printf("func1: new value: %d\n", arg_ptr->a);
}
void expore_proc_mem(void (*fn)(void *), void * data) {
off_t frame_pointer, stack_start;
char buffer[PAGE_SIZE];
const char * path = "/proc/self/mem";
int child_pid, status;
int parent_to_child[2];
int child_to_parent[2];
arg_t * arg_ptr;
off_t child_offset;
asm volatile ("mov %%"BP", %0" : "=m" (frame_pointer));
stack_start = PAGE_ROUND_DOWN(frame_pointer);
printf("Stack_start: %lx\n",
(unsigned long)stack_start);
arg_ptr = (arg_t *)data;
child_offset =
OFFSET_IN_PAGE((off_t)&arg_ptr->a);
printf("Address of arg_ptr->a: %p\n",
&arg_ptr->a);
pipe(parent_to_child);
pipe(child_to_parent);
bool msg;
int child_mem_fd;
char child_path[0x20];
child_pid = fork();
if (child_pid == -1) {
perror("fork");
exit(EXIT_FAILURE);
}
if (!child_pid) {
close(child_to_parent[0]);
close(parent_to_child[1]);
printf("CHILD (pid %d, parent pid %d).\n",
getpid(), getppid());
fn(data);
msg = true;
write(child_to_parent[1], &msg, 1);
child_mem_fd = open("/proc/self/mem", O_RDONLY);
if (child_mem_fd == -1) {
perror("open (child)");
exit(EXIT_FAILURE);
}
printf("CHILD: child_mem_fd: %d\n", child_mem_fd);
if (lseek(child_mem_fd, stack_start, SEEK_SET) == (off_t)-1) {
perror("lseek");
exit(EXIT_FAILURE);
}
if (read(child_mem_fd, buffer, sizeof(buffer))
!= sizeof(buffer)) {
perror("read");
exit(EXIT_FAILURE);
}
printf("CHILD: new value %d\n",
*(int *)(buffer + child_offset));
read(parent_to_child[0], &msg, 1);
exit(EXIT_SUCCESS);
}
else {
printf("PARENT (pid %d, child pid %d)\n",
getpid(), child_pid);
printf("PARENT: child_offset: %lx\n",
child_offset);
read(child_to_parent[0], &msg, 1);
printf("PARENT: message from child: %d\n", msg);
snprintf(child_path, 0x20, "/proc/%d/mem", child_pid);
printf("PARENT: child_path: %s\n", child_path);
child_mem_fd = open(path, O_RDONLY);
if (child_mem_fd == -1) {
perror("open (child)");
exit(EXIT_FAILURE);
}
printf("PARENT: child_mem_fd: %d\n", child_mem_fd);
if (lseek(child_mem_fd, stack_start, SEEK_SET) == (off_t)-1) {
perror("lseek");
exit(EXIT_FAILURE);
}
if (read(child_mem_fd, buffer, sizeof(buffer))
!= sizeof(buffer)) {
perror("read");
exit(EXIT_FAILURE);
}
printf("PARENT: new value %d\n",
*(int *)(buffer + child_offset));
close(child_mem_fd);
printf("ENDING CHILD PROCESS.\n");
write(parent_to_child[1], &msg, 1);
if (waitpid(child_pid, &status, 0) == -1) {
perror("waitpid");
exit(EXIT_FAILURE);
}
}
}
int main(void) {
arg_t arg;
arg.a = 42;
printf("In main: address of arg.a: %p\n", &arg.a);
explore_proc_mem(&func1, &arg.a);
return EXIT_SUCCESS;
}
This program produces the output below. Notice that the value of a (boldfaced) differs between parent's and child's reading of the /proc/child_pid/mem file.
In main: address of arg.a: 0x7ffffe1964f0
Stack_start: 7ffffe196000
Address of arg_ptr->a: 0x7ffffe1964f0
PARENT (pid 20376, child pid 20377)
PARENT: child_offset: 4f0
CHILD (pid 20377, parent pid 20376).
func1: old value: 42
func1: address: 0x7ffffe1964f0
func1: new value: 53
PARENT: message from child: 1
CHILD: child_mem_fd: 4
PARENT: child_path: /proc/20377/mem
CHILD: new value 53
PARENT: child_mem_fd: 7
PARENT: new value 42
ENDING CHILD PROCESS.
There's one silly mistake in this code:
const char * path = "/proc/self/mem";
...
snprintf(child_path, 0x20, "/proc/%d/mem", child_pid);
printf("PARENT: child_path: %s\n", child_path);
child_mem_fd = open(path, O_RDONLY);
So you always end up reading parent's memory here. However after changing this, I get:
CHILD: child_mem_fd: 4
CHILD: new value 53
read (parent): No such process
And I don't know why it could happen - maybe /proc is too slow in refreshing the entries? (it's from perror("read") in the parent - had to add a comment to see which one fails) But that seems weird, since the seek worked - as well as open itself.
That question doesn't seem to be new either: http://lkml.indiana.edu/hypermail/linux/kernel/0007.1/0939.html (ESRCH is "no such process")
Actually a better link is: http://www.webservertalk.com/archive242-2004-7-295131.html - there was an issue with marking processes pthread-attach-safe. You can find there Alan Cox sending someone to Solar Designer... for me that spells "here be dragons" and that it's not solvable if you don't hack kernels in your sleep :(
Maybe it's enough for you to check what is gdb doing in that case and replicating it? (Probably it just goes via ptrace(PTRACE_PEEKDATA,...))
The solution is to use ptrace to synchronize parent with child. Even though I am already communicating between parent and child (and the man page for ptrace says that it causes the two processes to behave as if they were parent and child), and even though the child is blocking on the read call, the child has apparently not "stopped" enough for Linux to allow the parent to read the child's /proc/child_pid/mem file. But if the parent first calls ptrace (after it receives the message over the pipe) with PTRACE_ATTACH, then it can open the file--and get the correct contents! Then the parent calls ptrace again, with PTRACE_DETACH, before sending the message back to the child to terminate.

Resources