Futex Error in C application that is using mutex and epoll - linux

I am running epoll to process network connection. Everything is working perfectly I think.
Now i am trying to connect to the Postgresql Database from the child process. I have 4 child process.
Sometimes when i try to connect to the server. It hangs.
Sometimes i get this error
fserver: tpp.c:84: __pthread_tpp_change_priority: Assertion `new_prio
Would appreciate it if someone would be able to advice on my code. Thanks
If i remove the Postgresql Connection code , the problem disappears.
/******CHILD PROCESS **************/
void childProcess()
{
const char conninfo[] = "postgresql://postgres#localhost?port=5432&dbname=ram&password=xxx";
PGconn *conn = PQconnectdb(conninfo);
const ConnStatusType connStatusType = PQstatus(conn);
if (connStatusType == CONNECTION_BAD) {
printf("pg connection not OK");
if (conn) {
PQfinish(conn);
}
exit(1);
} else {
printf("pg connection OK");
}
int efd = epoll_create1(0);
if(efd == -1)
{
perror("Epoll Creation Failed\n");
}
nonblocksocket(&p2c_var);
struct epoll_event event0;
event0.data.fd = p2c_var;
event0.events = EPOLLIN | EPOLLET;
if(epoll_ctl(efd,EPOLL_CTL_ADD,p2c_var,&event0) == -1)
{
perror("First Failed to add file descriptor to epoll watch list\n");
}
nonblocksocket(&c2p_var);
struct epoll_event event1;
event1.data.fd = c2p_var;
event1.events = EPOLLOUT | EPOLLET;
if(epoll_ctl(efd,EPOLL_CTL_ADD,c2p_var,&event1) == -1)
{
perror("Second Failed to add file descriptor to epoll watch list\n");
}
struct epoll_event *ttlevents;
ttlevents = calloc(MAXEVENTS, sizeof(event0));
int head = 0,tail = 0;
pthread_mutex_t buflock;
pthread_mutex_t fdlock;
int bufhead=0,buftail=0;
struct fifoarr *fifoptr = (struct fifoarr *) calloc(QSIZE,sizeof(struct fifoarr));
int co = 0;
char st[1050];
memset(st,0,sizeof(st));
st[0] = 1;
while(1)
{
int eventcount = epoll_wait(efd,ttlevents,MAXEVENTS,-1);
for(int i =0;i<eventcount;i++)
{
if ((ttlevents[i].events & EPOLLERR) || (ttlevents[i].events & EPOLLHUP))
{
/* An error has occured on this fd, or the socket is not ready for reading (why were we notified then?) */
fprintf (stderr, "epoll error\n");
close (ttlevents[i].data.fd);
continue;
}
if(ttlevents[i].events & EPOLLOUT)
{
if(ttlevents[i].data.fd == c2p_var)
{
pthread_mutex_lock(&buflock);
int bufval = popfd(&bufhead,&buftail);
if(bufval != -1)
{
write(c2p_var,fifoptr[bufval].buffer,sizeof(fifoptr[bufval].buffer));
}
pthread_mutex_unlock(&buflock);
}
}
if(ttlevents[i].events & EPOLLIN)
{
if(ttlevents[i].data.fd == p2c_var)
{
char buffer[1050];
while(1)
{
memset(buffer,0,sizeof(buffer));
int c = read(ttlevents[i].data.fd,buffer,sizeof(buffer));
if(c == -1)
{
if(errno == EAGAIN)
{
printf("Completed Reading From Parent Process\n");
break;
}
}
if(c == 0)
{
printf("Parent terminated Connection.\n");
break;
}
if(buffer[0] != 1)
{
pthread_mutex_lock(&buflock);
int bufval = pushfd(&bufhead);
if(bufval != -1)
{ memset(fifoptr[bufval].buffer,0,sizeof(fifoptr[bufval].buffer)); memcpy(fifoptr[bufval].buffer,buffer,sizeof(buffer));
// processRequest(fifoptr[bufval].buffer,conn);
write(c2p_var,&st,sizeof(st));
}
pthread_mutex_unlock(&buflock);
}//buffer[0]
}//While
}
}//if epollin
} //for
}//while(1)
PQfinish(conn);
}
The program tends to hang.

Well it is working now.
It seems that mutex is the cause of the problem.
Thanks for all that looked at my code.

Related

Win32 multithreaded sockets

I have a server class that has a method called handle_client as follows:
void server::handle_client()
{
do {
// Accept a client socket
EnterCriticalSection(&listenSocketCriticalSection);
SOCKET clientSocket = accept(listenSocket, NULL, NULL);
LeaveCriticalSection(&listenSocketCriticalSection);
// ... rest of the client handling code that reads
// from the client socket and sends appropriate response
// ...
} while(true);
}
I have a run method as follows:
void server::run()
{
// create the threads
for (int i = 0; i < THREAD_POOL_SIZE; i++) {
DWORD dwThreadId;
thread_pool_handle[i] = CreateThread(NULL, 0, thread_function, this, 0, &dwThreadId);
}
WaitForMultipleObjects(THREAD_POOL_SIZE, thread_pool_handle, true, INFINITE);
}
I have a thread_function as follows:
DWORD WINAPI thread_function(LPVOID lpParam)
{
server* pServer = (server*)lpParam;
pServer->handle_client();
}
I am creating a pool of threads that are all waiting for a client socket connection to be accepted. Since I have wrapped the accept within a critical section, only one thread will succeed at a time. Once a client socket is accepted by the server thread, that thread continues to go on to handle the request. The idea is that the thread will loop back indefinitely to the accept call after completing a request.
Questions:
Is the Critical Section necessary? (I think so, because otherwise the accept call from the multiple threads on the same listenSocket would clobber things. Is this correct?)
If handle_client loops indefinitely, what is the best way to cleanly terminate all the threads and exit the server? Should I use a special message from the client to trigger the thread terminations? Any other suggestions?
How should I handle the server process termination gracefully (as it pertains to the thread pool)?
It Is recommended to use Select model to store socket objects in multithreaded sockets. In Select model, you can use FD_CLR() to clear sockets when there are no network events.
I have the code for the server with the select socket, You can try to run and modify.
#include <iostream>
#include<WinSock2.h>
#include<windows.h>
#include<WS2tcpip.h>
#pragma comment(lib,"ws2_32.lib")
#pragma warning(disable:4996)//inet_addr
int main()
{
//1.Obtain version info
WSADATA wsaData = { 0 };
SOCKET hServer = { 0 };
WSAStartup(MAKEWORD(2, 2), &wsaData);
if (LOBYTE(wsaData.wVersion) != 2 || HIBYTE(wsaData.wVersion) != 2)
{
printf("version failed %d\n", GetLastError());
return -1;
}
else
{
printf("version succeed \n");
}
//2.create socket
hServer = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
if (hServer == SOCKET_ERROR)
{
printf("create socket tcp failed %d\n", GetLastError());
return -1;
}
else
{
printf("create socket tcp succeed \n");
}
//3. Create a protocol address family
sockaddr_in ServerAddr = { 0 };
ServerAddr.sin_family = AF_INET6;
ServerAddr.sin_zero[8];
ServerAddr.sin_port = htons(8888);
ServerAddr.sin_addr.S_un.S_addr = inet_addr("192.168.2.50");;//modify your address
//4.bind
int nRet = bind(hServer, (sockaddr*)&ServerAddr, sizeof ServerAddr);
if (nRet == SOCKET_ERROR)
{
printf("bind failed %d\n", GetLastError());
closesocket(hServer);
WSACleanup();
return -1;
}
else
{
printf("bind succeed \n");
}
//5.listen
nRet = listen(hServer, 3);
if (nRet == SOCKET_ERROR)
{
printf("listen failed %d\n", GetLastError());
closesocket(hServer);
WSACleanup();
return -1;
}
else
{
printf("listen succeed \n");
}
sockaddr_in clientAddr = { 0 };// The protocol address family used to receive the client
int len = sizeof(clientAddr);// The size of the accepted client protocol address family information
// Create a select model to store socket objects
FD_SET fd_read;
FD_ZERO(&fd_read);
FD_SET(hServer, &fd_read);
//6. Accept client connections
while (1) {
FD_SET fd_tmp = fd_read;// Read backup can only be in
const timeval tv = { 1,0 };
int Ret = select(NULL, &fd_tmp, NULL, NULL, &tv);
if (Ret == 0) // No network events, TMP is automatically deleted
{
Sleep(1000);
continue;
}
for (int i = 0; i < fd_tmp.fd_count; i++)
{
// If there are network events for a listening socket, it proves that a client is connecting to the socket
if (fd_tmp.fd_array[i] == hServer)
{
SOCKET hclient;
hclient = accept(hServer, (sockaddr*)&clientAddr, &len);// If you do not want to store the protocol address family information of the client, you can pass a NULL address
if (hclient == SOCKET_ERROR)
{
printf("recieve information of client failed %d\n", GetLastError());
closesocket(hServer);
return -1;
}
printf("connecting: %s******** \n", inet_ntoa(clientAddr.sin_addr));
FD_SET(hclient, &fd_read);
}
else // The client socket has network events that prove that the client is sending data and the server is accepting the data
{
char buff[32] = { 0 };
int nRet = recv(fd_tmp.fd_array[i], (char*)buff, 32, NULL);
if (nRet > 0)
{
printf("message: %s\n", buff);
}
else// Removes the current socket fd_array from the fd_read
{
FD_CLR(fd_tmp.fd_array[i], &fd_read);
printf("Disconnect \n", GetLastError());
closesocket(hServer);
closesocket(fd_read.fd_array[i]);
break;
}
}
break;
}
}
//7.close socket
WSACleanup();
getchar();
return 0;
}

How can I wait for both a file-descriptor and child state change simultanously?

In Linux, one can wait on any FD using select, poll or epoll.
It is also possible to wait for child-processes to change state using wait, waitpid or waitid.
However, I can't figure a way to combine these operations, i.e., to block the calling process until either some FD becomes ready or a child process changes state.
I can use polling, by repeatedly calling non-blocking epoll then waitid, but that is wasteful.
It is possible to create a pidfd for a child process (which is accepted by epoll), but pidfd only supports waiting for child termination, while I wish to wait for any state change (specifically, for ptrace stops).
Is this not possible in Linux?
You can wait for any child status change with signalfd() and make dummy read, then get actual status with waitpid():
sigset_t mask, old_set;
sigemptyset(&mask);
sigaddset(&mask, SIGCHLD);
sigprocmask(SIG_BLOCK, &mask, &old_set);
int sigfd = signalfd(-1, &mask, SFD_CLOEXEC);
if (sigfd == -1) {
perror("signalfd");
return 1;
}
for (int i = 0; i < 10; ++i) {
pid_t pid = fork();
if (pid == -1) {
perror("fork");
}
if (pid == 0) {
// Child process: restore blocked signals before exec() etc
sigprocmask(SIG_SETMASK, &old_set, NULL);
sleep(i % 3);
switch (i % 3) {
case 0:
raise(SIGSTOP);
break;
case 1:
raise(SIGABRT);
break;
}
exit(i);
}
printf("Spawned child %i with pid %u\n", i, pid);
}
for (;;) {
struct pollfd fds[] = {
{ .fd = STDIN_FILENO, .events = POLL_IN },
{ .fd = sigfd, .events = POLL_IN }
};
if (poll(fds, sizeof(fds)/sizeof(*fds), -1) == -1) {
perror("poll");
break;
}
if (fds[0].revents & POLL_IN) {
char buf[4096];
int ret = read(STDIN_FILENO, buf, sizeof(buf));
printf("Data from stdin: ");
fflush(stdout);
write(STDOUT_FILENO, buf, ret);
}
if (fds[1].revents & POLL_IN)
{
struct signalfd_siginfo fdsi;
read(sigfd, &fdsi, sizeof(fdsi));
for (;;) {
int status;
pid_t pid = waitpid(-1, &status, WNOHANG | WUNTRACED | WCONTINUED);
if (pid == -1) {
if (errno != ECHILD) {
perror("waitpid");
}
break;
}
if (pid == 0) {
break;
}
printf("Child %u ", pid);
if (WIFEXITED(status)) {
printf("exited with status %i\n", WEXITSTATUS(status));
} else if (WIFSIGNALED(status)) {
printf("terminated by signal %i\n", WTERMSIG(status));
} else if (WIFSTOPPED(status)) {
printf("stopped by signal %i\n", WSTOPSIG(status));
} else if (WIFCONTINUED(status)) {
printf("continued\n");
} else {
printf("status unknown\n");
}
}
}
}
close(sigfd);

LINUX msgget and ftok

Hello I'm trying IPC practice. But I have problem... Here is the code
Send_process.c (path : /home/John/h1/send_process.c)
struct mymsgbuf {
long mtype;
char mtext[80];
};
int main(void) {
key_t key;
int msgid;
struct mymsgbuf mesg;
char inputmsg[80];
if(msgid = msgget((key_t)123, IPC_CREAT | 0666) == -1) {
perror("Failed to create new MessageQueue");
exit(1);
}
printf("%d\n", msgid);
mesg.mtype = 1;
while(1) {
printf("Enter Sending Message. (Input exit, programm terminate) : ");
scanf("%s", inputmsg);
strcpy(mesg.mtext, inputmsg);
if(msgsnd(msgid, (void *)&mesg, sizeof(mesg.mtext), IPC_NOWAIT) == -1) {
perror("msgsnd");
exit(1);
}
if(strcmp(inputmsg, "exit") == 0) {
printf("Sending Process Terminated\n");
break;
}
}
return 0;
}
Receive_process.c (path : /home/John/h1/send_process.c)
struct mymsgbuf {
long mtype;
char mtext[80];
};
int main(void) {
struct mymsgbuf inmsg;
key_t key;
int msgid;
key = ftok("/home/John/h1/receive_process.c", 123);
if((msgid = msgget(key,0666)) < 0 ) { //here is error
perror("msgget");
exit(1);
}
printf("%d\n", msgid);
while(1) {
if(msgrcv(msgid, &inmsg, sizeof(inmsg.mtext), 0,0) == -1) {
perror("Message Receive");
exit(1);
}
printf("Received Message. Message is [%s]\n", inmsg.mtext);
if(strcmp(inmsg.mtext, "exit") == 0 ) {
printf("Receive_process end\n");
exit(0);
}
}
return 0;
}
Send_process is works very well, but Receive_process not works well. What is the problem?

Why is the Write system call in not writing anything to the sockets file?

I am writing a simple Server Client program that exchanges the data. After I write to the socket file, the write doesn't fail or it is not even partial write. but when I check the details of the socket file using ls -l , I still see it's size as zero and server doesn't recieve anything. Can anyone help me out with what I am doing wrong in here??
This is server.c
int main()
{
int socket_fd = 0;
struct sockaddr_un addr;
int result = -1;
char buffer[MAX_BUFFER_SIZE];
printf("Creating a socket\n");
socket_fd = socket(AF_UNIX,SOCK_STREAM,0);
if(socket_fd == -1)
{
perror("SOCKET");
return 0;
}
printf("Socket has been created %d\n",socket_fd);
addr.sun_family = AF_UNIX;
strncpy(addr.sun_path,_SOCKET_PATH,sizeof(addr.sun_path)-1);
printf("PATH : %s\n",addr.sun_path);
if(remove(_SOCKET_PATH) == -1)
{
perror("REMOVE");
return 0;
}
printf("Binding the socket\n");
result = bind(socket_fd,(struct sockaddr *)&addr,sizeof(struct sockaddr_un));
if(result == -1)
{
perror("BIND");
return 0;
}
printf("Binding the socket is done\n");
printf("Listening to the socket\n");
if(listen(socket_fd,1) == -1)
{
perror("Listen");
return 0;
}
if((result = accept(socket_fd,NULL,NULL)) == -1)
{
perror("ACCEPT");
return 0;
}
printf("Connection Accepted\n");
while (1)
{
while (result = read(socket_fd,buffer,sizeof(buffer)-1) > 0)
{
printf("Client said : %s\n",buffer);
}
}
}
This is client.c
int main()
{
int socket_fd = 0;
struct sockaddr_un addr;
int result = -1;
char buffer[MAX_BUFFER_SIZE];
printf("Creating a socket\n");
socket_fd = socket(AF_UNIX,SOCK_STREAM,0);
if(socket_fd == -1)
{
perror("SOCKET");
return 0;
}
printf("Socket has been found %d\n",socket_fd);
addr.sun_family = AF_UNIX;
strncpy(addr.sun_path,_SOCKET_PATH,sizeof(addr.sun_path)-1);
printf("Connecting to the socket\n");
result = connect(socket_fd,(struct sockaddr *)&addr,sizeof(struct sockaddr_un));
if(result == -1)
{
perror("CONNECT");
return 0;
}
printf("The client is connected to the server.\n");
while (1)
{
memset(buffer,0,sizeof(buffer));
scanf("%s",buffer);
printf("DATA WRITTEN %s,%d\n",buffer,strlen(buffer)+1);
result = write(socket_fd,buffer,strlen(buffer)+1);
printf("result = %d\n",result);
sleep(5);
}
}
Thanks for any help!
if((result = accept(socket_fd,NULL,NULL)) == -1)
...
while (result = read(socket_fd,buffer,sizeof(buffer)-1) > 0)
You are trying to read from the server socket (socket_fd). Instead you need to read from the new socket returned by accept, i.e. what you call result in result = accept.... To cite from man accept:
On success, these system calls return a nonnegative integer that is a
file descriptor for the accepted socket. On error, -1 is returned,
and errno is set appropriately.

Make new connections when blocked in a select() call

I have 2 threads:
Thread A:
It's the select() loop. Perform socket handling for reading operations, such as accepting new connections, receiving data.
while (1) {
FD_ZERO(&fdReadSet);
numActiveSockets = 0;
for (std::unordered_map<SOCKET, TcpSocket*>::iterator it = m_sock_table.begin(); it != m_sock_table.end(); it++)
{
numActiveSockets++;
FD_SET(it->first, &fdReadSet);
}
int ret;
bool hasListen = false;
if (( ret = select(numActiveSockets, &fdReadSet, NULL, NULL, NULL)) == SOCKET_ERROR) {
printf("Select Failed, Error code = %d\n", WSAGetLastError());
return -1;
}
for (std::unordered_map<SOCKET, TcpSocket*>::iterator it = m_sock_table.begin(); it != m_sock_table.end(); it++)
{
if (FD_ISSET(it->first, &fdReadSet))
{
if (it->first == TcpSocket::m_listen_sock)
{
if (!hasListen)
{
sockaddr_in sock_addr;
int sockLength = sizeof(sock_addr);
SOCKET sock = accept(it->first, (sockaddr *) &sock_addr, &sockLength);
TcpSocket * socket = new TcpSocket();
socket->m_sock = sock;
m_sock_table[sock] = socket;
it = m_sock_table.begin();
hasListen = true;
}
}
else
{
char * buffer = it->second->GetWriteBuffer();
int numRead = recv(it->first, buffer, SOCKET_BUFFER_SIZE, 0);
if (numRead == SOCKET_ERROR)
{
int err = WSAGetLastError();
if (err == WSAECONNRESET)
{
printf("Connection [%i]: RESET Received. Closing Socket\n", it->first);
closesocket(it->first);
it = socketVector.erase(it->first); // iterator invalidated after erase
}
else
{
printf("Recv Failed. Error code = %d\n", err);
return -1;
}
}
else if (numRead == 0)//connection close
{
printf("Connection [%i]: Graceful exit. Closing Socket\n", it->first);
closesocket(it->first);
it = socketVector.erase(it->first); // iterator invalidated after erase
}
else {
/* Process received data */
}
}
}
}
}
Thread B:
Allow the application to perform connect() to establish new connections. If a connect() is successful, it will the add the returned socket to m_sock_table.
I have a socket table called m_sock_table which holds all the sockets. I use this m_sock_table to initialize the fdReadSet to be used in select().
-----------Problem-----------------
If thread A is blocked by select(), and at the same time thread B establish a new connection through connect(), the application wouldn't be able to receive data from the new connection, because fdReadset has not been updated withed the new connected socket.
What would be a good way to solve this problem? Or the design is just wrong from the start?
You could use a signal that doesn't do anything other than interrupting the system call:
#include <signal.h>
void do_nothing() { }
struct sigaction sa;
sa.sa_handler = do_nothing;
sigemptyset(sa.sa_mask);
#ifdef SA_INTERRUPT
sa.sa_flags = SA_INTERRUPT;
#else
sa.sa_flags = 0;
#endif
sigaction(SIGUSR1, &sa, 0);
Then, in thread B, after starting a new connection, send the signal, after making sure thread A will handle it:
/* need only be done once, but needed in every thread other than A */
sigset_t sigs;
sigemptyset(&sigs);
sigaddset(&sigs, SIGUSR1)
pthread_sigmask(SIG_BLOCK, &sigs, 0);
/* each time we create a new connection */
kill(getpid, SIGUSR1);
With the above, select will return with an EINTR error -- so check for that and loop (adding the new connection to the set).

Resources