EBADF while recv after epoll_wait - linux

Question

i've got a following problem: i have a epoll code which receives connections:
while (1) {
int nfds = epoll_wait(epollfd, events, 4096, -1);
if (nfds == -1) {
if (errno == EINTR)
continue;
perror("epoll_wait");
exit(EXIT_FAILURE);
}
for (int i = 0; i < nfds; i++) {
if (events[i].data.fd == server_sock) {
client_sock = accept(server_sock,
(struct sockaddr *)&client_name,
(socklen_t *)(&client_name_len));
if (client_sock == -1) //server overloaded
continue;
ev.events = EPOLLIN | EPOLLERR;
#ifdef CORE_NONBLOCKING_SOCKETS
Arch::set_nonblocking(client_sock);
ev.events |= EPOLLET; //input data and connection closing
#endif
#ifdef EPOLLRDHUP
ev.events |= EPOLLRDHUP ;//
#else
//for old libraries
ev.events |= EPOLLHUP;//
#endif
ev.data.fd = client_sock;
if (epoll_ctl(epollfd, EPOLL_CTL_ADD, client_sock, &ev) == -1) {
perror("epoll_ctl: client_socket");
exit(EXIT_FAILURE);
}
accept_request(client_sock);
} else {
#ifdef EPOLLRDHUP
if (events[i].events & EPOLLRDHUP) {
std::cout << "EPOLLRDHUP on " << events[i].data.fd << std::endl;
listener->disconnectDriver(events[i].data.fd);
}
#else
if (events[i].events & EPOLLHUP) {
std::cout << "EPOLLHUP on " << events[i].data.fd << std::endl;
listener->disconnectDriver(events[i].data.fd);
}
#endif
if (events[i].events & EPOLLIN) {
std::cout << "debug EPOLLIN on " << events[i].data.fd << std::endl;
accept_request(events[i].data.fd);
}
if (events[i].events & EPOLLERR) {
std::cout << "debug EPOLLERR on " << events[i].data.fd << std::endl;
listener->disconnectDriver(events[i].data.fd);
}
}
}
when i received input connection i trying to read all buff data:
void get_all_buf(int sock, std::string & inStr) {
int n = 1;
int total = 0;
char c;
char temp[1024*1024];
bzero(temp, sizeof(temp));
do {
#ifdef CORE_NONBLOCKING_SOCKETS
timespec time_to_wait;
time_to_wait.tv_nsec = 10000000;
time_to_wait.tv_sec = 0;
timespec tm;
time_t begin = time(NULL);
do {
#endif
n = recv(sock, &temp[total], sizeof(temp), 0);
#ifdef CORE_NONBLOCKING_SOCKETS
nanosleep(&time_to_wait, &tm);
time_t end = time(NULL);
if ((end - begin) > MAX_CLIENT_TIME) {
inStr = std::string();
return;
}
} while (n < 0 && errno == EAGAIN); //nonblocking sockets in edge-triggered mode
#endif
if (n > 0) {
total += n;
} else if (n == 0) {
//TODO: error handling
//debug
std::cout << "possibly no one byte was received" << std::endl;
break;
} else if (n < 0) {
//TODO: error handling
//debug
std::cout << "error while receiving data" << std::endl;
if (errno == EBADF) {
std::cout << "recv returns with EBADF: " << strerror(errno) << std::endl;
} else if (errno == EFAULT) {
std::cout << "recv returns with EFAULT: " << strerror(errno) << std::endl;
} else if (errno == EINTR) {
std::cout << "recv returns with EINTR: " << strerror(errno) << std::endl;
} else if (errno == EINVAL) {
std::cout << "recv returns with EINVAL: " << strerror(errno) << std::endl;
}
//end debug
break;
}
} while (!strstr(temp, "</packet>"));
inStr = temp;
};
within accept_request function. but sometime i get following in my debug output:
packet type='connect'
size of vector<Driver> in getDriversWithMoney is 1
epoll_wait detected activity of 164 counter i = 0 nfds = 1
EPOLLRDHUP on 164
disconnectDriver (fd = 164)
driver 1 disconnected
debug EPOLLIN on 164
error while receiving data
recv returns with EBADF: Invalid file descriptor
which means that someone was connected first than disconnected and when he's trying to connect again recv returns EBADF. what i did wrong? please help me.
P.S. on EPOLLRDHUP i just close file descriptor. epoll man says it's ok, because epoll removes closed fd from epoll_wait queue by itself.

Answer 1

When the remote host closes the socket, epoll() reports both a HUP and an EPOLLIN for the file descriptor.
You check for EPOLLRDHUP first, and close the socket; then you check for EPOLLIN, find that too, and try to call recv(). Since the socket has been closed, the file descriptor is no longer valid, and you get EBADF (the closed socket is removed from the epoll set, so subsequent epoll_wait() calls won't return it; but for the epoll_wait() that has already returned it's too late - the EPOLLIN is already waiting in your events[i].
You need to stop checking for events after you call disconnectDriver() (if it closed the file descriptor):
if (events[i].events & EPOLLRDHUP) {
std::cout << "EPOLLRDHUP on " << events[i].data.fd << std::endl;
listener->disconnectDriver(events[i].data.fd);
continue;
}