The fd in epoll will set to non-blocking，and regist with EPOLLIN event. if i don't read until EAGAIN each time,and the epoll_wait can receive the read event？I find a mail (golang-nuts) with a description like this:
One of the harder considerations about using edge-triggered epoll is you have to read()/write() until EGAIN, otherwise you will not receive another read/write event for the socket when you call epoll_wait.
I write a demo epollserver.cpp,
the server listens on port 12439 and reads only 1 character at a time,like:
while(1)
{
int num = epoll_wait(epfd, events, BUFSIZE, -1);
char buf[BUFSIZE] = {0};
for(int i = 0; i < num; i++)
{
if(events[i].events & EPOLLIN){
std::cout << "Ready to read!\n";
int size;
std::cout << "size=" << (size = read(events[i].data.fd, buf, 1) ,buf[size] = 0,size)
<< "\t buf=[" << buf << ']' << std::endl;
}
}
}
The client use nc -4 localhost 12439 and sends 4 characters at a time
However,the server epoll_wait can receive read event every time.
So，i wonder if i misanderstand something?
Yes, epoll_wait() will wake you up repeatedly until you have read all the input. This is the default behavior, unless you use the EPOLLET flag to enable edge-triggered mode.
Related
I am new in socket programming and at this moment I am confronted with a problem that I can not solve. I have read from several sources that the C++ standard template (STL) containers are not thread-safe, so that one as a programmer has to impose a mechanism that ensures that several threads do not modify the data of a container concurrently.
For instance, Thread safety std::vector push_back and reserve
I have used the std::mutex class to make sure that nobody writes data in the same container at the same time when programming threads. However, this is not working for me when I use sockets.
Suppose I have 4 clients, each one sending data (int) to the server in the following order:
client_0: 4
client_1: 8
client_2: 5
client_4: 7
Observe the following code for a simple server:
#define PORT 60000
#include <unistd.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <vector>
#include <string>
#include <iostream>
#include <mutex>
using namespace std;
vector<int> inputQueue; //<--------!
mutex mtx; //<---------------------!
void printVector(vector<int> input) {
cout << "inputQueue: [";
for (unsigned int i = 0; i < input.size(); i++ ) {
if (i != input.size() - 1)
cout << input[i] << ", ";
else
cout << input[i];
}
cout << "]." << endl;
}
int main(int argc, char const *argv[])
{
int server_fd, client_fd;
struct sockaddr_in address;
int opt = 1;
int addrlen = sizeof(address);
if ((server_fd = socket(AF_INET, SOCK_STREAM, 0)) == 0) {
perror("socket failed");
exit(EXIT_FAILURE);
}
if (setsockopt(server_fd, SOL_SOCKET, SO_REUSEADDR | SO_REUSEPORT, &opt, sizeof(opt))) {
perror("setsockopt");
exit(EXIT_FAILURE);
}
address.sin_family = AF_INET;
address.sin_addr.s_addr = INADDR_ANY;
address.sin_port = htons( PORT );
if (bind(server_fd, (struct sockaddr *)&address, sizeof(address))<0) {
perror("bind failed");
exit(EXIT_FAILURE);
}
if (listen(server_fd, 10) < 0) {
perror("listen");
exit(EXIT_FAILURE);
}
while(1) {
char buffer[4];
if ((client_fd = accept(server_fd, (struct sockaddr *)&address, (socklen_t*)&addrlen))<0) {
perror("accept");
exit(EXIT_FAILURE);
}
if (!fork()) {
recv(client_fd, buffer, 4, MSG_WAITALL);
int receivedInt = int(
(unsigned char)(buffer[0]) << 24 |
(unsigned char)(buffer[1]) << 16 |
(unsigned char)(buffer[2]) << 8 |
(unsigned char)(buffer[3])
);
mtx.lock(); //<-------------------------------------!
inputQueue.push_back(receivedInt); //<--------------!
cout << "Client context. Integer registered: " << receivedInt << ": inputQueue length is " << inputQueue.size() << endl;
printVector(inputQueue); //<------------------------!
mtx.unlock(); //<-----------------------------------!
close(server_fd); close(client_fd);
}
cout << "Server context: inputQueue length is " << inputQueue.size() << endl;
printVector(inputQueue);
}
return 0;
}
The server must receive data making sure that they do so in the same order and registering their respective data in a vector of integers, that is, std::vector<int> inputQueue, using the push_back() method, so that inputQueue = {4, 8, 5, 7} at the end of the reception of all the data by the clients.
I must clarify that inputQueue is a global variable, which when starting the execution of the server, does not contain elements, but they are added as the clients register.
The problem is that none of the client registers elements in inputQueue. Notice in the following code that, depending on where you put the cout << instruction, you can see that the inputQueue size is different. This shows that within the context of the client, each client overwrites the first element of inputQueue, but outside it none of the clients is able to register a single element in inputQueue.
Apparently, each socket has its own copy of inputQueue, so when it is destroyed, the modified copy of inputQueue is also destroyed.
Output is the following:
Server context: inputQueue length is 0
inputQueue: [].
Client context. Integer registered: 4: inputQueue length is 1
inputQueue: [4].
Server context: inputQueue length is 1
inputQueue: [4].
Server context: inputQueue length is 0
inputQueue: [].
Client context. Integer registered: 8: inputQueue length is 1
inputQueue: [8].
Server context: inputQueue length is 0
inputQueue: [].
Server context: inputQueue length is 1
inputQueue: [8].
Client context. Integer registered: 5: inputQueue length is 1
inputQueue: [5].
Server context: inputQueue length is 1
inputQueue: [5].
Server context: inputQueue length is 0
inputQueue: [].
Client context. Integer registered: 7: inputQueue length is 1
inputQueue: [7].
Server context: inputQueue length is 1
inputQueue: [7].
Does anyone have any idea why this happens and how could they solve it? I hope you can help me. Thank you
if (!fork()) {
fork() creates a completely new, independent process with its own virtual memory address space. The shown code, apparently, expects both the child process and the original process to be interacting through the same object, namely a vector, locked by a mutex.
That's not what happens. You now have two completely independent processes. This is no different than running your program twice, at the same time or in quck succession. Do you expect both running copies of your program to somehow share the same vector and mutex? Of course not.
What you are looking to do, instead, is to use std::thread to create a new execution thread in the same process. Your C++ book should have more information how to create new execution threads with std::thread.
Furthermore, even if you replace the fork() with an analogous execution thread: that will still not solve all the problems here. You will also need to correctly handle synchronization between multiple execution threads. Specifically: there are no guarantees whatsoever that a new execution thread will insert something into the vector, before the other execution thread attempts to printVector its contents. The new execution thread could manage to do that, before the original execution thread enters printVector. Or it may not, and printVector finds a completely empty vector, because the other execution thread hasn't managed to push something into it, quickly enough. You now have two completely independent execution threads running at the same time, and you have no guarantees as to which thread does what, first.
You can even get a different result every time you run the multithreaded version of the shown program (and you probably will).
When you are ready to begin tackling this new problem, your C++ book will explain how to use condition variables, together with mutexes, to correctly implement multi-threaded synchronization. Unfortunately, this is not a topic that can be completely covered in a brief answer on stackoverflow.com, but it should have several dedicated chapters in your C++ book, where you will find more information.
P.S. The only reason your output shows anything in the input queue is because there's nothing to stop the child process to continue executing the program when it exits its if statement, and ends up, itself, calling printVector. It's not coming from the parent process. Each child process ends up printing the value it itself inserted into its own vector.
As noted by Miles Budnek, you are creating a new child process. Sockets are global OS objects, so are working as expected. Your vector, and the memory it's stored in, is local to the process and therefore cannot be accessed by your new proc.
Consider looking into std::thread:
https://en.cppreference.com/w/cpp/thread/thread
One of the most used methods of starting a thread is with a lambda.
#include <thread>
#include <iostream>
auto print_number(int number) -> void
{
std::cout << number << std::endl; // This runs in the new thread.
}
int main()
{
int num = 12;
auto t = std::thread([num](){print_number(num);}); // Spawn new thread that calls the lambda
t.join(); // Wait for thread to finish execution
return 0;
}
I need to have a thread which executes a function in while loop(say increments an int value). In the main I need to have a while loop which executes some function(say a for loop which counts from 0 to 5) and then reads the current value of a variable in the thread. The thread must keep running its own while loop irrespective of whats going on in main. However the value of the thread variable must not change while main reads the variable.
I guess this problem can be solved using atomic. However this is a toy problem in which the variable in the thread is an int. In my actual problem the thread variable if of type Eigen::quaternionf or float[4]. So I need to ensure that the entire Eigen::quaternionf or float[4] is held constant when it is read from main.
The cout in the thread is only for debugging. If the code runs with thread safety, it can be removed. I read from another post that using cout in a thread safe manner may need to write a new wrapper around cout with a mutex. I want to avoid it.
My main concern is reading the variable in correct order in main.
My code fails(today is my first day with multithreading) and is as below along with observed output(selected parts). the code fails to keep the order of the output using cout(garbled output). Also I am not sure that the thread variable is correctly read by the main.
#include <thread>
#include <mutex>
#include <iostream>
int i = 0;
void safe_increment(std::mutex& i_mutex)
{
while(1)
{
std::lock_guard<std::mutex> lock(i_mutex);
++i;
std::cout << "thread: "<< std::this_thread::get_id() << ", i=" << i << '\n';
}
}
int main()
{
std::mutex i_mutex;
std::thread t1(safe_increment, std::ref(i_mutex));
while(1)
{
for(int k =0; k < 5; k++)
{
std::cout << "main: k =" << k << '\n';
}
std::lock_guard<std::mutex> lock(i_mutex);
std::cout << "main: i=" << i << '\n';
}
}
The output(selected parts) I get is
thread: 139711042705152, i=223893
thread: 139711042705152, i=223894
thread: 139711042705152, i=223895
main: i=223895
main: k =0
thread: main: k =1139711042705152
main: k =2
main: k =3
, i=main: k =4
223896
thread: 139711042705152, i=223897
thread: 139711042705152, i=223898
thread: 139711042705152, i=224801
thread: 139711042705152, i=224802
main: i=224802
main: k =0
main: k =1
thread: main: k =2
main: k =3
main: k =4
139711042705152, i=224803
thread: 139711042705152, i=224804
thread: 139711042705152, i=224805
i is properly synchronized with the mutex. well done! obviously this runs until you force it to stop, so when you do find a better way to end execution, be sure to join on your thread.
to fix the garbling, you need to synchronize on std::cout:
int main()
{
std::mutex i_mutex;
std::thread t1(safe_increment, std::ref(i_mutex));
while(1)
{
std::lock_guard<std::mutex> lock(i_mutex);//moved up here to sync on std::cout << k
for(int k =0; k < 5; k++)
{
std::cout << "main: k =" << k << '\n';
}
std::cout << "main: i=" << i << '\n';
if (i > 100) break;
}
t1.join(); //thread will continue and main will wait until it is done
//your thread needs to have some way out of its while(1) as well.
}
the thread can maybe be this:
void safe_increment(std::mutex& i_mutex)
{
while(1)
{
std::lock_guard<std::mutex> lock(i_mutex);
++i;
std::cout << "thread: "<< std::this_thread::get_id() << ", i=" << i << '\n';
if (i > 111) break;
}
}
I have a multi threaded program and I am profiling time taken starting before all pthread_create's and after all pthread_join's.
Now I find that this time, lets call it X, which is shown below in "Done in xms" is actually user + sys time of time output. In my app the number argument to a.out controls how many threads to spawn. ./a.out 1 spawn 1 pthread and ./a.out 2 spawns 2 threads where each thread does the same amount of work.
I was expecting X to be the real time instead of user + sys time. Can someone please tell me why this is not so? Then this really means my app is indeed running parallel without any locking between threads.
[jithin#whatsoeverclever tests]$ time ./a.out 1
Done in 320ms
real 0m0.347s
user 0m0.300s
sys 0m0.046s
[jithin#whatsoeverclever tests]$ time ./a.out 2
Done in 450ms
real 0m0.266s
user 0m0.383s
sys 0m0.087s
[jithin#whatsoeverclever tests]$ time ./a.out 3
Done in 630ms
real 0m0.310s
user 0m0.532s
sys 0m0.105s
Code
int main(int argc, char **argv) {
//Read the words
getWords();
//Set number of words to use
int maxWords = words.size();
if(argc > 1) {
int numWords = atoi(argv[1]);
if(numWords > 0 && numWords < maxWords) maxWords = numWords;
}
//Init model
model = new Model(MODEL_PATH);
pthread_t *threads = new pthread_t[maxWords];
pthread_attr_t attr;
void *status;
// Initialize and set thread joinable
pthread_attr_init(&attr);
pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE);
int rc;
clock_t startTime = clock();
for(unsigned i=0; i<maxWords; i++) {
//create thread
rc = pthread_create(&threads[i], NULL, processWord, (void *)&words[i] );
if (rc){
cout << "Error:unable to create thread: " << i << "," << rc << endl;
exit(-1);
}
}
// free attribute and wait for the other threads
pthread_attr_destroy(&attr);
for(unsigned i=0; i<maxWords; i++) {
rc = pthread_join(threads[i], &status);
if (rc){
cout << "Error:unable to join thread: " << i << "," << rc << endl;
exit(-1);
}
}
clock_t endTime = clock();
float diff = (((float)endTime - (float)startTime) / 1000000.0F ) * 1000;
cout<<"Done in "<< diff << "ms\n";
delete[] threads;
delete model;
}
The clock function is specifically documented to return the processor time used by a process. If you want to measure wall time elapsed, it's not the right function.
I'm trying to read a processes memory on Linux (Xubuntu, to be precise). I'm pretty new to Linux, though I've done this same read using Win32API ReadProcessMemory() before in Windows. The general idea is that I'm trying to develop some software for a game which will get my stats and upload them to a server, which will track my progress and keep a log of it. The end goal is to make a bot which will automatically play and farm data about the game. In order to do this, I need to be able to access the processes memory. In Windows, that's dead easy. In Linux, it's proving a little more complex.
I've found a memory address which contains information I want to read. The information is an int32, and it is stored at 84a1bd8. I found it using GameConqueror 0.13. The address remains correct after restarting, so it appears there is no ASLR (as there was in Windows). I also know the ProcessID (I can find this using task manager for now, though if someone knows a simple way to get a PID by either ClassName, Exe name, or similar, that would be great too!) So, that looks like it should be all I really need to use PTRACE_PEEKDATA to read the memory, right? Well, that's the problem, it doesn't appear to be. My code looks like this:
#include <iostream>
#include <string>
#include <sys/ptrace.h>
#include <errno.h>
using namespace std;
int main()
{
pid_t pid = 4847;
int addr = 0x84a1bd8;
long ret = ptrace(PTRACE_TRACEME, pid, NULL, NULL);
cout << "ptrace Status: " << ret << endl;
cout << "Errno: " << errno << endl;
ret = ptrace(PTRACE_PEEKDATA, pid, (void*)addr, NULL);
cout << "ptrace Status: " << ret << endl;
cout << "Errno: " << errno << endl;
ret = ptrace(PTRACE_DETACH, pid, NULL, NULL);
cout << "ptrace Status: " << ret << endl;
cout << "Errno: " << errno << endl;
return 0;
}
The output looks like this:
ptrace Status: 0
Errno: 0
ptrace Status: -1
Errno: 3
ptrace Status: -1
Errno: 3
Being quite new to Linux, I don't know where I'm to find error codes and how I can work out what this error actually means, and nor do I know if I am even declaring the address correctly. Should I declare it as an int in it's decimal equivalent? Is there anything I'm missing?
Thanks for your time
Found the solution to be that when using ptrace() you must call in an order:
ptrace(PTRACE_ATTACH, pid, NULL, NULL)
ptrace(PTRACE_PEEKDATA, pid, addr, NULL)
ptrace(PTRACE_DETACH, pid, NULL, NULL)
So the simple answer: You need to attach and detach before and after reading the memory.
It may also be useful to know that between the attach and detach commands, the process will sleep, meaning this method isn't so good for my purpose, but may be useful to others :)
Thanks to #PeterL. for your help.
Everyone knows the classic model of a process listening for connections on a socket and forking a new process to handle each new connection. Normal practice is for the parent process to immediately call close on the newly created socket, decrementing the handle count so that only the child has a handle to the new socket.
I've read that the only difference between a process and a thread in Linux is that threads share the same memory. In this case I'm assuming spawning a new thread to handle a new connection also duplicates file descriptors and would also require the 'parent' thread to close it's copy of the socket?
No. Threads share the same memory, so they share the same variables. If you close socket in parent thread, it will be also closed in child thread.
EDIT:
man fork: The child inherits copies of the parent’s set of open file descriptors.
man pthreads: threads share a range of other attributes (i.e., these attributes are process-wide rather than per-thread): [...] open file descriptors
And some code:
#include <cstring>
#include <iostream>
using namespace std;
#include <errno.h>
#include <fcntl.h>
#include <pthread.h>
#include <unistd.h>
// global variable
int fd = -1;
void * threadProc(void * param) {
cout << "thread: begin" << endl;
sleep(2);
int rc = close(fd);
if (rc == -1) {
int errsv = errno;
cout << "thread: close() failed: " << strerror(errsv) << endl;
}
else {
cout << "thread: file is closed" << endl;
}
cout << "thread: end" << endl;
}
int main() {
int rc = open("/etc/passwd", O_RDONLY);
fd = rc;
pthread_t threadId;
rc = pthread_create(&threadId, NULL, &threadProc, NULL);
sleep(1);
rc = close(fd);
if (rc == -1) {
int errsv = errno;
cout << "main: close() failed: " << strerror(errsv) << endl;
return 0;
}
else {
cout << "main: file is closed" << endl;
}
sleep(2);
}
Output is:
thread: begin
main: file is closed
thread: close() failed: Bad file descriptor
thread: end
In principle, Linux clone() can implement not only a new process (like fork()), or a new thread (like pthread_create perhaps), but also anything in between.
In practice, it is only ever used for one or the other. Threads created with pthread_create share the file descriptors with all other threads in the process (not just the parent). This is non-negotiable.
Sharing a file descriptor and having a copy is different. If you have a copy (like fork()) then all copies must be closed before the file handle goes away. If you share the FD in a thread, once one closes it, it's gone.
On Linux threads are implemented via the clone syscall using the CLONE_FILES flag:
If CLONE_FILES is set, the calling
process and the child processes share
the same file descriptor table. Any
file descriptor created by the calling
process or by the child process is
also valid in the other process.
Similarly, if one of the processes
closes a file descriptor, or changes
its associated flags (using the
fcntl(2) F_SETFD operation), the other
process is also affected.
Also have a look at the glibc source code for the details of how it is used in createthread.c:
int clone_flags = (CLONE_VM | CLONE_FS | CLONE_FILES | CLONE_SIGNAL
| CLONE_SETTLS | CLONE_PARENT_SETTID
| CLONE_CHILD_CLEARTID | CLONE_SYSVSEM
#if __ASSUME_NO_CLONE_DETACHED == 0
| CLONE_DETACHED
#endif
| 0);