I have created one application with server and client class which have methods for creating either creating a tcp socket or udp socket. Now my requirement is i have created two application instances of this application. Since application is in c++ in unix environment I am using putty software to run the application. I have opened two instances of putty. But now my requirement is as follows:
There can be multiple communication instances between the 2 application instances
Each communication instance, There can be multiple communication instances between the 2 application instances
Each communication instance, can be either UDP or TCP (determined from the config file)be either UDP or TCP (determined from the config fil
Anybody who knows how to create such multiple instances.
Hmm, so there are two processes, but they want the processes to be able to communicate with each other via more than one pair of sockets? i.e. you could have two (or more) TCP socket connections between the two processes, and/or two (or more) pairs of UDP sockets sending packets back and forth.
If my above paragraph is correct (i.e. if I haven't misunderstood the request), that is certainly possible, although it's not terribly obvious what advantage you'd gain by doing it. Nevertheless, what you'd need to do is have each instance of your application create multiple sockets (either by socket()+bind() for a UDP socket, or by socket()+bind()+listen()+accept() for accepting an incoming TCP connection, or by socket()+connect() to initiate a TCP connection to the other program instance.
The tricky part with managing multiple sockets is handling the waiting correctly. With just one socket you can often get away with using the default blocking I/O semantics, and that way you can end up treating the socket something like a file, and just let each send() or recv() operation (etc) take however long it needs to take to complete before it returns to your calling function.
With more than one socket, on the other hand, you typically want to be able to respond to data on any of the sockets that are ready, which means that you can't just block waiting on any one particular socket, because if you do that, you may end up stuck waiting for a long time (potentially forever!) before that blocking call returns, and in the meantime you are unable to handle any data coming in from any of the other sockets. (The problem becomes particularly obvious when one of the connections is to a computer whose plug was just pulled, as it will typically take the TCP stack several minutes to figure out that the remote computer has gone away)
To deal with the problem, you'll typically want to either use non-blocking I/O and a socket-multiplexing call (e.g. poll() or select() or kqueue()), or spawn multiple threads and let each thread handle a single socket. Neither approach is particularly easy -- the socket-multiplexing approach works well once you get the hang of it, but the multiplexing calls' semantics are somewhat complex, and it takes a while to understand fully how it is intended to work. Non-blocking I/O complicates things further, since it means your code has to correctly deal with partial reads and writes. The multithreading approach seems simpler at first, but it has its own much larger and more subtle set of 'gotchas' (race conditions, deadlocks) that can cause much pain in the long run if you aren't very careful about what the threads are doing and how.
ps Since you're in a Unix environment, a third possible approach would be to fork() a child process for each socket. This would be similar to the multithreading approach, except a bit safer since your "threads" would actually be processes and each would have their own separate memory space, and thus they'd be less likely to trip over each other while doing their work. The downside would be higher memory usage, and also it becomes a bit harder (and slower) for the processes to communicate with each other due to the process space separation.
Related
I am currently developing a networking software that uses a datagram socket (UDP) to send data to clients. Whenever I'd like to send data to a client I am currently invoking sendto() and passing the respective parameters. Yet I am wondering whether or not making a blocking call to sendto() from multiple threads at the same time is good idea or whether data might get interleaved or corrupted in some other way.
I have already found this answer: is winsock2 thread safe? but I am not sure if this holds true for sendto() as it does for send()
System calls are not atomic, you can't assume they are thread safe.
Thread safety depends on the system implementation. But thread safety just means you won't encounter crashes or memory corruption, it doesn't tell you anything about the behaviour. For example, you may have data interleaved in what you sent, with no respect to your threads calling order.
If you're working on Windows, Winsock2 seems to be thread safe on recent versions of the os. But once again, it doesn't mean it'll behave as you expect.
Rather than using several threads to send to or receive from a socket, you should consider using IO ports, which are meant for multithreading and asynchronous processing.
It is a system call, and system calls are atomic, and therefore thread-safe.
It is UDP, and UDP send()/sendto()/sendmsg() sends a single datagram, and UDP guarantees datagram integrity, if it arrives at all.
But IMHO two threads writing to the same socket is probably never going to work at the application level without extreme care at a higher level.
I am trying to write a server program which supports one client till now and over the few days i was trying to develop it, I concluded i needed threads. The reason for such a decision was since I take input from a wifi socket and later process it and finally write to a file, the processing time is slow and hence i needed a input thread -> circular buffer -> output thread pattern with producer consumer model which is quite common in network programming.
Now, The situation becomes complicated, as I need to manage client disconnection and re connection. I thought of using pthread_exit() and cleaning up all the semaphores and then re initializing them each time the single client re connects.
My question is that is this a efficient approach i.e. everytime killing the threads and semaphores and re creating them. Are there any better solutions.
Thanks.
My question is that is this a efficient approach i.e. everytime killing the threads and semaphores and re creating them. Are there any better solutions.
Learn how to use non-blocking sockets and an event loop. Or use a library that provides TCP sessions for you using non-blocking sockets under the hood. Such as boost::asio.
Learn how to use multi-threading without polluting your code with any synchronization primitives by using message passing to communicate between threads, not shared state. The event loop library you use for non-blocking I/O should also provide means for cross-thread message passing.
Some comments and suggestions.
1-In TCP detecting that the other side has silently disconnected it very difficult if not impossible. A client could disconnect sending a RST TCP message to the server or sending a FIN message, this is the good case. Sometimes the client can disconnect without notice (crash, cable disconnection, etc).
One suggestion here is that you consider the way client and server will communicate. For example, you can use function “select” to set a timeout for receiving a message from client and detect a silent client.
Additionally, depending on the programming language and operating system you may need to handle broken pipe (SIGPIPE) signal (in Linux, with C/C++), for a server trying to send a message through a connection closed by the client.
2-Regarding semaphores, you shouldn’t need to clean semaphores in any especial way when a client disconnect. By applying common good practices of locking and unlocking mutexes should be enough. Also with resources like file descriptors, you need to release them before ending the thread either by returning from the thread start function or with pthread_exit. Maybe I didn’t understand this part of the question.
3-Regarding threads: if you work with multiple threads to optimum is to have a pool of pre-created consumer/worker threads that will check the circular buffer to consume the next available connection. Creating and destroying threads is costly for the operating system.
Threads are resource consuming and you may exhaust operating system resources if you need to create 1,000 threads for example.
Another alternative, is to have only one consumer thread that manages all connections (sockets) asynchronously: a) Each connection has its own state. b) The main thread goes through all connections and use function “select” to detect when connection reads or a writes are ready. 3)Use of non-blocking sockets but this is not essential because from select you know which sockets are ready and will not block.
You can use functions select, poll, epoll.
One link about select and non-blocking sockets: Using select() for non-blocking sockets
Other link with an example: http://linux.die.net/man/2/select
I've read the C10K doc as well as many related papers on scaling up a socket server. All roads point to the following:
Avoid the classic mistake of "thread per connection".
Prefer epoll over select.
Likewise, legacy async io mechanism in unix may be hard to use.
My simple TCP server just listens for client connections on a listen socket on a dedicated port. Upon receiving a new connection, parses the request, and sends a response back. Then gracefully closes the socket.
I think I have a good handle on how to scale this up on a single thread using epoll. Just one loop that calls epoll_wait for the listen socket as well as for the existing client connections. Upon return, the code will handle new creating new client connections as well as managing state of existing connections depending on which socket just got signaled. And perhaps some logic to manage connection timeouts, graceful closing of sockets, and efficient resource allocation for each connection. Seems straightforward enough.
But what if I want to scale this to take advantage of multiple threads and multiple cpu cores? The core idea that springs to mind is this:
One dedicated thread for listening for incoming connections on the TCP listen socket. Then a set of N threads (or thread pool) to handle all the active concurrent client connections. Then invent some thread safe way in which the listen thread will "dispatch" the new connection (socket) to one of the available worker threads. (ala IOCP in Windows). The worker thread will use an epoll loop on all the connections it is handling to do what the single threaded approach would do.
Am I on the right track? Or is there a standard design pattern for doing a TCP server with epoll on multiple threads?
Suggestions on how the listen thread would dispatch a new connection to the thread pool?
Firstly, note that it's C*10K*. Don't concern yourself if you're less than about 100 (on a typical system). Even then it depends on what your sockets are doing.
Yes, but keep in mind that epoll manipulation requires system calls, and their cost may or may not be more expensive than the cost of managing a few fd_sets yourself. The same goes for poll. At low counts its cheaper to be doing the processing in user space each iteration.
Asynchronous IO is very painful when you're not constrained to just a few sockets that you can juggle as required. Most people cope by using event loops, but this fragments and inverts your program flow. It also usually requires making use of large, unwieldy frameworks for this purpose since a reliable and fast event loop is not easy to get right.
The first question is, do you need this? If you're handily coping with the existing traffic by spawning off threads to handle each incoming request, then keep doing it this way. The code will be simpler for it, and all your libraries will play nicely.
As I mentioned above, juggling simultaneous requests can be complex. If you want to do this in a single loop, you'll also need to make guarantees about CPU starvation when generating your responses.
The dispatch model you proposed is the typical first step solution if your responses are expensive to generate. You can either fork or use threads. The cost of forking or generating a thread should not be a consideration in selecting a pooling mechanism: rather you should use such a mechanism to limit or order the load placed on the system.
Batching sockets onto multiple epoll loops is excessive. Use multiple processes if you're this desperate. Note that it's possible to accept on a socket from multiple threads and processes.
I would guess you are on the right track. But I also think details depend upon the particular situation (bandwidh, request patterns, indifidual request processing, etc.). I think you should try, and benchmark carefully.
I'm looking using to transfer an accept()ed socket between processes using sendmsg(). In short, I'm trying to build a simple load balancer that can deal with a large number of connections without having to buffer the stream data.
Is this a good idea when dealing with a large number (let's say hundreds) of concurrent TCP connections? If it matters, my system is Gentoo Linux
You can share the file descriptor as per the previous answer here.
Personally, I've always implemented servers using pre-fork. The parent sets up the listening socket, spawns (pre-forks) children, and each child does a blocking accept. I used pipes for parent <-> child communication.
Until someone does a benchmark and establishes how "hard" it is to send a file descriptor, this remains speculation (someone might pop up: "Hey, sending the descriptor like that is dirt-cheap"). But here goes.
You will (likely, read above) be better off if you just use threads. You can have the following workflow:
Start a pool of threads that just wait around for work. Alternatively you can just spawn a new thread when a request arrives (it's cheaper than you think)
Use epoll(7) to wait for traffic (wait for connections + interesting traffic)
When interesting traffic arrives you can just dispatch a "job" to one of the threads.
Now, this does circumvent the whole descriptor sending part. So what's the catch ? The catch is that if one of the threads crashes, the whole process crashes. So it is up to you to benchmark and decide what's best for your server.
Personally I would do it the way I outlined it above. Another point: if the workers are children of the process doing the accept, sending the descriptor is unnecessary.
Context: OS: Linux (Ubuntu), language: C (actually Lua, but this should not matter).
I would prefer a ZeroMQ-based solution, but will accept anything sane enough.
Note: For technical reasons I can not use POSIX signals here.
I have several identical long-living processes on a single machine ("workers").
From time to time I need to deliver a control message to each of processes via a command-line tool. Example:
$ command-and-control worker-type run-collect-garbage
Each of workers on this machine should receive a run-collect-garbage message. Note: it would be perfect if the solution would somehow work for all workers on all machines in the cluster, but I can write that part myself.
This is easily done if I will store some information about running workers. For example keep the PIDs for them in a known location and open a control Unix domain socket on a known path with a PID somewhere in it. Or open TCP socket and store host and port somewhere.
But this would require careful management of the stored information — e.g. what if worker process suddenly dies? (Nothing unmanageable, but, still, extra fuss.) Also, the information needs to be stored somewhere, thus adding an extra bit of complexity.
Is there a good way to do this in PUB/SUB style? That is, workers are subscribers, command-and-control tool is a publisher, and all they know is a single "channel url", so to say, on which to come for messages.
Additional requirements:
Messages to the control channel must wake up workers from the poll (select, whatever)
loop.
Message delivery must be guaranteed, and it must reach each and every worker that is listening.
Worker should have a way to monitor for messages without blocking — ideally by the poll/select/whatever loop mentioned above.
Ideally, worker process should be "server" in a sense — he should not bother about keeping connections to the "channel server" (if any) persistent etc. — or this should be done transparently by the framework.
Usually such a pattern requires a proxy for the publisher, i.e. you send to the proxy which immediately accepts delivery and then that reliably forwads to the end subscriber workers. The ZeroMQ guide covers a few different methods of implementing this.
http://zguide.zeromq.org/page:all
Given your requirements, Steve's suggestion does seem the simplest: run a daemon which listens on two known sockets - the workers connect to that and the command tool pushes to it which redistributes to connected workers.
You could do something complicated that would probably work, by effectively nominating one of the workers. For example, on startup workers attempt to bind() a PUB ipc:// socket somewhere accessible, like tmp. The one that wins bind()s a second IPC as a PULL socket and acts as a forwarder device on top of it's normal duties, the others connect() to the original IPC. The command line tool connect()s to the second IPC, and pushes it's message. The risk there is that the winner dies, leaving a locked file. You could identify this in the command line tool, rebind then sleep (to allow the connections to be established). Still, that's all a little bit complex, I think I'd go with a proxy!
I think what you're describing would fit well with a gearmand/supervisord implementation.
Gearman is a great task queue manager and supervisord would allow you to make sure that the process(es) are all running. It's TCP based too so you could have clients/workers on different machines.
http://gearman.org/
http://supervisord.org/
I recently set something up with multiple gearmand nodes, linked to multiple workers so that there's no single point of failure
edit: Sorry - my bad, I just re-read and saw that this might not be ideal.
Redis has some nice and simple looking pub/sub functionality that I've not used yet but sounds promising.
Use a mulitcast PUB/SUB. You'll have to make sure the pgm option is compiled into your ZeroMQ distribution (man 7 zmq_pgm).