I am trying to study all the different ways I can find in order for a process on a linux machine to establish IPC with a second process (not a child) on the same machine. I did find that socket can be used given I know the path on file system the second process is listening to.
Is IPC communication with an second process possible in other ways ? I don't want the first process to know the pid/uid of the second process. The scenario is more towards communicating with an untrusted process, by a different author, on the same machine, but still have some information like where the socket in the second process is listening ?
Shared Memory and Sockets can be used for IPC communications between the processes that are not related to each other. Pipes can be used for IPC communications between the parent and child processes.
Shared memory is the fastest form of interprocess communication. In all other methods, the system calls copy the data from the memory area of one process to another process. The drawback of shared memory is, you need to implement synchronization methods to avoid race condition.
Sockets interfaces enables communication in a connection oriented way between the process across the network as well as locally. UNIX domain sockets provides local IPC using a known file path.
Possible ways:
Shared memory (shmget, shmctl, shmat, shmdt)
FIFO queues aka named pipes (mkfifo)
Message queues (msgget, msgsnd, msgrcv, msgctl)
Semaphores - for synchronization (semget, semctl, semop)
Hint:
Useful commands: ipcs, ipcmk, ipcrm
Also, you can use mmap POSIX standard call with flag MAP_SHARED specified. Here https://www.cs.purdue.edu/homes/fahmy/cs503/mmap.txt you can find an example of using it.
Related
I have doubt in using Linux Pipes for IPC. My question is
Can Linux pipes can be used to communicate between the processes running on different machines?.
Thanks,
No, you can't use only pipe to communicate between different machines, because pipe is defined as local machine communication method (IEEE standard says that it creates two file descriptors in current process. Descriptors usually can't be send to other machine, only inherited from parent or passed via local machine sockets).
But you can try to use pipe to some external socket program, like netcat, which will resend all data over tcp socket, and remote netcat will replay it back into the program.
And if you are developing some application, it can be better to use tcp sockets directly.
PS: The IPC - Inter-process communication - AFAIK means communications between different processes on one (same) machine (linux IPC from Linux Programmer's Guide 1995).
PPS: If sockets are hard to work with them directly, you may choose some Message Passing library or standard. For example MPI standard (OpenMPI, MPICH libraries) is often used to communicate between many machines in tightly-coupled computing clusters, and there are some popular interfaces like RPC (Remote procedure call, several implementations) or ZeroMQ
Pipe is only used for communication between related process on the same host (eg. parent and child process).
I'm writing a daemon that interfaces with a USB device (an Arduino). This daemon is continuously aware of the current state.
Now I want to be able to interface with this daemon through a client program, also to be written in Perl. This client must be able to query the daemon for its current state and it must be able to update the daemon with settings.
I'm on Linux (x86_64)
I don't want to use an intermediate file and preferably simultaneous queries are easily implemented.
What is the name of such a mechanism? What Perl libraries can I use or should I avoid? What should I DuckDuckGo for?
Probably, you need to implement an event loop to allow doing the tasks of your USB device communication and serve information to the new interface. This concept will change the way you solved the problem, but I think is the better approach.
You can search at CPAN for modules like POE and AnyEvent
The idea is to build an event loop that handles a TCP socket in order to send & receive information from te interface
There's a process and a lot of child processes (Node.js)
which method is preferable to use for real-time communication between processes within a single machine: Linux IPC or TCP / UDP?
What are the limitations of the IPC?
Suitable whether IPC to transfer large amounts of information with minimum delay?
AFAIK, TCP/IP (even on localhost) is significantly slower than Linux pipes or socketpairs, so you should avoid TCP/IP (and probably even UDP/IP) if speed is a major concern.
When we doing network programming, no matter you use multi-process, multi-thread or select/poll(epoll), there is only one process/thread to deal with accept the connection on same port. And if you want to take advantage of multi-cores, you need to create worker processes/threads. But what about the bound is dealing with network connections? Is there a way to take advantage of multi-core when dealing with network connections?
I found some materials. And seems this is hard to complete.
Three-way hand shaking will be implicit done by the kernel. And in smp structure operating system will be divided into several critical zones. The same critical zone can't be run on more than one core at the same time.
All modern operating systems that run on PC hardware already have their network stacks heavily optimized for multi-core CPUs. For example, the packet handling code that pushes data to and from the network card is going to be independent of the TCP/IP stack code so a hardware interrupt can run to completion without disturbing the TCP code.
For most real-world applications though, the bulk of the work is between the packets. Data that comes in has to be processed and data that goes out has to be generated. That's up to application code, and that code can take advantage of multiple cores either by using multiple threads or multiple processes. How you do that best is very application and operating system specific. Windows, for example, has I/O completion ports which combine job discovery with multi-threaded job dispatch. Linux has epoll.
With just the network traffic, that's almost soley done by the network card (i.e. not the computer's CPU). Communication with the network card is usually single-threaded (queued by the OS so you can send/receive on multiple threads) because a NIC can only push/pop stuff off it's stack one-at-a-time.
It's up to your process to do what it needs in response to received data. That can be done on one thread and you can spawn other threads upon receive of data on that master thread and divide work up that way. If you have a language that supports asynchronous communications, I would try to get it to do most of the work to use multiple threads.
I'm preparing to write a multithread network application. At the moment I'm wondering what's the best thread pattern for my program. Whole application will handle up to 1000 descriptors (local files, network connections on various protocols and additional descriptors for timers and signals handling). Application will be optimized for Linux. Program will run on regular personal computers, so I assume, that they will have at least Pentium 4.
Here's my current idea:
One thread will handle network I/O
using epoll.
Second thread will
handle local-like I/O (disk I/O,
timers, signal handling) using epoll
Third thread
will handle UI (CLI, GTK+ or Qt)
Handling each network connection in separate thread will kill CPU because of too many context switches.
Maybe there's better way to do this?
Do you know any documents/books about designing multirhread applications? I'm looking for answers on questions like: What's the rational number of threads? etc.
You're on the right track. You want to use a thread pool pattern to handle the networking rather than one thread per network connection.
This website may also be helpful to you and lists the most common design patterns and in what situations they can be used.
http://sourcemaking.com/design_patterns/
To handle the disk I/O you might like to consider using mmap under linux. It's very fast and efficient. That way, you will let the kernel do the work and you probably won't need a separate thread for that.
I'm currently playing with Boost::asio which seems to be quite good. It uses epoll on linux. As it appears you are using a cross platform gui toolkit like Qt, then boost asio will also provide cross platform support so you will be able to use it on windows or linux. I think there might be a cross platform mmap too.