I am learning linux programming and want to do the following. I would like to create a mini-logger that will work like syslog. I want to be able to replace syslog (not in practice but just to understand at every level how things work).
So in my code, I would write
#include "miniLogger.h"
....
....
miniLogger(DEBUG, "sample debug message");
----
----
Now, I am guessing I would need some kind of daemon to listen for incoming messages from my miniLogger and I have no experience with daemons. Can you point me in the right direction or give me a quick overview how messages can move from my API into a configurable destination.
I read the man pages but I need more of an overview of how APIs communicate with daemons in general.
syslogd listens for log messages over /dev/log, which is a unix domain socket. The socket is datagram-oriented, meaning the protocol is similar to udp.
Your log daemon should open the socket, set the socket to server mode, open a log file in write mode, ask to get notified of packets, parse the messages safely, and write them to the file. The important system calls for doing socket io are described in man 7 socket. To get notified of incoming data on the socket, you can use epoll or select.
syslog commonly uses a PF_LOCAL socket at /dev/log.
Related
Having looked at code for sometime now, I see most coders have used sockets for IPC over Pipes (Or FIFO to be specific).
Considering that there's only one client and one server, isn't it better to use FIFO over sockets?
Please educate me in this matter.
FIFO has following benefits:
they are atomic for writing if data len is less than PIPE_BUF
splice is almost garanted working with SPLICE_F_MOVE (no user space data copy kernel will move data between pipes)
they are more easy to set up in comparison with sockets
But on the other hand it is unidirectional i.e. you most likely need 2 separate fifo:
for writing data from client to server
for writing data from server to client
Or use one fifo but reopen it as need i.e. get data on server - reopen fifo WR_ONLY on server, reopen fifo RD_ONLY on client, get data on client and do vice versa after reading data from server.
http://man7.org/linux/man-pages/man7/pipe.7.html
Socket is bi-directional, pipe/fifo is uni-directional.
Socket can be stream or datagram, pipe/fifo are always streams.
Socket and fifo do not require related processes, whereas unnamed pipes do.
Socket can handle more than one peer.
I want to connect to Unix Domain Socket created by ZeroMQ (IPC model) via command nc. I can connect, but when I sending some messages then, my deamon, which is listening to this socket, is not getting any message...
I'm using nc like:
nc -U /path/to/socket
Very well, here's a longer version.
ZeroMQ implements a message queue transport system over the top of stream connections like sockets, named pipes, etc. To do this it runs a protocol called ZMTP over the top of the stream, which provides all the message demarcation, communication patterns, and so forth. It also has to deal with protocol errors in order to give itself some resiliency.
Comparison to a Web Browser
It's the same idea to a web browser and web server communicating using http over a socket. Http is used to transport html files. If you look at the data flowing over the socket you see the html mixed up with the messages involved in running the http protocol. And because http is a text based protocol, it looks kinda OK to the human eye.
Talking the Same Language
Thus when a program that uses the zmq libraries for communication connects a socket / named pipe / etc, it will be expecting to exchange data over that connection in the way defined by the ZMTP protocol (in the same way a web browser is expecting to talk to a server using http). If the program at the other end is also using zmq, then they're both talking the same protocol and everything is good.
Incompatible Protocols
However, if you connect a program that doesn't of itself use the ZMTP protocol such as a web browser, and that sends a http request, it's unlikely to mean anything. And the zmq library routines will no doubt receive the bytes that make up the http request, attempt to interpret it, fail to understand it, and ultimately reject it as garbage.
Similarly if the program that uses the zmq library wants to send messages, nothing will happen unless the underlying ZMTP protocol driver is content that it is communicating with something else that talks ZMTP. If anything at all emerges from netcap, it won't look anything like the message you were sending (it'll be jumbled up with the bytes that ZMTP uses).
Human Equivalent
The equivalent is an Englishman called Bob picking up the phone and dialling the number for his English friend called Alice living in Paris. However, if a Frenchman called Charlie answers the phone by mistake (wrong number), it'll be very difficult for them to exchange information. Meanwhile Eve, who's tapped the phone line, is laughing her head off at the ineptitude of these two people's failed attempt to communicate. (I make sweeping and partly justifiable generalisations about us Englishmen's poor ability to speak any other language).
Way Forward
There's a ZMQ binding available for almost everything, possibly even bash. Whatever it is you're trying to accomplish it's probably well worth while getting a decent binding of ZMQ for the programming or scripting language your using, and use that to provide a proper ZMQ endpoint.
I am working on a feature that relies on socket communication. One test case I am considering is abrupt socket failures. for ex: I want to close the socket the client has established from the shell, when the client & server are in the middle of some exchange. Is this doable though a shell command ?
Do you want to close it (per your title), or abruptly interrupt it in order to test error-handling code (per your explanation)?
You should be able to interrupt communications on a particular socket using netfilter and appropriate match engines (e.g. pid and port number), and you definitely can run iptables from the shell.
If you want to close the socket, you can just terminate the peer process.
No, not on standard linux, there is no such facility.
Context: Linux (Ubuntu), C, ZeroMQ
I have a server which listens on ipc:// SUB ZeroMQ socket (which physically is a Unix domain socket).
I have a client which should connect to the socket, publish its message and disconnect.
The problem: If server is killed (or otherwise dies unnaturally), socket file stays in place. If client attempts to connect to this stale socket, it blocks in zmq_term().
I need to prevent client from blocking if server is not there, but guarantee delivery if server is alive but busy.
Assume that I can not track server lifetime by some external magic (e.g. by checking a PID file).
Any hints?
Non-portable solution seems to be to read /proc/net/unix and search there for a socket name.
Without showing your code all of this is guesswork... that said...
If you have a PUB/SUB pair, the PUB will hang around to make sure that its message gets through. Perhaps you're not using the right type of zmq pair. Sounds more like you have a REP/REQ pair instead.
This way, once you connect from the client (the REQ side), you can do a zmq_poll to determine if the socket is available for writing. If yes, then go ahead with your write, otherwise shutdown the client and handle the error condition (if it is an error in your system).
Maybe you can try to connect to the socket first, with your own native socket. If the connection succeeds, it's quite high possibility your publisher could work fine.
There is another solution. Don't use ipc:// sockets. Instead use something like tcp://127.0.0.101:10001. On most UNIXes that will be almost as fast as IPC because the OS recognizes that it is a local connection and shortcuts the full IP stack processing.
Is there an ipc option to get the last message in message queue but not removing it?
I want this to allow many clients reading same messages from the same server..
Edit:
Server and clients are on the same machine!
Thanks
I don't believe there is any way to do that using either system v or POSIX message queues. Furthermore, AFAIK neither API allows you to send messages to a remote machine, so unless your clients are running on the same host as the server, you will need to use a higher-level technology.