Find out whether tcp port is bound (not listening) using bash - linux

I am trying to determine whether a TCP port that was bound by a process, that was recently started, is actually in use by that particular process.
Take this program.cpp
int daemonport = 11234;
struct sockaddr_in loopback;
memset ((char*) &loopback, 0, sizeof (loopback));
socklen_t len = sizeof (loopback);
loopback.sin_family = AF_INET;
loopback.sin_port = htons (daemonport);
loopback.sin_addr.s_addr = htonl (INADDR_LOOPBACK);
daemonfd = socket (AF_INET, SOCK_STREAM, 0);
if (daemonfd < 0)
{
errx (EXIT_FAILURE, "Critical error");
}
if (bind (daemonfd, (struct sockaddr*) &loopback, sizeof (loopback)) != 0)
{
errx (EXIT_FAILURE, "Daemon already running, TCP port: '%d'", daemonport);
}
if (getsockname (daemonfd, (struct sockaddr*) &loopback, &len) != 0)
{
errx (EXIT_FAILURE, "Critical error");
}
printf ("%d\n", ntohs (loopback.sin_port));
if (daemon (1, 0) < 0)
{
close (daemonfd);
errx (EXIT_FAILURE, "Failed to daemonize!");
}
// event loop...
close (daemonfd);
Now with the tcp socket bound (but not listening) to port 11234 I want to check whether the port is bound by the process using a bash script.
I tried various netstat and lsof patterns w/o success:
netstat -a | grep ':11234' as well as lsof -i :11234.
They all don't print a line with the bound port.
But when I try to run the program a 2nd time it errors out with:
Daemon already running, TCP port: '11234'

Assuming Linux, start with this:
netstat --inet -n -a -p | grep ':myport'
and see what you're getting. The --inet keeps from showing IP6 and Unix domain sockets. -n shows numerical results and not names translated from the port number. -p tells you which process is listening on it.
If any of those lines lay "LISTEN" then a process is lisening on that port. However, any open connections using that port (even "TIME_WAIT") will prevent the port from being re-opened unless you use the SO_REUSEPORT option every time you bind to it.
If that command isn't showing you anything then nothing is listening on that port which means there must be a problem with your program.
You're printing an error message but assuming the problem is something already running. Print out the errno value (use perror(...)) so you can exactly what the problem is.

By way of example, to check to see if port 56789 is available locally:
port=56789
retval=$(python3 -c 'import socket; s=socket.socket(); s.bind(("", '"${port}"')); print(s.getsockname()[1]); s.close()' 2>/dev/null)
echo "$retval"
This will print a blank line if the port is already bound, and will print 56789 if it is not bound. If port 56789 was recently used and closed, but the TIME_WAIT period has not yet elapsed (typically one or two minutes), then the port will not be available and the above code will not echo 56789.
I realize that this is a bit of a cheat, because it also uses python, but it is bash scriptable if python 3 is available. No sudo required.

Related

Race condition when reading from /dev/pts/x

I read in this article that when a process reads from a tty device, say /dev/pts/1, the tty driver/line discipline (when in cooked mode) buffers the data, and only when enter is pressed, the data is passed to the process. I ran the following experiment:
I open a terminal, and note the tty it's using. Let's say it's /dev/pts/0.
Now I open another terminal, using /dev/pts/1 for the sake of the manner, and run a process, which only executes the following function:
// passing /dev/pts/0 to the function
int read_from_tty(char *tty)
{
int bytes_read = 0;
int fd = 0;
char buffer[100];
fd = open(tty, O_RDWR);
if(-1 == fd)
{
printf("Couldn't open pts\n");
return 1;
}
printf("File opened: %d\n", fd);
while(1)
{
bytes_read = read(fd, buffer, 100);
if(-1 == bytes_read)
{
perror("read");
}
if(-1 == write(1, buffer, bytes_read))
{
perror("write:");
}
memset(buffer, 0, 100);
}
}
Now I start typing characters in the /dev/pts/0 terminal, and I see that characters are appearing mostly in the /dev/pts/1 terminal, but one in something like 10 chars I see a char in the /dev/pts/0 terminal.
If what's written in the article is true, seemingly, the characters should have been buffered in the line discipline, and only when I press enter be passed to one of the reading processes (assuming the bash is just sitting blocked on read).
Could someone explain?
EDIT
Examining this a bit further. I add the following line to the code above:
...
bytes_read = read(fd, buffer, 100);
if(-1 == bytes_read)
{
perror("read");
}
printf("Bytes read: %d\n", bytes_read);
if(-1 == write(1, buffer, bytes_read))
{
...
I can see that when I read /dev/pts/0, it only reads 1 byte at a time. Whereas, if I run it with /dev/pts/1 (actually reading from stdin), it does read a whole line.
Could someone explain?
Actually, bash sets the terminal in non canonical mode when reading from the terminal and when it gets an end of line, it sets back the terminal in canonical mode to run the command line.
The same experience can be done with two terminals:
Terminal#1 (/dev/pts/6): launch strace /bin/bash
Terminal#2: launch strace cat /dev/pts/6
The bash shell on terminal#1 deactivates the canonical mode and calls pselect() to wait for the input:
$ strace /bin/bash
[...]
ioctl(0, TCGETS, {B38400 opost isig -icanon -echo ...}) = 0
[...]
pselect6(1, [0], NULL, NULL, NULL, {[], 8}
On terminal#2, the cat command merely calls a blocking read() to get chars from the terminal:
$ strace cat /dev/pts/6
[...]
openat(AT_FDCWD, "/dev/pts/6", O_RDONLY) = 3
[...]
read(3,
So, both bash and cat are reading concurrently on the terminal. When we type characters in terminal#1, pselect() returns to indicate that a character is available and then bash calls a blocking read() to get the char. But the concurrent read() from cat stills the characters between the calls to pselect() and read() by bash. Sometimes, bash is able to get a character before cat.
Here is an example where pselect() returns because a character is available (I typed "Y") and a following read() is called to get it on terminal#1:
pselect6(1, [0], NULL, NULL, NULL, {[], 8}) = 1 (in [0])
read(0,
But read() from cat on the other terminal succeeded to get the character before the read() of bash:
write(1, "Y", 1Y) = 1
read(3,
Sometimes, bash is able to get the typed character before cat. Generally, when it is blocked on its read() call (that is to say it missed the character detected by pselect() but it will be able to get one of the subsequent typed characters before one of the calls to read() by cat)...
Side note
When we launch stty -a under bash, the display shows that the terminal is in canonical mode:
$ stty -a
[...]
isig icanon iexten echo echoe echok -echonl -noflsh -xcase -tostop -echoprt echoctl echoke -flusho -extproc
This is because of the fact that bash reactivates the canonical mode before launching the stty command.

ssh port forwarding ("ssh -fNL") doesn't work via expect spawn to automatically provide password

I know that to do port forwarding, the command is ssh -L. I also use other options to decorate it. So for example, a final full command may look like this ssh -fCNL *:10000:127.0.0.1:10001 127.0.0.1. And everything just works after entering password.
Then, because there is not only one port need to be forwarded, I decide to leave the job to shell script and use expect(tcl) to provide passwords(all the same).
Although without a deep understanding of expect, I managed to write the code with the help of Internet. The script succeeds spawning ssh and provides correct password. But I end up finding there is no such process when I try to check using ps -ef | grep ssh and netstat -anp | grep 10000.
I give -v option to ssh and the output seems to be fine.
So where is the problem? I have searched through Internet but most of questions are not about port forwarding. I'm not sure whether it is proper to use expect while I just want to let script automatically provide password.
Here the script.
#!/bin/sh
# Port Forwarding
# set -x
## function definition
connection ()
{
ps -ef | grep -v grep | grep ssh | grep $1 | grep $2 > /dev/null
if [ $? -eq 0 ] ; then
echo "forward $1 -> $2 done"
exit 0
fi
# ssh-keygen -f "$HOME/.ssh/known_hosts" -R "127.0.0.1"
/usr/bin/expect << EOF
set timeout 30
spawn /usr/bin/ssh -v -fCNL *:$1:127.0.0.1:$2 127.0.0.1
expect {
"yes/no" {send "yes\r" ; exp_continue}
"password:" {send "1234567\r" ; exp_continue}
eof
}
catch wait result
exit [lindex \$result 3]
EOF
echo "expect ssh return $?"
echo "forward $1 -> $2 done"
}
## check expect available
which expect > /dev/null
if [ $? -ne 0 ] ; then
echo "command expect not available"
exit 1
fi
login_port="10000"
forward_port="10001"
## check whether the number of elements is equal
login_port_num=$(echo ${login_port} | wc -w)
forward_port_num=$(echo ${forward_port} | wc -w)
if [ ${login_port_num} -ne ${forward_port_num} ] ; then
echo "The numbers of login ports and forward ports are not equal"
exit 1
fi
port_num=${login_port_num}
## provide pair of arguments to ssh main function
index=1
while [ ${index} -le ${port_num} ] ; do
login_p=$(echo ${login_port} | awk '{print $'$index'}')
forward_p=$(echo ${forward_port} | awk '{print $'$index'}')
connection ${login_p} ${forward_p}
index=$((index + 1))
done
Here the output from script
spawn /usr/bin/ssh -v -fCNL *:10000:127.0.0.1:10001 127.0.0.1
OpenSSH_7.2p2 Ubuntu-4ubuntu2.10, OpenSSL 1.0.2g 1 Mar 2016
...
debug1: Next authentication method: password
wang#127.0.0.1's password:
debug1: Enabling compression at level 6.
debug1: Authentication succeeded (password).
Authenticated to 127.0.0.1 ([127.0.0.1]:22).
debug1: Local connections to *:10000 forwarded to remote address 127.0.0.1:10001
debug1: Local forwarding listening on 0.0.0.0 port 10000.
debug1: channel 0: new [port listener]
debug1: Local forwarding listening on :: port 10000.
debug1: channel 1: new [port listener]
debug1: Requesting no-more-sessions#openssh.com
debug1: forking to background
expect ssh return 0
forward 10000 -> 10001 done
This should work for you:
spawn -ignore SIGHUP ssh -f ...
UPDATE:
Another workaround is:
spawn bash -c "ssh -f ...; sleep 1"
UPDATE 2 (a bit explanation):
ssh -f calls daemon() to make itself a daemon. See ssh.c in the souce code:
/* Do fork() after authentication. Used by "ssh -f" */
static void
fork_postauth(void)
{
if (need_controlpersist_detach)
control_persist_detach();
debug("forking to background");
fork_after_authentication_flag = 0;
if (daemon(1, 1) == -1)
fatal("daemon() failed: %.200s", strerror(errno));
}
daemon() is implemented like this:
int
daemon(int nochdir, int noclose)
{
int fd;
switch (fork()) {
case -1:
return (-1);
case 0:
break;
default:
_exit(0);
}
if (setsid() == -1)
return (-1);
if (!nochdir)
(void)chdir("/");
if (!noclose && (fd = open(_PATH_DEVNULL, O_RDWR, 0)) != -1) {
(void)dup2(fd, STDIN_FILENO);
(void)dup2(fd, STDOUT_FILENO);
(void)dup2(fd, STDERR_FILENO);
if (fd > 2)
(void)close (fd);
}
return (0);
}
There's a race condition (not sure if its the correct term for here) between _exit() in the parent process and setsid() in the child process. Here _exit() would always complete first since "the function _exit() terminates the calling process immediately" and setsid() is much more heavy weight. So when the parent process exits, setsid() is not effective yet and the child process is still in the same session as the parent process. According to the apue book (I'm referring to the 2005 edition, Chapter 10: Signals), SIGHUP "is also generated if the session leader terminates. In this case, the signal is sent to each process in the foreground process group."
In brief:
Expect allocates a pty and runs ssh on the pty. Here, ssh would be running in a new session and be the session leader.
ssh -f calls daemon(). The parent process (session leader) calls _exit(). At this time, the child process is still in the session so it'll get SIGHUP whose default behavior is to terminate the process.
How the workarounds works:
The nohup way (spawn -ignore SIGHUP) is to explicitly ask the process to ignore SIGHUP so it'll not be terminated.
For bash -c 'sshh -f ...; sleep 1', bash would be the session leader and sleep 1 in the end prevents the session leader from exiting too soon. So after sleep 1, the child ssh process's setsid() has already done and child ssh is already in a new process session.
UPDATE 3:
You can compile ssh with the following modification (in ssh.c) and verify:
static int
my_daemon(int nochdir, int noclose)
{
int fd;
switch (fork()) {
case -1:
return (-1);
case 0:
break;
default:
// wait a while for child's setsid() to complete
sleep(1);
// ^^^^^^^^
_exit(0);
}
if (setsid() == -1)
return (-1);
if (!nochdir)
(void)chdir("/");
if (!noclose && (fd = open(_PATH_DEVNULL, O_RDWR, 0)) != -1) {
(void)dup2(fd, STDIN_FILENO);
(void)dup2(fd, STDOUT_FILENO);
(void)dup2(fd, STDERR_FILENO);
if (fd > 2)
(void)close (fd);
}
return (0);
}
/* Do fork() after authentication. Used by "ssh -f" */
static void
fork_postauth(void)
{
if (need_controlpersist_detach)
control_persist_detach();
debug("forking to background");
fork_after_authentication_flag = 0;
if (my_daemon(1, 1) == -1)
// ^^^^^^^^^
fatal("my_daemon() failed: %.200s", strerror(errno));
}

Who runs first in fork, with contradicting results

I have this simple test:
int main() {
int res = fork();
if (res == 0) { // child
printf("Son running now, pid = %d\n", getpid());
}
else { // parent
printf("Parent running now, pid = %d\n", getpid());
wait(NULL);
}
return 0;
}
When I run it a hundred times, i.e. run this command,
for ((i=0;i<100;i++)); do echo ${i}:; ./test; done
I get:
0:
Parent running now, pid = 1775
Son running now, pid = 1776
1:
Parent running now, pid = 1777
Son running now, pid = 1778
2:
Parent running now, pid = 1779
Son running now, pid = 1780
and so on; whereas when I first write to a file and then read the file, i.e. run this command,
for ((i=0;i<100;i++)); do echo ${i}:; ./test; done > forout
cat forout
I get it flipped! That is,
0:
Son running now, pid = 1776
Parent running now, pid = 1775
1:
Son running now, pid = 1778
Parent running now, pid = 1777
2:
Son running now, pid = 1780
Parent running now, pid = 1779
I know about the scheduler. What does this result not mean, in terms of who runs first after forking?
The forking function, do_fork() (at kernel/fork.c) ends with setting the need_resched flag to 1, with the comment by kernel developers saying, "let the child process run first."
I guessed that this has something to do with the buffers that the printf writes to.
Also, is it true to say that the input redirection (>) writes everything to a buffer first and only then copies to the file? And even so, why would this change the order of the prints?
Note: I am running the test on a single-core virtual machine with a Linux kernel v2.4.14.
Thank you for your time.
When you redirect, glibc detects that stdout is not tty turns on output buffering for efficiency. The buffer is therefore not written until the process exits. You can see this with e.g.:
int main() {
printf("hello world\n");
sleep(60);
}
When you run it interactively, it prints "hello world" and waits. When you redirect to a file, you will see that nothing is written for 60 seconds:
$ ./foo > file & tail -f file
(no output for 60 seconds)
Since your parent process waits for the child, it will necessarily always exit last, and therefore flush its output last.

Wrong exit status from expect script

I developed this expect script, TELNET_TEST.expect to test a TELNET connection on a remote machine.
This script should connect via telnet on a target machine, wait for the login prompt, send the password and then exit.
This script does work and you can see in example 1 that the script does successfully login via telnet then exit, but something very confusing is going on, (to me).
Why do I get an exit status 1? I believe I should be getting an exit of status 0...
Please let me know why I am getting an exit of status 1? Also, what would I need to change in my script in order to get the exit code I am anticipating?
My expect script:
more TELNET_TEST.expect
#!/usr/bin/expect --
set LOGIN [lindex $argv 0]
set PASSWORD [lindex $argv 1]
set IP [lindex $argv 2]
set timeout 20
spawn telnet -l $LOGIN $IP
expect -re "(Password:|word:)"
send $PASSWORD\r
expect -re "(#|>)"
send exit\r
expect {
timeout {error "incorrect password"; exit 1}
eof
}
catch wait result
set STATUS [ lindex $result 3 ]
exit $STATUS
EXAMPLE1
Running the expect script from my Linux machine I get an exit status 1 even though the telnet login is ok.
./var/TELNET_TEST.expect root pass123 198.23.234.12
.
spawn telnet -l root pass123
Trying 198.23.234.12...
Connected to 198.23.234.12.
Escape character is '^]'.
Digital UNIX (machine1001) (ttyp0)
login: root
Password:
Last login: Mon Jul 14 16:40:15 from 198.23.234.12
Digital UNIX V4.0F (Rev. 1229); Wed Nov 23 15:08:48 IST 2005
****************************************************************************
Wide Area Networking Support V3.0-2 (ECO 3) for Digital UNIX is installed.
You have new mail.
machine1001> Connection closed by foreign host.
[root#LINUX_XOR]# echo $?
1
I see that in the transcript of you session:
machine1001> Connection closed by foreign host.
Exit code 1 is the exit code for "Connection closed by foreign host". That is the "correct" code when the connection is closed by the "other side" (in that case, in response to your exit command).
As far as I can tell, if you want an exit code of 0, you need to enter command mode in your telnet client and send the quit command. That way, the connection is closed by the client not by the foreign host. But is this really more "normal" than the other way?
From the sources of GNU telnet (inetutils-1.9), in the file commands.c:
int
tn (int argc, char *argv[])
{
....
.... many many lines of code here
....
close (net);
ExitString ("Connection closed by foreign host.\n", 1);
return 0;
}
and (utilities.c):
void
ExitString (char *string, int returnCode)
{
SetForExit ();
fwrite (string, 1, strlen (string), stderr);
exit (returnCode);
}

Linux non-blocking fifo (on demand logging)

I like to log a programs output 'on demand'. Eg. the output is logged to the terminal, but another process can hook on the current output at any time.
The classic way would be:
myprogram 2>&1 | tee /tmp/mylog
and on demand
tail /tmp/mylog
However, this would create a ever growing log file even if not used until the drive runs out of space. So my attempt was:
mkfifo /tmp/mylog
myprogram 2>&1 | tee /tmp/mylog
and on demand
cat /tmp/mylog
Now I can read /tmp/mylog at any time. However, any output blocks the program until the /tmp/mylog is read. I like the fifo to flush any incoming data not read back. How to do that?
Inspired by your question I've written a simple program that will let you do this:
$ myprogram 2>&1 | ftee /tmp/mylog
It behaves similarly to tee but clones the stdin to stdout and to a named pipe (a requirement for now) without blocking. This means that if you want to log this way it may happen that you're gonna lose your log data, but I guess it's acceptable in your scenario.
The trick is to block SIGPIPE signal and to ignore error on writing to a broken fifo. This sample may be optimized in various ways of course, but so far, it does the job I guess.
/* ftee - clone stdin to stdout and to a named pipe
(c) racic#stackoverflow
WTFPL Licence */
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <errno.h>
#include <signal.h>
#include <unistd.h>
int main(int argc, char *argv[])
{
int readfd, writefd;
struct stat status;
char *fifonam;
char buffer[BUFSIZ];
ssize_t bytes;
signal(SIGPIPE, SIG_IGN);
if(2!=argc)
{
printf("Usage:\n someprog 2>&1 | %s FIFO\n FIFO - path to a"
" named pipe, required argument\n", argv[0]);
exit(EXIT_FAILURE);
}
fifonam = argv[1];
readfd = open(fifonam, O_RDONLY | O_NONBLOCK);
if(-1==readfd)
{
perror("ftee: readfd: open()");
exit(EXIT_FAILURE);
}
if(-1==fstat(readfd, &status))
{
perror("ftee: fstat");
close(readfd);
exit(EXIT_FAILURE);
}
if(!S_ISFIFO(status.st_mode))
{
printf("ftee: %s in not a fifo!\n", fifonam);
close(readfd);
exit(EXIT_FAILURE);
}
writefd = open(fifonam, O_WRONLY | O_NONBLOCK);
if(-1==writefd)
{
perror("ftee: writefd: open()");
close(readfd);
exit(EXIT_FAILURE);
}
close(readfd);
while(1)
{
bytes = read(STDIN_FILENO, buffer, sizeof(buffer));
if (bytes < 0 && errno == EINTR)
continue;
if (bytes <= 0)
break;
bytes = write(STDOUT_FILENO, buffer, bytes);
if(-1==bytes)
perror("ftee: writing to stdout");
bytes = write(writefd, buffer, bytes);
if(-1==bytes);//Ignoring the errors
}
close(writefd);
return(0);
}
You can compile it with this standard command:
$ gcc ftee.c -o ftee
You can quickly verify it by running e.g.:
$ ping www.google.com | ftee /tmp/mylog
$ cat /tmp/mylog
Also note - this is no multiplexer. You can only have one process doing $ cat /tmp/mylog at a time.
This is a (very) old thread, but I've run into a similar problem of late. In fact, what I needed is a cloning of stdin to stdout with a copy to a pipe that is non blocking. the proposed ftee in the first answer really helped there, but was (for my use case) too volatile. Meaning I lost data I could have processed if I had gotten to it in time.
The scenario I was faced with is that I have a process (some_process) that aggregates some data and writes its results every three seconds to stdout. The (simplified) setup looked like this (in the real setup I am using a named pipe):
some_process | ftee >(onlineAnalysis.pl > results) | gzip > raw_data.gz
Now, raw_data.gz has to be compressed and has to be complete. ftee does this job very well. But the pipe I am using in the middle was too slow to grab the data flushed out - but it was fast enough to process everything if it could get to it, which was tested with a normal tee. However, a normal tee blocks if anything happens to the unnamed pipe, and as I want to be able to hook in on demand, tee is not an option. Back to the topic: It got better when I put a buffer in between, resulting in:
some_process | ftee >(mbuffer -m 32M| onlineAnalysis.pl > results) | gzip > raw_data.gz
But that was still losing data I could have processed. So I went ahead and extended the ftee proposed before to a buffered version (bftee). It still has all the same properties, but uses an (inefficient ?) internal buffer in case a write fails. It still loses data if the buffer runs full, but it works beautifully for my case. As always there is a lot of room for improvement, but as I copied the code off of here I'd like to share it back to people that might have a use for it.
/* bftee - clone stdin to stdout and to a buffered, non-blocking pipe
(c) racic#stackoverflow
(c) fabraxias#stackoverflow
WTFPL Licence */
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <errno.h>
#include <signal.h>
#include <unistd.h>
// the number of sBuffers that are being held at a maximum
#define BUFFER_SIZE 4096
#define BLOCK_SIZE 2048
typedef struct {
char data[BLOCK_SIZE];
int bytes;
} sBuffer;
typedef struct {
sBuffer *data; //array of buffers
int bufferSize; // number of buffer in data
int start; // index of the current start buffer
int end; // index of the current end buffer
int active; // number of active buffer (currently in use)
int maxUse; // maximum number of buffers ever used
int drops; // number of discarded buffer due to overflow
int sWrites; // number of buffer written to stdout
int pWrites; // number of buffers written to pipe
} sQueue;
void InitQueue(sQueue*, int); // initialized the Queue
void PushToQueue(sQueue*, sBuffer*, int); // pushes a buffer into Queue at the end
sBuffer *RetrieveFromQueue(sQueue*); // returns the first entry of the buffer and removes it or NULL is buffer is empty
sBuffer *PeakAtQueue(sQueue*); // returns the first entry of the buffer but does not remove it. Returns NULL on an empty buffer
void ShrinkInQueue(sQueue *queue, int); // shrinks the first entry of the buffer by n-bytes. Buffer is removed if it is empty
void DelFromQueue(sQueue *queue); // removes the first entry of the queue
static void sigUSR1(int); // signal handled for SUGUSR1 - used for stats output to stderr
static void sigINT(int); // signla handler for SIGKILL/SIGTERM - allows for a graceful stop ?
sQueue queue; // Buffer storing the overflow
volatile int quit; // for quiting the main loop
int main(int argc, char *argv[])
{
int readfd, writefd;
struct stat status;
char *fifonam;
sBuffer buffer;
ssize_t bytes;
int bufferSize = BUFFER_SIZE;
signal(SIGPIPE, SIG_IGN);
signal(SIGUSR1, sigUSR1);
signal(SIGTERM, sigINT);
signal(SIGINT, sigINT);
/** Handle commandline args and open the pipe for non blocking writing **/
if(argc < 2 || argc > 3)
{
printf("Usage:\n someprog 2>&1 | %s FIFO [BufferSize]\n"
"FIFO - path to a named pipe, required argument\n"
"BufferSize - temporary Internal buffer size in case write to FIFO fails\n", argv[0]);
exit(EXIT_FAILURE);
}
fifonam = argv[1];
if (argc == 3) {
bufferSize = atoi(argv[2]);
if (bufferSize == 0) bufferSize = BUFFER_SIZE;
}
readfd = open(fifonam, O_RDONLY | O_NONBLOCK);
if(-1==readfd)
{
perror("bftee: readfd: open()");
exit(EXIT_FAILURE);
}
if(-1==fstat(readfd, &status))
{
perror("bftee: fstat");
close(readfd);
exit(EXIT_FAILURE);
}
if(!S_ISFIFO(status.st_mode))
{
printf("bftee: %s in not a fifo!\n", fifonam);
close(readfd);
exit(EXIT_FAILURE);
}
writefd = open(fifonam, O_WRONLY | O_NONBLOCK);
if(-1==writefd)
{
perror("bftee: writefd: open()");
close(readfd);
exit(EXIT_FAILURE);
}
close(readfd);
InitQueue(&queue, bufferSize);
quit = 0;
while(!quit)
{
// read from STDIN
bytes = read(STDIN_FILENO, buffer.data, sizeof(buffer.data));
// if read failed due to interrupt, then retry, otherwise STDIN has closed and we should stop reading
if (bytes < 0 && errno == EINTR) continue;
if (bytes <= 0) break;
// save the number if read bytes in the current buffer to be processed
buffer.bytes = bytes;
// this is a blocking write. As long as buffer is smaller than 4096 Bytes, the write is atomic to a pipe in Linux
// thus, this cannot be interrupted. however, to be save this should handle the error cases of partial or interrupted write none the less.
bytes = write(STDOUT_FILENO, buffer.data, buffer.bytes);
queue.sWrites++;
if(-1==bytes) {
perror("ftee: writing to stdout");
break;
}
sBuffer *tmpBuffer = NULL;
// if the queue is empty (tmpBuffer gets set to NULL) the this does nothing - otherwise it tries to write
// the buffered data to the pipe. This continues until the Buffer is empty or the write fails.
// NOTE: bytes cannot be -1 (that would have failed just before) when the loop is entered.
while ((bytes != -1) && (tmpBuffer = PeakAtQueue(&queue)) != NULL) {
// write the oldest buffer to the pipe
bytes = write(writefd, tmpBuffer->data, tmpBuffer->bytes);
// the written bytes are equal to the buffer size, the write is successful - remove the buffer and continue
if (bytes == tmpBuffer->bytes) {
DelFromQueue(&queue);
queue.pWrites++;
} else if (bytes > 0) {
// on a positive bytes value there was a partial write. we shrink the current buffer
// and handle this as a write failure
ShrinkInQueue(&queue, bytes);
bytes = -1;
}
}
// There are several cases here:
// 1.) The Queue is empty -> bytes is still set from the write to STDOUT. in this case, we try to write the read data directly to the pipe
// 2.) The Queue was not empty but is now -> bytes is set from the last write (which was successful) and is bigger 0. also try to write the data
// 3.) The Queue was not empty and still is not -> there was a write error before (even partial), and bytes is -1. Thus this line is skipped.
if (bytes != -1) bytes = write(writefd, buffer.data, buffer.bytes);
// again, there are several cases what can happen here
// 1.) the write before was successful -> in this case bytes is equal to buffer.bytes and nothing happens
// 2.) the write just before is partial or failed all together - bytes is either -1 or smaller than buffer.bytes -> add the remaining data to the queue
// 3.) the write before did not happen as the buffer flush already had an error. In this case bytes is -1 -> add the remaining data to the queue
if (bytes != buffer.bytes)
PushToQueue(&queue, &buffer, bytes);
else
queue.pWrites++;
}
// once we are done with STDIN, try to flush the buffer to the named pipe
if (queue.active > 0) {
//set output buffer to block - here we wait until we can write everything to the named pipe
// --> this does not seem to work - just in case there is a busy loop that waits for buffer flush aswell.
int saved_flags = fcntl(writefd, F_GETFL);
int new_flags = saved_flags & ~O_NONBLOCK;
int res = fcntl(writefd, F_SETFL, new_flags);
sBuffer *tmpBuffer = NULL;
//TODO: this does not handle partial writes yet
while ((tmpBuffer = PeakAtQueue(&queue)) != NULL) {
int bytes = write(writefd, tmpBuffer->data, tmpBuffer->bytes);
if (bytes != -1) DelFromQueue(&queue);
}
}
close(writefd);
}
/** init a given Queue **/
void InitQueue (sQueue *queue, int bufferSize) {
queue->data = calloc(bufferSize, sizeof(sBuffer));
queue->bufferSize = bufferSize;
queue->start = 0;
queue->end = 0;
queue->active = 0;
queue->maxUse = 0;
queue->drops = 0;
queue->sWrites = 0;
queue->pWrites = 0;
}
/** push a buffer into the Queue**/
void PushToQueue(sQueue *queue, sBuffer *p, int offset)
{
if (offset < 0) offset = 0; // offset cannot be smaller than 0 - if that is the case, we were given an error code. Set it to 0 instead
if (offset == p->bytes) return; // in this case there are 0 bytes to add to the queue. Nothing to write
// this should never happen - offset cannot be bigger than the buffer itself. Panic action
if (offset > p->bytes) {perror("got more bytes to buffer than we read\n"); exit(EXIT_FAILURE);}
// debug output on a partial write. TODO: remove this line
// if (offset > 0 ) fprintf(stderr, "partial write to buffer\n");
// copy the data from the buffer into the queue and remember its size
memcpy(queue->data[queue->end].data, p->data + offset , p->bytes-offset);
queue->data[queue->end].bytes = p->bytes - offset;
// move the buffer forward
queue->end = (queue->end + 1) % queue->bufferSize;
// there is still space in the buffer
if (queue->active < queue->bufferSize)
{
queue->active++;
if (queue->active > queue->maxUse) queue->maxUse = queue->active;
} else {
// Overwriting the oldest. Move start to next-oldest
queue->start = (queue->start + 1) % queue->bufferSize;
queue->drops++;
}
}
/** return the oldest entry in the Queue and remove it or return NULL in case the Queue is empty **/
sBuffer *RetrieveFromQueue(sQueue *queue)
{
if (!queue->active) { return NULL; }
queue->start = (queue->start + 1) % queue->bufferSize;
queue->active--;
return &(queue->data[queue->start]);
}
/** return the oldest entry in the Queue or NULL if the Queue is empty. Does not remove the entry **/
sBuffer *PeakAtQueue(sQueue *queue)
{
if (!queue->active) { return NULL; }
return &(queue->data[queue->start]);
}
/*** Shrinks the oldest entry i the Queue by bytes. Removes the entry if buffer of the oldest entry runs empty*/
void ShrinkInQueue(sQueue *queue, int bytes) {
// cannot remove negative amount of bytes - this is an error case. Ignore it
if (bytes <= 0) return;
// remove the entry if the offset is equal to the buffer size
if (queue->data[queue->start].bytes == bytes) {
DelFromQueue(queue);
return;
};
// this is a partial delete
if (queue->data[queue->start].bytes > bytes) {
//shift the memory by the offset
memmove(queue->data[queue->start].data, queue->data[queue->start].data + bytes, queue->data[queue->start].bytes - bytes);
queue->data[queue->start].bytes = queue->data[queue->start].bytes - bytes;
return;
}
// panic is the are to remove more than we have the buffer
if (queue->data[queue->start].bytes < bytes) {
perror("we wrote more than we had - this should never happen\n");
exit(EXIT_FAILURE);
return;
}
}
/** delete the oldest entry from the queue. Do nothing if the Queue is empty **/
void DelFromQueue(sQueue *queue)
{
if (queue->active > 0) {
queue->start = (queue->start + 1) % queue->bufferSize;
queue->active--;
}
}
/** Stats output on SIGUSR1 **/
static void sigUSR1(int signo) {
fprintf(stderr, "Buffer use: %i (%i/%i), STDOUT: %i PIPE: %i:%i\n", queue.active, queue.maxUse, queue.bufferSize, queue.sWrites, queue.pWrites, queue.drops);
}
/** handle signal for terminating **/
static void sigINT(int signo) {
quit++;
if (quit > 1) exit(EXIT_FAILURE);
}
This version takes one more (optional) argument which specifies the number of the blocks that are to buffered for the pipe. My sample call now looks like this:
some_process | bftee >(onlineAnalysis.pl > results) 16384 | gzip > raw_data.gz
resulting in 16384 blocks to be buffered before discards happen. this uses about 32 Mbyte more memory, but... who cares ?
Of course, in the real environment I am using a named pipe so that I can attach and detach as needed. There is looks like this:
mkfifo named_pipe
some_process | bftee named_pipe 16384 | gzip > raw_data.gz &
cat named_pipe | onlineAnalysis.pl > results
Also, the process reacts on signals as follows:
SIGUSR1 -> print counters to STDERR
SIGTERM, SIGINT -> first exits the main loop and flushed the buffer to the pipe, the second terminated the program immediatly.
Maybe this helps someone in the future...
Enjoy
However, this would create a ever growing log file even if not used until the drive runs out of space.
Why not periodically rotate the logs? There's even a program to do it for you logrotate.
There's also a system for generating log messages and doing different things with them according to type. It's called syslog.
You could even combine the two. Have your program generate syslog messages, configure syslog to place them in a file and use logrotate to ensure they don't fill the disk.
If it turned out that you were writing for a small embedded system and the program's output is heavy there are a variety of techniques you might consider.
Remote syslog: send the syslog messages to a syslog server on the network.
Use the severity levels availble in syslog to do different things with the messages. E.g. discard "INFO" but log and forward "ERR" or greater. E.g. to console
Use a signal handler in your program to reread configuration on HUP and vary log generation "on demand" this way.
Have your program listen on a unix socket and write messages down it when open. You could even implement and interactive console into your program this way.
Using a configuration file, provide granular control of logging output.
It seems like bash <> redirection operator (3.6.10 Opening File Descriptors for Reading and WritingSee) makes writing to file/fifo opened with it non-blocking.
This should work:
$ mkfifo /tmp/mylog
$ exec 4<>/tmp/mylog
$ myprogram 2>&1 | tee >&4
$ cat /tmp/mylog # on demend
Solution given by gniourf_gniourf on #bash IRC channel.
BusyBox often used on embedded devices can create a ram buffered log by
syslogd -C
which can be filled by
logger
and read by
logread
Works quite well, but only provides one global log.
The logging could be directed to a UDP socket. Since UDP is connection-less, it won't block the sending program. Of course logs will be lost if the receiver or network can't keep up.
myprogram 2>&1 | socat - udp-datagram:localhost:3333
Then when you want to observe the logging:
socat udp-recv:3333 -
There are some other cool benefits like being able to attach multiple listeners at the same time or broadcast to multiple devices.
If you can install screen on the embedded device then you can run 'myprogram' in it and detach it, and reattach it anytime you want to see the log. Something like:
$ screen -t sometitle myprogram
Hit Ctrl+A, then d to detach it.
Whenever you want to see the output, reattach it:
$ screen -DR sometitle
Hit Ctrl-A, then d to detach it again.
This way you won't have to worry about the program output using disk space at all.
The problem with the given fifo approach is that the whole thing will hang when the pipe buffer is getting filled up and no reading process is taking place.
For the fifo approach to work I think you would have to implement a named pipe client-server model similar to the one mentioned in BASH: Best architecture for reading from two input streams (see slightly modified code below, sample code 2).
For a workaround you could also use a while ... read construct instead of teeing stdout to a named pipe by implementing a counting mechanism inside the while ... read loop that will overwrite the log file periodically by a specified number of lines. This would prevent an ever growing log file (sample code 1).
# sample code 1
# terminal window 1
rm -f /tmp/mylog
touch /tmp/mylog
while sleep 2; do date '+%Y-%m-%d_%H.%M.%S'; done 2>&1 | while IFS="" read -r line; do
lno=$((lno+1))
#echo $lno
array[${lno}]="${line}"
if [[ $lno -eq 10 ]]; then
lno=$((lno+1))
array[${lno}]="-------------"
printf '%s\n' "${array[#]}" > /tmp/mylog
unset lno array
fi
printf '%s\n' "${line}"
done
# terminal window 2
tail -f /tmp/mylog
#------------------------
# sample code 2
# code taken from:
# https://stackoverflow.com/questions/6702474/bash-best-architecture-for-reading-from-two-input-streams
# terminal window 1
# server
(
rm -f /tmp/to /tmp/from
mkfifo /tmp/to /tmp/from
while true; do
while IFS="" read -r -d $'\n' line; do
printf '%s\n' "${line}"
done </tmp/to >/tmp/from &
bgpid=$!
exec 3>/tmp/to
exec 4</tmp/from
trap "kill -TERM $bgpid; exit" 0 1 2 3 13 15
wait "$bgpid"
echo "restarting..."
done
) &
serverpid=$!
#kill -TERM $serverpid
# client
(
exec 3>/tmp/to;
exec 4</tmp/from;
while IFS="" read -r -d $'\n' <&4 line; do
if [[ "${line:0:1}" == $'\177' ]]; then
printf 'line from stdin: %s\n' "${line:1}" > /dev/null
else
printf 'line from fifo: %s\n' "$line" > /dev/null
fi
done &
trap "kill -TERM $"'!; exit' 1 2 3 13 15
while IFS="" read -r -d $'\n' line; do
# can we make it atomic?
# sleep 0.5
# dd if=/tmp/to iflag=nonblock of=/dev/null # flush fifo
printf '\177%s\n' "${line}"
done >&3
) &
# kill -TERM $!
# terminal window 2
# tests
echo hello > /tmp/to
yes 1 | nl > /tmp/to
yes 1 | nl | tee /tmp/to
while sleep 2; do date '+%Y-%m-%d_%H.%M.%S'; done 2>&1 | tee -a /tmp/to
# terminal window 3
cat /tmp/to | head -n 10
If your process writes to any log file and then wipes the file and starts again every now and again, so it doesn't get too big, or uses logrotate.
tail --follow=name --retry my.log
Is all you need. You will get as much scroll-back as your terminal.
Nothing non standard is needed. I've not tried it with small log files but all our logs rotate like this and I have never noticed loosing lines.
To follow in Fabraxias foot steps I'm going to share my small modification of racic's code. In one of my use cases I needed to suppress the writes to STDOUT, so I've added another parameter: swallow_stdout. If that is not 0, then output to STDOUT will be turned off.
Since I'm no C coder I've added comments while reading the code, maybe they are useful for others.
/* ftee - clone stdin to stdout and to a named pipe
(c) racic#stackoverflow
WTFPL Licence */
// gcc /tmp/ftee.c -o /usr/local/bin/ftee
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <errno.h>
#include <signal.h>
#include <unistd.h>
int main(int argc, char *argv[])
{
int readfd, writefd; // read & write file descriptors
struct stat status; // read file descriptor status
char *fifonam; // name of the pipe
int swallow_stdout; // 0 = write to STDOUT
char buffer[BUFSIZ]; // read/write buffer
ssize_t bytes; // bytes read/written
signal(SIGPIPE, SIG_IGN);
if(3!=argc)
{
printf("Usage:\n someprog 2>&1 | %s [FIFO] [swallow_stdout] \n"
"FIFO - path to a named pipe (created beforehand with mkfifo), required argument\n"
"swallow_stdout - 0 = output to PIPE and STDOUT, 1 = output to PIPE only, required argument\n", argv[0]);
exit(EXIT_FAILURE);
}
fifonam = argv[1];
swallow_stdout = atoi(argv[2]);
readfd = open(fifonam, O_RDONLY | O_NONBLOCK); // open read file descriptor in non-blocking mode
if(-1==readfd) // read descriptor error!
{
perror("ftee: readfd: open()");
exit(EXIT_FAILURE);
}
if(-1==fstat(readfd, &status)) // read descriptor status error! (?)
{
perror("ftee: fstat");
close(readfd);
exit(EXIT_FAILURE);
}
if(!S_ISFIFO(status.st_mode)) // read descriptor is not a FIFO error!
{
printf("ftee: %s in not a fifo!\n", fifonam);
close(readfd);
exit(EXIT_FAILURE);
}
writefd = open(fifonam, O_WRONLY | O_NONBLOCK); // open write file descriptor non-blocking
if(-1==writefd) // write file descriptor error!
{
perror("ftee: writefd: open()");
close(readfd);
exit(EXIT_FAILURE);
}
close(readfd); // reading complete, close read file descriptor
while(1) // infinite loop
{
bytes = read(STDIN_FILENO, buffer, sizeof(buffer)); // read STDIN into buffer
if (bytes < 0 && errno == EINTR)
continue; // skip over errors
if (bytes <= 0)
break; // no more data coming in or uncaught error, let's quit since we can't write anything
if (swallow_stdout == 0)
bytes = write(STDOUT_FILENO, buffer, bytes); // write buffer to STDOUT
if(-1==bytes) // write error!
perror("ftee: writing to stdout");
bytes = write(writefd, buffer, bytes); // write a copy of the buffer to the write file descriptor
if(-1==bytes);// ignore errors
}
close(writefd); // close write file descriptor
return(0); // return exit code 0
}

Resources