So right now I have a server that is pinging an api and writes to stdout. I pipe that to nc as such.
node server.js | nc -lk 9999
I then have a job that listens to that and outputs 5 second snapshots of the data.
My question is that when I try to do that in multiple sessions with multiple server.js programs eg. server1.js, server2.js which all hit different parts of the api and pipe them all to port 9999. Only the first program gets picked up by the snapshot job. How do I pipe multiple outputs to:
nc -lk 9999
such that they all get handled as if it's the output of one program?
If you execute multiple times that command, it simply won't work. You cannot listen with several instances of netcat on the same port. You should get a nc: Address already in use error.
What you can do is something like:
mkfifo queue
nc -lk 9999 < queue
node server.js | cat > queue
Btw I did a nice and fully working Netcat porting in Node.js: https://github.com/roccomuso/netcat
This easily address the issue you're encountering, because you can implement the whole logic in JS without using the native nc bin or pipe data on the stdin.
Related
If I open an nc in listening on Terminal1
nc -l 35555
And perform a connection in this way on Terminal2
mkfifo /tmp/f
cat /tmp/f | nc 127.0.0.1 35555 > /tmp/f
Now I have few questions
On Terminal2 you can see something in waiting. What is that?
Is it the cat process or the nc process?
In which order are they executed?
If I write a string on Terminal1, that string just come back.
Why cat /tmp/f | nc 127.0.0.1 35555 > /tmp/f doesn't create a cycle?
The received string comes out from nc and goes in /tmp/f. Because of the cat that string begins input of nc, sent back to Terminal1 and again in /tmp/f etc. etc.
Why isn't the cat of a fifo a blocking operations? Technically it not end, so why the nc is executed?
Thanks.
Is it the cat process or the nc process?
It's both. cat is waiting for something to be written to the FIFO. nc is waiting for something to be written to its stdin or to receive something over the network socket.
In which order are they executed?
All commands in a pipeline are run concurrently.
Why cat /tmp/f | nc 127.0.0.1 35555 > /tmp/f doesn't create a cycle?
The first terminal is not sending back what it received from the network, it just prints it to the terminal. When you type something on terminal 1 it gets written to the network once. nc on terminal 2 writes it to /tmp/f. Then it reads it and sends it to the network. nc on terminal 1 reads it and prints it. That's the end.
-Why isn't the cat of a fifo a blocking operations? Technically it not end, so why the nc is executed?
Commands in a pipeline don't wait for the previous one to exit. That would make it impossible to write code like
tail -f filename | grep word
since tail -f never exits (until you kill it with Ctl-c, but that also kills grep). All the processes run, which allows the later programs to process partial output of the earlier programs.
So nc doesn't wait for cat to exit. It runs at the same time, and probably uses select() or epoll() to listen for input from both stdin and the network socket at the same time (it might also use separate threads for each). So while cat is blocking on the FIFO, it can read the network data from terminal 1, and then write it to the FIFO.
I have a command which out outputs certain data which i store in a ext file using a '>>' command.Now Instead of doing that I want to have a socket or a port on any server which will catch the output of the command.Basically i want to output all my script data to a socket or url which ever is possible.
Any help in this direction is most welcomed.
You can use socat to listening on a port 12345 and echo any data sent to it like this:
socat -u TCP-LISTEN:12345,keepalive,reuseaddr,fork STDOUT
If you want to capture it to a file as well (file.log), you can use the same command with tee:
socat -u TCP-LISTEN:12345,keepalive,reuseaddr,fork STDOUT | tee file.log
You can run your program to output to bash's TCP virtual device:
./prog > /dev/tcp/localhost/12345
If you don't want to use bash magic then you can also use socat to send the data:
./prog | socat - TCP-CONNECT:localhost:12345
The above example assume you are running your program and "logger" on the same system but you can replace "localhost" with the hostname or address of the system you wish to send to (where the socat is listening).
I am trying to listen continuously to port 5000 and print the first line of message but the program is stopping after one iteration. I am using the following command for it -
nc -k -l 5000 | head -n 1
I have used -k option with nc but still the program stops after one iteration. What is the problem in this command?
It is not so simple. The second nc could be started only after the full run of the previous. But it can't be done, because nc has a bug: it can't detect if the remote side closes its socket. Thus, the first nc never stops, thus the second can't be started.
The developers of the nc say, that it is an inherent problem of the TCP protocol, my opinion is that they haven't right [but it is only my opinion].
If you want to do practically a script listening on a TCP port, a working solution were to put this head -1 into a script, and calling this script from an inetd.
Some trickier hack could work as well, for example, killing the first nc after the first line, so:
nc -k -l 5000|(read;echo $REPLY;killall -9 nc)
Imho this tools aren't enough stable for productive environment, although they can be very funny. :-)
I have a linux machine which is listening for connections on port 4450. Where there is an incomming connection, this is supposed to send continuously over the socket the content of a file. Did you do this before ?
What I've done so far was to send once the content of the file like this:
x=$(filename); echo $x | nc -l 4450
On the client side I have an Android app, which connects to the server and then using a BufferedReader gets the data from the stream and processes it.
Any help would be highly appreciated.
Thanks
Use socat instead of netcat (nc). With socat you can do almost everything that can be done with netcat. But socat has a lot more features and is easier to use.
socat TCP-LISTEN:4450,fork OPEN:/tmp/filename,rdonly
You can also use the output of a command instead of some file contents:
socat TCP-LISTEN:4450,fork EXEC:/bin/date
Running in bash under Ubuntu:
I have a source that generates me some output, but not straight away. Let's assume it is a first netcat listening on a socket: netcat -l 12345.
And I would like to pipe it to an outgoing netcat (connecting over TCP), e.g. netcat -l 12345 | netcat localhost 54321. But the tricky bit is, that I know there is nothing listening for that incoming connection on localhost 54321 when I run the command, but I know there will be one when the first actual character arrives through the pipe.
So my question is: is there a way either:
to delay the execution of the outgoing netcat until the first character arrives into the pipe, or
to delay the outgoing netcat from trying to establish the TCP connection until it receives the first character on its standard input? (no straight option for that in man, switching to UDP is not acceptable)
Thanks in advance!
Edit: In reality, the source is more complex than a netcat, namely it is a listening netcat piped through all sort of stream modification.
Using the research you already did and that I commented to (by not knowing it was an answer to your own question), here is the full delayed_netcat.sh:
#!/bin/bash
read line
netcat "${#}" < <(echo $line ; cat)
This first waits for a line of input and later prepends that line using a simple echo to the "newly generated" input to the actual netcat. The rest of stdin is just redirected using cat which slurps it from stdin and adds it to the input of netcat. It also supports passing commandline options and arguments to the "real" netcat.
The usage is as follows:
netcat -l 12345 | cmd1 | cmd2 | ... | ./delayed_netcat.sh localhost 54321
The netcat is delayed till the first line is read. If you really want to start it after the first character is read the parts with read and echo need some rewrite.
Port Forwarding or Port Mapping with netcat:
ncat -l -p 12345 -c 'ncat localhost 54321'
Using socat:
socat TCP4-LISTEN:12345 TCP4:localhost:54321
This command exits after the first connection is done.
I have found an answer to my question, but it is awful... so still looking for something better.
netcat -l 12345 | gawk '(NR==1){print""}{print;fflush()}' | ./delayed_netcat.sh
where ./delayed_netcat.sh:
#!/bin/sh
read line
netcat localhost 12345
So the read line delays the netcat localhost 12345 by waiting for and consuming the first input line, and I use gawk '(NR==1){print""}{print;fflush()}' to insert an empty line just before the first record... I'm sure there is room for much improvement to that!