fellow coders.
I'm monitoring my outgoing traffic using libnetfilter_queue module and an iptables rule
ipatbles -I OUTPUT 1 -p all -j NFQUEUE --queue-num 11220
A certain app, called Jitsi (which runs on Java) is exhibiting such a strange behaviour a haven't encountered before:
My monitoring program which process NFQUEUE packets clearly shows that UDP packets are being sent out,
yet when I look into:
"/proc/net/udp" and "/proc/net/udp6" they are empty, moreover "/proc/net/protocols" has a column "sockets" for UDP and it is 0.
But the UDP packets keep getting sent.
Then after a minute or so, "/proc/net/udp" and "/proc/net/protocols" begin to show the correct information about UDP packets.
And again after a while there is no information in them while the UDP packets are being sent.
My only conclusion is that somehow it is possible for an application to send UDP packets without creating a socket and/or it is possible create a socket, then delete it (so that kernel thinks there are none) and still use some obscure method to send packets outside.
Could somebody with ideas about such behaviour land a hand, please?
Two ideas:
Try running the app through strace and take a look at that output.
You could also try to run it through systemtap with a filter for the socket operations.
From that link:
probe kernel.function("*#net/socket.c").call {
printf ("%s -> %s\n", thread_indent(1), probefunc())
}
probe kernel.function("*#net/socket.c").return {
printf ("%s <- %s\n", thread_indent(-1), probefunc())
}
Thank you Paul Rubel for giving me a hint in the right direction. strace showed that Java app was using IPv6 sockets. I had a closer look at /proc/net/udp6 and there those sockets were. I probably had too cursory a view the first time around chiefly because I didn't even expect to find them there. This is the first time I stumbled upon IPv4 packets over IPv6 sockets. But that is what Java does.
Cheers.
Related
Let's say I have a Modbus RTU temperature controller connected to a device that reads TCP packets from a network and sends the payload from said packets to the temperature controller over RS485. If I wanted to test such a setup by sending a Modbus RTU [Read Input Registers] request over TCP/IP using netcat on a Linux command line, what would be the full command for that, assuming that it is possible?
nc 192.168.0.5 2000 # ...what now?
Suppose that the following details apply:
device address: 1
function code: 3 (read input register)
memory address: 1000H
no. of registers: 1
server IP address: 192.168.0.5
client IP address: 192.168.0.4
Port on server: 2000
I feel like I can listen for the response using netcat, also. Could I just do that with:
nc -l 3456
Would that be right? To send a request and log the response like this, I guess I would have to specify the port to send the request on as 3456 (or whatever). Is there a way to do that?
Especially, I'm confused as to how to turn a Modbus message like [01][03][1000H][01][CRC] into its bytes equivalent.
Many thanks in advance.
You can send raw bytes with:
echo -e '\x04\x03...' | nc host port
Especially, I'm confused as to how to turn a Modbus message like [01][03][1000H][CNC] into its bytes equivalent.
That would be:
echo -e '\x01\x03\x10\x00...' | nc host port
I'm not going to calculate if for you because I see no point in doing something like this other than for fun.
You'll have to put all parts of the Modbus frame together, just google it. The most difficult part is to calculate the CRC at the end of the frame. I guess that's what you mean by CNC?
If you struggle I would suggest you try something like QModMaster. Look at the raw data with menu view► Raw Data and use those bytes.
Note that other than building the frames, QModMaster would be able to send them too and even listen for and receive responses. That should be your one-stop-shop for Modbus. You can also find many other tools like pymodbus, libmodbus, minimalmodbus and even command line tools for Windows and Linux.
If you edit your question to present a fair reason why you need to use nc (obviously other than for fun) maybe I can try to offer some more help.
I could not find any references on using netcat to send and/or receive Modbus frames. I'm not sure if that's because I did not search long enough or because it is pointless... Maybe it's both.
Should it be of use to anyone, the answer to this question is:
sudo echo -e -n '\x01\x03\x10\x00\x00\x01\x80\xCA' | nc 192.168.0.5 2000
-e allows the conversion of escaped chars
-n prevents a newline character being appended
Listening for incoming messages using Netcat doesn't work, apparently because the individual Netcat process wants to make the TCP connection itself, otherwise it isn't interested in anything arriving at the specified port. Still, I am able to see the reply coming back from the slave using Wireshark, and have also written to registers and successfully controlled the device.
This is my first time using raw sockets (yes, I need to use them as I must modify a field inside a network header) and all the documentation or tutorials I read describe a solution to sniff packets but that is not exactly what I need. I need to create a script which intercepts the packet, process it and sends it further to the destination, i.e. the packets should not reach the destination unless my script decides to.
In order to learn, I created a small prototype which detects pings and just prints "PING". I would expect ping not to work as I intercept the packets and I don't include the logic to send them to its destination. However ping is working (again, it seems as it is just sniffing/mirroring packets). My goal is that the ping packets are "trapped" in my script and I don't know how to do that. This is what I do in my current python script (I avoid writing how I do the decode for simplicity)
sock = socket.socket(socket.AF_PACKET, socket.SOCK_RAW, socket.ntohs(0x0003))
sock.bind((eth0, 0))
packet = sock.recvfrom(65565)
decode_eth(packet)
decode_ip(packet)
if (ipheader.ip_proto == 1):
print("\nPING")
Can somebody explain how can I achieve my goal or point me to the right documentation?
Your description seems to be different from what your title suggest. My understanding is that you want to receive, modify and possibly drop incoming network packets. And this is to be done on Linux. In that case I suggest you use a netfilter prerouting hook, which will make things a lot simpler (and likely more stable). Netfilter is well documented, a nice overview including information related to your requirements can be seen here. The important function to use is nf_register_hook(), read the answer to this question to get an idea of how to set things up.
I suppose that your Linux box is configured as a router (not a bridge). The packet will pass through your Linux because you have enabled IP Forwarding. So there are two solution:
Solution 1:
Disable IP Forwarding and then receive the packet from one interface and do the appropriate task (forwarding to another interface or dropping it).
Solution 2:
Use NetFilterQueue.
Install it on your Linux box (Debian/Ubuntu in my example):
apt-get install build-essential python-dev libnetfilter-queue-dev
Use iptables to send packets coming from input interface (eth0 in my example):
iptables -I INPUT -i eth0 -j NFQUEUE --queue-num 1
Run this script to handle packets forwarded to the Queue No.1 :
from netfilterqueue import NetfilterQueue
def print_and_accept(pkt):
print pkt
pkt.accept()
nfqueue = NetfilterQueue()
nfqueue.bind(1, print_and_accept)
try:
nfqueue.run()
except KeyboardInterrupt:
print
Note that pkt.drop() cause dropping the packet. Also you should accept/drop every packet.
I'm working on sending large data files between two Linux computers via a 10 Gigabit Ethernet cable and netcat with a UDP transfer, but seem to be having issues.
After running several tests, I've come to the conclusion that netcat is the issue. I've tested the UDP transfer using [UDT][1], [Tsunami-UDP]2, and a Python UDT transfer as well, and all of which have not had any packet loss issues.
On the server side, we've been doing:
cat "bigfile.txt" | pv | nc -u IP PORT
then on the client side, we've been doing:
nc -u -l PORT > "outputFile.txt"
A few things that we've noticed:
On one of the computers, regardless of whether it's the client or server, it just "hangs". That is to say, even once the transfer is complete, Linux doesn't kill the process and move to the next line in the terminal.
If we run pipe view on the receiving side as well, the incoming data rate is significantly lower than what the sending side thinks it's sending.
Running Wireshark doesn't show any packet loss.
Running the system performance monitor in Linux shows that the incoming data rate (for the receiving side) is the same as the outgoing data rate from the sending side. This is in contrast to what pipe view thinks (see #2)
We're not sure where the issue is with netcat, and if there is a way around it. Any help/insights would be greatly appreciated.
Also, for what it's worth, using netcat with a TCP transfer works fine. And, I do understand that UDP isn't known for reliability, and that packet loss should be expected, but it's the protocol we must use.
Thanks
It could well be that the sending instance is sending the data too fast for the receiving instance. Note that this can occur even if you see no drops on the receiving NIC (as you seem to be saying), because the loss can occur at OS level instead. Your OS could have its UDP buffers overflowing. Run this command:
watch -d "cat /proc/net/snmp | grep -w Udp"
To see if your RcvbufErrors field is non-zero and/or growing while your file transfer is going on.
This answer (How to send only one UDP packet with netcat?) says that nc sends one packet per line. Assuming that's true, this could lead to a significantly higher number of packets than your other transfer mechanisms. Presumably, as #Smeeheey suggested, you're running out of receive buffers on the receiving end.
To cause your sending end to exit, you can add -q 1 to the command line (exit 1 second after seeing end of file).
But there's no way that the the receiving end nc can know when the transfer is complete. This is why these other mechanisms are "protocols" -- they have mechanisms built into them to communicate the bounds of a file. Raw UDP has no concept of end of file.
Tuning the Linux networking stack is a bit complicated, as there are many components to tune to figure out where data is being dropped.
If possible/feasible, I'd recommend that you start by monitoring packet drops throughout the entire network stack. Once you've done that, you can determine where exactly packets are being dropped and then adjust tuning parameters as needed. There are a lot of different files to measure with lots of different fields. I wrote a detailed blog post about monitoring and tuning each component of the Linux networking stack from top to bottom. It's a bit difficult to summarize all the information there, but take a look, I think it can help guide you.
I want to capture all tcp packets on port 80 that are going to a forward address and pass them into a user-space program for mangling. I know how to do this with an IPtables rule such as
iptables -A FORWARD ... -j NFQUEUE --queue-num 10
but i can't seem to find any examples, and there is a time restriction, so I can't just sift through wireshark code. I'm looking for examples, or even a place to start. Can I use sockets? or is there a kernel data structure that I can use to pop packets off of the stack, and once I'm done, how can I pass the packets back to netfilter?
You can write a kernel module that registers for queued packets for a certain protocol.
You can read the netfilter-hacking-HOWTO, page 25 has a section "Processing Queued Packets". This section discusses at a higher level but answers everything you asked.
Download it from:
http://www.netfilter.org/documentation/index.html#documentation-howto
you can use nf_reinject() to let the packet get processed further.
I don't want to repeat whatever is in this guide but this should be enough to start with.
Ok, I realize this situation is somewhat unusual, but I need to establish a TCP connection (the 3-way handshake) using only raw sockets (in C, in linux) -- i.e. I need to construct the IP headers and TCP headers myself. I'm writing a server (so I have to first respond to the incoming SYN packet), and for whatever reason I can't seem to get it right. Yes, I realize that a SOCK_STREAM will handle this for me, but for reasons I don't want to go into that isn't an option.
The tutorials I've found online on using raw sockets all describe how to build a SYN flooder, but this is somewhat easier than actually establishing a TCP connection, since you don't have to construct a response based on the original packet. I've gotten the SYN flooder examples working, and I can read the incoming SYN packet just fine from the raw socket, but I'm still having trouble creating a valid SYN/ACK response to an incoming SYN from the client.
So, does anyone know a good tutorial on using raw sockets that goes beyond creating a SYN flooder, or does anyone have some code that could do this (using SOCK_RAW, and not SOCK_STREAM)? I would be very grateful.
MarkR is absolutely right -- the problem is that the kernel is sending reset packets in response to the initial packet because it thinks the port is closed. The kernel is beating me to the response and the connection dies. I was using tcpdump to monitor the connection already -- I should have been more observant and noticed that there were TWO replies one of which was a reset that was screwing things up, as well as the response my program created. D'OH!
The solution that seems to work best is to use an iptables rule, as suggested by MarkR, to block the outbound packets. However, there's an easier way to do it than using the mark option, as suggested. I just match whether the reset TCP flag is set. During the course of a normal connection this is unlikely to be needed, and it doesn't really matter to my application if I block all outbound reset packets from the port being used. This effectively blocks the kernel's unwanted response, but not my own packets. If the port my program is listening on is 9999 then the iptables rule looks like this:
iptables -t filter -I OUTPUT -p tcp --sport 9999 --tcp-flags RST RST -j DROP
You want to implement part of a TCP stack in userspace... this is ok, some other apps do this.
One problem you will come across is that the kernel will be sending out (generally negative, unhelpful) replies to incoming packets. This is going to screw up any communication you attempt to initiate.
One way to avoid this is to use an IP address and interface that the kernel does not have its own IP stack using- which is fine but you will need to deal with link-layer stuff (specifically, arp) yourself. That would require a socket lower than IPPROTO_IP, SOCK_RAW - you need a packet socket (I think).
It may also be possible to block the kernel's responses using an iptables rule- but I rather suspect that the rules will apply to your own packets as well somehow, unless you can manage to get them treated differently (perhaps applying a netfilter "mark" to your own packets?)
Read the man pages
socket(7)
ip(7)
packet(7)
Which explain about various options and ioctls which apply to types of sockets.
Of course you'll need a tool like Wireshark to inspect what's going on. You will need several machines to test this, I recommend using vmware (or similar) to reduce the amount of hardware required.
Sorry I can't recommend a specific tutorial.
Good luck.
I realise that this is an old thread, but here's a tutorial that goes beyond the normal SYN flooders: http://www.enderunix.org/docs/en/rawipspoof/
Hope it might be of help to someone.
I can't help you out on any tutorials.
But I can give you some advice on the tools that you could use to assist in debugging.
First off, as bmdhacks has suggested, get yourself a copy of wireshark (or tcpdump - but wireshark is easier to use). Capture a good handshake. Make sure that you save this.
Capture one of your handshakes that fails. Wireshark has quite good packet parsing and error checking, so if there's a straightforward error it will probably tell you.
Next, get yourself a copy of tcpreplay. This should also include a tool called "tcprewrite".
tcprewrite will allow you to split your previously saved capture files into two - one for each side of the handshake.
You can then use tcpreplay to play back one side of the handshake so you have a consistent set of packets to play with.
Then you use wireshark (again) to check your responses.
I don't have a tutorial, but I recently used Wireshark to good effect to debug some raw sockets programming I was doing. If you capture the packets you're sending, wireshark will do a good job of showing you if they're malformed or not. It's useful for comparing to a normal connection too.
There are structures for IP and TCP headers declared in netinet/ip.h & netinet/tcp.h respectively. You may want to look at the other headers in this directory for extra macros & stuff that may be of use.
You send a packet with the SYN flag set and a random sequence number (x). You should receive a SYN+ACK from the other side. This packet will have an acknowledgement number (y) that indicates the next sequence number the other side is expecting to receive as well as another sequence number (z). You send back an ACK packet that has sequence number x+1 and ack number z+1 to complete the connection.
You also need to make sure you calculate appropriate TCP/IP checksums & fill out the remainder of the header for the packets you send. Also, don't forget about things like host & network byte order.
TCP is defined in RFC 793, available here: http://www.faqs.org/rfcs/rfc793.html
Depending on what you're trying to do it may be easier to get existing software to handle the TCP handshaking for you.
One open source IP stack is lwIP (http://savannah.nongnu.org/projects/lwip/) which provides a full tcp/ip stack. It is very possible to get it running in user mode using either SOCK_RAW or pcap.
if you are using raw sockets, if you send using different source mac address to the actual one, linux will ignore the response packet and not send an rst.