Sending raw data in Scapy does not work correctly - scapy

I use Scapy to create an initial OpenVPN packet and send it to OpenVPN server (acting as a client). OpenVPN part of the packet I'm just reusing from old captured connection, but its irrelevant here.
Thing is, I add a payload of 42bytes but for some reason when I capture packet with Wireshark, I can see 84bytes of OpenVPN stuff. Last half of that is correct payload I sent, but I can't figure out what is the first half. All other layers (Ethernet, IP, UDP) have correct size.
#!/usr/bin/env python
import socket
from scapy.all import *
mysocket=socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
mysocket.connect(('192.168.138.129', 1194))
mystream=StreamSocket(mysocket)
ascapypacket=Ether()/IP(dst="192.168.138.129")/UDP(dport=1194, len=50)/Raw(load="\x38\x81\x38\x14\x62\x1d\x67\x46\x2d\xde\x86\x73\x4d\x2c\xbf\xf1\x51\xb2\xb1\x23\x1b\x61\xe4\x23\x08\xa2\x72\x81\x8e\x00\x00\x00\x01\x50\xff\x26\x2c\x00\x00\x00\x00\x00")
etherLoad = len(ascapypacket.getlayer(Ether)) # display size
print etherLoad
ipLoad = len(ascapypacket.getlayer(IP)) # display size
print ipLoad
udpLoad = len(ascapypacket.getlayer(UDP)) # display size
print udpLoad
rawLoad = len(ascapypacket.getlayer(Raw)) # display size
print rawLoad
mystream.send(ascapypacket)
I made an image. Here you can see green stuff is correct - first part is IP and UDP layers, and 2nd green part is my OpenVPN payload, but I don't understand what is the red part.
Edit: If I don't send that Raw payload I still get those 42 bytes for some reason.

You've created an ordinary UDP datagram socket:
mysocket=socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
This socket manages the ethernet, IP & UDP layers by itself, with guidance from the user through various auxiliary methods and arguments, such as the connect method you've in fact used in your code snippet:
mysocket.connect(('192.168.138.129', 1194))
Its various send methods, even when encapsulated as part of a scapy's StreamSocket object, are expecting to receive as their "data-to-send" argument just the application payload layered above the UDP layer.
However, you're passing on to it the entire protocol stack payload, i.e. the ethernet, IP & UDP headers, which is misinterpreted to be part of the payload data that you wish to send to the other side:
ascapypacket=Ether()/IP(dst="192.168.138.129")/UDP(dport=1194, len=50)/Raw(load="\x38\x81\x38\x14\x62\x1d\x67\x46\x2d\xde\x86\x73\x4d\x2c\xbf\xf1\x51\xb2\xb1\x23\x1b\x61\xe4\x23\x08\xa2\x72\x81\x8e\x00\x00\x00\x01\x50\xff\x26\x2c\x00\x00\x00\x00\x00")
Thus, the data you've marked in red is actually the payload data you've yourself set, before it is followed by the OpenVPN part:
Ether()/IP(dst="192.168.138.129")/UDP(dport=1194, len=50)
The first part marked in green, which you've mistakenly identified as created by yourself, is actually generated by the socket object (the kernel, the appropriate driver and the underlying hardware, to be more accurate).
Depending on your needs, you should either instantiate your socket as a raw one:
mysocket = socket(socket.AF_PACKET, socket.SOCK_RAW)
or set the payload accordingly as just the OpenVPN data:
ascapypacket=Raw(load="\x38\x81\x38\x14\x62\x1d\x67\x46\x2d\xde\x86\x73\x4d\x2c\xbf\xf1\x51\xb2\xb1\x23\x1b\x61\xe4\x23\x08\xa2\x72\x81\x8e\x00\x00\x00\x01\x50\xff\x26\x2c\x00\x00\x00\x00\x00")

Related

How to create an ICMPv6 scapy object from raw bytestring from a socket(AF_INET6, SOCK_RAW, IPPROTO_ICMPv6)?

I want to check/analyse ICMPv6 router advertisements (RAs) in a Python program using Python, where I get the (not so) raw packet data as follows:
import socket
sock = socket.socket(socket.AF_INET6, socket.SOCK_RAW, socket.IPPROTO_ICMPV6)
p, addr = sock.recvfrom(4096)
When an ICMPv6 packet gets received, the packet p will contain only(!) the ICMPv6 part itself, but not any other outer elements, so there's no IPv6 header, no Ethernet header, ...
How do I create the correct Scapy ICMPv6-derived packet class from my (no so) raw packet data? _ICMPv6 seems to be just a fallback class. Is there some kind of factory to get the correct sub class (such as ICMPv6ND_RA) from?
After reading the source code of Scapy's inet6.py I noticed that there's a more "direct" -- but not necessarily "cleaner" -- way to get the correct Scapy ICMPv6 message object than Cukic0d's fine answer: that's in the sense that there's an alternative way that doesn't need the "fake" outer IPv6 object shell (albeit I really appreciate Cukic0d's answer, as it would be a good place to dump the sender address into!):
from scapy.layers.inet6 import *
icmpv6_packet = icmp6typescls.get(p[0], ICMPv6Unknown)(p)
This uses the icmp6typescls dictionary mapping the ICMPv6 (message) type, which is in the first octet of the packet returned from the socket), to its corresponding Scapy ICMPv6 message class ... and then calls the constructor, handing the payload over to it.
The class ICMPv6Unknown is the default class Scapy uses when the ICMPv6 (message) type cannot be mapped onto a known message class. This ensures that there's always a sensible constructor to be called, even if we don't understand the packet received in its full details.
It is useally easier in scapy to dissect the packet from a lower layer (second or third), because of how different the dissecting methods can be.
You could try a hacky
a = IPv6(nh=58)
cls = a.default_payload_class(p)
pkt = cls(p)

UDP socket file transfer python 3.5

How do i transfer a large file (video,audio) from my client to server in the local host using UDP sockets in python 3.5? I was able to send a small .txt file but not other file types. Please give me suggestions.
Thank you!
Here is my code to transfer a text file.
CLIENT CODE:
import socket
import sys
s=socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
host = '127.0.0.1'
port=6000
msg="Trial msg"
msg=msg.encode('utf-8')
while 1:
s.sendto(msg,(host,port))
data, servaddr = s.recvfrom(1024)
data=data.decode('utf-8')
print("Server reply:", data)
break
s.settimeout(5)
filehandle=open("testing.txt","rb")
finalmsg=filehandle.read(1024)
s.sendto(finalmsg, (host,port))
SERVER CODE:
import socket
host='127.0.0.1'
port=6000
s=socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
s.bind(("",port))
print("waiting on port:", port)
while 1:
data, clientaddr= s.recvfrom(1024)
data=data.decode('utf-8')
print(data)
s.settimeout(4)
break
reply="Got it thanks!"
reply=reply.encode('utf-8')
s.sendto(reply,clientaddr)
clientmsg, clientaddr=s.recvfrom(1024)
Don't use UDP for transferring large files, use TCP.
UDP does not garauntee all packets you send will arrive, or if they will arrive in order, they may even be duplicated. Furthermore UDP is not suited to large transfers because 1) it has no congestion control so you will just flood the network and the packets will be dropped, and, 2) you would have to break up your packets into smaller ones usually about 1400 bytes is recommended to keep under MTU otherwise if you rely on IP fragmentation and one fragment is lost your whole file is lost .. You would have to write custom code to fix all these issues with UDP since file transfers require everything to be sent reliably.
TCP on the other hand already does all this, it is reliable, has congestion control and is ubiquitous - you are viewing this web page over HTTP which is on top of TCP.
If you must use UDP instead of TCP or an application level protocol then, you should be able to 'generate redundant blocks' with a package like zfec so that you can reconstruct the original data even if not all of the packets are received.

Can I intercept network packets with a raw socket (not only sniff)?

This is my first time using raw sockets (yes, I need to use them as I must modify a field inside a network header) and all the documentation or tutorials I read describe a solution to sniff packets but that is not exactly what I need. I need to create a script which intercepts the packet, process it and sends it further to the destination, i.e. the packets should not reach the destination unless my script decides to.
In order to learn, I created a small prototype which detects pings and just prints "PING". I would expect ping not to work as I intercept the packets and I don't include the logic to send them to its destination. However ping is working (again, it seems as it is just sniffing/mirroring packets). My goal is that the ping packets are "trapped" in my script and I don't know how to do that. This is what I do in my current python script (I avoid writing how I do the decode for simplicity)
sock = socket.socket(socket.AF_PACKET, socket.SOCK_RAW, socket.ntohs(0x0003))
sock.bind((eth0, 0))
packet = sock.recvfrom(65565)
decode_eth(packet)
decode_ip(packet)
if (ipheader.ip_proto == 1):
print("\nPING")
Can somebody explain how can I achieve my goal or point me to the right documentation?
Your description seems to be different from what your title suggest. My understanding is that you want to receive, modify and possibly drop incoming network packets. And this is to be done on Linux. In that case I suggest you use a netfilter prerouting hook, which will make things a lot simpler (and likely more stable). Netfilter is well documented, a nice overview including information related to your requirements can be seen here. The important function to use is nf_register_hook(), read the answer to this question to get an idea of how to set things up.
I suppose that your Linux box is configured as a router (not a bridge). The packet will pass through your Linux because you have enabled IP Forwarding. So there are two solution:
Solution 1:
Disable IP Forwarding and then receive the packet from one interface and do the appropriate task (forwarding to another interface or dropping it).
Solution 2:
Use NetFilterQueue.
Install it on your Linux box (Debian/Ubuntu in my example):
apt-get install build-essential python-dev libnetfilter-queue-dev
Use iptables to send packets coming from input interface (eth0 in my example):
iptables -I INPUT -i eth0 -j NFQUEUE --queue-num 1
Run this script to handle packets forwarded to the Queue No.1 :
from netfilterqueue import NetfilterQueue
def print_and_accept(pkt):
print pkt
pkt.accept()
nfqueue = NetfilterQueue()
nfqueue.bind(1, print_and_accept)
try:
nfqueue.run()
except KeyboardInterrupt:
print
Note that pkt.drop() cause dropping the packet. Also you should accept/drop every packet.

Finding out the number of dropped packets in raw sockets

I am developing a program that sniffs network packets using a raw socket (AF_PACKET, SOCK_RAW) and processes them in some way.
I am not sure whether my program runs fast enough and succeeds to capture all packets on the socket. I am worried that the recieve buffer for this socket occainally gets full (due to traffic bursts) and some packets are dropped.
How do I know if packets were dropped due to lack of space in the
socket's receive buffer?
I have tried running ss -f link -nlp.
This outputs the number of bytes that are currently stored in the revice buffer for that socket, but I can not tell if any packets were dropped.
I am using Ubuntu 14.04.2 LTS (GNU/Linux 3.13.0-52-generic x86_64).
Thanks.
I was having a similar problem as you. I knew that tcpdump was able to to generate statistics about packet drops, so I tried to figure out how it did that. By looking at the code of tcpdump, I noticed that it is not generating those statistic by itself, but that it is using the libpcap library to get those statistics. The libpcap is on the other hand getting those statistics by accessing the if_packet.h header and calling the PACKET_STATISTICS socket option (at least I think so, but I'm no C expert).
Therefore, I saw only two solutions to the problem:
I had to interact somehow with the linux header files from my Pyhton script to get the packet statistics, which seemed a bit complicated.
Use the Python version of libpcap which is pypcap to get those information.
Since I had no clue how to do the first thing, I implemented the second option. Here is an example how to get packet statistics using pypcap and how to get the packet data using dpkg:
import pcap
import dpkt
import socket
pc=pcap.pcap(name="eth0", timeout_ms=10000, immediate=True)
def packet_handler(ts,pkt):
#printing packet statistic (packets received, packets dropped, packets dropped by interface
print pc.stats()
#example packet parsing using dpkt
eth=dpkt.ethernet.Ethernet(pkt)
if eth.type != dpkt.ethernet.ETH_TYPE_IP:
return
ip =eth.data
layer4=ip.data
ipsrc=socket.inet_ntoa(ip.src)
ipdst=socket.inet_ntoa(ip.dst)
pc.loop(0,packet_handler)
tpacket_stats structure is defined in linux/packet.h header file
Create variable using the tpacket_stats structre and pass it to getSockOpt with PACKET_STATISTICS SOL_SOCKET options will give packets received and dropped count.
-- some times drop can be due to buffer size
-- so if you want to decrease the drop count check increasing the buffersize using setsockopt function
First off, switch your operating system.
You need a reliable, network oriented operating system. Not some pink fluffy "ease of use" with "security" functionality enabled. NetBSD or Gentoo/ArchLinux (the bare installations, not the GUI kitted ones).
Start a simultaneous tcpdump on a network tap and capture the traffic you're supposed to receive along side of your program and compare the results.
There's no efficient way to check if you've received all the packets you intended to on the receiving end since the packets might be dropped on a lower level than you anticipate.
Also this is a question for Unix # StackOverflow, there's no programming here what I can see, at least there's no code.
The only certain way to verify packet drops is to have a much more beefy sender (perhaps a farm of machines that send packets) to a single client, record every packet sent to your reciever. Have the statistical data analyzed and compared against your senders and see how much you dropped.
The cheaper way is to buy a network tap or even more ad-hoc enable port mirroring in your switch if possible. This enables you to dump as much traffic as possible into a second machine.
This will give you a more accurate result because your application machine will be busy as it is taking care of incoming traffic and processing it.
Further more, this is why network taps are effective because they split the communication up into two channels, the receiving and sending directions of your traffic if you will. This enables you to capture traffic on two separate machines (also using tcpdump, but instead of a mirrored port, you get a more accurate traffic mirroring).
So either use port mirroring
Or you buy one of these:

sendto on Tru64 is returning ENOBUF

I am currently running an old system on Tru64 which involves lots of UDP sockets using the sendto() function. The sockets are used in our code to send messages to/from various processes and then eventually on to a thick client app that is connected remotely. Occasionally the socket to the thick client gets stuck, this can cause some of these messages to get built up. My question is how can I determine the current buffer size, and how do I determine the maximum message buffer. The code below gives a snippet of how I set up the port and use the sendto function.
/* need to adjust the maximum size we can send on this */
/* as it needs to be able to cope with the biggest */
/* messages we send */
lenlen = sizeof(len) ;
/* allow double for when the system is under load */
int lenlen, len ;
lenlen = sizeof(len) ;
len = 2 * 32000;
msg_socket = socket( AF_UNIX,SOCK_DGRAM, 0);
result = setsockopt(msg_socket, SOL_SOCKET, SO_SNDBUF, (char *)&len, lenlen) ;
result = sendto( msg_socket,
(char *)message,
(int)message_len,
flags,
dest_addr,
addrlen);
Note. We have ported this application to Linux and the problem does not seem to appear there.
Any help would be greatly appreciated.
Regards
UDP send buffer size is different from TCP - it just limits the size of the datagram. Quoting Stevens UNP Vol. 1:
...
A UDP socket has a send buffer size (which we can change with SO_SNDBUF socket option, Section 7.5), but this is simply an upper limit on the maximum-sized UDP datagram that can be written to the socket. If an application writes a datagram larger than the socket send buffer size, EMSGSIZE is returned. Since UDP is unreliable, it does not need to keep a copy of the application's data and does not need an actual send buffer. (The application data is normally copied into a kernel buffer of some form as it passes down the protocol stack, but this copy is discarded by the datalink layer after the data is transmitted.)
UDP simply prepends 8-byte header and passes the datagram to IP. IPv4 or IPv6 prepends its header, determines the outgoing interface by performing the routing function, and then either adds the datagram to the datalink output queue (if it fits within the MTU) or fragments the datagram and adds each fragment to the datalink output queue. If a UDO application sends large datagrams (say 2,000-byte datagrams), there's a much higher probability of fragmentation than with TCP. because TCP breaks the application data into MSS-sized chunks, something that has no counterpart in UDP.
The successful return from write to a UDP socket tells us that either the datagram or all fragments of the datagram have been added to the datalink output queue. If there is no room on the queue for the datagram or one of its fragments, ENOBUFS is often returned to the application.
Unfortunately, some implementations do not return this error, giving the application no indication that the datagram was discarded without even being transmitted.
The last footnote needs attention - but it looks like Tru64 has this error code listed in the manual page.
The proper way of doing it though is to queue your outstanding messages in the application itself and to carefully check return values and the errno after each system call. This still does not guarantee delivery (since UDP receivers might drop the packets without any notice to the senders). Check the UDP packet discard counters with netstat -s on both/all sides, see if they are growing. There is really no way around this besides switching to TCP or implementing your own timeout/ack and re-transmission logic.
You should probably be using some sort of congestion control to avoid overloading the network. By far the easiest way to do this is to use TCP instead of UDP.
It fails less often on Linux because UDP sockets wait for space in the local network interface queue on Linux (unless you set them non-blocking). However, with any operating system, if the overfull queue is not in the local system, the packet will be dropped silently.

Resources