I'm trying to emulate packet loss for my project. I'm using TCP/IP protocol. netem tool provides such functionality. The delay works in the loopback IP but I couldn't make packet loss to work. According to the website of netem, the packet loss is activated as follows:
tc qdisc change dev lo root netem loss 5%
In the client/server app using TCP/IP socket in c programming, the client sends this message "Echo this !", the echoed message from the server received by the client is intact. As far as I know, TCP/IP guarantees the delivery of packets. Is emulating packet loss impossible with TCP/IP protocol?
If the packet is lost, TCP will send it again after some delay. If it gets lost again, it will send it again. And so on, up to a maximum of 10 minutes or so, after which it just gives up.
5% packet loss is not completely terrible and your message is likely to get through after one or two resends - or zero. Also notice your whole message fits in a packet, so your programs only send a few packets in total (your message plus extra ones to connect and disconnect) and it's quite likely that none of them will be lost.
You can try sending a longer message (like a megabyte), and you can try cranking the packet loss up to 25% or 50% (or even higher!). It should take a lot longer to send the message, even without any delay in the network, but your message should get through eventually, unless TCP decides to just give up and disconnect you.
Related
Let’s say client opens a tcp connection to server.
Say client sends 100 packets.
10 of them reached server and were picked by application.
50 of them reached server but not yet picked up by application
40 are still sitting in client
socket buffer because the servers receive window is full.
Let’s say now client closes the connection.
Question —
Does application get the 50 packets before it is told that the connection is closed?
Does the client kernel send the remaining 40 packets first to client before it sends the FIN packet?
Now to complicate matters, if there is lot of packet loss, what happens to the remaining 40 packets and the FIN. Does it close it?
Does application get the 50 packets before it is told that the connection is closed?
It does.
Does the client kernel send the remaining 40 packets first to client before it sends the FIN packet?
It does.
Now to complicate matters, if there is lot of packet loss, what happens to the remaining 40 packets and the FIN. Does it close it?
The kernel will keep trying to send the outstanding data. The fact that you closed the socket doesn't change things, unless you altered socket options to change this behaviour.
I've written my own packet sniffer in Linux.
I open a socket with socket(AF_PACKET, SOCK_RAW, htons(ETH_P_ALL)) and then process the Ethernet packets - unpacking ARP packets, IP packets (and ICMP / TCP / UDP packets inside those).
This is all working fine so far.
Now I can read packets like this - and I can also inject packets by wrapping up a suitable Ethernet packet and sending it.
But what I'd like is a means to block packets - to consume them, as it were, so that they don't get further delivered into the system.
That is, if a TCP packet is being sent to port 80, then I can see the packet with my packet sniffer and it'll get delivered to the web server in the usual fashion.
But, basically, I'd like it that if I spot something wrong with the packet - not coming from the right MAC address, malformed in some way, or just breaking security policy - that I can just "consume" the packet, and it won't get further delivered onto the web server.
Because I can read packets and write packets - if I can also just block packets as well, then I'll have all I need.
Basically, I don't just want to monitor network traffic, but sometimes have control over it. E.g. "re-route" a packet by consuming the original incoming packet and then writing out a new slightly-altered packet to a different address. Or just plain block packets that shouldn't be being delivered at all.
My application is to be a general "network traffic management" program. Monitors and logs traffic. But also controls it too - blocking packets as a firewall, re-routing packets as a load balancer.
In other words, I've got a packet sniffer - but if it sniffs something that smells bad, then I'd like it to be able to stop that packet. Discard it early, so it's not further delivered anywhere.
(Being able to alter packets on the way through might be handy too - but if I can block, then there's always the possibility to just block the original packet completely, but then write out a new altered packet in its place.)
What you are looking for is libnetfilter_queue. The documentation is still incredibly bad, but the code in this example should get you started.
I used this library to develop a project that queued network packets and replayed them at a later time.
A bit of a tangent, but it was relevant when I was resolving my problem. Blocking raw packets is relatively complicated, so it might make sense to consider doing that at a different layer. In other words, does your cloud provider let you set up firewall rules to drop specific kind of traffic?
In my case it was easier to do, which is why I'm suggesting such a lateral solution.
I noticed that for the transfer of one-packet file from remote FTP site to localhost on Linux, Wireshark can always capture the packets in correct order but not in tcpdump/libpcap or simple recvfrom on RAW_PACKET with promiscuous mode on.
In the former, the "transfer complete" response is always before the single data packet (in different connections so no TCP reordering), but in the latter the data packet always arrives first - which is clearly wrong according to the protocol and the implementation of FTP servers since "transfer complete" is sent after the data is sent out - and if the client has received it before data it'd stop waiting for data connection, which didn't happen since I can see the data clearly. So the libpcap/tcpdump actually captures packets in wrong order but no such problem in Wireshark?
How is this possible?? Wireshark also uses libpcap...
For FTP protocol payload is transferred in separate TCP connection, there is no promises about order of packets in parallel TCP connection (actually there is no promise in order of packets even in same TCP connection, your host should order them).
server has two open sockets.
it writes file to data socket
immediately after that it writes "transfer complete" to control sockets.
the difference between 2 & 3 is several microseconds.
Packets run thru the internet, they could even choose different paths
They came to your machine in random order.
p.s. also tcpdump file has number of packet and timestamp. Packets are not necessarily written sorted by timestamps. Wireshark could order them when showing, take a look on order field.
I am trying to generate a series of packets to simulate the TCP 3-way handshake procedure, my first step is to capture the real connecting packets, and try to re-send the same packets from the same machine, but it didn't work at first.
finally I found it out that the packet I captured with tcpdump is not exactly what my computer sent out, the TCP's checksum field is changed and it lead me to thinkk that I can establish a tcp connection even the TCP checksum is incorrect.
so my question is how is the checksum field calculated? is it modified by tcpdump or hardware? why is it changed? Is it a bug of tcpdump? or it's because the calculation is omitted.
the following is the screenshot I captured from my host machine and a virtual machinne, you can see that the same packet captured on differnet machine are all the same except for the TCP checksum.
and the small window is my virtual machine, I used command "ssh 10.82.25.138" from the host to generate these packets
What you are seeing may be the result of checksum offloading. To quote from the wireshark wiki (http://wiki.wireshark.org/CaptureSetup/Offloading):
Most modern operating systems support some form of network offloading,
where some network processing happens on the NIC instead of the CPU.
Normally this is a great thing. It can free up resources on the rest
of the system and let it handle more connections. If you're trying to
capture traffic it can result in false errors and strange or even
missing traffic.
On systems that support checksum offloading, IP, TCP, and UDP
checksums are calculated on the NIC just before they're transmitted on
the wire. In Wireshark these show up as outgoing packets marked black
with red Text and the note [incorrect, should be xxxx (maybe caused by
"TCP checksum offload"?)].
Wireshark captures packets before they are sent to the network
adapter. It won't see the correct checksum because it has not been
calculated yet. Even worse, most OSes don't bother initialize this
data so you're probably seeing little chunks of memory that you
shouldn't.
Although this is for wireshark, the same principle applies. In your host machine, you see the wrong checksum because it just hasn't been filled in yet. It looks right on the guest, because before it's sent out on the "wire" it is filled in. Try disabling checksum offloading on the interface which is handling this traffic, e.g.:
ethtool -K eth0 rx off tx off
if it's eth0.
I observed a surprising thing that when there are both udp-based and tcp-based applications sending packets, if the upd-based application sent the packets so fast that the bandwith are nearly filled with udp packets, then the tcp packets would be very hard to send out.
The surprising thing is that though tcp-based application is able to send a few packets out (observed by the return value of write()), the receiver of the tcp packets never receives them. Why? Is that because the tcp-packets arenot finally sent out by the network card? or the tcp packets are actually dropped by routers?
Thanks,
Steve
First, the return value of write() is not an indicator of whether packets were sent. It just indicates that the data was buffered.
Second, if you are saturating the network with UDP packets there will be a lot of packet loss, and TCP being adaptive will adapt to that by sending packets out more slowly. If the packet loss gets too high TCP can basically stop altogether. The solution is not to saturate the network with UDP packets.
This is a simplified answer. There are many articles you can read up on.
UDP is a layer built upon IP. Ditto for TCP. The network card just sends out IP packets. You can look up the various structures for these packets.
TCP is a protocol that uses IP packets but uses a mechanism to try to ensure delivery and rearranges packets in the correct order. See the article on Wikipedia.
Routers are free to drop packets. This can occur when the network is overloaded, network connections are down or the IP packet is corrupted.
So to answer your question their is no preference between UDP or IP to be transmitted from one end to the other.