I understand that UDP does not guarantee order of packets, but that is at Transport layer.
Does the Linux kernel guarantee ordered UDP packets to user space programs ?
I read that services like DNS and DHCP use UDP and they have daemons running in user
space for that.
If Linux does not provide ordered UDP packets to user space then how do services like
DNS and DHCP work reliably ?
Thanks
Related
I want to build an overlay network through VXLAN multicast to achieve communication between virtual machines, but I found that multicast packets can only be transmitted on the same subnet. In order to allow virtual machines on hosts that are not on the same subnet to communicate, I am thinking whether "capture and forward packets" would work. That is, grab a UDP packet with a destination address of 239.1.1.1 and a port of 4789 on hostA on network1, and send it to hostB on network2, and let hostB send the multicast packet. Then I found that the hosts on network2 all can catch this packet with wireshark, but no host corresponds to it. I wonder if Linux has a mechanism to discard fake multicast packets? If this is the case, how should this mechanism be prevented?
Unhandled, multicast is essentially broadcast. For IPv4 multicast that broadcast effect can be mitigated with IGMP. On switched networks with semi-intelligent switches there may be IGMP snooping functionality to further aid in this. Provided this exists, an end-device must subscribe to a multicast group by sending an IGMP join for the given group to "unfilter" that traffic towards itself. Routing multicast between subnets can be done with PIM or DVRMP implementations, or even static multicast routing daemons.
Only exception to this filtering is the 224.0.0.x range, which is reserved for link-local communication, usually IETF protocols. Traffic to these groups must never be filtered in any way.
Hence, to prevent filtering, either the end devices join the group (recommended!), or you send traffic to a group in the reserved range, e.g. on 224.0.0.1 the all-hosts group. (It's ugly and you may trigger ugly bugs on devices in the LAN, but it works.)
I have 2 Ubuntu 16.04 machines "A" and "B" in my local network. "A" periodically sends multicast UDP packet targeted at 225.0.0.37.
On system "B" I can verify through wireshark that those packets are hitting my NIC. I want to write a C program which creates a SOCK_DGRAM socket and listens to these packets without sending IGMP join packet in the network. If I write such a program the kernel doesn't allow these packets to reach my application. They get filtered on IP layer 3 (its my guess).
Is there a way of achieving this? I don't want any IGMP packet leaving my system but still want to capture the UDP traffic. One option that I know works is to use SOCK_RAW, but i want to achieve this from SOCK_DGRAM only.
I know that os keeps igmp subscription entries in /proc/net/igmp and /proc/net/dev_mcast. And these entries are created only if I send an IP_ADD_MEMBERSHIP request from an interface from a program. And these entries get cleared as soon as the program terminates. Maybe there is a way to fool kernel to create this entry.
I have a typical requirement, I want to have a converter module, that can convert UDP packet to TCP packet.
And I need to do it before the packet can be processed in IP layer.
I will explain this complete scenario using an example
Lets say we have 3 machines A, B and C.
A sent an UDP packet
B received UDP packet
At B, when packet is being given to IP layer (from Link / MAC / Ethernet layer) , I want to get hold of packet. I want to delete the UDP and IP header in packet. I want to add TCP and IP header (assuming C is the destination host).
Now from B machine, packet is sent to C machine
Can somebody help me how this can be done.
I am using linux machines.
Though libpcap can be one of the option (from wireshark), but it is not suitable for me because of performance reasons.
I want a very light weight solution for this problem.
Can't be done. TCP is a stateful, reliable, connection-oriented byte-stream protocol. UDP is a stateless, unreliable, unconnected packet protocol.
The best you can do is, on machine B, open a new socket/TCP connection to C, accept socket/UDP packets from A, and write the contents of those packets to the TCP stream. Data flowing the other direction is a bit more difficult because you have to create UDP packets to A no larger than the maximum UDP packet size supported by your systems.
You can not do it directly but if you will use a UDP VPN than you can do it very easily.
Just connect you system/PC with UDP VPN network and it work as you are want.
check below my image for more information, i am also using same.
I need to create a virtual IP network over TCP connection. The hosting system is Linux, with the TUN/TAP kernel driver, it's quite easy to receive & re-inject IP packets of the virtual network.
The difficult part is to transmit the received IP packets to another host. For some non-technical reasons, I can only transmit the packets over TCP protocol but not UDP. Transmit IP packets over UDP is easy, but with TCP it becomes tricky, here's the reason:
UDP protocol doesn't support retransmission/reordering, just like IP. So, if one UDP packet is sent for every received virtual IP packet, the kernel TCP/IP protocol stack would still see virtual IP packet loss/duplication/reordering(those are required for TCP/IP to work well, if those "features" are missing, the TCP connection speed on the virtual network would suffer). If IP packets are transmitted over TCP all required "features" will be missing, unless they are simulated some how.
It seems I have to fake some kind of packet duplication/loss/reordering on TCP connection, or patch the kernel TCP/IP protocol stack. Both options aren't easy.
Is there any other simpler solution to my problem ? or did I just go into a completely wrong direction ? I'm all ears.
==== UPDATE ====
I'm thinking about using raw IP socket (which could get rid of all the TCP retransmission/reordering stuff on the physical network easily while still using TCP packets) to transmit the received virtual network IP packets. But on the receiving host, how can I only receive the packets I'm interested in and return all other IP packets to the kernel TCP/IP stack ?
First of all, you do not want to make a VPN over TCP because you would end up with tcp-over-tcp eventually. The main issue is that the timers of your inner TCP and outer TCP might differ significantly which negatively impacts your TCP session reliability. You can find a bit longer explanation here.
UDP protocol doesn't support retransmission/reordering, just like IP. So, if one UDP packet is sent for every received virtual IP packet, the kernel TCP/IP protocol stack would still see virtual IP packet loss/duplication/reordering(those are required for TCP/IP to work well, if those "features" are missing, the TCP connection speed on the virtual network would suffer). If IP packets are transmitted over TCP all required "features" will be missing, unless they are simulated some how.
This does not make sense, if your outer layer uses TCP as a transport mechanism, nothing stops your inner layer to still use the full ip/tcp stack, including those features. They can conflict badly like I said, but it's not that this functionality disappears or breaks completely.
It seems like you actually want to use TCP just to have the headers and ignore the actual protocol, this would indeed avoid the issues with tcp over tcp. However, once again this is a very bad idea. Flow processing for firewalls, NAT, DPI, tcp boosters, becomes more and more common, if you fake TCP packets you might up stressing those boxes, possibly detoriating your own connection once again.
So you should ask yourself why you can't use UDP, and if no alternative protocol (header) is okay, like GRE or L2TP.
I observed a surprising thing that when there are both udp-based and tcp-based applications sending packets, if the upd-based application sent the packets so fast that the bandwith are nearly filled with udp packets, then the tcp packets would be very hard to send out.
The surprising thing is that though tcp-based application is able to send a few packets out (observed by the return value of write()), the receiver of the tcp packets never receives them. Why? Is that because the tcp-packets arenot finally sent out by the network card? or the tcp packets are actually dropped by routers?
Thanks,
Steve
First, the return value of write() is not an indicator of whether packets were sent. It just indicates that the data was buffered.
Second, if you are saturating the network with UDP packets there will be a lot of packet loss, and TCP being adaptive will adapt to that by sending packets out more slowly. If the packet loss gets too high TCP can basically stop altogether. The solution is not to saturate the network with UDP packets.
This is a simplified answer. There are many articles you can read up on.
UDP is a layer built upon IP. Ditto for TCP. The network card just sends out IP packets. You can look up the various structures for these packets.
TCP is a protocol that uses IP packets but uses a mechanism to try to ensure delivery and rearranges packets in the correct order. See the article on Wikipedia.
Routers are free to drop packets. This can occur when the network is overloaded, network connections are down or the IP packet is corrupted.
So to answer your question their is no preference between UDP or IP to be transmitted from one end to the other.