I use dpdk to capture packet:
$RTE_TARGET/app/testpmd -c '0xf' -n 4 --vdev 'eth_pcap0,rx_iface=eth0,tx_pcap=/path/to/file_tx.pcap' -- --port-topology=chained
eth0: using kernel driver.
I don't known stream of packet.
NICs--->Linux kernel---> Application
or
NICs--->DPDK library---> Application
The flow is a combination of your two options:
NIC --> Kernel --> DPDK Library (via the pcap PMD) --> Application (testpmd)
Related
I am new to DPDK and trying to run L3FWD app using scapy to send traffic to it.
I have two Hosts. Host A(Ubuntu 4.15.0-154-generic) for Scapy to send the traffic. Host B (Ubuntu 5.11.0-25-generic) for DPDK(21.08.0) and Host B has vfio-pci module and two NICs (Ethernet Controller XXV710) binded to it.
I have huge pages inserted like below
mkdir -p /dev/hugepages
mountpoint -q /dev/hugepages || mount -t hugetlbfs nodev /dev/hugepages
echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages
I verified the connectivity between Hosts using Scapy on Host A and tcpdump on Host B. (Traffic is coming in from Host A to B)
I build the DPDK and its example apps. I am trying to run L3FWD with this arguments on Host B.
./dpdk-l3fwd -l 1,2 -n 4 -- -p 0x3 --config="(0,0,1),(1,0,2)"
And I tried many ways of sending the traffic from Host A to B using Scapy like below and still not able to see the output of L3FWD on Host B.
way 1)
sendp(Ether()/IP(src="1X.1X.2x.1x"), iface="enp25s0f0",count=1000)
using sendp command, I didn't see traffic on Host B with L3FWD running. Please note, for src ip add i have replaced numeric values with x here
way 2)
send(IP(src="1x.1x.2x.1x"), iface="enp25s0f0",count=1000)
using send command, I didn't see traffic on Host B with L3FWD running
way 3)>>
x = Ether(src='xc:xd:xe:a9:x9:x0', dst='xC:xx:xx:Ax:Bx:x1')
sendp(x, iface='enp25s0f0',count=10000)
This also didn't work
pls Note I have replaced above actual MAC and IP addr with few 'x'.
Output of L3FWD is as below
./build/examples/dpdk-l3fwd -l 1,2 -n 4 -- -p 0x3 --config="(0,0,1),(1,0,2)"
EAL: Detected 56 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
**EAL: No available 1048576 kB hugepages reported**
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_i40e (8086:158b) device: 0000:18:00.0 (socket 0)
EAL: Probe PCI driver: net_i40e (8086:158b) device: 0000:18:00.1 (socket 0)
**TELEMETRY: No legacy callbacks, legacy socket not created**
Neither LPM, EM, or FIB selected, defaulting to LPM
Initializing port 0 ... Creating queues: nb_rxq=1 nb_txq=2... Port 0 modified RSS hash function based on hardware support,requested:0xa38c configured:0x2288
Address:xx:xx:xx:xx:xx:xx, Destination:02:00:00:00:00:00, Allocated mbuf pool on socket 0
LPM: Adding route 198.18.0.0 / 24 (0)
LPM: Adding route 198.18.1.0 / 24 (1)
LPM: Adding route 2001:200:: / 64 (0)
LPM: Adding route 2001:200:0:1:: / 64 (1)
txq=1,0,0 txq=2,1,0
Initializing port 1 ... Creating queues: nb_rxq=1 nb_txq=2... Port 1 modified RSS hash function based on hardware support,requested:0xa38c configured:0x2288
Address:xx:xx:xx:xx:xx:xx, Destination:02:00:00:00:00:01, txq=1,0,0 txq=2,1,0
Initializing rx queues on lcore 1 ... rxq=0,0,0
Initializing rx queues on lcore 2 ... rxq=1,0,0
Checking link status........done
Port 0 Link up at 25 Gbps FDX Autoneg
Port 1 Link up at 25 Gbps FDX Autoneg
L3FWD: entering main loop on lcore 1
L3FWD: -- lcoreid=1 portid=0 rxqueueid=0
L3FWD: entering main loop on lcore 2
L3FWD: -- lcoreid=2 portid=1 rxqueueid=0
Output of L3FWD doesn't go forward after this point.
Can any of you please help me find out where i am going wrong. Or help me know how to send traffic from Host A to run L3FWD app on Host B.
Thanks for your responses.
[EDIT] the solution mentioned in the answer is tried and tested on both HOST and Virtual machines and it works.
There are a couple of ways to check if L3fwd is actually receiving and sending the matches rule traffic. One can check the statistics
using DPDK secondary application dpdk-procinfo for DPDK ports
checking the statistics at the HOST-B PF port via ethtool
checking stats at the sender (scapy) on Host-A
For your specific use cases there 2 factors to match
Since DPDK ports on Host-B is created via VF over single PF interface, you would at least have to send packets with VF-1 MAC address to allow the packet to be received in.
For packets to send out from VF-2, the mac address of the packet has to be modified to allow be forwarded out from Host-B PF to Host-A.
Note: If you were having 2 separate PF NIC on Host-B one need to configure ports on Host-A to promiscuous mode to receive with any MAC address.
Solution:
Program Scapy to send ETH-IP packet with MAC address of VF-1 (assuming MAC-VLAN is not enabled on PF) and Dest IP Address of LPM table entry as 198.18.X.X (where X stands for wild card byte value)
check with the dpdk secondary application for Port-0 (VF-1) and Port-1 (VF-2) to confirm packets are indeed received and transmitted out.
Check on Host-B statistics of PF using cat /proc/net/dev to see if packets are received, transmitted or dropped.
Note: the sample program can be easily edited with print and rte_pktmbuf_dump too.
I am performing a dpdk experiment. In my setup, I have two physical machines, Host1 and Host2 with 2 10Gbps NICs on each. One interface of Host1 is bounded with dpdk and generating traffic using pktgen. Both interfaces of Host2 are bounded with dpdk and l3fwd is running as packet forwarding application. Second NIC of Host2 is used to capture the packets. I want to breakdown the delay experienced by a packet by seeing the time spent in each interface of Host2.
Is there any way to capture packets of dpdk interfaces using l3fwd as packet forwarding applications?
For DPDK interfaces you can make use DPDK-PDUMP capture to get packets from DPDK bonded nic. Refer https://doc.dpdk.org/guides-16.07/sample_app_ug/pdump.html.
Application l3fwd is to be modified with rte_pdump_init API call right after rte_eal_init. This will enable multi-proecss communication channel, there by when dpdk-pdump (secondary) application is run rte_ring and packet copy is enabled to copy the content over.
Note: please check DPDK PDUmp App on usage. FOr example to copy packets from port 0 and queue 1 use sudo ./[path to applciation]/dpdk-pdump -- --pdump 'port=0,queue=1,rx-dev=/tmp/port0_queue1.pcap'
pdump is good tool to capture packets at any port binded to dpdk. Launch the pdump tool as follows:
sudo ./build/app/dpdk-pdump -- --pdump 'port=0,queue=*,rx-dev=/tmp/capture.pcap'
and after packets are received, run the following command in home/temp directory to view them
tcpdump -nr ./capture.pcap
I'm doing this project, where I have to send a bunch of packets (using rte_eth_tx_burst) ,from one network card, to another network card. these 2 network cards are linked（by Fiber I guess? it was not me who linked them.）, so if I just use tcpdump to listen to the other network card, I can capture the whole thing, and store them into a pcap file.
My test environment:
1.a Linux machine, with 2 pairs of linked network cards: eth1 and eth2 are linked, eth3 and eth4 are linked also.
2.my program, listens to eth1.
3.I send a pcap file using eth2 (cmd: tcpreplay -i eth2 test.pcap)
4.my program gets the packet stream, frame by frame, for every frame it recieves, it sends it out, from eth3, using rte_eth_tx_burst() provided by dpdk.
5.I run tcpdump to listen to eth4(cmd:tcpdump -i eth4 -w recieve.pcap -B 1000)
My current problem is:
1. the frames are not in their original orders. (but the contents are the same.)
2. the time gap between frames are not the original gaps.(but I guess this is inevitable).
question:
1.does dpdk guarantees the send order for packets you put in the send queue.
2.perhaps it's because that tcpdump doesn't guarantee the order?
3.or is this problem unsolvable?
Yes, the order of packets send by tx_burst should be guaranteed, provided you're just using one queue.
The out of place packet you are seeing is a reply from eth4, as tcpdump also records outgoing traffic. Remember that the TCP stack is still working as usual on eth4.
Why not use DPDK to capture the packets too? It'd be more reliable and you'd only see incoming packets. Theres an utility for that: pdump. Or you can choose only incoming direction in tcpdump using -Q in.
Also, maybe you already know this, but there's an example app bundled with DPDK that does exactly what you're trying to do: L2FWD.
I have a custom build linux 2.6 running on a imx CPU. CAN communication with external nodes works like a charm, but the frames that I send to the nodes from my app is not printed when using "candump can0" in the terminal (when I run the same app on my host using a usb-CAN dongle this problem does not occur).
When extended CAN frames are sent they show up using candump.
I understand that it might be convenient that messages that is sent is filtered, but I should be possible to choose.
Any clues are welcome. I'm thinking that it might be some filtering in the driver that causes this.
The FLexCAN driver has the following config files:
abort br_propseg dump_rx_mb loopback state
wakeup bcc br_pseg1 dump_xmit_mb maxmb
std_msg xmit_maxmb bitrate br_pseg2
ext_msg modals subsystem boff_rec br_rjw
fifo power tsyn br_clksrc driver
listen smp uevent br_presdiv
dump_reg local_priority srx_dis wak_src
The can-utils package requires SocketCAN, which was integrated into the stock kernel 2.6.30.
If you are using SocketCAN, you should be able to turn on loopback to receive messages that are being transmitted from your interface (and they must actually be transmitted on the wires to be received via loopback).
From bash:
ifconfig can0 down
ip link set can0 type can loopback on
ifconfig can0 up
candump can0|grep C0FFEE
and in another terminal:
cansend can0 00C0FFEE#00112233445566
Alright, I have been at this issue many times but given up. But know I stumbled on the solution:
The setting file called srx_dis in the FlexCAN folder controls this behaviour.
So, if you want to be able to read the CAN frames you send on the imx53_loco, do the following:
ifconfig can0 down
echo 0 > /sys/bus/platform/devices/FlexCAN.0/srx_dis
ifconfig can0 up
The reason I want to do this is for creating a service tool on the same device which can log all CAN frames.
I have a setup having Ipsec Tunnel between two end systems. When I capture packets on outgoing interface e.g eth0 I can only see encrypted ESP packets going out of my system. I have to go and look at the receiving end to see what was actually transmitted i.e on the receiving end I can see packets after decryption. similarly for the ack sent by receiver. How it works actually in the linux kernel ? Is there a way to see the packets on sender side in tcpdump before they get encrypted ?
You have three questions in your question :
How to see outgoing ESP packets in tcpdump before they get encrypted ?
How it works actually in the linux kernel ?
Is there a way to see the packets on sender side in tcpdump before they get encrypted ?
The answer to the third question is
It depends on your kernel and which implementation you are using.
If you are using KLIPS in 2.4 Kernel as part of any of the *swan solutions, then Yes.
Otherwise if you are using the 2.6 kernel which uses the KAME / *BSD model then
No, unless you change your Ipssec or iptables configuration so that you send the packets to another interface unencrypted and then forward the unencrypted packets to an interface which they will be encrypted on.
The answer to the first question is change your iptables configuration and IPSec configuration.
To send the data from one interface to another interface on the same host.
To setup forwarding of data from that second interface to your destination
configure the ip address of the second interface in the security policy to create the tunnel on the second interface.
This way could can tcpdump on the encrypted interface without changing anything and before the data is encrypted.
For the KLIPS and 2.4 kernel it is just as simple as specific the interface in tcpdump as ipsec0 using the -i switch.
Other options to see the data unencrypted involve using tcpdump with -E flag and relevant parameters (see man page) which decrypts the data after it is encrypted or to use tcpdump to capture the encrypted data and then decrypt it in Wireshark (more information regarding ESP decryption of IPSec in Wireshark is located here).
Alternatively you can place instrumentation using printk or printk_times in the source code to xfrm or esp_input.
Regarding the second question, there are several pictures which explain how it works at a high level including the ones explaining the Solaris Implementation as part of Oracle's System Administration Guide: IP Services and Linux's Foundation Network Flow through the kernel diagram.
Also see the following reference. Basically either manual configuration using setkey or automatic configuration using an IKE daemon in userspace passes the information to the XFRM which then transforms the packet before it reaches netfilter prerouting.
See here for more information on the history of KLIPS and KAME.
Assuming you are on Linux:
My approach is to place iptables rule with NFQUEUE target on the correct place in the processing pipeline. Then use tshark with -i nfqueue to capture packets.