I have to make a network monitoring application, under Visual C++ (Windows)
What libraries or APIs do you recommend me to use?
Transport Driver Interface (TDI) filters allow you to monitor network traffic.
You could also try using libpcap which powers ethereal/wireshark. This is based on Network Driver Interface Specification (NDIS).
Few options, depending on what you really want to monitor:
Net-SNMP (or other SNMP library) for talking with the network equipment and getting the data from various MIBs (for example you want to collect interface counters, cpu utilization, and other goodies provided by the vendors)
WinPcap for doing more low-level, packet oriented things (for example capture certain types of traffic using the power and speed of BPF filters)
If you are interested in collecting specific type of information (NetFlow, sFlow) then I cannot help you much. There are ready tools but I'm not sure if there is any in form of a library.
RRDtool (If you want to show the user some nice plots)
Related
I learned the hard way that modern Windows machines do not permit sending TCP data over raw sockets after trying to perform the TCP handshake myself in Python. And yet Scapy, a Python library, is able to do it seemingly just fine. Other libraries, like Npcap and WinPcap, also seem to be able to send raw TCP data just fine on Windows. How is this possible? What are these libraries doing under the hood that enables them to bypass this limitation?
WinPcap (the windows implementation of libpcap) authors say in their website:
WinPcap consists of a driver that extends the operating system to provide low-level network access and a library that is used to easily access low-level network layers.
So the answer to your question would be: in windows, the implementation of libpcap (which is what Scapy uses according to their site) uses a driver to get access to the low-level networking stuff
I want to know what is the difference between iperf tool and Ixchariot tool.
Both are used for performance testing. I want to know difference and features of both tools.
Thanks,
Devanathan.D
Both iperf and IxChariot do network performance monitoring, much in the same way that GIMP and Photoshop both manipulate images. IxChariot is a commercial, enterprise-level, customizable tool. On the other hand, iperf is open source freeware and as such, you get what you pay for. The main advantage of iperf is that it's free. The application is a simple command line executable which can act as either a server or client, and is available on a variety of platforms. There is also a Java-based GUI called jperf which is convenient in that it presents all of iperf's features without requiring the user to recall command line arguments. It seems that jperf may no longer be maintained and some have reported difficulty making it work with more recent versions of iPerf, particularly iPerf3.
IxChariot is more advanced tool that can instantly assess network performance, including wireless performance. Performance endpoints run on mobile, PC, Mac or in any hypervisor or cloud service and allows central management of any platform. It also delivers full application emulation and key performance metrics, including throughput, packet loss, jitter, delay, MOS, and OTT video like Netflix, YouTube.
Some can get by with iperf and similar freeware, just like some can get by with GIMP instead of Photoshop. However, just like most serious graphic artists will eventually migrate to Photoshop, most organizations that really need robust network performance monitoring will eventually turn to commercial, enterprise grade solutions like IxChariot particularly if they need endpoint hardware appliances, advanced UDP, multi-codec voice, video, configurable mixed traffic, configurable TOS/DSCP or mixed traffic with different COS (TOS/DSCP).
Full disclosure - I am an Network Engineer at Ixia.
iperf is free. Thats the main difference to be truthful
I spent the last days reading through man pages, documentations and anything else google brought up, but I suppose I'm even more confused now than I was at the beginning.
Here is what I want to do: I want to send and receive data packets with my own layer 3-x protocol(s) via a wireless interface (802.11) on Linux systems with C/C++.
So far, so good. I do not require beacons, association or any AP/SSID related stuff. However, for data transmissions I'd like the MAC layer to behave "as usual", meaning unicast packets are ACK'd, retransmissions, backoff etc. I'd also like to enjoy the extended QoS capabilites (802.11e with 4 queues and different access categories). Promiscuous mode on the other hand is not a concern, I require only broadcast packets and packets sent to the specific station.
What would be the right way to go about it? Most of the documentation out there on raw socket access seems to be focused on network sniffing and that does not help. I've been playing around with the monitor mode for some time now, but from what I've read so far, received packets are not ACK'd in monitor mode etc.
Without monitor mode, what would be the alternative? Using ad hoc mode and unix raw sockets? Or do I have to fiddle around with the drivers?
I'm not looking for a complete solution, just some good ideas, where to start. I read through the man pages for socket(2), socket(7) and packet(7) but that did not help concerning the behaviour of the MAC layer in different modes.
Thanks in advance.
802.11 is layer 2 (and 1) protocol specification. It was designed in a way, which allows higher-layer protocols to treat it as Ethernet network. Addressing and behaviour is generally the same. So for a layer 3 protocol you should not be concerned about 802.11 at all and write your code as if you were expecting it to run on Ethernet network.
To make it work you should first connect to a wireless network of some sort (which is conceptually equal to plugging a wire into a Ethernet card). Here you may choose ad-hoc (aka IBSS) or infrastructural (aka BSS) network (or PBSS once 802.11ad is approved ;).
Operating cards without any sort of association with network (just spitting out packets on air) is not a good idea for a couple of reasons. Most importantly it's very hardware dependent and unreliable. You can still do it using RF mon (AKA monitor mode) interface on one side and packet injection (using radiotap header) on the other but I don't recommend that. Even if you have a set of identical cards you'll most likely encounter hard to explain and random behaviour at some point. 802.11 NICs are just not designed for this kind of operation and keep different mount of state inside firmware (read about FullMAC vs. SoftMAC cards). Even SoftMAC cards differ significantly. For example theoretically in monitor mode, as you said, card should not ACK received packets. There are cards though that will ACK received frame anyway, because they base their decision exclusively on the fact that said frame is addressed to them. Some cards may even try to ACK all frames they see. Similar thing will happen with retransmissions: some cards will send injected packet only once (that's how it should work). In other NICs, retransmissions are handled by hardware (and firmware) and driver cannot turn it off, so you will get automatic retransmission even with injected data.
Sticking with layer 3 and using existing modes (like ad hoc), will give you all capabilities you want and more (QoS etc.). Ethernet frame that you send to interface will be "translated" by the kernel to 802.11 format with QoS mapping etc.
If you want to find out about MAC behaviour in various modes you'll have to either read the mac80211 code or 802.11 standard itself. http://linuxwireless.org wiki my help you with a few things, but kernel hackers are usually to busy to write documentation other than comments in the code ;)
L3 protocol implementation itself can be also done either in kernel or user mode (using raw sockets). As usual kernel-side will be harder to do, but more powerful.
Because you want to create own network layer protocol (replacement for IP), the keyword is: "raw ethernet socket". So ignore "Raw IP socket" stuff.
This is where to start:
int sockfd = socket( PF_PACKET, SOCK_RAW, htons(XXX) );
Correct man page is: packet(7).
Find more information by googling with the keyword.
One quite complete example here.
Edit: The link to the example seems to be currently broken: another examples
Probably you want something like libpcap.
Libpcap allows you to read/inject raw packets from/into a network interface.
First, there’s something you should be aware of when trying to transmit raw 802.12 frames- the device driver must support packet injection.
You mentioned monitor mode, which is at a high level the rx equivalent of the injection capability- which is not a “mode”, jist a capability/feature. I say this because some 892.11 device drivers on Linux either:
Support monitor mode and frame injection
Support monitor mode and not frame injection
Support neither
I don’t know any straightforward way to check if the driver supports frame injection aside from attempting frame injection and sniffing the air on another device to confirm it was seen.
Monitor mode is usually easy to check by using sudo wlan0 set monitor and seeing what the return code and/or output is.
It’s been a few years since I’ve worked on this but at the time, very few devices supported monitor mode and frame injection “out of the box”. Many only supported monitor mode with a modified version of the vendor or kernel driver
You’ll want to make sure your device has a driver available that fully supports both. This sort of task (frame monitoring and injection) is common for Penetration Testers who tend to use Kali Linux, which is really just an Ubuntu distribution with a bunch of “hacking” tools and (modified) 802.11 device drivers preloaded and in its repositories. You can often save time finding a well supported card by using a search engine to find the device and driver recommended for Kali users
I’m bringing this monitor/injection capability up explicitly because when I first worked on a similar project a few years ago, I needed to use a patched version of the official kernel driver to support monitor mode- it was an rtl8812au chipset. At that time, I made an incorrect assumption that monitor mode support in the driver implied full injection support. I spent 2 days banging my head against the wall, convinced my frames weren’t built correctly in my application, causing no frames to leave the card. Turned out I needed a more recent branch of the driver I was using to get the full injection support. This driver in particular supports both monitor mode and frame injection now. The most frustrating thing about diagnosing that problem was that I did not receive any errors from system calls or in kernel messages while trying to transmit the frames- they were just being silently discarded somewhere, presumably in the driver
To your main question about how to do this- the answer is almost certainly libpcap if you’re writing your application in C/C++ as libpcap provides not only packet capture APIs but also packet injection APIs
If you do it in Python, scapy is an excellent option. The benefit of Python/scapy is that
Python code is much quicker to write than C
scapy provides a significant amount of classes that you can use to intuitively create a frame layer by layer
Because the layers are implemented as classes, you can also extend and “register” existing classes to make certain frames easier to create (or parse when received)
You can do this in straight C using the UNIX sockets API with raw sockets directly- but you’ll have to deal with things that libpcap exists to abstract from you- like underlying system calls that may be required when doing raw frame transmission, aside from the standard socket(), send(), recv() calls. I’m speculating that there are a handful of ioctl calls you may need at the least, specific to the kernel 802.11x subsystem/framework- and these ioctl() calls and their values may not be completely portable across different major kernel versions. I’ll admit I ended up not using the pure C (without libpcap) approach, so I’m not 100% sure about this potential problem. It’s something you should look more into if you plan to do it without libpcap. I don’t recommend it unless you have a really good reason to
It sounds like you are getting the media and transport layers mixed up.
802.11 is what's commonly referred to as a "link", "physical", or "media" layer, meaning it only deals with the transmission of raw datagrams.
Concepts like ACKs, retransmissions, backoff (flow control) apply to the "transport" layer, and those particular terms are strongly associated with TCP/IP.
Implementing your own complete transport layer from scratch is very difficult and almost certainly not what you want to do. If instead you want to use the existing TCP/IP stack on top of your own custom interpretation of 802.11, then you probably want to create a virtual network interface. This would act as an intermediary between TCP/IP and the media layer.
Hopefully this gives you some better context and keywords to look for.
I need to expose certain monitoring statistics from my application and I'm wondering what the most widespread framework or protocol is for doing this?
SNMP is widely used and a standard protocol. It's implemented in computers, routers, hubs, printers and practically anything connected to the net. Although it's called the SImple Network Management Protocol it's not restricted to network management.
It's an open standard and consequently there are a huge array of management/monitoring solutions, from simple shell scripts and libraries up to enterprise monitoring suites (e.g. HP Openview).
You can query synchronously for data or receive events (in SNMP-speak, traps). Each device will report a common set of data (relating mainly to the network status of that device) plus enterprise-specific data (e.g. CPU usage, printer status etc.).
It runs over UDP, and message consistency is a responsibility of the implementing library. This is a little unusual, but it's designed to operate even when the network is not functioning correctly (e.g. flooded with traffic/misconfigured etc.) and decisions about retry strategies, timeouts etc. need to be taken at the application level (unlike TCP).
We implemented a server application available on Windows only. Now we like to port it to Linux, HP-UX and AIX, too. This application provides internal statistics through performance counters into the Windows Performance Monitor.
To be more precise: The application is a data base, and we like to provide information like number of connected users or number of requests executed to the administrator. So these are "new" information, proprietary to our application. But we like to make them available in the same environment where the operating system delivers information like the CPU, etc. The goal is to make them easily readable for the administrator.
What is the appropriate and commonly used performance monitor under Linux, HP-UX and AIX?
I would say: that depends on which performance you want to monitor. Used CPU time? Free RAM? Disk IO? Number of beers in your freezer...
But regardless of this you can look at any files below /proc. I'm not sure for HP, but at least Linux and AIX should have that tree (if it's not deactivated at kernel compile time).
Management is where most OSes depart from one another. For this reason there are not many tools that are common between all the OSes.
Additionally, Unix tools follow the single process single responsibility idiom where one tool gets cpu info, another gets memory etc.
The only tool i have seen in the Unix world that gets all this info in one place is top. Almost all sys admins are familiar with this tool and works on all the flavors of OSes you are interested in. It also has the additional advantage of being open source. You could simply extend this tool to expose the counters you are interested in and ship it along with your application.
Another way to do this might be to expose your counters through SNMP and leave it to some third party SNMP tool like HP open view that can collect and present a consistent view along with other management info. This might be a more enterprisy solution, which might appeal to the marketing folks.
I would also say its a good idea to write a standalone console tool that admins can use from their custom home grown scripts (there are many firsm out there with super human admins / over paid it staff that does this).
All together would be a healthy solution for your requirement i think.
The most standard unix tools for such data are the *stat (iostat, vmstat, netstat) tools and sar. On Linux you'll find all this information in /proc, but most Unixes don't have /proc nicely filled with what you are looking for. The mentioned tools are quite standardized and can be used to gather the data you need.