Pfsense Iftop bandwidth monitoring - firewall

Experts,
I installed a pfsense in my server to be my firewall. I am new to iftop. When I execute the iftop command, I would like to understand what is the bottom form where it says:
rate xxxx yyyyy zzzz
xxx1 yyyy1 zzz1
xxx2 yyy2 zzz2
What does it mean ?
Best
Anderson

From the man page:
At the bottom of the display, various totals are shown, including peak traffic over the last 40s, total traffic transferred (after filtering), and total transfer rates averaged over 2s, 10s and 40s.
The three rows are labelled on the left hand side (TX = outbound, RX = inbound, TOTAL = total)

Related

Need to measure pairwire (ip based) bandwidth used over time when using TC

I need to measure the datarates of packets between multiple servers. I need pairwise bandwidths between the servers (if possible even the ports), not the overall datarate per interface on each server.
Example output
Timestamp
Server A to B
Server B to A
Server A to C
Server C to A
0
1
2
1
5
1
5
3
7
1
What I tried or thought of
tcpdump - I was capturing all the packets and looking at ip.len for getting the datarates. It worked quite well till I started testing along with TC.
Turns out tcpdump captures packets at a lower layer than TC. So, the bandwidths I measure using this can't see the limit set by TC.
netstat - I tried using this by greping the output and look at Recv-Q and Send-Q columns. But later I found out that it reports the bytes that have been received and are buffered, waiting for the local process that is using this connection to read and consume them. I won't be able to use them to get bandwidth being used.
iftop - Amazing GUI and has all the things I need. But no way to get the output in a good way to process. Might also overwhelm the storage because of the amount of extra text it stores along with.
bwm-ng - Gives overall datarate per interface on each server but not pairwise.
Please let me know if there are any other ways to achieve what I need.
Thanks in advance for your help.

What does BandwidthIn and BandwidthOut graph represent for a service?

I have a service and its bandwidth graph looks like this
What does it represent.? I am using tutum which shows me these graphs.!
Should I worry about it.? Please Explain! Any help is appreciated.!
Bandwidth is the the amount of data sent (Out) or received (In) in a period of time. Mbps stands for Mega bits per seconds, i.e., how many bits did you send or receive during that past whole second.
I am sure you heard about xxx Mpbs from your internet provider, in which case, it correspond to the maximum speed you can have, but you are not required to use the whole bandwidth all the time.
Same thing on Tutum, depending on your hosting provider / instance type you will also have a maximum Mbps bandwidth, but at any given t time, you are using YY Mbps out of your XX Mpbs maximum.
As the graph increase, it simply means that you send/receive more data, which can mean that you have a higher traffic or you are doing some kind of networking activity.

Using Unix TC to shape high bandwidth traffic

We actually have a 10Gb/s servers and 1Gb/s servers that coexist together (temporary migrating solution) [UDP traffic]. We would like to shape the traffic coming from the 10Gb/s servers in order to avoid big bursts that the 1G servers could not handle.
It seems that "tc" cannot do the job with a tbf (or maybe we use it the wrong way). For instance on our 10G servers we tried the following:
sudo tc qdisc add dev eth5 root tbf rate 950mbit latency 1s burst 50mbit peakrate 1000mbit mtu 1500
Here we normally set the peakrate at 1mb (which normally can't generate burst > 1mb/s).
Unfortunately, that does not work, in fact after using this tc config, we lower our main bandwidth to at max 2Mb/s..
Our only clue for this strange behavior is that sentence in the tc manual:
"To achieve perfection, the second bucket may contain only a single packet, which leads to the earlier mentioned 1mbit/s limit.
This limit is caused by the fact that the kernel can only throttle for at minimum 1 'jiffy', which depends on HZ as 1/HZ. For perfect shaping, only a single packet can get sent per jiffy - for HZ=100, this means 100 packets of on average 1000 bytes each, which roughly corresponds to 1mbit/s. "
So, it's sure we can't have a peakrate > 1Mbit/s ?
Maybe, there is another completely different way to achieve our goal, if anyone has a suggestion that would help me achieve our goal.. =) ?
Kind regards
Why do you have a 1s latency? Seems WAY too high for a 1 Gbit link

How to automate measuring of bandwidth usage between two hosts

I have an application that has a TCP client and a server. I set up the client and server on separate machines. Now I want to measure how much bandwidth is being consumed ( bytes sent and received during a single run of the application). I have discovered that wireshark is one such tool that can help me get this statistic. However, wireshark seems to be GUI dependent. What I wanted was a way to automate the measuring and reporting of this statistic. I dont care about the information about individual packets captured by wireshark. I dont need that information. Is there some way to run wireshark so that all it does is write to a file, the total bytes sent and received between two hosts while the application was running on both ends?
Also, is there a better way to capture this statistic ? Through netstat or /proc/dev/net or any other tool ?
Both my machines have ubuntu 10.04 or later running on them.
Bro is an appropriate tool to measure connection-oriented statistics. You can either record a trace of your application communication or analyze it in realtime:
bro -r <trace>
bro -i <interface>
Thereafter, have a look at the connection log (conn.log) in the same directory for the amount of bytes sent and received by the application. Specifically, you're interested in the TCP payload size, which conn.log exposes via the columns orig_bytes and resp_bytes. Here is an example:
bro-cut id.orig_h id.resp_h conn_state orig_bytes resp_bytes < conn.log | head
which yields the following output:
192.168.1.102 192.168.1.1 SF 301 300
192.168.1.103 192.168.1.255 S0 350 0
192.168.1.102 192.168.1.255 S0 350 0
192.168.1.103 192.168.1.255 S0 560 0
192.168.1.102 192.168.1.255 S0 348 0
192.168.1.104 192.168.1.255 S0 350 0
192.168.1.104 192.168.1.255 S0 549 0
192.168.1.103 192.168.1.1 SF 303 300
192.168.1.102 192.168.1.255 S0 - -
192.168.1.104 192.168.1.1 SF 311 300
Each row represents a single connection, transport-layer ports omitted. The last two columns represent the bytes sent by the originator (first column) and responder (second column). The column conn_state represents the connection status. Please refer to the documentation for all possible field values. Some important values are:
S0: Connection attempt seen, no reply.
S1: Connection established, not terminated.
SF: Normal establishment and termination. Note that this is the same symbol as for state S1. You can tell the two apart because for S1 there will not be any byte counts in the summary, while for SF there will be.
REJ: Connection attempt rejected.

Bluetooth Ping Latency

I am currently working on a project involving a Lego Mindstorms kit. The brick is the NXT and I was curious about the bluetooth ping rates.
I ran a test of 100 pings on it and got some interesting results. The latencies seemed to fall into bands. I increased to 10,000 pings and it highlighted this trend even more clearly. Does anyone know what could cause this to happen?
In case it is relevant, the distance between the sender and receiver was about 3 metres.
Few reasons :
Buffering and internal timers to flush buffers can cause it.
Also depends on the ping intervals (i.e. time between subsequent pings), as the link might go to power save mode during inactivity and it will take a fine time to come back up.
Size of the ping packets
What bluetooth profile is used here ?

Resources