I'm working on a project where my client is billed exorbitant rates for data transfer on a boat. When they are in port, they use 3g and when they are out at sea they use sattelite.
Every 30 minutes I need to check to see what network I am attached to (moving vessel) but I need to give them specific information on how much data is actually used to make these calls.
I was wondering if anyone knew of any way to get the exact bytes that were sent out and received via terminal response.
Right now I am running this command to get the IP address that my ISP has assigned me.
dig +short myip.opendns.com #resolver1.opendns.com
To identify which network is used right now you may check route table
netstat -r | grep default
You will see default interface used for connection.
There are multiple commands that will show you statistics for interface. E.g.
ip -s link show dev eth0
where eth0 interface identified from command above.
or
ethtool -S eth0
If you want to get data independently from interface(all data stats from boot) you may use IpExt sectoin of
netstat -s
All those metrics will provide system wide counters. For inspecting specific app you may use iptables stats. There are owner module in iptables-extensions that may help. Here are example commands:
# sudo su
# iptables -A OUTPUT -m owner --uid-owner 1000 -j CONNMARK --set-mark 1
# iptables -A INPUT -m connmark --mark 1
# iptables -A OUTPUT -m connmark --mark 1
# iptables -nvL | grep -e Chain -e "connmark match 0x1"
Iptables will allow you to clear counters whenever it needed. Also owner module allow you match packets associated with user group, process id and socket.
Related
I've been using /sys/class/net/eno1/statistics/rx_bytes and tx_bytes to gather stats on my network interface. The trouble is, that network has a device (a Silicon Dust HDHOMERUN HDTV tuner) which constantly streams UDP packets at a very high rate that I don't want to monitor. I'd like to remove that traffic from the monitor - perhaps by only looking at TCP packets.
Is there any way to separate out the TCP and UDP stats?
netstat -st gives some info but it's somewhat cryptic - just how big is a 'segment'? The MTU? The man page is silent on that.
$ netstat -st | grep 'segments received'
25449056 segments received
1683 bad segments received
$ netstat -st | grep 'segments sent out'
37860139 segments sent out
Based on this answer from serverfault. If you are using iptables you can add a rule to each of the INPUT and OUTPUT chains which will count every packet which carries TCP in the payload. It is possible that you will need to invoke every iptables command with sudo.
Create the rules:
# Match all TCP-carrying packets incoming on 'eno1' iface
iptables -I INPUT -i eno1 -p tcp
# Match all TCP-carrying packets outgoing through 'eno1' iface
iptables -I OUTPUT -o eno1 -p tcp
Afterwards, you can use iptables -nvxL INPUT or OUTPUT to be presented with the number of bytes processed by the rule:
Chain INPUT (policy ACCEPT 9387 packets, 7868103 bytes)
pkts bytes target prot opt in out source destination
10582 9874623 tcp -- eno1 * 0.0.0.0/0 0.0.0.0/0
In case you already have other rules defined it might be handy to create a separate chain entirely. This is also described in the answer i referenced, though you also want the -i and -o options in the in/out chains respectively. These allow you to filter on a single interface (use -i for INPUT and -o for OUTPUT).
iptables -N count_in # create custom chain named 'count_in'
iptables -A count_in -j RETURN # append RETURN action to chain 'count_in'
iptables -I INPUT -j count_in # insert chain at the top of chain INPUT
iptables -I count_in 1 -i eno1 -p tcp # insert rule that matches all tcp packets on eno1
# and has no action (does nothing)
iptables -nvxL count_in # list chain 'count_in' rules
I am not sure whether the "bytes" counter includes the IP header, or just the TCP segment bytes but it is still probably the closest metric to what you want to measure (TCP-only rx/tx bytes).
Additionally keep in mind that oftentimes rules defined with iptables are not actually saved and will get deleted on a system reboot. To enable them persistently on every reboot you may use the iptables-save and iptables-restore commands. To learn their usage you should probably look in your Linux distro's documentation as well as iptables manual.
Finally, AFAIK iptables is considered legacy by now and it is being slowly replaced by nftables. I myself still have iptables installed in my system by default. If you want to switch/are already using nftables, then you need to translate above commands to the syntax supported by the nft command. There is a utility called iptables-translate available which may help with this. It's purpose is to translate old iptables commands to equivalent nft commands. I mention this mostly for the sake of completeness, you should be just fine using iptables for your particular task if you have it installed.
You can use iptraf-ng.
Install with:
sudo apt install iptraf-ng
This will give you statistics per protocol (IPv4/IPv6/TCP/UDP/ICMP/...) on a specific interface:
sudo iptraf-ng -d eth0
You can also use this to have details per ports:
sudo iptraf-ng -s eth0
I am trying to forward/clone traffic from my host machine to my docker container using IPtables command.
I am able to receive traffic inside my container via iptables TEE command. However, this command changes the ethernet header by replacing SRC ethernet mac with host ethernet mac. I am interested in collecting this data for my project.
Is there any other way I can achieve this?
Commands used:
1. iptables -t mangle -I PREROUTING -i <host_interface_name>-p tcp -j TEE --gateway <container_ip>
2. iptables -t nat -A PREROUTING -p tcp -j DNAT --to-destination <container_ip:port>
IPtables operate at the network layer and route the packet from the host where the rules were added. Therefore, we cannot avoid update of the source mac. I've tried using TPROXY, FORWARD, ACCEPT. Found the documentation for this at https://ipset.netfilter.org/iptables-extensions.man.html#lbDU
Achieved my requirement using : Linux TC. Simple inbuild Linux Traffic Controller can be used for shaping traffic moving through your interfaces.
https://man7.org/linux/man-pages/man8/tc-mirred.8.html
I am trying to capture the DHCP frames for analysis using the following command in my mac book.
sudo tshark -i en0 -f "port 67 or port 68" -a duration:300 -w /tmp/dump.pcap
I use the following command to get all the fields of all protocols in the packet but it is not printing any value. Is the capture filter option for the DHCP frame is correct? Any help is appreciated?
sudo tshark -T text -r /tmp/dump.pcap -V
Answer
Yes, your commands are OK. Maybe no DHCP packets arrived and therefore not captured. Try to force a DHCP activity by commands in second teminal window of the same device:
sudo dhclient -r
sudo dhclient
Warning: Do not apply these commands if you are connected remotely. First command releases the IP address and your connection will be interrupted without a possibility to put second command and get address back remotely.
Some details concerning data capture
The thsark filters have the same syntax as Wireshark.
Threre exist 2 (or 3) filter types:
capture filter, -f tshark option: It selects which packets will be captured and which not. This is useful e.g. for getting lower capture file size.
display filter, -Y tshark option: It selects which packets will be displayed from all captured ones.
You can combine both types.
Examples:
tshark -i eth0 -n -Y "ip.addr==8.8.8.8"
tshark -i eth0 -n -Y "ip.addr==8.8.8.8" -f "udp port 53"
tshark -i eth0 -n -Y "ip.addr==8.8.8.8 and udp.port==53"
All packets are captured, but only the 8.8.8.8 IP address packets are displayed.
Only the DNS packets are captured, and only the 8.8.8.8 IP address packets from captured are displayed.
All packets are captured, but only the 8.8.8.8 IP address packets having UDP port 53 (i.e. DNS) are displayed. Compare different syntax of the port filtering between the display and the capture filters in line above.
All other options like -a, -b, -w, -s can be applied too.
The tcpdump application is usefull too. It is available in most Linux systems even very small or special. It does not have a display filter option. Only capture filters can be applied. Other options are missing: -a, -b ...
sudo tcpdump -i eth0 -w /tmp/dhcp.pcap "udp port 67 or udp port 68"
Assuming that I am the Server, and I want to watch bandwidth of downloading and uploading for the specific MAC address. With the uploading monitor chain. I use this:
iptables -N clientA_upload and then iptables -A FORWARD -m mac --mac-source 00:11:22:33:44:55:66 and it works just fine. - But when it comes to the downloading chain. I use iptables -A FORWARD -m mac --mac-destination 00:11:22:33:44:55:66 and the iptables doesn't support the mac-destination. Please help me out
P/s: I just want to monitor by MAC address. Not ip address. Because in android OS. It doesn't support byte countering using IP address. So please help !!!
For the missing --mac-destination the trick is to combine iptables --mac-source with CONNMARK:
First use --mac-source to match packets coming from the mac address you're interested in.
use CONNMARK to mark the whole connection, ie both directions (!) and
now check packets going in the other direction with the connection mark.
# lan interface
if_lan=eth0
# packets going to mac address will pass through this:
iptables -t mangle -N clientA_download
# mark connections involving mac address:
iptables -t mangle -A PREROUTING -i $if_lan -m state --state NEW -m mac --mac-source 00:11:22:33:44:55 -j CONNMARK --set-mark 1234
# match packets going to mac address:
iptables -t mangle -A POSTROUTING -o $if_lan -m connmark --mark 1234 -j clientA_download
Initially i thought this would only work for tcp connections originating from the lan, but given the definition of --state NEW it should work in both directions for both tcp and udp (!)
For counters see also ipset which is very nice for this.
Policy Routing on Linux based on Sender MAC Address was the inspiration for this answer.
There is no such thing as --mac-destination. You have to move to ebtables for that.
You are confusing downloading and uploading rules.
Rule 1: iptables -A FORWARD -m mac --mac-source 00:11:22:33:44:55:66
is appended to the ipchain and checks the given mac in forwarding chain.
Now you need to check your mac in input chain, so instead of applying the second rule in FORWARD chain, apply it in INPUT chain:
Rule 2: iptables -I INPUT -m mac --mac-destination 00:11:22:33:44:55:66
Im writing a bash scripting to account traffic in my network server:
WAN:eth1 -> GNU/Linux Server:eth0 -> Users
The GNU/Linux server uses squid, bind, QoS, mysql, lighttpd.
After an IP exceed the established quota a new QoS rule is applied for that IP (user) too exist one "flag" to decide when is restored the IP counter to Zero.
Some IPs and subnets work without quotas, other gruop of ips/subnets work with new QoS after quota is exceeded, and now I wanna work with a third group with redirection after quota is exceeded.
When an IP exceed the established quota all http traffic must be redirected to host (lighttpd runing on GNU/Linux ) and DROP all other traffic generated for that IP. In webserver exist a webpage with: "You exceed your daily quote of traffic, please wait "x" hours or call to your provider to purchase an extra navigation package" or something like that.
Is possible using a chain, or how can I do that?.
The most topics that I found in Internet, are related to block all and create a new chain to let out to Internet (not work for me). And other redirect only IP by IP, but how can I create something that a "chain" and attach the IPs to must me redirected to can after restore that IPs easly?
Thanks for help and sorry for my poor English :S.
Are you looking for something like this?
iptables -t nat -A PREROUTING -s 192.168.100.66 -p tcp --dport 80 -j REDIRECT --to-ports 80
iptables -I INPUT 1 -i lo -s 192.168.100.66 -j ACCEPT
iptables -I INPUT 2 -i eth1 -d 192.168.100.66 -j DROP
This will redirect packets from 192.168.100.66 on port 80 to the local webserver on the loopback interface, allow that conversation, then reject all other packets being routed to 192.168.100.66 on the WAN interface.
To restore the connection back to normal you will want to delete those firewall entries:
iptables -t nat -D PREROUTING -s 192.168.100.66 -p tcp --dport 80 -j REDIRECT --to-ports 80
iptables -D INPUT -i lo -s 192.168.100.66 -j ACCEPT
iptables -D INPUT -i eth1 -d 192.168.100.66 -j DROP
Note that iptables itself (well, the xtables-addons extension set providing quota2) can already do the quota matching magic and you can (re)set the values through procfs, combined with REDIRECT as #resmon6 says:
-t nat -s user1addr -m quota2 --name user1 ! --quota 0 -j REDIRECT...
-t nat -s user2addr -m quota2 --name user2 ! --quota 0 -j REDIRECT...
The syntax is a arguably a little odd right now (0 is the initial value only and is independent from the runtime quota test involving the negational !. Noticing this just now, a patch may make it in to unroll this confusing syntax in the future).