I'm writing a testing tool that requires known traffic to be captured from a NIC (using libpcap), then fed into the application we are trying to test.
What I'm attempting to set-up is a web server (in this case, lighttpd) and a client (curl) running on the same machine, on an isolated test network. A script will drive the entire setup, and the goal is to be able to specify a number of clients as well as a set of files for each client to download from the web server.
My initial approach was to simply use the loopback (lo) interface... run the web server on 127.0.0.1, have the clients fetch their files from http://127.0.0.1, and run my libpcap-based tool on the lo interface. This works ok, apart from the fact that the loopback interface doesn't emulate a real Ethernet interface. The main problem with that is that packets are all inconsistent sizes... 32kbytes and bigger, and somewhat random... it's also not possible to lower the MTU on lo (well, you can, but it has no effect!).
I also tried running it on my real interface (eth0), but since it's an internal web client talking to an internal web server, traffic never leaves the interface, so libpcap never sees it.
So then I turned to tun/tap. I used socat to bind two tun interfaces together with a tcp connection, so in effect, i had:
10.0.1.1/24 <-> tun0 <-socat-> tcp connection <-socat-> tun1 <-> 10.0.2.1/24
This seems like a really neat solution... tun/tap devices emulate real Ethernet devices, so i can run my web server on tun0 (10.0.1.1) and my capture tool on tun0, and bind my clients to tun1 (10.0.2.1)... I can even use tc to apply shaping rules to this traffic and create a virtual WAN inside my linux box... but it just doesn't work...
Here are the socat commands I used:
$ socat -d TCP-LISTEN:11443,reuseaddr TUN:10.0.1.1/24,up &
$ socat TCP:127.0.0.1:11443 TUN:10.0.2.1/24,up &
Which produces 2 tun interfaces (tun0 and tun1), with their respective IP addresses.
If I run ping -I tun1 10.0.1.1, there is no response, but when i tcpdump -n -i tun0, i see the ICMP echo requests making it to the other side, just no sign of the response coming back.
# tcpdump -i tun0 -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tun0, link-type RAW (Raw IP), capture size 65535 bytes
16:49:16.772718 IP 10.0.2.1 > 10.0.1.1: ICMP echo request, id 4062, seq 5, length 64
<--- insert sound of crickets here (chirp, chirp)
So am I missing something obvious or is this the wrong approach? Is there something else i can try (e.g. 2 physical interfaces, eth0 and eth1???).
The easiest way is just to use 2 machines, but I want all of this self-contained, so it can all be scripted and automated on a single machine, without and other dependencies...
UPDATE:
There is no need for the 2 socats to be connected with a tcp connection, it's possible (and preferable for me) to do this:
socat TUN:10.0.1.1/24,up TUN:10.0.2.1/24,up &
The same problem still exists though...
OK, so I found a solution using Linux network namespaces (netns). There is a helpful article about how to use it here: http://code.google.com/p/coreemu/wiki/Namespaces
This is what I did for my setup....
First, download and install CORE: http://cs.itd.nrl.navy.mil/work/core/index.php
Next, run this script:
#!/bin/sh
core-cleanup.sh > /dev/null 2>&1
ip link set vbridge down > /dev/null 2>&1
brctl delbr vbridge > /dev/null 2>&1
# create a server node namespace container - node 0
vnoded -c /tmp/n0.ctl -l /tmp/n0.log -p /tmp/n0.pid > /dev/null
# create a virtual Ethernet (veth) pair, installing one end into node 0
ip link add name veth0 type veth peer name n0.0
ip link set n0.0 netns `cat /tmp/n0.pid`
vcmd -c /tmp/n0.ctl -- ip link set n0.0 name eth0
vcmd -c /tmp/n0.ctl -- ifconfig eth0 10.0.0.1/24 up
# start web server on node 0
vcmd -I -c /tmp/n0.ctl -- lighttpd -f /etc/lighttpd/lighttpd.conf
# create client node namespace container - node 1
vnoded -c /tmp/n1.ctl -l /tmp/n1.log -p /tmp/n1.pid > /dev/null
# create a virtual Ethernet (veth) pair, installing one end into node 1
ip link add name veth1 type veth peer name n1.0
ip link set n1.0 netns `cat /tmp/n1.pid`
vcmd -c /tmp/n1.ctl -- ip link set n1.0 name eth0
vcmd -c /tmp/n1.ctl -- ifconfig eth0 10.0.0.2/24 up
# bridge together nodes using the other end of each veth pair
brctl addbr vbridge
brctl setfd vbridge 0
brctl addif vbridge veth0
brctl addif vbridge veth1
ip link set veth0 up
ip link set veth1 up
ip link set vbridge up
This basically sets up 2 virtual/isolated/name-spaced networks on your Linux host, in this case, node 0 and node 1. A web server is started on node 0.
All you need to do now is run curl on node 1:
vcmd -c /tmp/n1.ctl -- curl --output /dev/null http://10.0.0.1
And monitor the traffic with tcpdump:
tcpdump -s 1514 -i veth0 -n
This seems to work quite well... still experimenting, but looks like it will solve my problem.
Related
I am running a system with the following configuration:
NODE A <-> PROXY <-> SWITCH <-> PROXY <-> NODE B
Where node A and its proxy share a common private network, node b and its proxy share a common private network, and the proxy's are connected to the switch.
I have configured each as such:
#nodeA
#!/bin/sh
#Aliases
alias ip='sudo ip'
alias iptables='sudo iptables'
#interfaceconfiguration
ip link set enp0s8 up
ip addr add 192.168.0.1/29 dev enp0s8
iptables -t nat --flush
sudo route add default gw 192.168.0.2 enp0s8
iptables --policy INPUT ACCEPT
iptables --policy FORWARD ACCEPT
iptables --policy OUTPUT ACCEPT
#proxy a
#!/bin/sh
#Aliases
alias ip='sudo ip'
alias iptables='sudo iptables'
#interfaceconfiguration
ip link set enp0s8 up
ip link set enp0s9 up
ip addr add 192.167.0.1/26 dev enp0s9
ip addr add 192.168.0.2/29 dev enp0s8
sudo sysctl net.ipv4.ip_forward=1
iptables -t nat --flush
sudo route add default gw 192.167.0.2 enp0s9
sudo ip route add 192.168.0.0/29 via 192.168.0.2 dev enp0s8
iptables --policy INPUT ACCEPT
iptables --policy FORWARD ACCEPT
iptables --policy OUTPUT ACCEPT
#proxy B
#!/bin/sh
#Aliases
alias ip='sudo ip'
alias iptables='sudo iptables'
#interfaceconfiguration
ip link set enp0s8 up
ip link set enp0s9 up
ip addr add 192.167.0.2/26 dev enp0s9
ip addr add 192.168.0.10/29 dev enp0s8
sudo sysctl net.ipv4.ip_forward=1
iptables -t nat --flush
sudo route add default gw 192.167.0.1 enp0s9
sudo ip route add 192.168.0.8/29 via 192.168.0.10 dev enp0s8
iptables --policy INPUT ACCEPT
iptables --policy FORWARD ACCEPT
#node B
#!/bin/sh
#Aliases
alias ip='sudo ip'
alias iptables='sudo iptables'
#interfaceconfiguration
ip link set enp0s8 up
ip addr add 192.168.0.9/29 dev enp0s8
iptables -t nat --flush
sudo route add default gw 192.168.0.10 enp0s8
iptables --policy INPUT ACCEPT
iptables --policy FORWARD ACCEPT
iptables --policy OUTPUT ACCEPT
iptables --policy OUTPUT ACCEPT
I then have this simple server client code running on NODE A and NODE B just to have some traffic going through the network:
import socket
import sys
#NODE A - SERVER
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
server_address = ('192.168.0.1', 10000)
sock.bind(server_address)
while True:
data, address = sock.recvfrom(4096)
print(data.decode())
import socket
import sys
#NODE B - CLIENT
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
server_address = ('192.168.0.1', 10000)
while True:
sock.sendto(input("Message to send:").encode(), server_address)
My question is, how can I, at the proxy level intercept these messages to bring them up to the application level so that I can work with them as to add a layer of encryption for example then send them back out to their destination so that the proxy on the other side can decrypt the message.
Thank you
This is a perfectly good debugging loop:
while True:
data, address = sock.recvfrom(4096)
print(data.decode())
Once you see that the right datagrams are arriving,
that loop will be a natural place to forward datagrams
to your favorite destination.
Ensuring that your listening socket was properly bound can be a tricky thing,
given that you have several addresses and interfaces to worry about.
Consider using $ sudo tcpdump -i en0 udp port 10000 to examine
datagrams in flight, before you even run that debugging loop.
You can use the command on the sending host,
on an intermediate transit host,
and on the destination host.
By viewing $ ifconfig -a you may find that an interface besides en0,
perhaps a tunnel interface, is relevant for reading the packets you're interested in.
Once you know certain packets are flowing,
that tends to make it easier to correctly specify the bind() parameters.
I think you're approaching this from the wrong way around. Namely, if you want to "work with" packets on the proxy, then you don't want to be telling the kernel to do any IP level forwarding
the Linux tun/tap interface would let you do this, see here for an example of how this is used with a VPN. although there are Python packages that help if you really want Python code to be doing the work
as a minor side note, I've generally found it an anti-pattern to put sudo onto basically every line of a script as you have. I'd just run the entire script via sudo and just have the code in it assume this is the case.
I have 2 IP addresses on my server.
$curl --interface eth0 ifconfig.co
111.111.111.111
$curl --interface eth0:0 ifconfig.co
222.222.222.222
So via curl I can switch via interfaces so my IP address changes when I visit any url based on eth IP
How can I use same thing via ffmpeg?
For example, if want to access this video via ffmpeg (it uses 111.111.111.111 to access it)
ffmpeg -i 123.com/video.mp4
how can I access same video from my second IP 222.222.222.222 ?
Is there any command like this, perhaps?
ffmpeg --interface eth0:0 -i 123.com/video.mp4
ffmpeg has no such feature. One approach would be to set up a second entry in your /etc/hosts file and access it:
$ cat /etc/hosts
111.111.111.111 123.com
222.222.222.222 456.com
Then access which ever interface you want, using the appropriate DNS entry.
$ ffmpeg -i 123.com/...
$ ffmpeg -i 456.com/...
In a separate question (link below), it was suggested that I obtain the network interface card number and subnet mask from a remote SSH login, rather than prompting the user for it. All I have is the IP address of my remote host. I need to obtain the subnet mask to determine if the remote and local host are on the same subnet, and the interface card number to set up and configure a virtual IP for aliasing purposes.
Could someone suggest how I might be able to parse out the necessary information and return it to my shell script that initiates the remote ssh connection? My target hosts have Linux or AIX as the operating system. I am familiar with the netstat function, but I'm not sure if parsing information from this is valid, or if there is a better way to get what I need that will work for both Linux and AIX operating systems.
Thanks for any help you can provide!
How to return from shell 'read' command passed in expect script?
--UPDATE--
AIX ifconfig -a:
$ ifconfig -a
en0: flags=1e080863,480<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPR
T,64BIT,CHECKSUM_OFFLOAD(ACTIVE),CHAIN>
inet 10.105.65.131 netmask 0xffff0000 broadcast 10.105.255.255
tcp_sendspace 262144 tcp_recvspace 262144 rfc1323 1
lo0: flags=e08084b,c0<UP,BROADCAST,LOOPBACK,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64
BIT,LARGESEND,CHAIN>
inet 127.0.0.1 netmask 0xff000000 broadcast 127.255.255.255
inet6 ::1%1/0
tcp_sendspace 131072 tcp_recvspace 131072 rfc1323 1
AIX netstat -rn:
$ netstat -rn
Routing tables
Destination Gateway Flags Refs Use If Exp Groups
Route tree for Protocol Family 2 (Internet):
default 10.105.65.1 UG 31 39412125 en0 - -
10.105.0.0 10.105.65.131 UHSb 0 0 en0 - - =
>
10.105/16 10.105.65.131 U 219 985607244 en0 - -
10.105.65.131 127.0.0.1 UGHS 5 1326738 lo0 - -
10.105.255.255 10.105.65.131 UHSb 3 6926640 en0 - -
127/8 127.0.0.1 U 36 11962928 lo0 - -
Route tree for Protocol Family 24 (Internet v6):
::1%1 ::1%1 UH 1 393270 lo0 - -
I tried route get and that doesn't work on my AIX box to tell me the route. The only thing I can get to work is netstat -rn. I'm not sure if there is another command similar to ip route that would work.
The Linux boxes support both ip and ifconfig.
I am not sure what to do when there are multiple network interface cards, as I do not know which one really should be used when setting up a virtual IP.
The Linux setup is more what I am concerned with, as I will be eventually adding in the AIX support for my software installation script later and can do more research on it then.
For Linux, I might do this something like so (using bash extensions, so invoked using a #!/bin/bash shebang, or piping the script over stdin to an interpreter invoked as ssh "$hostname" bash <<'EOF'):
internet_address=8.8.8.8 # read data for the NIC used to route here
dev_re='dev ([^[:space:]]+)($|[[:space:]])'
read default_route < <(ip -o route get "$internet_address")
[[ $default_route =~ $dev_re ]] && devname=${BASH_REMATCH[1]}
IFS=$'\n' read -r -d '' -a addresses < \
<(netstat -rn |
awk -v dev="$devname" '$8 == dev && ($2 == "0.0.0.0" || $2 == "default") { print $1 }')
# emit this output however you like
printf '%s\n' "$dev_re" "${addresses[#]}"
I've been looking into LXC containers and I was wondering as to whether or not it is possible to use an LXC container like an ordinary VPS?
What I mean is;
How do I assign an external IP address to an LXC container?
How do I ssh into an LXC container directly?
I'm quite new to LXC containers so please let me know if there are any other differences I should be aware of.
lxc-create -t download -n cn_name
lxc-start -n cn_name -d
lxc-attach -n cn_name
then in container cn_name install openssh server so you can use ssh then reboot it or restart ssh service.
To make any container "services" available to the world configure port forwarding from the host to the container.
For instance if you had a web server in a container, to forward port 80 from the host ip 192.168.1.1 to a container with ip 10.0.3.1 you can use the iptables rule below.
iptables -t nat -I PREROUTING -i eth0 -p TCP -d 191.168.1.1/32 --dport 80 -j DNAT --to-destination 10.0.3.1:80
now the web server on port 80 of the container will be available via port 80 of the host OS.
It sounds like what you want is to bridge the host NIC to the container. In that case, the first thing you need to do is create a bridge. Do this by first ensuring bridge-utils is installed on the system, then open /etc/networking/interfaces for editing and change this:
auto eth0
iface eth0 inet dhcp
to this:
auto br0
iface br0 inet dhcp
bridge-interfaces eth0
bridge-ports eth0
up ifconfig eth0 up
iface eth0 inet manual
If your NIC is not named eth0, you should replace eth0 with whatever your NIC is named (mine is named enp5s0). Once you've made the change, you can start the bridge by issuing the command
sudo ifup br0
Assuming all went well, you should maintain internet access and even your ssh session should stay online during the process. I recommend you have physical access to the host because messing up the above steps could block the host from internet access. You can verify your setup is correct by running ifconfig and checking that br0 has an assigned IP address while eth0 does not.
Once that's all set up, open up /etc/lxc/default.conf and change
lxc.network.link = lxcbr0
to
lxc.network.link = br0
And that's it. Any containers that you launch will automatically bridge to eth0 and will effectively exist on the same LAN as the host. At this point, you can install ssh if it's not already and ssh into the container using its newly assigned IP address.
"Converting eth0 to br0 and getting all your LXC or LXD onto your LAN"
I have been using wire-shark to analyse the packets of socket programs, Now i want to see the traffic of other hosts traffic, as i found that i need to use monitor mode that is only supported in Linux platform, so i tried but i couldn't capture any packets that is transferred in my network, listing as 0 packets captured.
Scenario:
I'm having a network consisting of 50+ hosts (all are powered by windows Except mine), my IP address is 192.168.1.10, when i initiate a communication between any 192.168.1.xx it showing the captured traffic.
But my requirement is to monitor the traffic of 192.168.1.21 b/w 192.168.1.22 from my host i,e. from 192.168.1.10.
1: is it possible to capture the traffic as i mentioned?
2: If it is possible then is wire-shark is right tool for it (or should i have to use differnt one)?
3: if it is not possible, then why?
Just adapt this a bit with your own filters and ips : (on local host)
ssh -l root <REMOTE HOST> tshark -w - not tcp port 22 | wireshark -k -i -
or using bash :
wireshark -k -i <(ssh -l root <REMOTE HOST> tshark -w - not tcp port 22)
You can use tcpdump instead of tshark if needed :
ssh -l root <REMOTE HOST> tcpdump -U -s0 -w - -i eth0 'port 22' |
wireshark -k -i -
You are connected to a switch which is "switching" traffic. It bases the traffic you see on your mac address. It will NOT send you traffic that is not destined to your mac address. If you want to monitor all the traffic you need to configure your switch to use a "port mirror" and plug your sniffer into that port. There is no software that you can install on your machine that will circumvent the way network switching works.
http://en.wikipedia.org/wiki/Port_mirroring