Is there a way to double the number of ephemeral ports and work around the 16-bit limit? I have tried creating virtual ethernet interfaces over eth0, and hope that would lift the limit. Although the application is utilizing the new virtual IPs in outbound traffic, it seems still hitting the same ephemeral-port limit. I suppose the virtual ports have 1-to-1 mapping to the ports on the physical interface.
ifconfig eth0:1 10.10.10.210 netmask 255.255.255.192
ifconfig eht0:2 10.10.10.211 netmask 255.255.255.192
Could someone please advise how I could double the total number of ephemeral ports in Linux, without adding an extra NIC?
(FYI, I have tried increasing ulimit/max open file, changing the port range, enabling tcp recycle/timestamps, reducing tcp fin timeout... I suppose we simply need more than 65k ports for this proxy machine.)
If you create virtual interfaces over eth0, then you should be able to assign different IP addresses to those interfaces. With that, you can use the same ephemeral port numbers (they are allocated in the kernel, so you dont really have much control) for multiple sockets each bound to different addresses -- you will probably need to set SO_REUSEADDR option. The reason this will work is because for incoming packets (UDP/TCP), the flow is identified by looking at both local source IP and the port number.
And as #Duck mentioned, since TCP/UDP headers allocate only 16 bits for port numbers, there is not much point in increasing the ephemeral range in the local stack.
It's a limitation of the network protocols. Both TCP & UDP, for instance, have 16 bit source and destination ports. Even if you could increase the number of ports no one could address them.
It seems there's a way, but it's not for free. It's called "bind before connect". See this short but dense article, which sums it up very nicely.
Having multiple virtual IPs is just a start. Quoting the linked article:
On Linux the ephemeral port range is a global resource, it's not a specific setting local to an IP address.
So that's bad and you have to improve your starting position with few right settings (where most of them you already found) and get around the global limit with a clever socket allocation technique. The result is that you'll control all the outgoing IPs manually. This also seems not to cope well with other apps on the system using the traditional "connect" way.
It turns out that you cannot use 0 for binding ephemeral port if you want to exceed the 65535 limit. Instead, you need to use an explicit port number.
And also turning on tcp_tw_reuse might be helpful: http://krenel.org/tcp-time_wait-and-ephemeral-ports-bad-friends.html
Related
To my CentOS server, TCP connection is provided over port 643 and UDP connection is provided over port 6194. I want to add 1 hour drop rule to each IP address that consumes 50MB traffic from each of these ports.
Can I do this using iptables or tc? If I can how do I do it? I don't know enough about the subject, can you help me please?
It is possible to limit incoming and outgoing bandwidth and latency with tc (Traffic Control). This means you can control the throughput, the data amount over time only.
According your description for 50MB/hr you would need to set something like 125kBps for your rate. Since it is bandwidth limitation to make sure that only a specific amount of traffic can be used, there is no time limitation.
Further Questions and Answers on this topic are
Limiting interface bandwidth with tc
Limit network bandwith for an IP
How to limit network bandwidth
A more Advanced Solution could be
Rate limit network but allow bursting per TCP connection before limiting
Even if it is possible to cut of TCP/IP connections via in example cutter tool or set block time with iptables, I am not aware of any production ready solution for controlling the time of a network session. You may also have a look into wondershaper or trickle.
I am creating a tcp connection using the function socket(), bind(), and then listen().
Our customers would like to be able to define an IP address of the server at runtime. Is there a way of changing the IP at runtime or must it be done in the BIOS?
Thanks for any tips
I've changed the IP address using ifAddrSet(..) many times. Usually I call this function from within the startup script before my application is running so I have no idea how calling this function affects already connected sockets.
But have a look at the functions provided by ifLib.h. I'm sure you'll find something that suits your needs (ifAddrAdd(..) looks promising).
I am not sure what you mean by defining 'IP address of the server at runtime?'. Obviously for a given socket it's IP address cannot be changed. It's an endpoint of a connection, it cannot be changed run time. If you just want to assign multiple IP addresses to a host that's possible.
In general - you can add as many IP addresses as you want to your machine (ok not exactly) but certainly a hundred or so (ie. statically allocated). That's not the problem (management of that is a nightmare, but sure not impossible). The problem is how those IP addresses are reached, that is not in your control, that depends upon the settings on client especially the routing entries. eg. you could use all of the IP addresses in a Subnet (say 10.1.2/24).
Not recommended - but possible.
Once you have those IP addresses - you bind on the port and address as INADDR_ANY, which says accept connection on 'any ' of the local addresses. On which address the connection was made to can be determined on server using getsockname.
I have a laptop that is connected to my organization's network using one or more network adapters. I am trying to write a tool that will continuously monitor the connectivity status and connection quality of each network. However my networking knowledge to limited and the terminology confuses me.
Specifically finding all the network adapters. Someone suggested that I use the command ifconfig and it gave me what are called "interface identifiers".
ex:
['lo0', 'gif0', 'stf0', 'en0', 'en1', 'en2', 'bridge0', 'p2p0']
I'm not quite sure how this helps me solve my problem because I don't know what interface identifiers are and I am not sure how to leverage this information. My assumption is that they represent a computer or a router in the network.
If someone could clear this up or explain it to me in layman's terms that would be really helpful.
First of all, you need to understand that there may be physical network cards(OR/AND logical network adapters) present in the computer to identify connection/manage connection.
Next, you have an incorrect notion about interface identifier.
What you talked about(eth,virbr,lo) are interfaces. In IPv4 addressing scheme, we don't have interface ID. We have interface ID's in IPv6 addresses.
As mentioned in The Payoff of IPv6’s Very Large Address Size
In IPv4, IP addresses have no relationship to the addresses used for underlying data link layer network technologies. A host that connects to a TCP/IP network using an Ethernet network interface card (NIC) has an Ethernet MAC address and an IP address, but the two numbers are distinct and unrelated in any way.
With the overhaul of addressing in IPv6, an opportunity presented itself to create a better way of mapping IP unicast addresses and physical network addresses. Implementing this superior mapping technique was one of the reasons why IPv6 addresses were made so large. With 128 total bits, even with a full 48 bits reserved for network prefix and 16 bits for site subnet, we are still left with 64 bits to use for the interface identifier, which is analogous to the host ID under IPv4.
Having so many bits at our disposal gives us great flexibility. Instead of using arbitrary “made-up” identifiers for hosts, we can base the interface ID on the underlying data link layer hardware address, as long as that address is no greater than 64 bits in length. Since virtually all devices use layer two addresses of 64 bits or fewer, there is no problem in using those addresses for the interface identifier in IP addresses. This provides an immediate benefit: it makes networks easier to administer, since we don't have to record two arbitrary numbers for each host. The IP address can be derived from the MAC address and the network identifier. It also means we can in the future tell the IP address from the MAC address and vice-versa.
Visit this link for more clear understanding about interface ID.
Now,returning to clear your confusion,
all of the connections(interfaces) such as Ethernet-0,Ethernet-1,WiFi-1,etc. have their own interface identifier.You can think of them as a kind of special identification number which identifies the kind of interfaces available at that moment!
When you type ifconfig in Linux, it displays the status of the currently active interfaces.
Now,coming on the example part, let's say you have two Ethernet connections on your system, say, eth0 and eth1(these are interface names) ---so ifconfig will print these two as a result of it's output!
So,to identify these two separate interfaces,there must be an
interface identifier.The interface identifier(generally 64-bit) is
either automatically generated from the interface's MAC address using
the modified EUI-64 format, obtained from a DHCPv6 server,
automatically established randomly, or assigned manually.
Also,the interfaces which you have mentioned are some of the most-commonly used interfaces :-
'lo0', 'gif0', 'stf0', 'en0', 'en1', 'en2', 'bridge0', 'p2p0'
lo0---local network connection(0 for 1st connection of lan)
en0---ethernet connection(0,1,2 for 1st,2nd and 3rd connection on Ethernet)
bridge0---a bridged connection to this machine
p2p0---a peer-to-peer connection
don't know about gif,stf.Please note that there are logical connections/virtual connections,instead of limitation of physical connections(using NIC cards) too!
I discovered that there are man entries for gif and stf -- on OSX, at least. These are generic tunnel interface, and IPv6 to IPv4 tunnel interface ("Six To Four"), respectively.
Greetings,
For various reasons, my connection to the internet looks like this:
[DSL Modem in Bridge Mode] <-ethernet-> (eth0)[Linux system](eth1) <-ethernet-> [Wireless Router]
(Where the Linux system is running PPPoE, BIND, DHCP, etc.)
In order to diagnose a recent problem, I needed to connect to the web interface on the DSL modem. In order to do this I have to connect from a specific address range and as I am running PPPoE on eth0, I haven't assigned an address to it nor even turn it on. (The modem's web interface is at a fixed IP address regardless of what mode the modem is in and only answers to traffic from a fixed address range)
So anyway, to connect to the modem, and not finding anything helpful on the internet, I just tried assigning an IP address to eth0 after already starting PPPoE (like this: ifconfig eth0 192.168.1.5 netmask 255.255.255.0 up). I didn't really think that it would actually work. But it did. I.e. PPPoE and a static address assigned to eth0 at the same time and both worked correctly. Thus my question.. Should it? Is it safe to do this long-term or am I just lucky that it works long enough for me to get that which I need to done?
Thanks!
It's fine. PPPoE and IP are carried in Ethernet frames of different types.
What could be the maximum number of concurrent Clients (using different port number) that could communicate to a Server on the same port (Single socket) ? What are the factors that could influence this count ? I am looking for this information w.r.t telnet in Linux environment.
This depends in part on your operating system.
There is however no limit on a specific port. There is a limit on the number of concurrent connections however, typically limited by the number of file descriptors the kernel supports (eg 2048).
The thing to remember is that a TCP connection is unique and a connection is a pair of end points (local and remote IP address and port) so it doesn't matter if 1000 connections connect to the same port on a server because the connections are all still unique because the other end is different.
The other limit to be aware of is that a machine can only make about 64K outbound connections or the kernel limit on connections, whichever is lower. That's because port is an unsigned 16 bit number (0-65535) and each outbound connection uses one of those ports.
You can extend this by giving a machine additional IP addresses. Each IP address is another address space of 64K addresses.
More than you care about. Or rather.
More than your code can actually handle (for other reasons)
More than your clients will actually make
More than you can handle on a single box for performance reasons
More than you need on a single box because your load balancers will distribute them amongst several for availability reasons anyway
I can guarantee that it is more than all of those. There are scalability limitations with large numbers of sockets, which can be worked around (Google for the c10k problem). In practice it is possible to have more than 10,000 sockets usefully used by a single process under Linux. If you have multiple processes per server, you can increase that up again.
It is not necessary to use a single port, as your dedicated load-balancers will be able to round-robin several ports if needed.
If you are running a service for many 10s of 1000s of client processes, it is probably fairly important that it keeps working, therefore you will need several servers for redunancy ANYWAY. Therefore you won't have a problem deploying a few more servers.
I did a testing on Windows, doing multiple loopback connections onto a single socket. Windows refused to allocate anything after 16372 mark.