My Azure app hosts multiple ZeroMQ Sockets which bind to several tcp ports.
It worked fine when I developed it locally, but they weren't accessible once uploaded to Azure.
Unfortunately, after adding the ports to the Azure ServiceDefinition (to allow access once uploaded to azure) every time I am starting the app locally, it complains about the ports being already in use. I guess it has to do with the (debug/local) load balancer mirroring the azure behavior.
Did I do something wrong or is this expected behavior? If the latter is true, how does one handle this kind of situation? I guess I could use different ports for the sockets and specify them as private ports in the endpoints but that feels more like a workaround.
Thanks & Regards
The endpoints you add (in your case tcp) are exposed externally with the port number you specify. You can forcibly map these endpoints to specific ports, or you can let them be assigned dynamically, which requires you to then ask the RoleEnvironment for the assigned internal-use port.
If, for example, you created an Input endpoint called "ZeroMQ," you'd discover the port to use with something like this, whether the ports were forcibly mapped or you simply let them get dynamically mapped:
var zeromqPort = RoleEnvironment.CurrentRoleInstance.InstanceEndpoints["ZeroMQ"].IPEndpoint.Port;
Try to use the ports the environment reports you should use. I think they are different from the outside ports when using the emulator. The ports can be retrieved from the ServiceEnvironment.
Are you running more than one instance of the role? In the compute emulator, the internal endpoints for different role instances will end up being the same port on different IP addresses. If you try to just open the port without listening on a specific IP address, you'll probably end up with a conflict between multiple instances. (E.g., they're all trying to just open port 5555, instead of one opening 127.0.0.2:5555 and one openining 127.0.0.3:5555.)
Related
I am writing an app that is to be running on a Windows PC. I need to create a server socket listening on 127.0.0.1, and another client socket which is to connect with this server socket.
Since the data exchange between the two sockets are within the same machine and there is no client connecting from outside of the machine, what port to use is insignificant, as long as the two sockets use the same port number.
So, how do I decide which port number to use? Shall it be a hard-coded port number such as 49500? What if another unrelated app on this machine happen to use this port number? Or shall I get the list of all used ports and programmatically pick an unused port?
Just want to know what is the best approach. Thanks.
ports within 0 to 1023 are generally controlled and you should assign your socket with higher port numbers, although in that range ports within 1024 and 49151 can be registered for others to be informed about that and not use them.
if you want to avoid conflicts you can see registered ports on your machine and assign a port number to your socket which is empty but ports higher than that (49152 to 65535) are completely free and are not even registered.
generally, it is not common to worry about that. for example, two major applications like VMware and apache web server operate on the same port number (443), and if you want to use VMware workstation and Xampp (which works with apache) you have to simply make one of them listen on another port and its not a big deal. so in my opinion the best practice is to let your users change this via a config file or something similar.
for further information, you can search google. for instance this link might be useful:
https://www.sciencedirect.com/topics/computer-science/registered-port#:~:text=Well%2Dknown%20ports%E2%80%94Ports%20in,1023%20are%20assigned%20and%20controlled.&text=Registered%20ports%E2%80%94Ports%20in%20the,be%20registered%20to%20prevent%20duplication.&text=Dynamic%20ports%E2%80%94Ports%20in%20the,assigned%2C%20controlled%2C%20or%20registered.
I would like to create a secure internal route between two applications within the same space/organization. It should never be possible to reach the Node.js application from the outside. My Java application connects via HTTP to the Node application (running on express).
I have now tried to setup the desired configuration by creating a route called example-route.apps.internal and assigned it to the Node application. As a next step, I've opened the port (I've tried 443, 80, 8080) in the network configuration of the Java application (with the destination being the Node app). I restaged both applications.
Then, I opened a Java connection to the link http://example-route.apps.internal/test123. I've also tried to use https. The result was the same. Java refused to conncet to this URL.
Now, the following questions:
How can I properly set up this communication? Should I resolve this internal DNS somehow? Which port is the correct one if I just use the port of the env variable? How should I read this port from the other application?
How secure is the communication, if HTTP is used instead of HTTPS? (I assume HTTPS is not possible internally). Is it as safe as an HTTPS connection from the outside? Which devices are between, how far out does the connection go?
Thank you!
I think you're almost there.
Then, I opened a Java connection to the link http://example-route.apps.internal/test123. I've also tried to use https. The result was the same. Java refused to conncet to this URL.
You should use http://example-route.apps.internal:8080/test123. Your app is set to listen on $PORT, which is always 8080 in current versions of CF.
Normally you don't need to worry about this because your traffic goes in through Gorouter which translates for you (maps external port 80 -> internal 8080). With internal routes, traffic is direct so there is no transformation. That's why you need to use port 8080 in your URL.
Alternatively, you could use a service discovery mechanism like Eureka or Consul, but it's not a requirement. In this case, the service would know it's listening on 8080 and register that in the registry.
As far as HTTPS, that's tricky. Your app is only listening on 80/HTTP. You would have to change it to listen on 443/HTTPS, but then you need certs and different server configuration. It's technically possible, but it's a whole can of worms.
In some newer versions, Envoy is present and accepts HTTPS traffic into a container, can make HTTPS easier but it's still not a slam dunk (at the time of writing, at least). I expect this will get better in the future.
Should I resolve this internal DNS somehow?
Internal DNS helps with locating your other apps, not the port. Otherwise you'd need to manage IP addresses, which change often, and that would require something like Eureka or Consul.
Which port is the correct one if I just use the port of the env variable?
See above.
How should I read this port from the other application?
It's always 8080 at the moment, and has been for multiple years. It's unlikely to change, so you could probably hard code or set it in a config file safely.
How secure is the communication, if HTTP is used instead of HTTPS? (I assume HTTPS is not possible internally).
Is it as safe as an HTTPS connection from the outside? Which devices are between, how far out does the connection go?
Traffic would not be accessible externally as it wouldn't leave the Cell in some cases or worst case it goes between two Cells, but traffic would be visible internally since it's not encrypted. That means you need to have more trust on your CF provider, who would have access to internal traffic.
If it were HTTPS, only someone with the key would be able to decrypt it. You would still have to trust your provider though as they could likely get the key & use it to decrypt traffic. It would just be more work for them than if traffic is unencrypted.
Hope that helps!
Let me be more specific about my question with an example: Let's say that I have a slew of little servers that all start up on different ports using TCPv4. These ports are going to be destination ports, of course. Let's further assume that these little servers don't just start up at boot time like a typical server, but rather they churn dynamically based on demand. They start up when needed, and may shut themselves down for a while, and then later start up again.
Now let's say that on this same computer, we also have lots of client processes making requests to server processes on other computers via TCPv4. When a client makes such a request, it is assigned a source port by the OS.
Let's say for the sake of this example that a client processes makes a web request to a RESTful server running on a different computer. Let's also say that the source port assigned by the OS to this request is port 7777.
For this example let's also say that while the above request is still occurring, one of our little servers wants to start up, and it wants to start up on destination port 7777.
My question is will this cause a conflict? I.e., will the server get an error because port 7777 is already in use? Or will everything be fine because these two different kinds of ports live in different address spaces that cannot conflict with each other?
One reason I'm worried about the potential for conflict here is that I've seen web pages that say that "ephemeral source port selection" is typically done in a port number range that begins at a relatively high number. Here is such a web page:
https://www.cymru.com/jtk/misc/ephemeralports.html
A natural assumption for why source ports would begin at high numbers, rather than just starting at 1, is to avoid conflict with the destination ports used by server processes. Though I haven't yet seen anything that explicitly comes out and says that this is the case.
P.S. There is, of course, a potential distinction between what the TCPv4 protocol spec has to say on this issue, and what OSes actually do. E.g., perhaps the protocol is agnostic, but OSes tend to only use a single address space? Or perhaps different OSes treat the issue differently?
Personally, I'm most interested at the moment in what Linux would do.
The TCP specification says that connections are identified by the tuple:
{local addr, local port, remote addr, remote port}
Based on this, there theoretically shouldn't be a conflict between a local port used in an existing connection and trying to bind that same port for a server to listen on, since the listening socket doesn't have a remote address/port (these are represented as wildcards in the spec).
However, most TCP implementations, including the Unix sockets API, are more strict than this. If the local port is already in use in any existing socket, you won't be able to bind it, you'll get the error EADDRINUSE. A special exception is made if the existing sockets are all in TIME_WAIT state and the new socket has the SO_REUSEADDR socket option; this is used to allow a server to restart while the sockets left over from a previous process are still waiting to time out.
For this reason, the port range is generally divided into ranges with different uses. When a socket doesn't bind a local port (either because it just called connect() without calling bind(), or by specifying IPPORT_ANY as the port in bind()), the port is chosen from the ephemeral range, which is usually very high numbered ports. Servers, on the other hand, are expected to bind to low-numbered ports. If network applications follow this convention, you should not run into conflicts.
I'm wondering how browsers internally works.
Now, connecting to different Web sites using 'Tabs' within the same browser can be handled in one of two ways:
1 - Using threads
2 - Using Different Source Port Numbers for each open tab
I know there might not be a single answer for this question, and it might differ from one browser to another, however all responses are appreciated.
Thanks
Thanks for everyone.I really appreciate that.
My question relates to the Source Port at the client side. I'm asking if the browser uses different source ports for each tab it opens, or the same source port for the entire process 'I mean window that includes different tabs', or the same source port for the entire windows?
Or, do web browsers use threads?
Threads and ports are separate and mostly unrelated concepts.
Threads are what the local computer processor does to handle computations, such as drawing to the screen or waiting for Internet traffic. There's probably a separate thread (and more) for those operations in each tab.
Ports are what the traffic itself is identified by (in TCP and UDP). In order to communicate your browser would open a local port (usually something big like ~5000, and that doesn't matter as long as its unique) and connect to the server on usually port 80 (the one the server is listening on). If your computer didn't know the remote port it couldn't connect, but its standard to use 80 for HTTP, for example.
Browsers open tabs in separate threads (and new ones even in separate processes for security and reliability reasons), and use separate ports on the client side. So yes, the answer is both threads and ports. They always use the same remote port unless you physically specify otherwise (for example, connecting to a website using https:// instead of http:// uses a separate port because that's how that protocol was made). You can specify a port to use in modern browsers with :# after the name, too. (example: http://www.google.com:81/, however that will fail because that's not what port they listen on!)
A quick check using netstat (or sockstat on BSD machines) reveals that different source port numbers are used for different connections. In that regard, you are right.
Firefox uses at least one thread for each tab. Each thread could open multiple connections for different data (for example, loading images from a media server and content from the web server). Each connection should have its own source port.
Depending on the browser it uses different threads or different processes for each tab. The local ports used probably don't have much to do with different tabs.
Copied from Cisco CCNA course:
For instance, assume a host is initiating a web page request from a web server. When the host initiates the web page request, the source port number is dynamically generated by the host to uniquely identify the conversation. Each request generated by a host will use a different dynamically created source port number. This process allows multiple conversations to occur simultaneously.
In the request, the destination port number is what identifies the type of service being requested of the destination web server.. For example, when a client specifies port 80 in the destination port, the server that receives the message knows that web services are being requested.
no! it usually uses port 80 by default unless specified. for example www.someweb.com:8080.
Tabs within browser i am assuming ran on different threads
some guys use a firewall on their laptops which not only blocks their own local incoming ports (except those they need for their application) but also blocks messages unless they are issued from a distinct port number. We're talking about a local UDP server which is listening to UDP broadcasts.
The problem is that the remote client uses a random port, say 1024, which is blocked unless they tell the firewall to accept it.
What puzzles me is that as far as I know from using sockets in my programs is that usually the client gets its port number from the OS, whereas only when you have a server, you bind your socket to a distinct port, right?
In my literature and in tutorials and code snippets in the web I haven't found any clue that clients should be using fixed port numbers at all.
So how is this in reality? Am I probably missing a point?
Are there client applications around using fixed ports?
Is is actually useful to block remote ports with a firewall?
And if yes, what level of added security does this give to you?
Thanks for enlightenment in beforehand...
Although the default API's allow the network stack to select a local port for client connections, clients may specify a fixed port for various reasons.
Some specifications (FTP) specify a fixed port for clients. Most servers don't care if clients get this correct.
Some clients use a fixed pool of ports for egress from a LAN to the Internet. This allows firewall rules to more completely lock down outbound traffic.
Source ports are sometimes uses as a weak type of "security through obscurity".
You always get a random address and/or port when not explicitly having bound to one before sending.
Daemons are usually bound to a fixed port, so that:
you can actually contact them without having to try all possible ports or utilize a secondary resolver (remember the SUNRPC portmapping crap?)
and because a TCP socket is not allowed to listen() if it has not bound to a port, IIRC.
Are there client applications around using fixed ports?
Some can be configured so, like BIND9.
useful to block remote ports with a firewall?
No, because your peer may choose any port of his. Block him and you'll lose a customer, so to speak.