Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
What happens exactly when I hit the link Google.com on my browser. Can anybody please explain it technically how all this work.
Thanks
Reference Taken from
Hyper Text Transfer Protocol (HTTP) is a protocol used for transferring web pages (like the one you're reading right now). A protocol is really nothing but a standard way of doing things. If you were to meet the President of the United States, or the king of a country, there would be specific procedures that you'd have to follow. You couldn't just walk up and say "hey dude". There would be a specific way to walk, to talk, a standard greeting, and a standard way to end the conversation. Protocols in the TCP/IP stack serve the same purpose.
The TCP/IP stack has four layers: Application, Transport, Internet, and Network. At each layer there are different protocols that are used to standardize the flow of information, and each one is a computer program (running on your computer) that's used to format the information into a packet as it's moving down the TCP/IP stack. A packet is a combination of the Application Layer data, the Transport Layer header (TCP or UDP), and the IP layer header (the Network Layer takes the packet and turns it into a frame).
The Application Layer
...consists of all applications that use the network to transfer data. It does not care about how the data gets between two points and it knows very little about the status of the network. Applications pass data to the next layer in the TCP/IP stack and then continue to perform other functions until a reply is received. The Application Layer uses host names (like www.dalantech.com) for addressing. Examples of application layer protocols: Hyper Text Transfer Protocol (HTTP -web browsing), Simple Mail Transfer Protocol (SMTP -electronic mail), Domain Name Services (DNS -resolving a host name to an IP address), to name just a few.
The main purpose of the Application Layer is to provide a common command language and syntax between applications that are running on different operating systems -kind of like an interpreter. The data that is sent by an application that uses the network is formatted to conform to one of several set standards. The receiving computer can understand the data that is being sent even if it is running a different operating system than the sender due to the standards that all network applications conform to.
The Transport Layer
...is responsible for assigning source and destination port numbers to applications. Port numbers are used by the Transport Layer for addressing and they range from 1 to 65,535. Port numbers from 0 to 1023 are called "well known ports". The numbers below 256 are reserved for public (standard) services that run at the Application Layer. Here are a few: 25 for SMTP, 53 for DNS (udp for domain resolution and tcp for zone transfers) , and 80 for HTTP. The port numbers from 256 to 1023 are assigned by the IANA to companies for the applications that they sell.
Port numbers from 1024 to 65,535 are used for client side applications -the web browser you are using to read this page, for example. Windows will only assign port numbers up to 5000 -more than enough port numbers for a Windows based PC. Each application has a unique port number assigned to it by the transport layer so that as data is received by the Transport Layer it knows which application to give the data to. An example is when you have more than one browser window running. Each window is a separate instance of the program that you use to surf the web, and each one has a different port number assigned to it so you can go to www.dalantech.com in one browser window and this site does not load into another browser window. Applications like FireFox that use tabbed windows simply have a unique port number assigned to each tab
The Internet Layer
...is the "glue" that holds networking together. It permits the sending, receiving, and routing of data.
The Network Layer
...consists of your Network Interface Card (NIC) and the cable connected to it. It is the physical medium that is used to transmit and receive data. The Network Layer uses Media Access Control (MAC) addresses, discussed earlier, for addressing. The MAC address is fixed at the time an interface was manufactured and cannot be changed. There are a few exceptions, like DSL routers that allow you to clone the MAC address of the NIC in your PC.
For more info:
Protocols
TCP/IP
Related
I investigated a lot about this topic but most of the guides just teach how to exchange data between devices on the same network and, regarding exchanging data between devices on different networks, no source was totally clear to me. I hope with this question somebody can give me (and other users) a good overview. If you have any guide or book about it I’d be super interested (for Java would also be fine).
First of all I’m interested in the difference between programs that
need to exchange data quickly (it may be an online videogame) versus
programs that need to exchange data accurately (it may be a message
app). My understanding is that the difference between the two is the
protocol used: in the first case is UDP (where no checks are done to
ensure there is no packets loss), in the second case is TCP (where
checks are done and data is exchanged more slowly). Is this correct?
So in an hypothetical Python script in the first case the socket
created would look like this:
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
While in the second case would look like this:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
My understanding is that to exchange data between different networks
you have to use port forwarding (very good explanation here),
concept that is clear to me. However, do you have any source that
suggests how to do it in Python? Also, is port forwarding
everything you need to do in order to exchange data between
different networks? Finally, I’m not sure I understand the role UPnP
plays in port forwarding. Based on this question it seems UPnP
is a way to automatically port forwarding. Is it correct? Can I use
miniupnpc library to do it automatically?
Finally, if I switch off and on my router, the private IP addresses
assigned to the devices connected to the network change (so the
private IP of my phone connected to my home WiFi could change, for
example, from 192.168.1.2 to 192.168.1.11). Does this represent a
problem in networking programming? If I set on the router a certain
port and the traffic that comes to that port is directed to a
certain private IP address and then this IP changes I suppose there is a
problem. Is this correct? If it is what is the solution?
Your understanding of use cases for UDP and TCP seem roughly
accurate. UDP ensure lower latency (not always) so for apps that
require lowest latency possible while also not caring about missed
packets, UDP is used. So if you think about video streaming, once a
packet is missed, it makes no sense to hold up every future packet
for that one old packet. This is because a small amount of data that
is missed doesn't really affect a user's watching experience. For
gaming, we want the newest data as soon as possible, so waiting for
old data also doesn't matter. But if you're implementing a protocol
or something that requires all data to be transmitted, TCP makes
sense since its absolutely vital that all information gets to the
receiver and in order.
There are a few methods to exchange data between two private networks. Port forwarding is certainly one method, and both machines on either network would have to have port forwarding. I don't know anything about automated port forwarding like you mention, but you can go into your router settings and set it up pretty easily. Another method of talking across networks is something like webRTC. Its a protocol that uses the STUN TURN and ICE protocols to perform something called NAT traversal. Short story shorter, it tricks your routers into letting your machines talk to each other(analogous to temporary port forwarding).
You're right that this could be an issue. However you should be able to setup static IP addresses in your router. So you can assign one machine to have a static IP address, setup port forwarding, and bam you have a permanent(hopefully) open connection.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 10 months ago.
Improve this question
I've searched extensively on this topic, yet I'm still confused about what exactly a 'port' is.
I know what it is 'conceptually', and I know that there are many different ports, and that some numbers, like 80 are reserved for certain uses, and I know that by typing in the 'netstat' commands in the CMD I can check if they are in use.
But I still don't understand what a port is. Does it 'physically' exist on my computer? Because all I see is one Ethernet connection line that I plug in, not some 6000 different ports that do different things.
And I don't really get how it's got to do with my internet browser. When I click on my 'Chrome.exe.' icon, does that 'open' the 80 -number port?
To reach a service on any networked computer you need to know 3 parameters. Some of them you provide explicitly and some of them are deduced by operating system implicitly. The three parameters are IP Address, Protocol, Port number. So any running service can be uniquely described by a combination of these three numbers.
Imagine, that we did not use port numbers. That would mean that on each computer we could have only one service using any of the protocols. So a web server could only serve HTTP protocol, mail server could only provide SMTP services and so on.
Usage of ports allows us to multiplex services on same computer. A port number is nothing more than a number, indicating what service should get a data packet. So we can have HTTP server, SMTP server and SSH server all running on a same computer and all using same protocol - namely TCP.
Also, these ports have nothing to do with a physical ports you see on your computer. Once again - they are just a number, allowing to distinguish which service (i.e. program) should receive which packet.
For example when I enter www.stackoverflow.com in my web-browsers address bar, my OS sends a HTTP request to {IP:104.16.34.249, Proto: TCP, Port: 80}. How does it know these numbers? IP address is resolved from the host name using the DNS protocol. Web browser is running HTTP protocol, which is using TCP. Port 80 is default port for HTTP. Behind the scenes, each packet will have destination address IP set to the above value and TCP payload will have destination port set to 80. On the SO computer, there will be a web-server running, which is listening for data received on port 80 and replies to my packet.
A port number is a 16-bit unsigned integer, thus ranging from 0 to 65535. For TCP, port number 0 is reserved and cannot be used, while for UDP, the source port is optional and a value of zero means no port.
A process associates its input or output channels via an Internet socket, which is a type of file descriptor, associated with a transport protocol, an IP address, and a port number. This is known as binding.
A socket is used by a process to send and receive data via the network. The operating system's networking software has the task of transmitting outgoing data from all application ports onto the network, and forwarding arriving network packets to processes by matching the packet's IP address and port number to a socket. For TCP, only one process may bind to a specific IP address and port combination.
Common application failures, sometimes called port conflicts, occur when multiple programs attempt to use the same port number on the same IP address with the same protocol.
https://en.wikipedia.org/wiki/Port_(computer_networking)
For short, ports are just logical numbers (not exists physically) related to services (or server software) whats are running above the operating system (ftp server, web server etc). When there is an incoming connection (or any traffic) addressed to a specific port, the OS can forward the traffic to the proper service for handling.
By default browsers are addressing the given site with port number 80 so
http://www.randomsite.com and http://www.randomsite.com:80 are identical.
If you request something from the internet then it has to send data to your request but the data is been sent in the form of data packets and the server needs some address to where it has to send the data packets and here comes the role of IP address.
An IP address identifies a machine in an IP network and is used to determine the destination of a data packet.
So the data will reach to your system or computer but still it doesn't know which application it has to send the data so here we use port number to identify the applications.
Port numbers identify a particular application or service on a system.
I recall reading an article about a proposed way to do this. If I recall correctly, the researchers successfully created a connection to a client on another network without port forwarding by sending HTTP packets to each other (Alice pretends that Bob is an HTTP web server while Bob pretends Alice is a web server).
I'm not sure if that makes sense, but does anyone know where I can find the article or does anyone have any other ideas how to connect two clients together without a central server or port forwarding?
Is it even possible?
Edit: I would know the IPs of both computers and port the program listens on.
It is possible. I see at least 2 parts to your question. (It is not going to be HTTP packet. It is a lot more complex than that.)
First off, I believe you might be talking about a concept called decentralized P2P network. The main idea behind a decentralized peer-to-peer network is the fact that nodes conjoint in such a network will not require central server or group of servers.
As you might already know, most common centralized peer-to-peer networks require such centralized system to exchange and maintain interconnectivity among nodes. The basic concept is such, a new node will connect to one of the main servers to retrieve information about other nodes on the network to maintain its connectivity and availability. The central system gets maintained through servers constantly synchronizing network state, relevant information, and central coordination among each other.
Decentralized network, on the other hand, does not have any structure or predetermined core. This peer-to-peer model is also called unstructured P2P networks. Any new node will copy or inherit original links from the "parent" node and will form its own list over time. There are several categories of decentralization of such unstructured networks.
Interestingly enough, the absence of central command and control system makes it solution of choice for modern malware botnets. A great example could be Storm botnet, which employed so-called Passive P2P Monitor (PPM). PPM was able to locate the infected hosts and build peer list regardless whether or not infected hosts are behind a firewall or NAT. Wikipedia's article Storm botnet is an interesting read. There is also great collaborative study called Towards Complete Node Enumeration in a Peer-to-Peer Botnet, which provides excellent conceptual analysis and techniques employed by Storm botnet network.
Second of all, you might be talking about UDP hole punching. This is a technique or algorithm used to maintain connectivity between 2 hosts behind NATed router/gateway using 3rd comment host by means of a third rendezvous server.
There is a great paper by Bryan Ford, Pyda Srisuresh, and Dan Kegel called Peer-to-Peer Communication Across Network Address Translators.
As answered, a peer-to-peer connection requires establishment of a connection between two (presumably) residential computers, which will necessitate punching holes through both of their firewalls. For a concrete example of hole punching, see pwnat: "The only tool to punch holes through firewalls/NATs without a third party". The process, put simply, goes like this:
The "server" (who doesn't know the client's IP address, but the client knows the server's) pings a very specific ICMP Echo Request packet to 1.2.3.4 every 30 seconds. The NAT, during translation, takes note of this packet in case it gets a response.
The client sends an ICMP Time Exceeded packet to the server, which is a type of packet that usually contains the packet that failed to deliver. The client, knowing in advance the exact packet that the server has been sending to 1.2.3.4, embeds that whole packet in the Data field.
The NAT recognizes the Echo Request packet and happily relays the whole Time Exceeded packet, source IP and all, to the correct user, i.e. the server. Voila, now the server knows the client's IP and port number.
Now that the server knows the address, it begins to continually send UDP packets to the client, despite the fact that the client's NAT did not expect them and will therefore ignore them all.
The client begins sending UDP packets to the server, which will be recognized by the server's NAT as a response to the server's packets and route them appropriately.
Now that the client is sending UDP packets to the server, the server's stream of UDP packets starts getting properly routed by the client's NAT.
And, in 6 easy steps, you have established a UDP connection between a client and a server penetrating two residential firewalls. Take that, ISP!
Every time I use a different router and different P2P program, I get the same problem - port forwarding. I then usually read random values of ports(TCP, UDP, whatever) and paste it into random places in my router setttings page and repeat this process until the damn thing starts working. As I am a bit tired of doing that i would like to understand the theory behind it a little bit, so that I can put the right things in right places immediately. Could anybody just explain it briefly to me in a few words? Apologies for lengthy description of the problem, but I didn't know how to describe the level of understanding that I am talking about in a more concise way.
Thanks.
Well, the router hides you from the outer world, so you can only make outgoing connections, for which router takes care of sending your requests to the outer world, receiving responses, and sending those back to you. No one can send a packet to you unless you have specifically asked for it—i.e. you can only receive responses.
In case on p2p, the ability to send packets to your machine is important if not vital. So what you do is ask router to forward (here! that's where the word comes from) all incoming packets to port X to your machine, port X.
Originally IP addresses were provided per device, now-a-days we tend to have 1 IP address per household (unless your doing something crazy), also called your external IP. Your external IP is your connection to the world via your router, but each computer within your network has it's own IP (called internal IP). Port forwarding allows the external world to establish communications with a specific computer.
A web server is a simple example, web services typically rely on port 80, what-if in your network you had 4 computers, 1 of which was your web server. How would the outside world know which PC to contact? Port Forwarding allows you to tell your router to direct internet traffic to that server.
How can an application be designed such that two peers can communicate directly with each other (assuming both know each other's IPs), but without outgoing connections? That's, no ports will be opened. Bitorrent for example does it, but multiplayer games (as far as I know) require port forwarding.
I'm not sure what you mean by No Outgoing Connections, I'm going to assume like everyone else you meant no Incoming Connections (they are behind a NAT/FW/etc).
The most common one mentioned so far is UPNP, which in this context is a protocol that allows you as a computer to talk to the Gateway and say forward me this port because I want someone on the outside to be able to talk to me. UPNP is also designed for other things, but this is the common thing for home networking (Actually it's one of many definitions).
There are also more common and slightly more reliable ways if you don't own the network. The most common is called STUN but if I recall correctly there are a few variants. Basically you use a third party server that allows incoming connections to try and coordinate a communication channel. Basically, what you do is send a UDP packet to you're peer, which will open up you're NAT for a response, but gets dropped on you're peer's NAT (since no forwarding rule exists yet). Through the connection to the intermediary, they are then told to do the same, which now opens up their NAT, and matches the existing rule in you're NAT. Now the communications can proceed. Their is a variant of this which will allow a TCP/IP connection as well by sending SYN and SYN-ACK messages with some coordination.
The Wikipedia articles I've linked to has links to the relevant rfc's for these protocols on precisely how they work. Essentially it comes down to, there isn't an easy answer, as this is a very network centric problem.
You need a "meeting point" in the network somewhere: the participants "meet" at a "gateway" of some sort and the said "gateway function" takes care of the forwarding.
At least that's one way of doing it: I won't try to comment on the details of Bittorrent... I am sure you can google for links.
UPNP dealt with this mostly in the recent years, but the need to open ports is because the application has been coded to listen on a specific port for a response.
Ports beneath 1024 are called "registered" because they've been assigned a port number because a company paid for it. This doesn't mean you couldn't use port 53 for a webserver or SSH, just that most will assume when they see it that they are dealing with DNS. Ports above 1024 are unregistered, so there's no association - your web browser, be it Internet Explorer/Firefox/etc, is using an unregistered port to send the request to the StackOverflow webserver(s) on port 80. You can use:
netstat -a
..on windows hosts to see what network connections are currently established, including the port involved.
UPNP can be used to negotiate with the router to open and forward a port to your application. Even bit-torrent needs at least one of the peers to have an open port to enable p2p connections. There is no need for both peers to have an open port however, since they both communicate with the same server (tracker) that lets them negotiate and determine who has an open port.
An alternative is an echo-server / relay-server somewhere on the internet that both peers trust, and have that relay all the traffic.
The "problem" with this solution is that the echo-server needs to have lots of bandwidth to accomodate all connected peers since it relays all the traffic rather than establish p2p connections.
Check out EchoWare: http://www.echogent.com/tech.htm