My internet access is restricted to ports 80 and 443. These ports will only be given data if it comes with appropriate Content-Type so you cannot download any flv files for instance. Is there a solution out there or a mixture of solutions to route the whole traffic via a php script? So for instance there could be something running in the system which would capture the whole traffic, covert outgoing traffic to requests to a php script, then the script would act as my computer, obtain response and covert the response to text. Since text is allowed then the program running on my machine would convert text back to incoming traffic. A bit like proxy. Is it possible?
Related
I want to build a DSR load balancer for an application I am writing. I wont go into the application because it is irrelevant for this discussion. My goal is to create a simple load balancer that does direct server response for TCP packets. The idea is to receive all packets at the load balancer, then using something like round robin, select a server from a list of available servers which are defined in some config file. The next step would be to alter the packer received and change the destination ip to be equal to the chosen backend server. Finally, the packet will be sent over to the backend server using normal system calls for sending packets. Theoretically the backend server should receive the packet, and send one back to the original requester, and then the requester can communicate directly with the backend server rather than going through the load balancer.
I am concerned that this design will not work as I expect it to. The main question is, what happens when computer A send a packet to IP Y, but receives a packet back in the same TCP stream from a computer at IP X? Will it continue to send packets to IP Y? Or will it switch over to IP X?
So it turns out this is possible, but only halfway so, and I will explain what I mean by this. I have three processes, one which is netcat, used to initiate an tcp request, a second process, the dsr-lb, which receives packets on a certain port, changes the destination ip to a backend server(passed in via command line arg), and forwards it off using raw sockets, and a third process which is a basic echo server. I got this working on a local setup. The local setup consists of netcat running on my desktop, and dsr-lb and echo servers running on two different linux VMs on the desktop as well. The path of the packets was like this:
nc -> dsr-lb -> echo -> nc
When I said it only half works, what I meant was that outgoing traffic has to always go through the dsr-lb, but returning traffic can go directly to the client. The client does not send further traffic directly to the backend server, but still goes through the dsr-lb. This makes sense since the client opened a socket to the dsr-lb ip, and internally still remembers this ip, regardless of where the packet came from.
The comment saying "if its from a different IP, it's not the same stream. tcp is connection-based" is incorrect. I read through the linux source code, specifically the receive tcp packet portion, and it turns out that linux uses source ip, source port, destination ip, and destination port to calculate a hash which is uses to find the socket that should receive the traffic. However, if no such socket matches the hash, it tries again using only the destination ip and destination port and that is how this "magic" works. I have no idea if this would work on a windows machine though.
One caveat to this answer is that I also spun up two remote VMs and tried the same experiment, and it did not work. I am guessing it worked while all the machines were on the same switch, but there might be a little more work to do to get it to work if it goes through different routers. I am still trying to figure this out, but from using tcpdump to analyze the traffic, for some reason the dsr-lb is forwarding to the wrong port on the echo server. I am not sure if something is corrupted, or if the checksum is wrong after changing the destination ip and some router along the way is dropping it or changing it somehow(I suspect this might be the case) but hopefully I can get it working over an actual network.
The theory should still hold though. The IP layer is basically a packet forwarding layer and routers should not care about the contents of the packets, they should just forward packets based on their routing tables, so changing the destination of the packet while leaving the source the same should result in the source receiving any answer. The fact that the linux kernel ultimately resolves packets to sockets just using destination ip and port means the only real roadblock to this working does not really exist.
Also, if anyone is wondering why bother doing this, it may be useful for a loadbalancer in front of websocket servers. Its not as great as a direct connection from client to websocket server, but it is better than a loadbalancer that handles both requests and responses, which makes it more scalable, and more able to run on less resources.
I'm working on a project where some clients (embedded linux systems) needs to connect to a main server using so far at least two protocols: HTTPS and SSH. One of the requirement is that only one flow is allowed from every client to the server, so I've to found a way to make the two services works on the same port.
One solution that I was thinking about is to use the iptables markers: on the client side mark the packets for SSH with 0x1, the packets for HTTPS with 0x2 and then on the server side, based on the marker, redirect the packets to the right service listening on the local interface. Is it an acceptable solution? Are the markers kept by the network routers or is only something working locally on the same machine for iptables?
And anyway, if you can advice about a different solution, of course it's welcome!
More for other users finding this question in the future:
https://github.com/yrutschle/sslh has what you might need. I haven't used it (yet) but planning on it.
From the Github site:
sslh -- A ssl/ssh multiplexer
sslh accepts connections on specified ports, and forwards them further based on tests performed on the first data packet sent by the remote client.
Probes for HTTP, SSL, SSH, OpenVPN, tinc, XMPP are implemented, and any other protocol that can be tested using a regular expression, can be recognised. A typical use case is to allow serving several services on port 443 (e.g. to connect to SSH from inside a corporate firewall, which almost never block port 443) while still serving HTTPS on that port.
Hence sslh acts as a protocol demultiplexer, or a switchboard. Its name comes from its original function to serve SSH and HTTPS on the same port.
I am a bit new to server side scipts. I am fairly capable with javascript so I decided to take a look at node.js as opposed to php or python ect. Correct me if I am wrong but it seems that when I code my webserver I may freely choose the port number I listen to. Is there any significance to this port number or may I choose any random number I wish? Will anyone be able to send a request to my server regardless of the number I choose?
Thanks in advance!
If you want to run node.js directly without any supporting web server or reverse proxy (no nginx, varnish, apache, etc), you need to listen on port 80 for HTTP and (optionally) 443 for HTTPS if you want normal URLs to work. Otherwise users will need to type the port number in the URL like http://example.com:3000 which is unheard of for public-facing sites.
However, you almost certain DO want to use a separate web server as I describe in detail here, in which case any port over 1024 is fine.
If you have root access you can choose any port that's not already bound to a process.
If you do not have root access you can choose any port above 1024 that is not already bound to a process.
Port 80 is usually the one you want to use if you're serving up HTTP, however, you can access an HTTP server on any port via the URL port syntax. For example, a server on port 3000. http://yourdomain.com:3000
If you're running on Linux and you do not want to run your Node process as root, you can redirect port 80 traffic to another port.
I want to build a service on my Linux VPS which listens to a certain UDP port and does something with the (text)message which is captured. This processing consists of appending the message to a locally stored txt-file and send it as http, with a post variable to another server.
I've looked into Nginx but as far is can see this server can only be bound to receive http packets. Although it is asynchronous.
What is the best way to achieve this listening-service on linux? And which has the capabilities to do the above mentioned processing?
Is for instance node.js a possibilty? It looks great
For simplicity, you can use xinetd, and for the app you can use any scripting language, which will read the packet from the stdin and save it to the file.
I'm trying to create and run a Node.JS proxy in a machine that has a pool of IPv6 addresses. I want the proxy to randomly choose one of these addresses for each request (making it difficult for the websites to track record of users' requests).
With wget I can achieve this by using the attribute --bind-address as following:
wget --bind-address OUTGOING_IP http://www.example.com/
Is there any way to achieve the same behavior using Node.JS?
If you want to make outbound HTTP requests from different IPs, have a look for "localAddress" option under "http.request":
http://nodejs.org/docs/latest/api/http.html#http_http_request_options_callback
If you want to start a TCP server to listen on a particular IP bound to your host, you would probably want to specify it when you create the server [i.e. server.listen(PORT, HOST)]:
http://nodejs.org/docs/latest/api/net.html#net_class_net_server
-- ab1