Consider the following.
let bluebird = require('bluebird');
let fs = bluebird.promisifyAll(require('fs'));
let express = require('express');
let https = require('https');
let app = express();
let eventualKeys = bluebird.all(['key', 'crt'].map(x => fs.readFileAsync("server." + x)));
let eventualCredentials = eventualKeys.then(([key, cert]) => {
return {key: key, cert: cert};
});
let eventualHttpsServer = eventualCredentials.then(credentials => https.createServer(credentials, app));
eventualHttpsServer.then(httpsServer => httpsServer.listen(4443));
If I make a request to the server using https, everything works fine.
However if I make a request using http, it hangs indefinitely.
Obviously as it is an https server, it can't be expected to handle http requests. But is there a cleaner way of handling this? For instance, nginx replies to attempts to query the https port using http with a much less confusing "The plain HTTP request was sent to HTTPS port" message.
Also is this behavior likely to cause a resource leak on the server side?
This hangs because the express server is not responding to the client (because it is not listening to the port). As the server is not handling the request it will not cause a resource leak on the server, and the client is waiting as long as it can to give the server a chance to respond.
You could set up another server listening on the http port (80) to respond with a failure or a redirect (301) if you want to handle this kind of response. However if you are using nginx or apache it is recommended that you handle any such refusals or redirects with them, as they are less resource intensive than starting up a new node http instance just to drop a connection.
Related
I'm using Socket.IO to run a WebSocket server locally in NodeJS using the following code:
import express = require('express');
const path = require('path');
import http = require('http');
import { Socket } from 'socket.io';
const app = express();
const server = http.createServer(app);
const socketio = require('socket.io')(server);
app.get('/', (req, res) => {
res.send("Node Server is running");
});
server.listen(3000, function () {
console.log('Example app listening on port 3000!');
});
socketio.on("connection", (socket: Socket) => {
console.log(`connect ${socket.id}`);
console.log(`connect ${socket.handshake.url}`);
socket.on("disconnect", () => {
console.log(`disconnect ${socket.id}`);
});
});
Using a tool like Firecamp, I try to establish a connection on ws://localhost:3000, but to no avail. I eventually use the Socket.IO client to connect from a simple web page by running let socket = io(). It seems the only reason this works is because that call connects to the host serving the page by default, as stated here. Running console.log(socket) and looking at the output, I eventually find that the URL inside the engine field is ws://localhost:3000/socket.io/?EIO=4&transport=websocket&sid=qerg3iHm3IKMOjdNAAAA.
My question is why is the URL so complicated rather than simply ws://localhost:3000? And is there no easier way to get the URL instead of having to access it through dev tools?
A socket.io server does not accept generic webSocket connections. It only accepts socket.io connections as socket.io goes through an extra layer of preparation stuff (over http) before establishing the actual webSocket connection. It then also adds a layer on top of the regular webSocket packet format to support some of its features (such as message names).
When using a socket client to connect to a socket.io server in the default configuration, socket.io first makes a few regular http requests to the socket.io server and with those http requests it sends a few parameters. In your URL:
ws://localhost:3000/socket.io/?EIO=4&transport=websocket&sid=qerg3iHm3IKMOjdNAAAA
The path:
/socket.io/
Is the path that the socket.io server is looking for requests on as destined for the socket.io server. Since this is a unique path and not generally used by other requests, this allows you to share an http server between socket.io and other http requests. In fact, this is a common way to deploy a socket.io server (hooking into an http server that you are already using for http requests).
In fact, the path /socket.io/socket.io.js is also served by the socket.io server and that will return the client-side socket.io.js file. So, clients often use this in their HTML files:
<script src="/socket.io/socket.io.js"></script>
as a means of getting the socket.io client code. Again you see the use of the path prefix /socket.io on all socket.io related URLs.
In your original URL, you can see parameters for:
EIO=4 // engine.io protocol version
transport=websocket // desired transport once both sides agree
sid=qerg3iHm3IKMOjdNAAAA // client identifier so the server knows which client this
// is before the actual webSocket connection is established
Once both sides agree that the connection looks OK, then the client will make a webSocket connection to the server. In cases where webSocket connections are blocked (by network equipment that doesn't support them or blocks them), then socket.io will use a form of http polling where it repeatedly "polls" the server asking for any more data and it will attempt to simulate a continuous connection. The client configuration can avoid this http polling and go straight to a webSocket connection if you want, but you would give up the fallback behavior in case continuous webSocket connections are blocked.
And is there no easier way to get the URL instead of having to access it through dev tools?
Not really. This URL is not something you have to know at all. The socket.io client will construct this URL for you. You just specify http://localhost:3000 as the URL you want to connect to and the socket.io client will add the other parameters to it.
I need help creating a proxy server using node js to use with firefox.
the end goal is to create a proxy server that will tunnel the traffic through another proxy server (HTTP/SOCKS) and return the response back to firefox. like this
I wanna keep the original response received from the proxy server and also wanna support https websites as well.
Here is the code I came up with.
var http = require('http');
var request = require("request");
http.createServer(function(req, res){
const resu = request(req.url, {
// I wanna Fetch the proxy From database and use it here
proxy: "<Proxy URL>"
})
req.pipe(resu);
resu.pipe(res);
}).listen(8080);
But it has 2 problems.
It does not support https requests.
It also does not supports SOCKS 4/5 proxies.
EDIT: I tried to create a proxy server using this module. https://github.com/http-party/node-http-proxy
but the problem is we cannot specify any external proxy server to send connections through.
I have found a really super simple solution to the problem. We can just forward all packets as it is to the proxy server. and still can handle the server logic with ease.
var net = require('net');
const server = net.createServer()
server.on('connection', function(socket){
var laddr = socket.remoteAddress;
console.log(laddr)
var to = net.createConnection({
host: "<Proxy IP>",
port: <Proxy Port>
});
socket.pipe(to);
to.pipe(socket);
});
server.listen(3000, "0.0.0.0");
You have to use some middleware like http-proxy module.
Documentation here: https://www.npmjs.com/package/http-proxy
Install it using npm install node-http-proxy
This might help too: How to create a simple http proxy in node.js?
It must be a simple issue, but my knowledge of streams is limited.
HTTP/1 80 to HTTP/2 h2c proxy
script(not working):
const net = require('net');
const http = require('http');
const http2 = require('http2');
const socketPath = `/tmp/socket.test.${Date.now()}`;
// front http 80 server.
http.createServer((req, res) => {
const socket = net.createConnection(socketPath)
req.pipe(socket).pipe(res);
}).listen(80);
// private http2 socket server.
http2.createServer(function(socket) {
socket.write(`Echo from http2 server\r\n`);
socket.pipe(socket);
}).listen(socketPath);
HTTP/2 h2c to HTTP/2 h2c proxy
cli command to start request:
curl --http2-prior-knowledge -v http://localhost:3333/ --output -
script(not working):
const net = require('net');
const http = require('http');
const http2 = require('http2');
const socketPath = `/tmp/socket.test.${Date.now()}`;
const port = 3333;
const private = http2.createServer({allowHTTP1: true});
private.on('stream', (stream, headers) => {
console.log('private http2 request');
stream.end('HTTP/2');
});
private.listen(socketPath, () => console.log('private http2 server is listening', socketPath));
const public = http2.createServer({allowHTTP1: true});
public.on('stream', (stream, headers) => {
console.log('public http2 request');
const socket = net.connect(socketPath);
stream.pipe(socket).pipe(stream);
});
public.listen(port, () => console.log('public http2 server is listening port', port));
Finally, http2(h2c) to http2(h2c) (with unix socket) proxy works!
const net = require('net');
const http2 = require('http2');
const socketPath = `/tmp/socket.test.${Date.now()}`;
const port = 4444;
const priv = http2.createServer({});
priv.on('stream', (stream, headers) => {
console.log('private http2 request');
stream.end('HTTP/2');
});
priv.listen(socketPath, () => console.log('private http2 server is listening', socketPath));
const pub = http2.createServer({});
pub.on('stream', (stream, headers) => {
const clientSession = http2.connect('http://0.0.0.0', {
createConnection: () => net.connect({path: socketPath})
});
const req = clientSession.request({
':path': `/`,
});
req.pipe(stream).pipe(req);
});
pub.listen(port, () => console.log('public http2 server is listening port', port));
I am not a node expert, and I may have completely misunderstood what you are trying to do here, but I am really struggling to make sense of the question...
If you are trying to have Node act as a HTTP/2 proxy (so a client can connect via h2c to node and it passes on those details to another HTTP/2 aware server), then the way you are going about it seems... weird to say the least.
A proxy can be a Level 4 proxy (e.g. a TCP proxy), where it creates two TCP separate connections (one from client to proxy, and one from proxy to destination server) and sends the connects of those TCP packets between them without really inspecting or interfering with them, other then the TCP headers.
Alternative a proxy can be a Level 7 proxy (e.g. a HTTP proxy), where it creates two separate HTTP connections (one from client to proxy, and one from proxy to destination server) and sends HTTP messages between them, mapping the HTTP headers and details between them and sometimes changing details or even adding more headers (e.g. X-FORWARDED-FOR).
You seem to be trying to create some kind of hybrid between these two distinct and incompatible modes of working! You are hoping to create a HTTP or HTTP/2 server and then open a TCP socket and pass those TCP messages between them and hope this works? While this might possibly work over a simple protocol like HTTP/1 it is never going to work over HTTP/2!
For your first example a HTTP/1 instance is entirely different to HTTP/2. So to set these two up and expect them to work is flawed from the start. If one of your friends only spoke German and the other friend only spoke Spanish and you passed all the German messages, verbatim, unfiltered, and still in German to the Spanish speaker then would you expect the Spanish speaker to magically be able to understand them? Of course not! So you cannot connect HTTP/1 and HTTP/2 at a socket level - they are completely different protocols. You need your proxy to act like a translator between them.
In your second example I'm even more confused. I guess you are trying to create two HTTP/2 servers, and have the client connect to one, and then proxy requests over to the other? And presumably you would add some logic in there at some stage so only certain requests made it through otherwise this would be pointless. Regardless this will almost certainly not work. HTTP/2 is a complex protocol, in many ways more akin to TCP. So each packet needs to be given a unique stream id, and many other settings need to be negotiated between the two end points. So to assume that one HTTP/2 message will seamlessly translate to an identical HTTP/2 message on another HTTP/2 connection is extremely naive! Back to the language analogy, copying German messages verbatim to another German speaker who is perhaps hard of hearing but sitting closer to you, might work initially but as soon as one end fails to keep up, speaks a slightly different dialect, or asks you to repeat something they missed, the whole show comes tumbling down.
I would suggest you either want to make this a Level 4 proxy (so ignore HTTP and HTTP/2 and just use sockets) or you want this to be a HTTP proxy (in which case ingest each HTTP message, read it, and send a similar HTTP message on the downstream connection). You cannot have both.
I would also question why and if you need to do this? HTTP/2 is not universally supported, and gets most of it's gains between client and edge server (proxy in this case), so why do you feel the need to speak HTTP/2 all the way through? See this question for more details on that: HTTP/2 behind reverse proxy
Hope that helps and apologies if I have completely misunderstood your question or your intention!
I have the following code:
express = require('express');
app = express();
http = require('http').createServer(app);
io = require('socket.io')(http);
app.use(express.static(__dirname + '/'));
http.listen(80);
I know it creates a server that clients can connect to and it works. But I don't know what exactly happens. Can you explain in detail?
Also, why things don't work when I forget about Express.js and just use this line:
io = require('socket.io').listen(80);
It appears to listen for connections. However, inside the browser when I go to http://localhost/, nothing happens. My guess is that I don't specify the directory for my app like that:
app.use(express.static(__dirname + '/'));
Is that why I need Express? To specify the directory?
At the client, I use:
socket = io('http://localhost/'); // this
socket = io(); // or this
None of them work with the single line code at the server-side.
Also, why do I need an HTTP server when Socket.IO uses the WebSocket protocol?
When your browser goes to http://localhost/, you need a web server that's going to respond back to the browser with a web page. That's what Express and the express.static() lines were doing. When you remove those, you do indeed have a server listening for webSocket connections on a specific path, but you don't have anything serving web pages. So, when the browser goes to http://localhost/, there's nothing responding back with a plain web page.
Also, why do I need an HTTP server when Socket.IO uses the WebSocket
protocol?
All socket.io connections start with an HTTP request. socket.io is based on the webSocket protocol and all webSocket connections are initiated with an HTTP request. So, to accept a socket.io connection, you need a web server that responds to an HTTP request and you then need a web server that is smart enough to recognize a request for a webSocket connection so it can "upgrade" the protocol from HTTP to webSocket.
For a well written overview of how a webSocket connection is established, see this overview on MDN.
The socket.io infrastructure then runs on top of that webSocket once it is connected.
I know it creates a server that clients can connect to and it works.
But I don't know what exactly happens. Can you explain in detail?
Here's a line-by-line explanation of your code:
express = require('express');
This loads the Express library.
app = express();
This creates an Express app object which can be used as a webServer request handler.
http = require('http').createServer(app);
This creates a web server and passes it the Express app object as the webServer request handler.
io = require('socket.io')(http);
This hooks socket.io into your web server as another request handler so it can see any incoming http requests that are actually the first stage of starting a webSocket/socket.io connection.
app.use(express.static(__dirname + '/'));
This tells Express that if any request is made for a web page that it should look in the __dirname for a file that matches the requested path. If found, it should return that path.
http.listen(80);
This starts the web server listening on port 80.
None of them work with the single line code at the server-side.
Both of those lines of code to create a socket.io connection will work when used properly. You don't say how this code is being run. If you're trying to run this code from a web page that the browser loads from http://localhost/, then I've already explained why that web page won't load if you don't start Express. If you're trying to run those lines of code from a web page loaded some other way, then you're probably having a same-origin security issue were the browser by default won't let you access a domain that is different than the one the web page came from.
You need the express http server to deliver the socket client to the browser.
Express server starts on port 80
Browser connects to express on port 80, the socket.io server component delivers socket client javascript to the browser (http://localhost:80/socket.io/socket.io.js)
Socket client (running in browser) can then connect to socket.io server
I have a chatting application using node.js server, Few days back we moved from http to https server, Where https server is working fine on web but not on native app, so for this we have created new http server with different port number.
Now I have two node socket.io server, one is http who is working on port 3000(for mobile client) and another one is https who is working on 3001 port(for web client), both the server has a same code, but now I am not able to make communication between native and web user.
So is it possible for node server to listen HTTP and HTTPS request simultaneously on a same port.
or
Is there any way through which I can send HTTp server request to HTTPS server ?
Below is the code for HTTP server
var app = require('http').createServer(handler),
io =require('socket.io').listen(app);
var querystring = require('querystring');
var http=require('http');
var fs = require('fs');
var webservice_host="xxxxxxxxxxxx.com";
var port = process.argv[2] || 3000;
var authKey="";
I'm not sure if you can get the http and https servers to listen simultaneously on the same port and accept only its protocol.
Is there any way through which I can send HTTp server request to HTTPS server ?
Yes, check out this question that explains how to create an HTTP -> HTTPS proxy. How to use node-http-proxy for HTTP to HTTPS routing?