I try to consume an REST API with the node-http2 package.
The API supports HTTP/2.
I GET a file with two related resources:
GET /article/223
GET /comment/123
GET /comment/322
Shouldn't the two /comment requests be faster than the /article request because of http/2 multiplexed connections?
I already have a custom agent with keep-alive
new http2.Agent({ keepAlive: true });
Сan i check when a new connection gets opened or how many connections are open at a specific moment of time?
I think keep-alive is not needed:
"Under HTTP/2 keep alives are the default and HTTP/2 doesn't use the connection header" - HTTP2 and NGINX - when would I use a keepalive directive?
Monitoring connections:
"Other than using Wireshark 2.0, one of the easiest ways to actually see the individual binary frames is by using the net-internals tab of Google Chrome (type chrome://net-internals/#http2 into the address bar). The data can be hard to understand for large Web pages. Rebecca Murphey helpfully wrote a useful tool for displaying it visually in the command line."
https://blog.newrelic.com/2016/02/09/http2-best-practices-web-performance/
Related
I've been testing HTTP2 Multiplexing and HTTP2/Server Push in Node.js locally and inspected it in the waterfall network graph in Chrome Dev Tools.
While using my own Node.js Server Push with res.stream.pushStream I got an "Initiator: Push (index)" in DevTools like this:
The change was noticeable as below:
I did some more research and figured I should use a reverse proxy to do the job (for example Nginx instead) and connect to my Node.js via HTTP1.1 instead as the upstream and serve HTTP2 from my reverse proxy.
After setting up the nginx.conf with http2_push_preload on; i sent some headers like these from my Node.js backend:
res.setHeader("Link","</picture.jpg>; as=image; rel=preload");
To my suprise I didn't see the "Push / (index)" indicator but "Other" like on the screenshot below and the asset listed in the link header seemed to show up on the waterfall graph quicker with a slightly lower TTFB time compared to the rest of assets.
I've been also looking for a solution to serve HTTP/2 as the reverse proxy and to download the assets via HTTP/2 from the service directly without the TLS (HTC) but it seems there's nothing like this.
Getting back to my question: How should I go about testing the HTTP/2 Server Push? Is the "Initiator: Other" a misinterpretation from the devtools? It seems to be working but it doesn't report as Server Push.
Also, are there any projects/solutions that would let for a connection to a backend upstream via HTTP2 directly?
I'm pretty sure the latter image shows slower access because of the overhead of using a reverse proxy instead of connecting to the server directly and it's magnified by the HTTP1.1 usage on the target server.
Thanks to #Barry I figured out it's actually Chrome itself as the "Other" initiator and it indeed speeds up the process a bit by using the Resource Hints Link header and not HTTP/2 Server Push itself.
The problem was actually a bug I can't seem to reproduce and it works well after a restart of the OS. Afterall Nginx was the culprit of not using the headers correctly, which kicked in when parsed in Chrome.
I'm trying to understand what is the difference between HTTPS and http/2?
If I'm going to build a Node.js/express app, what should I use?
Can I use HTTPS with http/2?
Maybe if I use HTTPS, I don't need http/2 because it's the same, or HTTPS use http/2 under the hood?
I'm confused.
Someone is linked to me "difference between HTTP 1.1 and HTTP 2.0 [closed]", but I understand the difference between HTTP and HTTP2. I'm asking about HTTPS and HTTP/2
HTTP - A protocol used by clients (e.g. web browsers) to request resources from servers (e.g. web servers).
HTTPS - A way of encrypting HTTP. It basically wraps HTTP messages up in an encrypted format using SSL/TLS. The web is moving towards HTTPS more and more and web browsers are starting to put more and more warnings when a website is served over unencrypted HTTP. Unless you have a very good reason not to, use HTTPS on any websites you create now.
Digging into HTTP more we have:
HTTP/1.1 - this was the prevalent format of HTTP until recently. It is a text-based protocol and has some inefficiencies in it - especially when requesting lots of resources like a typical web page. HTTP/1.1 messages can be unencrypted (where web site addresses start http://) or encrypted with HTTPS (where web site address start with https://). The client uses the start of the URL to decide which protocol to use, usually defaulting to http:// if not provided.
HTTP/2 - a new version of HTTP released in 2015 which addresses some of the performance issues by moving away from a text based protocol to a binary protocol where each byte is clearly defined. This is easier to parse for clients and servers, leaves less room for errors and also allows multiplexing. HTTP/2, like HTTP/1.1, is available over unencrypted (http://) and encrypted (https://) channels but web browsers only support it over HTTPS, where it is decided whether to use HTTP/1.1 or HTTP/2 as part of the HTTPS negotiation at the start of the connection.
HTTP/2 is used by about a third of all websites at the time of writing (up to 50% of websites as of Jan 2020, and 67% of website requests). However not all clients support HTTP/2 so you should support HTTP/1.1 over HTTPS and HTTP/2 over HTTPS where possible (I believe node automatically does this for you when using the http module). I do not believe HTTP/1.1 will be retired any time soon. You should also consider supporting HTTP/1.1 over unencrypted HTTP and then redirect to HTTPS version (which will then use HTTP/1.1 or HTTP/2 as appropriate). A web server like Apache or Nginx in front of Node makes this easy.
HTTP/3 - the next version of HTTP, currently under development. It is expected to be finalised in 2020 though it will likely be late 2020 or even 2021 before you see this widely available in web servers and languages like node. It will be built on top of a UDP-based transport called QUIC (rather than the TCP-based protocol that HTTP/1.1 and HTTP/2 are based on top of). It will include part of HTTPS in the protocol so HTTP/3 will only be available over HTTPS.
In short you should use HTTP/1.1 over HTTPS, should consider HTTP/2 as well if easy to implement (not always possible as not quite ubiquitous yet - but getting there) and in future you might be using HTTP/3.
I suggest you get a firm understanding of all of these technologies (except maybe HTTP/3 just yet) if you want to do web development. It will stand you in good stead.
Wondering how to setup a persistent connection between client and server in Node.js.
The persistent connection should be able to send requests back and forth from both sides. It seems like it needs to have a mechanism for delineating each request, such as \r\n if requests are JSON, but not sure what best-practices are here if it's already been done before. Wondering how web-sockets handle this.
Mozilla says Keep-Alive shouldn't be used in production, so wondering if that still holds with HTTP2.
With EventSource you can receive server events, but wondering if there is a way to add client events to send to the server.
In the end, I would like to have a simple setup for two-way communication like WebSockets, but am not sure of best practices in HTTP2 and if it should just be done with WebSockets. I would prefer to try to do this without websockets.
https://daniel.haxx.se/blog/2016/06/15/no-websockets-over-http2/
With all due respect, I believe you're under the mistaken assumption that the HTTP/2 connection is page specific.
In actuality, HTTP/2 connection are often browser wide connections. It's not possible for an HTTP/2 push to know which page/tab will use the data (possibly more than one).
Also, see the discussion here that includes a myriad number of reasons as to why HTTP/2 couldn't (or shouldn't) be used as a Websocket alternative.
On the other hand, if HTTP/2 is "promised" and Websockets are unavailable, than polling might be an option.
Polling with HTTP/2 will be significantly more resource friendly than polling with HTTP/1.1, even though it would still be far more expensive than Websocket push (or poll) due to the extra header and authentication data (it will also decrease security, but that's probably something nobody really cares about all that much).
P.S.
Websockets over HTTP/2
This is a non-issue unless HTTP/1.1 is retired (in fact, IMHO, this is intentional and good).
It's true that HTTP/2 connections can't be "upgraded" (changed) to Websocket connections, nor can they tunnel Websocket data, but that means absolutely nothing.
New Websocket connections use the existing HTTP/1.1 upgrade handshake and that's it.
Any HTTP/2 server that supports Websockets will support the handshake.
Keep-Alive
Mozilla warns against the non-standard Keep-Alive header. However, the Connection: keep-alive standard for HTTP/1.1 (which is also the default for HTTP/1.1 clients) is definitely recommended.
SSE (Server Sent Events)
SSE pre-date Websockets and they didn't get much traction in the community, while Websockets were adopted with a vengeance.
wondering if there is a way to add client events to send to the server
Simple - send an HTTP request. The reply can be asynchronously received with SSE.
I like Websockets better. As far as server-side goes, I find them easier to manage and code.
I am checking node-http-proxy and nodejs-proxy to build a DIY reverse proxy/load balancer in Node.js. After coding a small version, I setup 2 WEBrick servers for the same Rails app so I could load balance (round robin) between them. However each HTTP request is sent to one or another server which is very inefficient since the loading process of CSS and Javascript files from the home page is performed with more than 25 GET requests.
I tried to play a bit with socket events but I didn't get anywhere because by default it uses keep-alive connections (possibly this is why nginx just support http/1.0).
Ok, so I am wondering how can my proxy send a block of HTTP requests (for instance loading a webpage entirely, etc) to only one server so I could send the next block to another server.
You need to consider stickiness or session persistence. This will ensure future connections after the first connection inbound will get 'stuck' to the chosen server for the duration of the session or until the persistence connection times out.
I set up a Node.js HTTP server. It listens to path '/' and returns an empty HTML template on a get request.
This template includes Require.js client script, which creates Socket.IO connection with a server.
Then all communication between client and server is provided by Web Sockets.
On connection, server requires authentication; if there are authentication cookies then client sends them to server for validation, if no cookies then client renders login view and waits for user input, etc.
So far everything works, after validating credentials I create a SID for user and use it to manage his access rights. Then I render main view and application starts.
Questions:
Is there a need to use HTTPS instead of HTTP since I'm only using HTTP for sending script to the client? (Note: I'm planning to use Local Storage instead of cookies)
Are the any downfalls in using pure Web Sockets without HTTP?
If it works, why nobody's using that?
Is there a need to use HTTPS instead of HTTP since I'm only using HTTP
for sending script to the client? (Note: I'm planning to use Local
Storage instead of cookies)
No, HTTP/HTTPS is required for handshake for websockets. Choice of HTTP or HTTPS is from security point of view. If you want to use it for simply sending script then there is no harm. If you want to implement user login / authentication in your pages then HTTPS should be used.
Are the any downfalls in using pure Web Sockets without HTTP?
Web sockets and HTTP are very different. If you use pure Web Sockets you will miss out on HTTP. HTTP is the preferred choice for cross-platform web services. It is good for document traversal/retrieval, but it is one way. Web socket provides full-duplex communications channels over a single TCP connection and allows us to get rid of the workarounds and hacks like Ajax, Reverse Ajax, Comet etc. Important thing to note is that both can coexist. So aim for web sockets without leaving out HTTP.
If it works, why nobody's using that?
We live in the age of HTTP, web sockets are relatively new. In the long term, web sockets will gain popularity and take up larger share of web services. Many browsers until recently did not support web sockets properly. See here, IE 10 is the latest and only version in IE to support web sockets. nginx, a wildly popular server did not support web sockets until Feb-March 2013. It will take time for web sockets to become mainstream but it will.
Your question is pretty similar to this one
Why use AJAX when WebSockets is available?
At the end of the day they were both created for different things although you can use web sockets for most, if not everything which can be done in normal HTTP requests.
I'd recommend using HTTPS as you do seem to be sending authentication data over websockets (which will also use the SSL, no?) but then it depends on your definition of 'need'.
Downfalls - Lack of support for older browsers
It's not used this this in many other situations because it's not necessary and it's still 'relatively new'.