We are considering using DynamicIpRestriction to deny excessive access from a single IP in Azure App Service.
As a question, in the case of HTTP/2, it is doubtful whether it is effective to deny excessive access using this function.
In the case of HTTP/1.1, the number of simultaneous connections is restricted to 6 due to client side (browser) restrictions.
We are aware that we can limit the number of concurrent requests by maxConcurrentRequests, but
In the case of HTTP/2, parallel requests can be made on the same socket, and there is no upper limit.
Could you tell me if there is a best practice to reject excessive access with DynamicIpRestricion when using HTTP/2?
Could you tell me if there is a best practice to reject excessive access with DynamicIpRestricion when using HTTP/2?
As per this Azure Blog article, HTTP/2 is an advanced version of HTTP/1.1 where only few changes made in it such as Http/2 has only 1 TCP/IP Connection, fully multiplexed, binary model, single connection usage for parallelism, overhead reduction by using header compression process.
As per my research, I didn’t find the alternative way for protecting the app service from attacks (DDoS) using the “Dynamic IP Restriction” feature and the process for it same as here after enabling the HTTP/2 and working as expected.
Related
I have been studying web sockets recently and plan to use them in my application even though the app is not realtime. I am mostly doing this because I want to try it out and further down the line it might open more possibilites for the app's functionality. Also I am not bothered about having an API for mobile at the moment but think it would still be possible to have some kind of api over web sockets if I needed it in the future.
However for in-production apps are there any real reasons why somebody would consider implementing websockets if there is no real-time element?
Are there any benefits over HTTP requests other than the real timeness of it?
HTTP requests include the full HTTP headers. Depending on the cookie load, this may reach a couple of KB per request. WebSocket protocol headers are minimal compared to that. If you have a lot of requests and care about bandwidth then going with WebSocket makes sense.
Additionally a HTTP connection is (traditionally) negotiated for each request, which means you have overhead on each request compared to WebSocket, which has persistent connections. Connection establishment takes time (hence the advantage in real-time applications), but it also uses resources on the server. Again, depending on your app's communication patterns, using WebSocket may make sense.
I have an API on my server(node) that writes new data into my database.
To use the API the user is required to provide a token which acts as an identifier. So if someone floods my database or abuses the api, I can tell who it is.
But, what are some techniques I can use to prevent the ability to flood or hang my server all together? Notice that most request to the API are done by the server itself, so, in theory I might get dozens of requests a second from my own server's address.
I'd love to get some references to reading materials.
Thanks!
You could use this module: https://www.npmjs.com/package/ddos to put limits depending on the user.
However you will still be exposed to larger scale ddos attacks. These attacks cannot be stopped at the node.js level since they often target infrastructure. This is another can of worms however.
Try to configure limits on proxy or/and load balancer.
Alternatively, you can use rate-limiter-flexible package to limit number of requests by user per N seconds.
There is also black-and-white list, so you're able to whitelist your server's IP.
I am developing a node/express RESTful server with normal routes and HTTP request and response. Now I am asked to extend the server to provide near real time data to clients using socket.io or some such thing.
I feel like the real time update requires a client and serve connection management and client state management on the server and that on its own is orthogonal to the RESTful aspects of my server. For a client, to get continuous data feed from a RESTful server, the client has to "poll" for it.
Is my statement correct? If not, is there a pattern for providing the two features?
Correct, RESTful services are contrary to the stateful client/server connection needed to give real time I/O. Without knowing how much data your talking about RESTful would be very expensive in I/O.
No, there's no contradiction. Each HTTP request takes a round-trip to set-up, and many more if you support SSL (which you should), or do most forms of authentication. Socket.io lets you minimize those setups, by leaving the connection open and authenticated, ready for the next request. You could do something fancy with the 100 Continue header, following section 8.2.3 of the spec: http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html
But, just doing normal RESTful https pull requests over socket.io should show a meaningful decrease in latency, by minimizing the need to re-establish connections. Of course, implementing full server push support will lower latency even more.
The tradeoff is increased complexity (though socket.io can be implemented in Express in a few lines), and increased memory consumption by leaving sockets open.
I have a server that is running on aws - it's load balanced to some ec2 instances that run node.js servers. The security groups are set up so that only the LB can hit them on the HTTP port.
I was tailing some log files, and saw a bunch (50 or so at a time, seemingly somewhat periodically) of requests to /manager/html - AFAIK this looks like an attempt to expose a vulnerability in my app or gain access to a database manager of some sort.
My questions are:
Am I being targeted or are these random crawlers? This is on a service that is not even launched yet, so it's definitely obscure. There's been a bit of press about the service, so it's feasible that a person would be aware of our domain, but this subdomain has not been made public.
Are there common conventions for not allowing these types of requests to hit my instances? Preferably, I'd be able to configure some sort of frequency or blacklist in my LB, and never have these types of requests hit an instance. Not sure how to detect malicious vs normal traffic though.
Should I be running a local proxy on my ec2 instances to avoid this type of thing? Are there any existing node.js solutions that can just refuse the requests at the app level? Is that a bad idea?
Bonus: If I were to log the origin of these requests, would that information be useful? Should I try to go rogue and hunt down the origin and send some hurt their way? Should I beeswithmachineguns the originating IP if it's a single origin? (I realize this is silly, but may inspire some fun answers).
Right now these requests are not effecting me, they get 401s or 404s, and it has virtually no impact on other clients. But if this were to go up in scale, what are my options?
There are too many random automated requests are being made, even I host a nodejs server, they try to use cgi and phpmyadmin/wordpress configs. You can just use basic rate limiting techniques (redis-throttle)[https://npmjs.org/package/redis-throttle] for both your NodeJS server and ssh fail2ban to protect yourself from simple DoS attacks.
Automatic requests cannot do harm unless NodeJS or the libraries you have as well known flaws, so you should be always input & security checking all over your server. You should not be worried if you coded well. (Don't dump errors to users, sanitize input etc.)
You can log your 401 and 404s for a week, and filter the most common ones via your LB. Hunting down the IPs and sources will not help you if you are not a hollywood producer or fighting terrorists, as yoır problem is not so imporant and most importantly these requests are mostly from botnets.
We had faced similar issues in the past and we had taken some preventive measures to stop such attacks though it can't guarantee to stop them completely but it showed significant measures in the reduction of such attacks.
http://uksysadmin.wordpress.com/2011/03/21/protecting-ssh-against-brute-force-attacks/
http://www.prolexic.com/knowledge-center-white-paper-ddos-mitigation-incident-response-plan-playbook.html
https://serverfault.com/questions/340307/how-can-i-prevent-a-ddos-attack-on-amazon-ec2
Hope this helps.
Consider running a proxy cache like Varnish in front of your app servers. Use it's VCL to allow access to only the URI you define and reject everything else, allow GET but block PUT and POST, etc... Can also be used to filter http response headers you return. This would let you mask your node.js server as apache for example. Many tuts out on the net to implement this.
My node.js server is experiencing times when it becomes slow or unresponsive, even occasionally resulting in 503 gateway timeouts when attempting to connect to the server.
I am 99% sure (based upon tests that I have run) that this lag is coming specifically from the large number of outbound requests I am making with the node-oauth module to contact external APIs (Facebook, Twitter, and many others). Admittedly, the number of outbound requests being made is relatively large (in the order of 30 or so per minute). Even worse, this frequently means that the corresponding inbound requests to my server can take ~5-10 seconds to complete. However, I had a previous version of my API which I had written in PHP which was able to handle this amount of outbound requests without any problem at all. Actually, the CPU usage for the same number (or even fewer) requests with my Node.js API is about 5x that of my PHP API.
So, I'm trying to isolate where I can improve upon this, and most importantly to make sure that 503 timeouts do not occur. Here's some stuff I've read about or experimented with:
This article (by LinkedIn) recommends turning off socket pooling. However, when I contacted the author of the popular nodejs-request module, his response was that this was a very poor idea.
I have heard it said that setting "http.globalAgent.maxSockets" to a large number can help, and indeed it did seem to reduce bottlenecking for me
I could go on, but in short, I have been able to find very little definitive information about how to optimize performance so these outbound connections do not lag my inbound requests from clients.
Thanks in advance for any thoughts or contributions.
FWIW, I'm using express and mongoose as well, and my servers are hosted on the Amazon Cloud (2x M1.Large for the node servers, 2x load balancers, and 3x M1.Small MongoDB instances).
It sounds to me that the Agent is capping your requests to the default level of 5 per-host. Your tests show that cranking up the agent's maxSockets helped... you should do that.
You can prove this is the issue by firing up a packet sniffer, or adding more debugging code to your application, to show that this is the limiting factor.
http://engineering.linkedin.com/nodejs/blazing-fast-nodejs-10-performance-tips-linkedin-mobile
Disable the agent altogether.