I've been running a website on an Ubuntu EC2 instance that serves a useful tool that I use on the go (just displaying changing data). Mostly my goal here is to learn server security. The server runs a NodeJS server with Express, kept alive with ForeverJS.
Over the past two weeks I've seen some typical weak tries at "hacking", if you would call it that. For example, requests thrown at "/wp-admin" and "/administrator/manifests/libraries/joomla.xml". Recently though, I've been getting requests that look like this:
[0mGET http://robercid.es/ [32m200 [0m1.019 ms - 10669[0m
[0mGET http://api.ipify.org/ [32m200 [0m0.668 ms - 10669[0m
It doesn't look like they go through, but I'm curious as to how this is accomplished, and also what the "hacker" is trying to accomplish.
Also, as for security, I think I've covered everything (SSH Keys, non-20 SSH port, all ports except 80 closed), but is there anything specific to route handling that I should do security wise? Any non-valid page gets a 404, inputs are sanitized, DB is limited to local network.
Its probably not much to worry about. IP addresses in EC2 are part of a big pool and get used by different customers. Sometimes those IP addresses end up hardcoded into applications or are used to resolve cached DNS lookups that are not respecting ttl.
If it becomes a problem, like getting an ip address that was used for a high volume API endpoint, then you can simply stop/start your instance to get a new IP. Or, just request a new elastic ip and assign it to your instance.
Related
I've been reading through Jonas Czech's popular How To Prevent Scraping guide to help prevent harvesting of content on a Laravel site I'm responsible for. Several of the guide's suggestions involve filtering requests by identifiers like IP address and user agent, and then either blocking them or showing challenges like reCAPTCHAs. Presumably this sort of filtering would be most efficiently done at the server configuration level using Nginx config files or Apache's htaccess, but can also be done in application code.
The problem is that I've seen it stated in a few places elsewhere that filtering requests on my own server - as opposed to on the cloud by putting Cloudflare in front of the site and using their many equivalent offerings - can result in the server quickly becoming overloaded with requests (and therefore DOSed or DDOSed).
If my site is running on a cheap VPS like a Digital Ocean droplet, is implementing such a feature likely to make the server fall over? Or in other words, does filtering incoming requests typically add a high overhead to the server's compute?
I have build a node js code for an API server. Part of one feature is that when it starts, it should be able to know its own IP, despite the type of setup of the server where it is running.
The classic scenario is not that hard (I think). There are several options, like using the os module and find the ip or the external interface. I am sure there are other ways and some might be better, but this is the way I have been doing it so far. Feel free to add alternatives as informative as possible.
There is this case that I stumbled on. In one case, the web server was running on a google cloud instance. This instance has two IPs, one internal and one external. What I want is the external IP. However, when I use the method above, the actual external IP is not part of the object returned. The internal IP is declared as being considered as non-internal. Even when I run different commands from within the server command line, the only IP returned is the one that is actually internal and cannot be used to access the node server.
From what I understand, the instance itself is not aware of it's external IP. There might be a dns (I think) that redirects requests made to the external IP towards the correct instance.
While reading in the internet I read that problems getting the server's correct external IP might also rise when using load balancing or proxies.
The solution I thought about is to have the node js code make a request towards a service that I will build. This service will treat the node js servers as clients, and will return their external IPs. From experiments that I have done, the req object contains among others the information of the client's IP. So I should check first req.connection.remoteAddress and then the first element of req.headers['x-forwarded-for']. Ideally the server would make a request towards itself, but
I know there are external API like https://api.ipify.org?format=json that do just that - return the actual IP. But I would very much like to have the node js servers independent of services I cannot control.
However, I really am hoping that there are better solutions out there than making a request from the server which returns the server IP.
However, I really am hoping that there are better solutions out there
than making a request from the server which returns the server IP.
It is not really possible, you always rely on some kind of external observer / external request.
While reading in the internet I read that problems getting the
server's correct external IP might also rise when using load balancing
or proxies.
This is because not in all scenarios your own device is able to be self-aware of its external ip. There might be sitting behind some network, that means external address assigned to devices that forwards the WAN to it. (example : router) so when you try to obtain external ip from the devices interface itself, you end up obtaining an ip but inside the scope of the routers LAN and not the one used for external requests .
So if you really want to
Have a method to use in all scenarios
Not rely on 3rd party services
Only Solution :
Build your own ip echo service (you maintain and can use for future projects).
i dont want to do something illegal with it(e.g. vote continuously, in fact, somebody is doing it), but i only feel curious about it. For i have learned TCP/IP, and i found there are many software such like "IP changer",using which you can submit a website with different IP. WOW it is really magic! so i analysed some possible mechanism about it. But every possible way was denied by me.
i thought that they might connect and disconnect the internet continuously. because each time they connect the Internet, the ISP will dispatch a new IP address, and the hacker can make use of the new IP to submit the website, and disconnected after submitting successfully, and then connect for the next time...But it is impossible to some extent, for if do like this, every submitting will last a long time, and it doesn't work in some areas.
Modify TCP/IP data packets.For some time i did think it might be all right. but then i denied it. Assuming that i would submit a website, and i changed the IP address of the data packet which i will submit to the web site. it seems that everything is OK, but the web server will send message to the fake IP, so i wont get any information from the website. but in some circumstances where we only needn't reply it should work. Right? netfilter and iptables in linux may realize it, but i am not sure because i dont know the tools very well.
Using proxy server. i also think it is impossible to some extent.is there any method to get lots of free proxy servers? and most free proxy servers is very unstabitily, for there is a possible circumstance that you cannot use the proxy server in one day.Of course, paid proxy server may be permanent. but with these money you can do something better.
IMO the three methods all have disadvantages. and the realization may be none of them. Can anybody tell me the real mechanism of the technique?
Use lots of proxy servers. That will do the trick and since they can be harvested quite easily that's not very hard. Proxy's can be installed on hacked websites for example.
The added question:
Using proxy server. i also think it is impossible to some extent.is there any method to get lots of free proxy servers?
By simply hacking lots of webservers, totally automated, this is possible. For example searching for bad Joomla installs could allow you to install software at each webserver. Also normal computers can be used off course. Like a botnet.
and most free proxy servers is very unstabitily, for there is a possible circumstance that you cannot use the proxy server in one day. Of course, paid proxy server may be permanent. but with these money you can do something better.
Stability is off course important but in this case not really actually. You just send out lots and lots and lots of requests. Don't care which one succeeds and which one doesn't. It doesn't matter for your target.
1. ISP reconnect
This will not work for some (most?) ISPs which will reassign the same IP on a reconnect (as my provider does). Even if it works, you are likely to get the same IP address after some reconnects.
2. IP spoofing
That's the term describing your second method. You change the src-address of the outgoing IP packet. There are two problems with that:
Most ISP's routers don't allow it. They detect that the src address can't come from inside their network, so they simply drop it.
If you have a machine that is allowed to do this (maybe a dedicated server), you can only fake exactly one IP frame. This allows you to, e.g. spoof a DNS request but as you said, you will never get the response. Especially you cannot establish a connection within a stateful protocol like TCP, because this requires a bidirectional handshake. So you can't, e.g., fake a HTTP request using this (even if you don't need the answer)
Proxying
This is the only method that works. You have several options here:
Use open proxy servers (can be found using a search engine, although some will identify themselves as proxies and provide the original IP in the X-Forwarded-For HTTP header, which makes them basically useless for this use case)
Use hacked servers/desktop machines as proxies (maybe from a botnet)
Use free networks like JAP or TOR (the latter of which is probably your best bet, because you can change the exit nodes using some trickery)
If you are going to do something illegal, you might as well go all the way in. There ARE people who run "botnets" which are basically just armies of a few hundred to a few thousand indfected computers (that's what most viruses do). The people who run these armies, actually can charge people a certain amount of money for their "slaves" to visit a website for you (and rate/vote whatever) so you get a few hundred or a few thousand more ratings...
I can't exactly tell where or how much these services cost, since I haven't done it myself, but I know for sure that people over at "H#ckf0rums.net" will do it for you.
It has recently come to my attention that setting up multiple A records for a hostname can be used not only for round-robin load-balancing but also for automatic failover.
So I tried testing it:
I loaded a page from our domain
Noted which of our servers had served the page
Turned off the web server on that host
Reloaded the page
And indeed the browser automatically tried a different server to load the page. This worked in Opera, Safari, IE, and Firefox. Only Chrome failed to try a different server.
But after leaving that server offline for a few minutes and looking at the access logs, I found that the number of requests to the other servers had not significantly increased. With 1 out of 3 servers offline, I had expected accesses to each of the remaining 2 servers to roughly increase by 50%, but instead I only saw 7-10%. That can only mean DNS-based failover does not work for the majority of browsers/visitors, which directly contradicts what I had just tested.
Does anyone have an idea what is up with DNS-based web browser failover? What possible reason could there be why automatic failover works for me but not the majority of our visitors?
What's happening is that the browsers are not doing automatic DNS failover.
If you have multiple A records on a domain then when your nameserver requests the IP for the domain you typed into your browser, it'll request one from the SOA. It could be any of those A records. Then it passes it along.
Some nameservers are 'smart' enough to request a new A record if the one it gets doesn't work and some aren't. So if you set multiple A records then you will have set up a pseudo redundancy failover, but only for those people with 'smart' nameservers. The rest get a toss of the dice on which IP they get and if it works then good, and if not then it will fail to load as it did for you in Chrome.
If you want to specifically test this then you can use your hosts file C:\Windows\system32\drivers\etc\hosts in Windows and /etc/hosts in Linux to specify what IP you want to go with what domain to see if you get a true failover - as what you'll run into in practicality is that DNS servers across the net will cache your domain name resolution based on its TTL. So if/when you get a real failure, that IP will still need to be resolve and be otherwise farmed out to another nameserver.
Another possible explanation is that, for most public websites, the bulk of traffic comes from bots not from browsers. Depending on the bot it is possible that they aren't quite as smart as the browsers when it comes to handling multiple A records for a domain.
Also, some bots use keep-alives to keep the TCP connections open & make multiple HTTP requests over the same connection. Given that the DNS lookup is only done when a connection is made, they will continue to make requests to the old IP address at least as long as the connection is kept open.
If the above explanation has any weight you should be able to see it in your logs by examining the user agent strings.
How can I make that a site automagically show a nice "Currently Offline" page when the server is down (I mean, the full server is down and the request can't reach IIS)
Changing the DNS manually is not an option.
Edit: I'm looking to some kind of DNS trick to redirect to other server in case the main server is down. I can make permanent changes to the DNS, but not manually as the server goes down.
I have used the uptime services at DNSMadeEasy to great success. In effect, they set the DNS TTL to a very low number (5 minutes). They take care of pinging your server.
In the event of outage, DNS queries get directed to the secondary IP. An excellent option for a "warm spare" in small shops with limited DNS requirements. I've used them for 3 years with not a single minute of downtime.
EDIT:
This allows for geographically redundant failover, which the NLB solution proposed does not address. If the network connection is down, both servers in a standard NLB configuration will be unreachable.
Some server needs to dish out the "currently offline page", so if your server is completely down, there will have to be some other server serving the file(s), so either you can set up a cluster of servers (even if just 2) and while the first one is down, the 2nd is configured only to return the "currently offline page". Once the 1st server is back up, you can take down the 2nd safetly (as server 1 will take all the load).
You probably need a second server with 100% uptime and then add some kind of failover load balancer. to it, and if the main server is online redirect to that and if it isn't redirect to itself showing a page saying server is down
I believe that if the server is down, there is nothing you can do.
The request will send up a 404 network error because when the web address is resolved to an IP, the IP that is being requested does not exist (because the server is down). If you can't change the DNS entry, then the client browser will continue to hit xxx.xxx.xxx.xxx and will never get a response.
If the server is up, but the website is down, you have options.
EDIT
Your edit mentions that you can make a permanent change the IP. But you would still need a two server setup in order to achieve what you are talking about. You can direct the DNS to a load balancer which would be able to direct the request to a server that is currently active. However, this still requires 100% uptime for the server that the DNS points to.
No matter what, if the server that the DNS is pointing to (which you must control, in order to redirect the traffic) is down, then all requests will receive a 404 network error.
EDIT Thanks to brian for pointing out my 404 error error.
Seriously, DNS is not the right answer to server load-balancing or fail-over. Too many systems (including stub clients and ISP recursive resolve) will cache records for much longer than the specified TTL.
If both servers are on the same network, use routing protocols to achieve fail-over by having both servers present the same IP address to the network, but where the fail-over server only takes over if it detects that the (supposedly) live server is offline.
If the servers are Unix, this is easily done by running Quagga on each server, and then using OSPF as the local routing protocol. I've personally used this for warm standby servers where the redundant system was actually in another data center, albeit one that was connected via a direct link to the main data center.
Certain DNS providers, such as AWS's Route 53, have a health-check option, which can be used to re-route to a static page. AWS has a how-to guide on setting this up.
I'm thinking if the site is load balanced the load balancer itself would detect that the web servers it's trying to redirect clients to are down, therefore it would send the user to a backup server with a message dictating technical problems.
Other than that.....
The only thing I can think is to control the calling page. Obviously that won't work in all circumstances... but if you know that most of your hits to this server will come from a particular source, then you could add a java script test to the source, and redirect to a "server down" page that is generated on a different server.
But if you are trying to handle all hits, from all sources (some of which you can't control), then I think you are out of luck. As other folks are saying - when a server is down, the browser gets a 404 error when it attempts a connection.
... perhaps there would be a way at a point in between to detect 404 errors being returned by servers and replacing them with a "server is down" web page. You'd need something like an HTML firewall or some other intermediate network gear between the server and the web client.