So, I have a express API that checks a list of blocked IPs, if the request IP matches one IP on the IPs blocked list, I want to sent a message to the end-user.
Now I'm implementing tests, I have been using supertest to make calls to my API endpoints
Let's say I want to make two calls the endpoint "/user":
one call with IP 127.0.0.1 (default)
one call with IP 127.0.1.1 (API should block this IP)
I found some old stackoverflow issues, did everything as shown in the answers, but none answers changed the supertest IP, therefore my API didn't return a message saying the IP was blocked
So, is there a way to automatically test IP bans?
Related
I have an AppEngine Node.js application running in a standard environment, and I'm having some trouble with cron verification. The docs say that you can verify that the IP address comes from 0.1.0.2. In the request logs I can see that the request IP is 0.1.0.2; however, in my fastify request object, request.ip is 127.0.0.1. Anyone know what could be happening here?
I was thinking that maybe there's some sidecar like nginx that is accepting the requests, but in that case, I would expect to x-forwarded-for to be defined, but it's not.
As per documentation, X-Forwarded-For value is the list of IP addresses through which the client request has been routed. You can see that the first IP (0.1.0.1) as expected, is the IP of the client that created the request. The subsequent IPs are the proxy servers that handled the request before it reached the application server (e.g.X-Forwarded-For: clientIp, proxy1Ip, proxy2Ip). Therefore, in this case the VM is seeing the remote IP from a Google Cloud internal Load Balancer address, but we do use X-Forwarded-For for things like this.
One quick solution is to only check for the X-Appengine-Cron header rather than also checking the IP address. The X-Appengine-Cron header is set internally by Google App Engine. If your request handler finds this header it can trust that the request is a cron request.
For more information, you can refer to these Stackoverflow Link1 and Link2 which may help you.
I am trying to make some tests on my backend protection logic, and I want to see if someone can change his ip manually and send a get request to my website that has to be assessable
only from certain ips neither from outside those range of ips,
let ipRange = ['75.21.323.32', '52.57.66.799','98.89.88.77']
const getIP = require('ipware')().get_ip;
app.use(function(req, res, next) {
const ip= getIP(req);
if(ipRange.includes(ip)){
// Allow acess
} else {
res.send({erorr : 'no ip acess'})
}
});
any idea how to make those tests using postman or similar tool that could allow you to manipulate you get request ip address if this could be even possible ?
It is not possible to send a GET request from a spoofed IP address because just establishing the TCP connection requires round-trip communication and if the attacker has spoofed the source IP address then the return data from setting up the TCP connection will not go back to the attacker, it will go to the spoofed IP address which will fail to complete the TCP connection. Note, UDP has more vulnerabilities in this regard. Thus, the only way the attacker could make that work is if they had control of your local router and thus could intercept the outgoing traffic that was going back to the spoofed IP address.
Even if they could successfully establish a TCP connection, the result of the GET request would again go back to the spoofed IP address, not the attacker's address.
In order to be compatible with local proxy servers (which are used in most hosting environments), your ipware library is also looking a variety of headers on the request that the proxy could use to tell your endpoint what the original IP address was. For this to be secure, you have to do several things:
Make sure that your instance of ipware is configured to ONLY use the one and only one header that your proxy is using to communicate the original IP address to you.
Make sure that your proxy is configured to only pass the IP address that it saw, not append to a list of IP addresses that may have already been present on the incoming request.
If you can't do #2, then you must make sure that you have configured ipware to use the most recent IP address only (the IP address that your proxy was connected to from) as this header can consist of a list of IP addresses (some of which could be spoofed - the ones that came from the request before it arrived at your proxy). Depending upon how your proxy works with appending of IP addresses, you may have to tell ipware whether to use the first or last IP address in the list (known as left-most or right-most). Proxies are not all the same in this implementation so you have to test how yours works. See here for examples of multiple IP addresses in the proxy header.
Make sure your proxy is not permitting spoofing of whichever header it is using to communicate the IP address.
I am using the ip library of npm.
I have two config files, one for React and one for Node, for the same application.
const ip = require('ip');
console.log(ip.address());
This returns different ip addresses for the React config file(inside the src folder-127.0.0.1) and Node server file(outside the src folder - IPv4 address).
The issue is that I am pretty sure that I ran the exact same code earlier and it gave me the same ip addresses for both as then I was able to access my webpages. I need the same ip to make requests to my node backend, I cannot afford it in production. Are there other definite methods of doing this?
You get different ip results because 2 method call ip.address() are using different network interfaces.
To make ip.address() return identical result, you can pass network interface name as the first parameter, such as en0:
const ip = require('ip');
console.log(ip.address('en0'));
p.s. To get all current networks interface names, os.networkInterfaces() can be used.
Update: OP try to get IP address in React code, in browser side. This is mission impossible. Otherwise, it would bring huge security problem.
Update 2: OP don't want to store endpoint IP address in frontend code for security reason, neither want to retrieve the IP address first (network overhead issue). In this case, you can make a proxy in server. All frontend know is interacting with current server, the data exchange is delivered by server as:
Browser <--> Server <--> Various endpoint IP
The steps are:
The server (that host the frontend code) get request from browser
Server check which endpoint would be used for that client
Server send the request to specific endpoint
Server get response from endpoint
Server return the response in above step to browser
I have 1 single machine with an IP 1.2.3.4. This machine has 2 web servers and an ftp server:
Web Server 1 listens to port 82; the domain for it: ws1.example.com
Web Server 2 listens to port 83; the domain for it: ws2.example.com
FTP Server listens to port 21; the domain for it: ftp.example.com
This is what the DNS mapping looks like:
ws1.example.com CNAME example.com
ws2.example.com CNAME example.com
ftp.example.com CNAME example.com
example.com A 1.2.3.4
Case 1: I make a request at the browser URL ws1.example.com:82 and the DNS redirects me to example.com but with the Host header: ws1.example.com.
Case 2: I make a request at the browser URL ws2.example.com:83 and the DNS redirects me to example.com but with the Host header: ws2.example.com.
In both the cases:
the request ultimately reaches the same physical machine
when the request arrives:
In Case 1, the request arrives at this machine and the request is attended to by the application that is listening on port 82 i.e. Web Server 1.
In Case 2, the request arrives at this machine and the request is attended to by the application that is listening on port 83 i.e. Web Server 2.
The Host header, as I understand, is used to inform the receiving host to identify which server (from the multiple servers that this IP has been hosting) is this request meant for and accordingly directs the request to the appropriate application.
My question is:
In this example, what is the purpose of the Host header as the same physical machine with the same IP has multiple applications listening at their corresponding ports. Once the request reaches this machine, the appropriate port will anyway pick up and the other applications will ignore the request as the port does not match the request. So, what purpose is the Host header serving here when apprpriate ports are anyway doing their job, right and well?
Can I infer that
CNAMES
Multiple Web Servers behind a single IP
subsequent resolution of a particular user request to the appropriate Web Server with the Host header
make sense only when you are using something like a Reverse Proxy e.g. 1 machine interfaces with the client and redirects user requests to the appropriate web server on separate machines all listening on the same port e.g. 80, each in the network behind the reverse proxy in which case you have ws1.example.com and ws2.exmple.com both be redirected to the reverse proxy example.com and this reverse proxy now forwards it to the appropriate host based on the Host header?
No DNS redirections
First an important terminology fix:
There are no "redirects" in the DNS. In your case, the DNS is just use to map a name to an IP. Sometimes, because of CNAME, a name is mapped to another name which is then mapped to an IP. It does not matter if there are intermediate steps like that, at the end a name maps to an IP (or there is a DNS resolution failure)
This also means that if the URL has a specific port, then that is not changed, the final IP will be queried over the port mentioned in the URL.
Redirections are an HTTP level feature: when querying a webserver for https://www.mygreatsite.example/foo it will reply with an HTTP return code of 301, 302, 303, 307 or 308 and giving you (the HTTP client, aka the browser) the new URL to go to.
HTTP virtual hosting
In the good old days, IP addresses were plenty. If you were hosting both www.site1.example and www.site2.example on the same physical box you could attach one different IP address to each.
Hence, in that specific case, in a way, the HTTP host header is useless, the mere fact of connecting either to 192.0.2.37 or 192.0.2.42 already lets you know which site you want.
In fact in HTTP/0.9 there was no host header, as there were no headers at all.
But then, with mass virtual hosting coming into play, and IPv4 addresses becoming scarce, you could not anymore attach one single IP address per site, since it was also a waste.
So you had, through the DNS, either directly or indirectly (CNAME records), both websites resolving to the same IP.
Hence when the HTTP client connected to the server, the server by default has no way to know which website do you want. That is why the HTTP host header filled by the client lets the server know which website you want to access, irrespective to its IP address, that was resolved earlier through the DNS.
By default HTTP uses port 80, so it is often not visible in the URLs.
Of course if you forced your clients to use http://www.site1.example:4569 on one side and http://www.anothersite2.com:9873 on another side, then you are right the host header would not be really needed.
Except that the plan falls down for many reasons:
Port numbers are not an infinite space either and many of them are already used typically for other things; so even if you extend this scheme at one point you could not attach new websites to the same IP
But more important than the previous technical point, for humans this will be a nightmare and many people will use forget the port number and then not coming to the appropriate website.
Hence typically it is not done like that, if you want to expose some given service over HTTP but in a non default port you typically install a reverse proxy in front of it. Or you do an HTTP redirection from http://www.coolpublicname.example/ to http://www.complicatedinternalname.example:9713, but then the client sees this naked truth.
HTTPS virtual hosting
In passing note that HTTPS added a level of complexity because the HTTPS webserver needs to send its certificate to the client, but since each website can have a different certificate it needs to know which website the client wants to use, which it could learn through the host HTTP header but then comes after the TLS handshake is finished, so in the early stage of the server sending a certificate this is not available yet.
So at the earliest times of HTTPS we were forced again to do IP-based virtual hosting and not name-based virtual hosting like it was possible in pure HTTP thanks to the host header.
The solution was found with a TLS extension, the Server Name Indication (SNI), something that the client sends early to the server and gives the website name, so that the server can send the appropriate certificate, and hence we are back in business in the name-based case where you can theoretically have an infinite number of names resolving to the same IP for them to be served by one given webserver.
I have tried to send a DNS packet to get an IP of some web-site.
In some cases, like google, the IP was right and when i typed it in the url line it sent me to google.
But in other cases (for example : stackoverflow.com) its gave me an IP that didin't linked to the web-site.
To be sure that my packet is right, i tried to do Nslookap in the command line, and the result was the same.
So i cant find the right IP adress of a web-site.
There is the message that appear when I'm trying to enter stakoverflow
Fastly error: unknown domain: 151.101.65.69.
Please check that this domain has been added to a service.
You (generally speaking) can not open the website just by entering the IP address in your browser's address bar because web servers (and possibly many other network components that are between you and the web server) often do not host only one web site on that IP address so they rely on exact domain name typed in address bar to serve the right content.
I think, it's caused by yours internet restriction. Try to contact your ISP (your internet provider) about this problem. He will probably know more about cause of this problem.
Short answer: you need a host header.
Long answer: Since HTTP/1.1 introduced in 1997 (and then updated in 1999 and in 2014), the request needs a host header. That allows the web server to route a request to a corresponding server configuration, a virtual server in Apache speak. Some servers don't have this configured and is allowing requests to any host to be served from the same web server configuration.
HTTP/1.1 also allowed multi-tenant proxies, as Fastly, to exist in the Internet. Fastly is a CDN - content delivery network - that allows to cache websites content on closer to users and deliver it locally (faster than from a cloud or a colo, thus the name).
When you're not specifying the domain for the request, it looks like your client (or a library) is using the IP address as the host header. That's why the response from Fastly talks about domain: unknown domain: 151.101.65.69.
While Fastly do support service pinning to a dedicated IP address, which would have worked for your request - it doesn't look like stackoverflow is using the feature as they might not need it.