How is it possible to get browser IP with Watir? I'm using proxy and I want to verify if it's working correctly.
Perhaps there is some other way if proxy is working?
Here's my current code:
profile = Selenium::WebDriver::Firefox::Profile.new
profile.proxy = Selenium::WebDriver::Proxy.new :http => 'my.proxy.com', :ssl => 'my.proxy.com'
browser = Watir::Browser.new :firefox, :profile => profile
browser.goto 'http://someurl.com'
The browser will open the url, although the proxy is not working.
Thanks for help
This is really not a pretty way of getting around this but you could use the following to get the ip.
browser.goto("http://www.whatsmyip.org/")
ip = browser.span(:id, "ip").text
As i said it is really not an ideal way but i am not sure if watir has access to the ip you are on.
Note that if you use the site above, please respect the author's wishes and do not generate a high volume of requests against the site. If you look at the source there, you will find this comment:
Please DO NOT program a bot to use this site to grab your IPs. It
kills my server and thats not nice. Just get some cheap or free web
hosting and make your own IP-only page to power your bot. Then you
won't even have to parse any html, just load the IP directly - better
for everyone!!
As good citizens of the net we need to respect that. I doubt he would be upset by a few hits a day, but if your scripts are doing this a lot, make your own reflector page to report your IP back to you.
You don't need Watir to go through a proxy to get the IP. You can use net/http, which has less overhead and is easier. BTW, I used whatsmyip.com here but I do not believe that it's so reliable. there are others including http://whatismyipaddress.com, http://show-ip.net, http://ipchicken.com, http://www.ipaddresslocation.org, http://www.myipaddress.com/show-my-ip-address/, http://www.lawrencegoetz.com/programs/ipinfo/, http://www.find-ip-address.org.
require 'net/http'
uri = URI("http://automation.whatismyip.com/n09230945.asp")
Net::HTTP::Proxy(proxyhost, proxyport, proxyuser, proxypassword).start(uri.host) do |http|
req = Net::HTTP::Get.new(uri.path=='' ? '/' : uri.path)
#ip = http.request(req).body.scan /\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}/
end
Related
I have a Meteor App that I'm whitelisting to just a specific IP.
So something like
handleRoute(req,res) => {
if (req.HEADERS[x-forwarder-for]) === WHITELISTED_IP) {
next(res,req)
} else {
res.writeHead(404);
res.end();
}
}
This works and you get a 404 page.
However, this can lead an attacker to know that the site at least exists. I'd like to obfuscate that further if possible.
Like, if you go to some obscure site that doesn't exist you'll probably see some splash page from your ISP. I'm guessing this is something the ISP put in place when DNS lookup fails.
I'm wondering if it's possible to still show that somehow. This would be using standard Node HTTP Request req,res API.
Thanks!
No, that's not possible. Once the control flow reaches your Node application, an attacker will know that it exists. They will be able to tell the difference between a page that is rendered by the browser on failure to look up a domain name in DNS, and a page you return to them. Besides, they won't be using browsers to investigate targets, so they will see quite a bit more than what a user in a browser would.
I think your best bet would be to copy & paste one of those annoying domain parking pages that web hosts put on a domain when it was purchased but isn't yet hosting a page yet. Ideally you would use the parking page of the domain registrar you used to acquire your domain because it will be the most believable. And of course, try to replicate the entire message (including headers), not just the HTTP body. Unlike the idea of serving a fake "can't resolve domain" page, this one should be entirely possible.
Recently, I made a setup where I pointed some websites to a redirect server. The redirect server in return served the website requests using ProxyPass directive of Apache2. It worked like a charm without even a single problem for my websites.
So, based on that I have got an idea to access internet via Apache2. Please note that this is because I do not have access to fast internet and every internet provider is so lousy and lame here to provide better connection speeds even for the lot of money I pay to them.
Now, https as better speends than VPN.
So, the idea is to get rid of VPN and SSH tunnel redirects and instead, resolve every domain on my Mac to a single server IP address which should be a redirect server and which can in turn bring me back every web request made from my Mac. Possible? This will make me to always use https to my own redirect server. https has better speed than VPN for me whenever I try and when I am on VPN things are too slow for me, may be because of level of encryption. Please note that I do not want solution using PPTP, L2TP and anything else which are lighter than OpenVPN (using Pritunl).
Please let me know if anything like that is possible and if yes then how.
Even though if it does not work, my mind always gets this idea every time. I just want someone to shed light on this and shut down my idea if its the worst by far. Thanks in advance.
Also, I have also seen some proxy sites where I put any website link on their website and their website works like a browser as if I am surfing on their remote server itself. May be something like that can be useful and speedy for me. But, I do not want to use them because I do not trust those sites for security. No way.
Got a solution myself without any kind of VPN.
Actually I needed to make my DNS secure and connections to my server Apps secure. So, for that I tried DNSCrypt-Proxy and its working great and resolving my DNS queries on HTTPS (443).
And, I am using an Addon on Chrome for "Always https" connections. I am blocking every request on http for Chrome using that Addon. Perfect!!!
So, now all surfing traffic on my Mac is going on HTTPS and is perfectly safe from hackers. I do not care for any other connections made by my other Mac Apps. I just care for security of my Apps while I am surfing them OR any payments I am making for shopping.
DNSCrypt-Proxy:
Please go to https://dnscrypt.org/#dnscrypt-osx and you will find all help there to how to install and run it on your Mac.
brew install dnscrypt-proxy --with-plugins
sudo dnscrypt-proxy --ephemeral-keys --resolver-name=cisco
^ You can find the resolver name in excel sheet that comes with this package.
And, just add an entry in your Network interfaces for DNS to point to 127.0.0.1, Please note that remove all other entries.
"Always HTTPS for Chrome":
https://chrome.google.com/webstore/detail/https-everywhere/gcbommkclmclpchllfjekcdonpmejbdp?hl=en
Enjoy perfect security on your Mac, if you do not care about IP address anonymity. Always use legal stuff!!!
I've got a requirement to detect if a webpage is being served on the internet or intranet, i.e. assuming a url of https://accessibleanyway.com, is the phone connected to the work wifi or to something else like their home wifi or the phone network?
What different ways are there to do this?
(1) Use WebRTC to get the local ip address. Not widely supported
(2) Try to access a local web page using jsonp/cors/iframe
The problem with 2 is that the webpage is https and the local resource is likely to be http which you can't do in IE afaik. If I make the local resource https then it's via a self cert which means installing CAs on the phones (can you buy certificates for the intranet anymore?)
Any suggestions?
The problem with (2) was that the same page was trying to use http and https, and even with an iframe you get issues.
What you could do instead is start on a http loading page, use an iframe to access a local resource which you can only access if you are on the intranet, jsonp will work fine for this. Once that's worked or failed, redirect to your start page with some token in the querystring to indicate that you are on the intranet or not
NB jumping from http to https would probably have some security issues if you are on the same website (authentication cookies being initially visible), but I would have thought it would be fine if you are going to a different one
Obviously there'll be some security needed around the token as otherwise the user could just generate their own but that's a different matter which depends on individual setups. It would obviously have to be generated by a server call, otherwise someone could just read the client code.
NB I think the IP address approach is never going to work as you have no way of knowing what a companies intranet setup looks like until you go there, so it's not a generic answer
I've set up a Raspberry Pi as a Wi-Fi access point. Everything works, including the captive portal. The web browser on each client is redirected to the login page, which functions correctly. I'm looking to modify the configuration of iptables and/or dnsmasq to make the client open a web browser on the captive portal automatically. Starbucks, McDonald's, etc. can all do it; I'm trying to figure out how to do it.
Here, here and here are partial explanations of how to achieve it, but I'm looking to understand it - not merely follow someone else's instructions - so that I can do it myself. I would like to write a HOWTO on the subject, partly because one doesn't exist yet (or if it does then I can't find it).
There are third-party apps such as Wifidog and Coovachilli, which seem to do the job, but I've failed to grasp how they do it. I believe it can be achieved by modifying the configuration of dnsmasq and iptables, but that's as far as I've gotten. it should do something like this:-
1) Regulate the data packets in such a way as to let the client's web browser realize that there's a captive portal; this will cause the client's web browser to open a window and direct it to the captive portal
2) Handle the captive portal; permit login; modify the settings of iptables to facilitate login; etc.
3) Redirect all traffic transparently after the login
Items 2 and 3 aren't a problem. I'm stuck on item 1. All advice is appreciated, including redirection to existing documentation. Thank you.
I do not know how WifiDog and CoovaChilli do their thing, but ChilliSpot (which CoovaChilli was originally based on) did something along these lines:
Open a raw socket bound to the internal interface
Capture all traffic bound to that interface
If it was authorized (eg. logged in), handle NAT and forward on out
If not authorized, block traffic
UNLESS
If it was not authorized AND HTTP, use some custom code to reply to the HTTP GET request with a 301 Redirect to point to the portal page itself, which would then allow for login.
That's the very simplified version of it, but I expect that most other captive portals will use very similar methods (especially the 301 Redirect). The absolute best way to find out would be to read a lot of code :)
Best of luck!
We're looking to do some scraping on a specific URL that uses cloudflare. Has anyone experienced issues using Zombie.js/user-agents while trying to crawl cloudflare hosted sites.
Would love some help!
I am trying to interface to an API on a client's site and I am getting a 403 error indeed. The request doesn't even reach my server.
Turning security to "essentially off" did not help. The final solution was to white-list the developer machine's IP.
The error is triggered on a single URL (json serving API) with a Java client with standards compliant libraries.
Solution:
1. try to set a rule to allow direct access for that URL
2. try setting security to weaker and weaker ("essentially off")
3. if both fails: try whitelisting
4. set up an alternate non-cloudflare url (direct.domain.com)
These will of course only work if you can negotiate with the site owners.
Backup solution: use an embedded browser that you can "frame" and "remote control" or a testing framework that does the same through a plugin, and extract the content from there (if you can)
Hope this helps.
You're probably triggering one of our security features by trying to scrape a site on us. The only option, really, would be to ask the site owner to whitelist your IP(s) to override the behavior.