Enter certain text to verify you are human
I have observed this on numerous websites, including SO. I dont know a thing about hacking, but I don't understand why is it required? Especially, in SO when making edits, sometimes it asks to enter some text to verify if it is Human or not.. I mean, how can a computer program (design for these kind of hacks) do edits (with SO's rules and restrictions on editing) , and then save them?
Is there any other reason, these verification things are added on any website?
SO isn't magic, it's just receiving some HTTP posts, and responding to them. Indeed, it won't even stop me from writing a script, though it will throw up the verification step if something indicates a nasty script (lots of posts per minute, lots of repetition). (Also, please don't call nasty tricks "hacking", it's not polite).
Edit: How to repeat the above from a script, with bits that would make it look like I did it, editted out:
POST http://stackoverflow.com/questions/11989856/answer/submit HTTP/1.1
Host: stackoverflow.com
User-Agent: Mozilla/5.0 (Windows NT 6.0; rv:14.0) Gecko/20100101 Firefox/14.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-IE,en;q=0.8,en-gb;q=0.5,en-us;q=0.3
Accept-Encoding: gzip, deflate
Connection: keep-alive
Referer: http://stackoverflow.com/questions/11989856/human-verfication
Cookie: [THIS BIT EDITED OUT]
Content-Type: application/x-www-form-urlencoded
Content-Length: 471
post-text=SO+isn%27t+magic%2C+it%27s+just+receiving+some+HTTP+posts%2C+and+responding+to+them.+Indeed%2C+it+won%27t+even+stop+me+from+writing+a+script%2C+though+it+will+throw+up+the+verification+step+if+something+indicates+a+nasty+script+%28lots+of+posts+per+minute%2C+lots+of+repetition%29.+%28Also%2C+please+don%27t+call+nasty+tricks+%22hacking%22%2C+it%27s+not+polite%29.&fkey=[THIS BIT EDITED OUT]&author=&i1l=[THIS BIT EDITED OUT]
My browser sent that to the site. Send something like that, and you'll have posted to this thread.
One can write scripts that make edits and save them too. If somebody wants to flood some particular website by doing this, he can do that by ways such as posting spam. To avoid such attacks, tasks like 'Enter certain text to verify if you are a human' are given.
Related
I'm trying to implement long polling for the first time, and I'm using XMLHttpRequest objects to do it. So far, I've been successful at getting events in Firefox and Internet Explorer 11, but Chrome strangely is the odd one out this time.
I can load one page and it runs just fine. It makes the request right away and starts processing and displaying events. If I open the page in a second tab, one of the pages starts seeing delays in receiving events. In the dev tools window, I see multiple requests with this kind of timing:
"Stalled" will range up to 20 seconds. It won't happen on every request, but will usually happen on several requests in a row, and in one tab.
At first I thought this was an issue with my server, but then I opened two IE tabs and two Firefox tabs, and they all connect and receive the same events without stalling. Only Chrome is having this kind of trouble.
I figure this is likely an issue with the way in which I'm making or serving up the request. For reference, the request headers look like this:
Connection: keep-alive
Last-Event-Id: 530
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.71 Safari/537.36
Accept: */*
DNT: 1
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8
The response looks like this:
HTTP/1.1 200 OK
Cache-Control: no-cache
Transfer-Encoding: chunked
Content-Type: text/event-stream
Expires: Tue, 16 Dec 2014 21:00:40 GMT
Server: Microsoft-HTTPAPI/2.0
Date: Tue, 16 Dec 2014 21:00:40 GMT
Connection: close
In spite of the headers involved, I'm not using the browser's native EventSource, but rather a polyfill that lets me set additional headers. The polyfill is using XMLHttpRequest under the covers, but it seems to me that no matter how the request is being made, it shouldn't stall for 20 seconds.
What might be causing Chrome to stall like this?
Edit: Chrome's chrome://net-internals/#events page shows that there's a timeout error involved:
t=33627 [st= 5] HTTP_CACHE_ADD_TO_ENTRY [dt=20001]
--> net_error = -409 (ERR_CACHE_LOCK_TIMEOUT)
The error message refers to a patch added to Chrome six months ago (https://codereview.chromium.org/345643003), which implements a 20-second timeout when the same resource is requested multiple times. In fact, one of the bugs the patch tries to fix (bug number 46104) refers to a similar situation, and the patch is meant to reduce the time spent waiting.
It's possible the answer (or workaround) here is just to make the requests look different, although perhaps Chrome could respect the "no-cache" header I'm setting.
Yes, this behavior is due to Chrome locking the cache and waiting to see the result of one request before requesting the same resource again. The answer is to find a way to make the requests unique. I added a random number to the query string, and everything is working now.
For future reference, this was Chrome 39.0.2171.95.
Edit: Since this answer, I've come to understand that "Cache-Control: no-cache" doesn't do what I thought it does. Despite its name, responses with this header can be cached. I haven't tried, but I wonder if using "Cache-Control: no-store", which does prevent caching, would fix the issue.
adding Cache-Control: no-cache, no-transform worked for me
I have decided to keep it simple and checked the response headers of a website that did not have this issue and I changed my response headers to match theirs:
Cache-Control: max-age=3, must-revalidate
I'm trying to protect a Classic ASP web application from HTTP Header Injected XSS attacks and am having trouble finding a solution that stops scripts found in the User Agent String.
Here is an example HTTP request to the web application:
HTTP Request
GET /WebApp/Login.aspx HTTP/1.1
Host: WebServer.Webapp.Com
User‐Agent: Mozilla/5.0 (X11; Linux x86_64; rv:41.0) Gecko/20100101 Firefox/41.0**alert(1)**
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept‐Language: en‐US,en;q=0.5
Accept‐Encoding: gzip, deflate
Cookie: ASP.NET_SessionId=foobarID
Connection: keep‐alive
Basically what we're trying to do is keep that alert script in the User Agent String from firing off when the page is loaded. I've been doing a lot of research and haven't been able to find too much help for this old app.
We do have validateRequest and EnableHeaderChecking set to true, But this script still executes. Any help is really appreciated.
The issue was from the user agent string (with the malicious script) being rendered on the page at the bottom for debug purposes. If you're having this issue, please check that you aren't displaying the object with the bad script on the page.
If you are, than remember to use HTML Encoding to render it safely.
Thanks to the_lotus and Lankymart for the quick answers.
I googled a lot and many answers are Yes. For example: Is GET data also encrypted in HTTPS? But the senior security engineer in our company told me the URL would not be encrypted.
Image that, if the URL was encrypted, how does the DNS server find the host and connect?
I think is this is very strong point although it's against most of the answers. So I'm really confused and my questions are:
Does https encrypt the everything in the request? (including the URL, host, path, parameters, headers)
If yes, how the DNS server decrypt the request and send it to the host server?
I tried to access https://www.amazon.com/gp/css/homepage.html/ref=ya_surl_youracct and my IE sent two requests to the server:
First:
CONNECT www.amazon.com:443 HTTP/1.0
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
Host: www.amazon.com
Content-Length: 0
DNT: 1
Connection: Keep-Alive
Pragma: no-cache
Second:
GET /gp/css/homepage.html/ref=ya_surl_youracct HTTP/1.1
Accept: text/html, application/xhtml+xml, */*
Accept-Language: en-US,zh-CN;q=0.5
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
Accept-Encoding: gzip, deflate
Host: www.amazon.com
DNT: 1
Connection: Keep-Alive
It seems my browser has requested twice: the first time is to establish the connection with host (without encryption) and the second time send an encrypted request over https? Am I right? If I am understanding this correctly, when a client call the RESTFUL API using https, it sends the requests (connection and get/post) twice every time?
The URL IS encrypted from the time it leaves the browser until it hits the destination server.
What happens is that the browser extracts the domain name and the port from the URL and uses that to resolve DNS itself. Then it starts an encrypted channel to the destination server IP:port. Then it sends a HTTP request through that encrypted channel.
The important part is anyone but you and the destination server can only see that you're connecting to a specific IP address and port. They can't tell anything else (like specific URLs, GET parameters, etc).
Attackers can't even see the domain in most cases (though they can infer it if there is actually a DNS lookup - if it wasn't cached).
The big thing to understand is that DNS (Domain Name Service) is a completely different service with a different protocol from HTTP. The browser makes DNS lookup requests to convert a domain name into an IP address. Then it uses that IP address to issue a HTTP request.
But at no time does the DNS server receive a HTTP request, and at no time does it actually do anything other than provide a domain-name - IP mapping for users.
While the other responses are correct so far as they go, there are many other considerations than just the encryption between the browser and the server. Here are some things to think about...
The IP address of the server is resolved.
The browser makes a TCP socket connection to the server's IP address using TLS. This is the CONNECT you see in your example.
The request is sent to the server over the encrypted session.
If this was all there is to it, you are done. No problem.
But wait, there's more!
Having the fields in a GET instead of a POST reveals sensitive data when...
Someone looks in the server logs. This might be a snoopy employee, but it can also be the NSA or other three-letter government agency, or the logs might become public record if subpoenaed in a trial.
An attacker causes the web site encryption to fall back to cleartext or a broken cipher. Have a look at the SSL checker from Qualsys labs to see if a site is vulnerable to this.
Any link on the page to an external site will show the URI of the page as the referrer. User ID and passwords are unintentionally yet commonly given away in this fashion to advertising networks. I sometimes spot these in my own blog.
The URL is available in the browser history and therefore accessible to scripts. If the computer is public (someone checks your web site from the guest PC in the hotel or airport lounge) the GET request leaks data to anyone else using that device.
As I mentioned, I sometimes find IDs, passwords and other sensitive info in the referrer logs of my blogs. In my case, I contact the owner of the referring site and tell them they are exposing their users to hacking. A less scrupulous person would add comments or updates to the site with links to their own web site, with the intention of harvesting the sensitive data in their referrer logs.
So your company's senior security engineer is correct that the URL is not encrypted in many places where it is extremely important to do so. You and the other respondents are also correct that it is encrypted in the very narrow use case of the browser talking to the server in context of a TLS session. Perhaps the confusion you mention has to do with the difference in the scope of these two use cases.
Please see also:
Testing for Exposed Session Variables (OTG-SESS-004)
Session Management - How to protect yourself (Note that "always use POST" is repeated over and over on this page.)
Client account hijacking through abusing session fixation on the provider
The URL (also known as "Uniform Resource Locator") contains four parts:
Protocol (e.g. https)
Host name (e.g. stackoverflow.com)
Port (not always included, typically 80 for http and 443 for https)
Path and file name or query
Some examples:
ftp://www.ftp.org/docs/test.txt
mailto:user#test101.com
news:soc.culture.Singapore
telnet://www.test101.com/
The URL as an entire unit is not actually encrypted because it is not passed in its entirety. The URL is actually pulled apart into bits and each part is used in different ways. E.g. the protocol portion will tell your browser how to use the rest of the URL, the host name will tell it how to look up the IP address of the intended recipient, and the port will tell it, well, which port to use. The only portion of the URL that is passed in the payload itself is the path and query, and that portion is encrypted.
If you take a look at an HTTP request in the raw, it looks something like this:
GET /docs/index.html HTTP/1.1
Host: www.test101.com
Accept: image/gif, image/jpeg, */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)
(blank line)
--Body goes here--
What you see in the example above is passed. Notice the full URL appears nowhere. The host header can actually be omitted completely (it is not used for routing). The only portion of the URL that appears here is to the right of the GET verb, and only includes the rightmost portion of the original URL. The protocol and the port number appear nowhere in the message itself.
Short answer: Everything to the right of the port number in the URL is included in the payload of the https request and is in fact encrypted.
I am interested in trying to figure out exactly what is going on when a user types in, for example
https://www.bing.com
which lands them on
http://www.bing.com
If you'll notice, www.bing.com apparently doesnt support https, so the page returned has no cert associated with it. Shouldn't the browser complain about this? What's more, is that when looking at the HTTP headers, I never actually see a ridirect or anything that indicates this page returned is not the https version (guess I was expecting some indication this happened).
For another example, gmail does something similar -
I go to https://gmail.com
and I end up on mail.google.com or accounts.google.com depending on whether I'm logged in or not. At least these sites give me a cert, unlike bing, but how come the browser doesn't complain that the URL's are mismatched? It seems like I should also get a cert for gmail.com is that case, right? (the cert on the gmail redirect is good for mail.google.com, but makes no mention wildcard or otherwise of gmail.com)
There's nothing special going on. It's a simple HTTP redirect, but you'll only see it if you ignore the SSL certificate error. (https://www.bing.com currently serves a certificate issued to akamai.) Remember, once you tell your browser to ignore the cert error, it will generally remember that choice for the rest of the session.
If you instruct your browser to ignore the SSL certificate error, the following happens inside a SSL-encrypted connection:
GET https://www.bing.com/ HTTP/1.1
Host: www.bing.com
Connection: keep-alive
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.73 Safari/537.36
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8,es;q=0.6
HTTP/1.1 302 Moved Temporarily
Server: AkamaiGHost
Content-Length: 0
Location: http://www.bing.com/
Date: Thu, 02 May 2013 22:02:28 GMT
Connection: keep-alive
There's no rule against a HTTPS site redirecting to plain HTTP1, so the browser just does a normal request for http://www.bing.com. Since we're now on a plain HTTP page, there's nothing to display (warning or otherwise) regarding certificates.
1 - except in certain situations involving POST requests, where some browsers issue warnings.
The other sites you mention work similarly, except the redirect from gmail.com is to https://mail.google.com. mail.google.com has its own certificate, distinct from https://www.gmail.com's certificate.
Reading over
http://getpocket.com/api/docs/
Is it safe to pass a password through the HTTP string? My understand is that this is not safe, even though it's HTTPS. Correct?
The API documentation states that you're passing over HTTPS. Actually all of the information whether GET or POST in the HTTP Header is part of the SSL Transport therefore the URL parameters are encrypted as well, so your GET parameters are encrypted. What can't be guaranteed is what your client will retain. Or if there was some other process that exposed some information such as when your server did a DNS lookup for the host name. Another example is if you have a browser and it keeps a history of everything you type in it including your https urls then you may compromise your security.
Below is the HTTP Header, your client will initiate a TCP connection and send something like the following:
GET /tutorials/other/top-20-mysql-best-practices/ HTTP/1.1
Host: net.tutsplus.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 (.NET CLR 3.5.30729)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Cookie: PHPSESSID=r2t5uvjq435r4q7ib3vtdjq120
Pragma: no-cache
Cache-Control: no-cache
SSL will dictate that all that information is encrypted along with anything that is sent back. I would say you're safe using this API, the only difference between the GET and the POST methods would be that in the POST the parameters would be in the body whereas with the GET the parameters are in the header. In both cases all the sensitive information is encrypted.
I agree in principle that it sounds unsafe. URLs can end up in all kinds of funny places in plain text (even over HTTPS), like logs. It would be best to avoid having it in plaintext anywhere.
You should probably talk to the API authors about whether there is an alternative strategy. For example, it looks like some of those methods support both POST and GET, in which case you could possibly POST password details, which is a relatively safe thing to do over an HTTPS connection.