There is such a windows application that communications with the server through https protocol, it is an auction tool and works only several hours per month. I have captured network packets (by windows network monitor) during one auction.
I am wondering whether it is possible to mimic this client, by analyzing the the packets I collected (or any packets I could collect in future auctions). I know from this wireshark artical "Secure Socket Layer (SSL)" that it should be possible (and without much effort) to descypt the entrypted messages from server, but how? And is it possible to dectrypt the messages sent by client to server, too?
So the whole reasoning behind SSL is that third party listeners who are trying to receive and decrypt packets between your client and the server won't be able to do so. The packets that your clients send will be encrypted and the server will need the appropriate key to decrypt the message which you could then analyze using wireshark. This article does a good job of explaining how HTTPS works. To answer your questions:
Is it possible to mimic the client of a https web service
Yep, this is available in a lot of different tools. A popular tool you could implement this with is called Jmeter. This article explains how you can send HTTPS requests to your server. Once you exchange the key pairs between your client (JMeter in this case) you will be able to decrypt messages on both ends which have been sent in an encrypted format over the wire.
Please let me know if you have any questions!
I am trying to verify a new functionality added to the router kernel. So lets assume I added a filter,iptable and such. I have live traffic. I tried to use tcp dump and look into the packet header and see if the new values in the header are there but I am getting random traffic passing through.
I wonder if it is possible to have tcpdump send traffic and dump it so I have specific packets I sent ? or any other linux commands I can use to do the process. I have a destination port that has no traffic but I don't know how I can send to it and capture those packets. I did some reading on sendip and tcpdump and couldnt help myself.
Send packets, tcpdump it or something like that, verify the packet header has what I want and move on with my life.
I hope I was clear enough.
I would like to create chain for Iptable which return URL and post and get from all packets.
I do not believe that netfilter/iptables is generally aware of application-level protocols like HTTP. There is some limited protocol dissection support that is used for connection tracking in some specific protocols (e.g. FTP), but that's about it. The way I see it, you have two options:
Redirect the packets that you want to inspect to userspace where you can have a helper process that will extract the information that you need. I am not aware of any existing application that will do that, however, so you may have to write it yourself.
Do the sane thing and just use a packet sniffer such as Wireshark or tcpdump, preferably with a capture filter that will only let HTTP traffic through. As a bonus, you can get a whole lot more information than just an HTTP request and a URL...
( 3 years later but I think useful for other people too )
barebone iptables are agnostic of application protocols, but there are iptables extensions
for example, see the extensions string and u32
Would like to understand whether the HTTPS body part of the Response is encrypted. Also, in a HTTPS request whether the header are transmitted as plain text / encrypted?
Is there any tool with which I can observe the raw HTTPS traffic without decrypting it.
HTTPS is HTTP over SSL. So the whole HTTP communication is encrypted.
As the other posts say - HTTPS is HTTP (plaintext) wrapped in SSL on top of the TCP/IP layer. Every part of the HTTP message is encrypted. So the stack looks like:
TCP/IP
SSL
HTTP
As far as encryption goes, there is no way to see any part of the HTTP message with SSL around it.
If you need to debug your traffic I suggest the following:
Use a network traffic watcher (like Ethereal) to watch the creation of connections. This will let you see the connection be initiated. It will show you the start of the SSL Handshake, details on failures, and when the session is set up, there will be chains of cipher text. The ciphertext is not very useful, but its presence lets you know data is going back and forth.
Deubg your http layer in the clear prior to setting up HTTPS. Every application or web server I've ever worked with has let me turn off HTTPS, and host the same set of URLs in the clear. Do this, and watch it with the same network tool.
If you get both sides talking with HTTP and everything breaks on HTTPS, it's time to look at either the SSL session establishment or anything in between the two points that may be interrupting the flow.
YES https flow is encrypted. When an https connection is initialized, it uses a strong encryption algorithm to handshake and agree with other part on a less strong, but much faster encryption algorithm for the flow.
To observe network packets, you can use sniffers like http://www.ethereal.com/.
When using HTTPS, the entire content of the request and reply are encrypted, including the headers and body. The HTTP protocol in plaintext happens on top of, TLS or SSL, so what's on the wire is encrypted.
The entire HTTP session is encrypted including both the header and the body.
Any packet sniffer should be able to show you the raw traffic, but it'll just look like random bytes to someone without a deep understanding of SSL, and even then you won't get beyond seeing the key exchange as a third party.
Any packet capture/sniffing tool can show you the raw HTTPS traffic. To view the actual contents (by decrypting it), use Fiddler.
Anything sent over https is encrypted using SSL transport
Try WireShark or Fiddler as helpful tools for this.
When sending data over HTTPS, I know the content is encrypted, however I hear mixed answers about whether the headers are encrypted, or how much of the header is encrypted.
How much of HTTPS headers are encrypted?
Including GET/POST request URLs, Cookies, etc.
All the HTTP headers are encrypted†.
That's why SSL on vhosts doesn't work too well - you need a dedicated IP address because the Host header is encrypted.
†The Server Name Identification (SNI) standard means that the hostname may not be encrypted if you're using TLS. Also, whether you're using SNI or not, the TCP and IP headers are never encrypted. (If they were, your packets would not be routable.)
The headers are entirely encrypted. The only information going over the network 'in the clear' is related to the SSL setup and D/H key exchange. This exchange is carefully designed not to yield any useful information to eavesdroppers, and once it has taken place, all data is encrypted.
New answer to old question, sorry. I thought I'd add my $.02
The OP asked if the headers were encrypted.
They are: in transit.
They are NOT: when not in transit.
So, your browser's URL (and title, in some cases) can display the querystring (which usually contain the most sensitive details) and some details in the header; the browser knows some header information (content type, unicode, etc); and browser history, password management, favorites/bookmarks, and cached pages will all contain the querystring. Server logs on the remote end can also contain querystring as well as some content details.
Also, the URL isn't always secure: the domain, protocol, and port are visible - otherwise routers don't know where to send your requests.
Also, if you've got an HTTP proxy, the proxy server knows the address, usually they don't know the full querystring.
So if the data is moving, it's generally protected. If it's not in transit, it's not encrypted.
Not to nit pick, but data at the end is also decrypted, and can be parsed, read, saved, forwarded, or discarded at will. And, malware at either end can take snapshots of data entering (or exiting) the SSL protocol - such as (bad) Javascript inside a page inside HTTPS which can surreptitiously make http (or https) calls to logging websites (since access to local harddrive is often restricted and not useful).
Also, cookies are not encrypted under the HTTPS protocol, either. Developers wanting to store sensitive data in cookies (or anywhere else for that matter) need to use their own encryption mechanism.
As to cache, most modern browsers won't cache HTTPS pages, but that fact is not defined by the HTTPS protocol, it is entirely dependent on the developer of a browser to be sure not to cache pages received through HTTPS.
So if you're worried about packet sniffing, you're probably okay. But if you're worried about malware or someone poking through your history, bookmarks, cookies, or cache, you are not out of the water yet.
HTTP version 1.1 added a special HTTP method, CONNECT - intended to create the SSL tunnel, including the necessary protocol handshake and cryptographic setup.
The regular requests thereafter all get sent wrapped in the SSL tunnel, headers and body inclusive.
HTTPS (HTTP over SSL) sends all HTTP content over a SSL tunel, so HTTP content and headers are encrypted as well.
With SSL the encryption is at the transport level, so it takes place before a request is sent.
So everything in the request is encrypted.
Yes, headers are encrypted. It's written here.
Everything in the HTTPS message is encrypted, including the headers, and the request/response load.
the URL is also encrypted, you really only have the IP, Port and if SNI, the host name that are unencrypted.
To understand, what is encrypted and what not, you need to know that SSL/TLS is the layer between the transport-layer and the application-layer.
in the case of HTTPS, HTTP is the application-layer, and TCP the transport-layer. That means, all Headers below the SSL-Level are unencrypted. Also, SSL itself may expose data. The exposed data includes(for each layer's Header):
NOTE: Additional Data might be exposed too, but this data is pretty sure to be exposed.
MAC:
Source MAC address(Current Hop)
Destination MAC address(Next Hop)
IP(assuming IPv4):
Destination IP address
Source IP address
IP Options(if set)
Type-Of-Service(TOS)
The number of hops the current packet passed, if TTL is set to 64
TCP:
Source Port
Destination Port
TCP-Options
Theoretically, you can encrypt the TCP-Headers, but that is hard to implement.
SSL:
Hostname(if SNI is being used)
Usually, a browser won't just connect to the destination host by IP immediantely using HTTPS, there are some earlier requests, that might expose the following information(if your client is not a browser, it might behave differently, but the DNS request is pretty common):
DNS:
This request is being sent to get the correct IP address of a server. It will include the hostname, and its result will include all IP addresses belonging to the server.
HTTP:
the first request to your server. A browser will only use SSL/TLS if instructed to, unencrypted HTTP is used first. Usually, this will result in a redirect to the seucre site. However, some headers might be included here already:
User-Agent(Specification of the client)
Host (Hostname)
Accept-Language (User-Language)