Does the encryption length of the hash encrypted influence his own security?
I mean if i use md5() or sha() if the returned hash has 35 chars or 55 chars does will this influence in some way security of hash?
Hash don't have a "security", as they generally can't be inverted.
What they do have is risk of collision, i.e. two different messages (filenames, passwords etc.) mapping to the same hash.
To prevent that, the more bits you have, the better, provided that hash distribution is as flat (spread out) as possible.
In that case, collision probability is approximately the reciprocal of the bit length: a 32 bit hash is 1/2**32, etc.
There are also other considerations: under certain circumstances it is possible to craft a collision against MD5, and SHA is therefore to ne preferred. The difficulty of the attack is very high, and so you can still use MD5 for most purposes except very high security scenarios.
Related
I am trying to understand SHA uniqueness in simple terms.
For example let us assume there are only messages with maximum length of 4 bits (binery) in whole world. Number of possible messages with different lengths is
2 for single bit length
2^2 for double bit length
2^3 for 3 bit length
2^4 for 4 bit length
that would be 2+4+8+16 = 30 (31 if we consider empty message 2^0 = 1)
Lets us consider SHA3(for example) with output length of 3bits (binery), so maximum possible number of digest are 8.
How can a digest be unique if we need to map 30 messages to 8, or why is it hard to find digest collision for 2 unique messages
I'm not sure what you mean by "SHA uniqueness". An SHA value (any version) is not unique, it cannot be, because it maps an infinite number of inputs (an input of any length) to a finite number of outputs.
A cryptographic hash function has three important properties (which make it a crypto hash, over a regular hash):
strong collision resistance: it is very difficult (computationally infeasible, ie. "not practically possible") to find two inputs that produce the same output (even if you can choose both)
weak collision resistance: for a given input, it is computationally infeasible to find another input that gives the same hash value (you can choose one input to match the output of a given input)
preimage resistance: for a hash value, it's computationally infeasible to find an input that produces that output (it's "one-way")
The only problem in your example is the size. With such small numbers it doesn't make sense of course. But if the hash value is say 512 bits, it suddenly gets really time consuming and hence practically impossible to brute force.
"SHA3 which has digest length of 3bits"
I think this question is based on one bit misunderstanding. SHA-3 is a family of hashes that has the same output bit size as SHA-2. SHA-2 has bit sizes 224, 256, 384 or 512 for SHA-224, SHA-256, SHA-384 and SHA-512 respectively.
Of course, SHA-2 already took those identifiers, so SHA-3 will have SHA3-224, SHA3-256, SHA3-384 and SHA3-512. There were some proposals to use a different acronym, but those failed.
Still, SHA-3 hashes have near infinite input, so there will be many hashes that map to the same value. However, since it is not possible reverse any SHA-3 algorithm, it should be impossible to find a collision. That is, unless SHA-3 is broken, as it is not provably secure.
Any SHA3 variant will have digests with more than 100 bits. The terminology has probably confused you, because SHA256 has 256 bits, while SHA3 is considered the third generation of SHA algorithms (and does NOT have 3 bits of lenght).
Generally speaking it's not hard to find a hash collision by brute-forcing (alas, it's time-consuming), what is difficult is producing a collision that is also meaningful in its context. For example, assume you have a source file for an important application, that hashes to a digest. If an attacker tried to alter the source file in a way to introduce a vulnerability, while also hashing to the same digest, he'd have to introduce a lot of random gibberish, making the attack obvious.
So this is a two-part question:
Are there any hashing functions that guarantee that for any combination of the same length, they generate a unique hash? As I remember - most are that way, but I just need to confirm this.
Based on the 1st question - so, given a file hash and a length - is it then theoretically possible to 'brute-force' all byte permutations of that same length until the same hash is generated - ie. the original file has been recreated?
PS. I am aware that this will take ages (if theoretically possible), but I think it would be feasible for small files (sizes < 1KB)
1KB, that'd be 1000^256, right? 1000 possible combinations of bytes (256 configurations each?). It's a real big number. 1 with 768 0s behind it.
If you were to generate all of them, one would be the right one, but you'd have some number of collisions.
According to this security.SE post, the collission rate for md5 (for example) is about 1 in 2^64. So, if we divide our original number by that, we'd get how many possible combinations, right? http://www.wolframalpha.com/input/?i=1000%5E256+%2F+2%5E64
~5.42 × 10^748
That is still a lot of files to check.
I'd feel a lot better if someone critiqued my math here, but the point is that your first point is not true because of collisions. You can use the same sort math for calculating two 1000 character passwords having the same hash. It's the birthday problem. Given 2 people, it is unlikely that we'd have the same birthday, but if you take a room full the probability of any two people having the same birthday increases very quickly. If you take all 1000 character passwords, some of them are going to collide. You are going from X bytes to 16 bytes. You can't fit all of the combinations into 16 bytes.
Expanding upon the response to your first point, one of the points of cryptographic hash functions is unpredictability. A function with zero collisions is a 1-1 (or one-to-one) function, so called because every input has exactly one output and every output has exactly one input.
In order for a function to accept arbitrary length & complexity inputs without generating a collision, it is easy to see that the function must have arbitrary length outputs. As Gray obliquely points out, most hash functions have fixed-length outputs. (There are apparently some new algorithms that support arbitrary length outputs, but they still don't guarantee 0 collisions.) The reason is not stated clearly in the common crypto literature, but consider the difference between hashing and encrypting.
In hashing, you have the message (the unaltered original) and the message digest (the output of the hash function. (Digest here having the meaning "a summation or condensation of a body of information.")
With encryption, you have the plain text and the cipher text. The implication is that the cipher text is of equal length and complexity as the original.
I look at it as a cryptographic hash function with 0 collisions is of equal complexity as encryption. (Note that I'm unsure of what the advantages of a variable-length hash output are, so I asked a question about it.)
Additionally, hash functions are susceptible to attacks by pre-computed rainbow tables, which is why all hash algorithms still considered secure employ extra random inputs, called salts. The reason encryption isn't susceptible to a similar attack is that the encryption key is kept secret and you can't pre-compute output values without knowing the key. Compare symmetric key encryption (where there is one key that must be kept secret) with public key encryption (where the encryption key is public and the decryption key is private).
The other thing that prevents encryption algorithms from pre-computation attacks is that the number of computations for arbitrary-length inputs grows exponentially, and it is literally impossible to store the output from every input you may be interested in.
Is there a function that generates a hash that has the exact lenght I want? I know that MD5 always has 16 bytes. But I want to define the lenght of the resulting hash.
Example:
hash('Something', 2) = 'gn'
hash('Something', 5) = 'a5d92'
hash('Something', 20) = 'RYNSl7cMObkPuXCK1GhF'
When the length increases, the result should be more secure from duplicates.
The upcoming SHAKE256 (or SHAKE128 for a security level of 128bit instead of 256bit), a so called extendable-output function (XOF), is exactly what you are looking for. It will be defined alongside with SHA3. There is already a draft online.
If you need an established solution now, follow CodesInChaos advice and truncate SHA512 if a maximum of 64byte is enough and otherwise seed a stream cipher with the output of a hash of the original data.
Technical disclaimer: After an output length of 512bit the "security against duplicates" (collision resistance) does not increase any more with longer output, as with SHAKE256 it has reached the maximum security level against collisions the primitive supports (256bit). (Note that because of the birthday paradox the security level of an ideal hash function with output length of n bit against collisions is only n/2 bit.) Any higher security level is pretty much meaningless anyway (probably 256bit is already an overkill) given that our solar system does not provide enough energy to even count from 0 to 2^256.
Please do not confuse "security levels" with key lengths: With symmetric algorithms one usually expects a security level equal to the key size, but with asymmetric algorithms the numbers are completely unrelated: A 512 bit RSA encryption scheme is far less secure than 128bit AES (i.e. 512bit RSA moduli can be factored by brute force already).
If a cryptographic primitive tries to achieve a "security level of n bits" it means that there are supposed to be no attacks against it that is faster than 2^n operations.
BLAKE2 can produce digests of any size between 1 and 64 bytes.
If you want a digest considered cryptographically secure, consider the Birthday problem and what other algorithms use — e.g. SHA-1 uses 20 bytes and is considered insecure, SHA-2 uses 28/32/48/64 bytes and is generally considered secure.
If you just want to avoid accidental collisions, still consider the Birthday problem (above), but 16 or even 8 bytes might be considered sufficient depending on the application (see table).
I was reading this article regarding the number of times you should hash your password
A salt is added to password before the password is hashed to safeguard against dictionary attacks and rainbow table attacks.
The commentors in the answer by ORIP stated
hashing a hash is not something you should do, as the possibility of
hash collision increase with each iteration which may reduce the
search space (salt doesn't help), but this is irrelevant for
password-based cryptography. To reach the 256-bit search space of this
hash you'd need a completely random password, 40 characters long, from
all available keyboard characters (log2(94^40))
The answer by erickson recommended
With pre-computation off the table, an attacker has compute the hash
on each attempt. How long it takes to find a password now depends
entirely on how long it takes to hash a candidate. This time is
increased by iteration of the hash function. The number iterations is
generally a parameter of the key derivation function; today, a lot of
mobile devices use 10,000 to 20,000 iterations, while a server might
use 100,000 or more. (The bcrypt algorithm uses the term "cost
factor", which is a logarithmic measure of the time required.)
My questions are
1) Why do we iterate over the hash function since each iteration reduces the search space and hence make it easier to crack the password
2) What does search space mean ??
3) Why is the reduction of search space irrelevant for password-based cryptography
4) When is reduction of search space relevant ??
.
Let's start with the basic question: What is a search space?
A search space is the set of all values that must be searched in order to find the one you want. In the case of AES-256, the total key space is 2^256. This is a really staggeringly large number. This is the number that most people are throwing around when they say that AES cannot be brute forced.
The search space of "8-letter sequences of lowercase letters" is 26^8, or about 200 billion (~2^37), which from a cryptographic point of view is a tiny, insignificant number that can be searched pretty quickly. It's less than 3 days at 1,000,000 checks per second. Real passwords are chosen out of much smaller sets, since most people don't type 8 totally random letters. (You can up this with upper case and numbers and symbols, but people pick from a tiny set of those, too.)
OK, so people like to type short, easy passwords, but we want to make them hard to brute-force. So we need a way to convert "easy to guess passwords" into "hard to guess key." We call this a Key Derivation Function (KDF). We need two things for it:
The KDF must be "computationally indistinguishable from random." This means that there is no inverse of the hash function that can be computed more quickly than a brute force search.
The KDF should take non-trivial time to compute, so that brute forcing the tiny password space is still very difficult. Ideally it should be made as difficult as brute forcing the entire key space, but it is rare to push it that far.
The first point is the answer to your question of "why don't we care about collisions?" It is because collisions, while they could possibly exist, cannot be predicted in an computationally efficient manner. If collisions could be efficiently predicted, then your KDF function is not indistinguishable from random.
A KDF is not the same as just "repeated hashing." Repeated hashing can be distinguished from random, and is subject to significant attacks (most notably length-extension attacks).
PBKDF2, as a specific KDF example, is proven to be computationally indistinguishable from random, as long as it is provided with a pseudorandom function (PRF). A PRF is defined as itself being computationally indistinguishable from random. PBDFK2 uses HMAC, which is proven to be a PRF as long as it is provided a hashing function that is at least weakly collision resistant (the requirement is actually a bit weaker than even that).
Note the word "proven" here. Good cryptography lives on top of mathematical security proofs. It is not just "tie a lot of knots and hope it holds."
So that's a little tiny bit of the math behind why we're not worried about collisions, but let's also consider some intuition about it.
The total number of 16-character (absurdly long) passwords that can be easily typed on a common English keyboard is about 95^16 or 2^105 (that doesn't count the 15, 14, 13, etc length passwords, but since 95^16 is almost two orders of magnitude larger than 95^15, it's close enough). Now, consider that for each password, we're going to randomly map it to 10,000 intermediate keys (via 10,000 iterations of PBKDF2). That gets us up to 2^118 random choices that we hope never collide in our hash. What are the chances?
Well, 2^256 (our total space) divided by 2^118 (our keys) is 2^138. That means we're using much less than 10^-41 of the space for all passwords that could even be remotely likely. If we're picking these randomly (and the definition of a PRF says we are), the chances of two colliding are, um, small. And if two somehow did, no attacker would ever be able to predict it.
Take away lesson: Use PBKDF2 (or another good KDF like scrypt or bcrypt) to convert passwords into keys. Use a lot of iterations (10,000-100,000 at a minimum). Do not worry about the collisions.
You may be interested in a little more discussion of this in Brute-Forcing Passwords.
As the second snippet said, each iteration makes each "guess" a hacker makes take longer, therefore increasing the total time it will take then to crack an average password.
Search space is all the possible hashes for a password after however many iterations you are using. Each iteration decreases the search space.
Because of #1, as the size of the search space decreases, the time to check each possibility increases, balancing out that negative effect.
According to the second snippet, answers #1 and #3 say it actually isn't.
I hope this makes sense, it's a very complicated topic.
The reason to iterate is to make it harder for an attacker to brute force the hash. If you have a single round of hashing for a value, then in order to precompute a table for cracking that hash, you need to do 1 * keyspace hashes. If you do 1000 hashes of the value, then it would require the work of 1000 * keyspace.
Search space generally refers to the total number of combinations of characters that could make up a password.
I would say that the reduction of search space is irrelevant because passwords are generally not cracked by attempting 0000000, then 0000001, etc. They are instead attempted to be cracked by using dictionaries and combinatorics. There is essentially a realm of passwords that are likely to get cracked (like "password", "abcdef1", "goshawks", etc.), but creating a larger work factor will make it much more difficult for an attacker to hit all of the likely passwords in the space. Combining that with a salt, means they have to do all of the work for those likely passwords, for every hash they want to crack.
The reduction in search space becomes relevant if you are trying to crack something that is random and could take up any value in the search space.
Well, from the discussion of hashing methods weaknesses, I've got that the only ol' good brute-force is efficient to break.
So, the question is:
Is there a hashing algorithm which is more rigid against brute-force than others?
In case of hashing passwords.
The only protection against brute force is the fact that it takes an inordinately long time to perform a brute force.
Brute force works by simply going through every possible input string and trying it, one at a time. There's no way to protect against simply trying every possible combination.
This question is a decade old and now I've got the answer.
Yes, there are bruteforce-proof algorithms. The key for such algo is being slow. It will do no harm if correctness will be verified in a few milliseconds. But it will drastically slow the brute-force. Moreover, those algorithms can adapt to the future CPU's performance increase. Such algorithms include
bcrypt
argon2
Particularly in PHP, the password_hash() function must be used for hashing passwords
If you know that the input space is small enough for a brute force attack to be feasible, then there are two options for protecting against brute-force attacks:
Artificially enlarging the input space. This isn't really feasible - Password salting looks like that at first glance, but it really only prevents attackers from amortizing the cost of a brute force attack across multiple targets.
Artificially slowing down the hashing through key strengthening or using a hash algorithm that is inherently slow to compute - presumably, it's only a small extra cost to have the hash take a relatively long time (say, a tenth of a second) in production. But a brute-force attacker incurs this cost billions of times.
So that's the answer: the slower a hash algorithm is to compute, the less susceptible it is against brute-forcing the input space
(Original Answer follows)
Any additional bit in the output format makes the algorithm twice as strong against a straightforward brute force attack.
But consider that if you had a trillion computers that could each try a trillion hashes per second, it would still take you over 100 trillion years to brute-force a 128 bit hash, and you'll realize that a straightforward brute-force attack on the output is simply not worth wasting any throughts on.
Of course, if the input of the hash has less than 128bits of entropy, then you can brute-force the input - this is why it's often feasible to brute-force password cracking (Nobody can actually remember a password with 128 bits of entropy).
Consider the output of the hash algorithms.
A MD5 (128 bit) or SHA-1 (160 bit) is certainly easier to brute-force than a SHA-2 (224, 384 or even 512 bit).
Of course, there can be other flaws (like in MD5 or a bit less in SHA-1) which weaken the algorithm a lot more.
As Codeka said, no hashing algorithm is 100% secure against brute force attacks. However, even with hardware-assisted password cracking (using the GPU to try passwords), the time it takes to crack a sufficiently long password is astronomical. If you have a password of 8ish characters, you could be vulnerable to a brute force attack. But if you add a few more characters, the time it takes to crack increases radically.
Of course, this doesn't mean you're safe from rainbow attacks. The solution to that is to add a salt to your password and use a hashing algorithm that isn't vulnerable to preimage attacks.
If you use a salted password of 12-14 characters, preferably hashed with an sha2 algo (or equivalent), you've got a pretty secure password.
Read more here: http://www.codinghorror.com/blog/2007/10/hardware-assisted-brute-force-attacks-still-for-dummies.html
All cryptographic systems are vulnerable to brute force. Another term for this is a "Trivial Attack".
A simple explanation for hashing is that all hashing algorithms we use accept an infinitely sized input and have a fixed sized output. This is an unavoidable collision, and for something like sha256 it takes 2^256 operations to find one naturally. md5() has a shortcut making it 2^39th operations to find a collision.
One thing you can do to make your passwords stronger is to hide your salt. A password hash cannot be broken until its salt is retrieved. John The Ripper can be given a Dictionary, a Salt and a Password to recover password hashes of any type. In this case sha256() and md5() will break in about the same amount of time. If the attacker doesn't have the salt he will have to make significantly more guesses. If your salt is the same size as sha256 (32 bytes) it will take (dictionary size)*2^256 guesses to break one password. This property of salts is the basis of CWE-760.
Brute force is the worst attack, nothing can be brute force proof...
right now ~80-90 bits is considered cryptographically safe from a brute force attack standpoint, so you only need 10 bytes if a Collision Resistant Hash function is perfect, but they aren't so you just do more bits...
the proof that nothing can be brute force proof is in the Pigeon Hole Principle.
since hash function H allows arbitrary sized input [0,1]^n and outputs constant output [0,1]^k when the size of input exceeds the output size:, n>k, there are necessarily some outputs that can be produced by more than one input.
you can visualize that with a square divided into 9 sub squares.
0 | 0 | 0
0 | 0 | 0
0 | 0 | 0
these are your 9 holes. We are a brute force attacker, we have unlimited chances to attack... we have unlimited pigeons... but we at most need 10 to find a collision...
after 4 pidgeons and a good collision resistant hashing algorithm:
P | 0 | 0
0 | P | P
0 | 0 | P
after 9 pidgeons:
P | P | P
P | P | P
P | P | P
so our 10th pigeon will necessarily be a collision, because all of the holes are full.
but it really isn't even that good, because of another numerical property called the Birthday Paradox where given a number of independent selections you will find a duplicate much much faster than it takes to fill all of your "holes".