Bcrypt Longer Passwords - security

So Bcrypt does have a limit for how long passwords can be. I have read many pages regarding this. The one thing I can't figure out is how most sites bypass this.
Most sites I have noticed don't have a max password length. Maybe I'm totally wrong about this but that is just what I have noticed. Bcrypt seems to be one of the most popular libraries for this type of thing.
So are all of these sites just not alerting users and Bcrypt is cutting the passwords to the max character limit and not alerting users? Or are they doing some special technique to allow for longer passwords?
I'm just trying to figure out how to best implement this. I would love to have no max character limit. But at the same time I want to be straight forward with users and if Bcrypt is cutting passwords users should know about that.
Any suggestions for how to handle this limitation in practice?

Even if the question is already answered, I would like to point out two things:
The length of passwords the user can enter should not be limited, that's correct. BCrypt has no problems to work with passwords longer then 72 characters though, it will just truncate the password to this length. So accept passwords of any length, pass them directly to BCrypt or use the scheme from Zaphs answer.
Hashing 72 character passwords is more than enough to be on the very safe side. Even 20 character passwords cannot realistically be brute-forced.
A 72 character password would allow for 1E129 combinations (without special characters). Very fast hashes can be calculated with 100Giga/second. Even in this worst case you would need about 1E110 years to expect a match, that is about 1000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 times the age of the universe!
Be aware when combining different hash algorithms:
If done correctly this can increase entropy, if the entered password is longer than 72 characters (which is rarely the case) and if the password is not random like (which is unlikely if somebody takes care to use such long passwords). A lot of ifs.
And there are pitfalls:
If you use the binary output of the SHA-512 then you could stumble over \0 characters, which can lead to unsecure hashes, see this article.
If you use the hex/base64 representation of the SHA, you limit the possible characters for the 72 places.

I see no problem with first running SHA-512.
According to the NIST SP 800-63-3 Draft document "Digital Authentication Guidelines" passwords SHALL accept (and use) at least 64 characters, if they accept more it must not be truncated.
In reality NIST is recommending use of PBKDF with any of SHA-1, SHA-2 family, SHA-3 family, even with SHA1 there will essentially be no collision and even if there is it is not a problem for password hashing. The key is the iteration count to slow down the attacker.
Read the answer comment by #ilkkachu in the links answer.

Related

Password symbols, length and 'strength'

Note: This doesn't explicitly relate to programming, but I was hoping this can be explained from a programmers point of view.
There are two things I simply don't understand about current 'password strength ratings'. This all pertains to brute force entry. (If these 'password strength ratings' relate to any other type of breach aside from using a common/popular password please let me know).
1) Why does it matter if I include numbers/symbols/uppercase letters as long as the password system allows for the possibility of using them?
For example lets just say:
a) The systems accepted characters are a-z, A-Z, 0-9, and their "shifted values" '!' to ')', so 72 possible symbols.
b) I use a password of length ten, so 72^10 possibilities.
c) My password is not in the top 10,000 most common/popular passwords used. So 72^10 - 10,000 possibilties remain.
Wouldn't an all lowercase password like 'sndkehtlyo' be identical strength as 'kJd$56H3di' since they both share the same possibility of including the additional characters? Doesn't the brute force algorithm have to include those numbers/symbols/uppercase regardless of whether or not I use them? It seems like these rating systems believe a brute force attempt will try all 26^n lowercase passwords first, all 52^n passwords second, then all 62^n passwords, etc, etc.
2) Why does that even matter? I have yet to come across any password system that doesn't lock you out after some small fixed number of attempts (usually 5). How can brute force approaches even work these days?
I feel like I am missing something fundemental here.
1) Cracking a password doesn't need to happen in one pass. A well implemented brute force crack may iterate first through small ranges of characters and then work its way into caps and numbers. Starting with the simplest ranges first (maybe just lowercase a-z) will find passwords of those unfortunate enough to have constructed a weak password. They may also start with dictionary attacks or Most-common-passwords-used attacks first as they take very little time.
2) Crackers aren't going to brute force right through some online service's login prompt. Anyone truly intent on getting access to an account would retrieve the hash of a user's password and crack it on their own machine, not over the internet. While there are practically infinite ways to hash a password there are some very common methods that can be identified by properties such as the hash's character length. You can read more about common hash algorithms in this Wikipedia article.
1) All man-made passwords are not totally random. In other words, taking the human factor (e.g. memorability), the probability distribution of a password space is not even.
2) The attempt times restriction is used for authentication, which is a means of Access Control. It has nothing to do with the password strength. It is the system level control method and it is usually configurable. Of course, it is an effective weapon against brute force attacks, but one can still design a system without that access control method. Also, hackers may not crack into the system directly but they could intercept the user data from the network which contains encrypted password or anything else and use brute force or other ways to crack it. So a high-strength password scheme, a high-security crypto method and a well-designed access system could live together to make a strong security system.
In general, with a brute force system, you are correct. But, a lot of automated password crackers out there begin their searches by trying common english words and their combinations. For example: sports teams, states, dates, etc etc... So by having those special characters it immediately eliminates a lot of those possibilities. Generally, if you're worried about brute force, a much longer password is more secure than a shorter one with special characters.

Keeping password length uniform internally and keeping salt secret?

When storing passwords, it's been said that Salt doesn't need to be secret and it's only purpose is to keep all Hash unique. It's also said that limiting password length is not a good practice but consider this example:
Before hashing, we make sure the plain text version is always 128 characters internally by trimming user input to max 100 then appending additional characters as our salt.
So if user inputs 20 characters, we append 108 random characters as salt. If user inputs 100, we append 28, and so on. The point is, the length of the plain text version should be 128 characters. In code it might look like this:
$salt = generate_salt($pass); // length varies as explained above
$hash = hash('sha512', $pass.$salt);
This way our "plain text" before hash will always be 128 characters.
We store $hash on Server A and store $salt on Server B.
Now let's assume the attacker gains access to hash DB (Server A) and manages to reverse the hashes. Looks good for him but the plain text version (or the reversed hashes) that he sees still looks like hashes since it's 128 characters. Since he doesn't know the salt he will never know the original password.
As an added challenge, due to the fact that SHA512 produces 128 characters he'll also never be sure if he already arrived at the plain text version since (like already mentioned) the plain text version looks like hashes. On plain sight he might think it's an iterated version, if so he'll probably continue to iterate, possibly indefinitely.
Is there any problem with this approach since in the event of hash reversal, keeping the salt secret gives extra security, and, keeping plain text length uniform arguably adds layer of obfuscation?
Note: This of course assumes your app has multiple failed login detection/prevention.
First of all, unless you really know what your doing, don't invent your own crypto system.
Use an existing one (like PBKDF2, bcrypt or scrypt), which have advantages over simple salted hashes, since they use more CPU time (all three) and/or memory (the latter two), which makes parallelization expensive.
The salt in a hash protects only against rainbow tables, not against brute-force attacks!.
If the attacker manages to reverse the hashes, he will gain knowledge of $pass.$salt (an, therefore, the password).
The purpose of a salt is to avoid an inexpensive creation of rainbow tables, that is, instead of just calculating the hashes of every probable password and comparing them to your database, an attacker has to do this for every different salt.
Keeping the salt secret has the theoretical advantage of making the attack even more expensive, since the attacker would also have to try every possible salt for every possible password.
In practice, however, he will probably be able to gain access to server B once he has access to server A.
If something looks like a hash is probably not important. Once the server is compromised, the attacker could probably find out what obfuscation techniques are used.
On a side note: SHA-512 produces 512 bits of output, which are 64 ASCII characters.
The first problem with this approach is that you're implementing your own cryptographic code. This is almost always a bad idea; crypto is hard, and very easy to screw up in subtle ways, and other people have put a lot of time and effort into implementing cryptographic primitives and services built on top of them so that you don't have to. But let's assume for the sake of argument that you really do need to do this :-).
The second thing is that you're truncating the user's input -- throwing away precious entropy -- for no benefit at all. You're prepared to generate up to 128 characters' worth of salt; why not just always do this and feed them (together with the password) into your hash? What do you gain by truncating? The only answer I see is that supposedly the original passwords "look like hashes since it's 128 characters", but that simply isn't true; your salted passwords still begin with actual password data, which typically looks very much unlike hashes.
The third thing -- which may just be a failure on my part to read your mind correctly -- is that it's not clear where these salts of yours are coming from. You describe them as "random" and say attackers won't know them; but then how does your authentication system get them? It seems like they're derived from the password, but in that case salting-plus-hashing is just a marginally more complicated hash function and you haven't really gained anything.
The general principle you've apparently rejected (or perhaps not encountered) is this: Always design cryptographic systems with the assumption that the attackers know everything they could possibly know, including all your source code. This is a good principle and you should not reject it. Relying on a small "extra layer of obfuscation" will not serve you well; in the worst case it will make bugs more likely (by complicating the system) and induce a sense of false security.
Salt doesn't need to be secret and it's only purpose is to keep all Hash unique
It also makes it 'slower' to brute-force a table of hashes, since you need a try per row for every hash + salt combination instead of just trying out one hash for the whole table (' select * from passwords where hash = 'xxx' ')
keeping the salt secret gives extra security, and, keeping plain text length uniform arguably adds layer of obfuscation?
The fact that your 'reversed hashes' look like hashes does not add real extra security (it's really just security by obscurity). Your webserver will need to connect to server A and server B (for authenticating a user / password combination) so when that server is compromised all hope is lost.
An article that may interest you is this blog post by Jeff Atwood. (edit: posted wrong codinghorror link)

Do similar passwords have similar hashes?

Our computer system at work requires users to change their password every few weeks, and you cannot have the same password as you had previously. It remembers something like 20 of your last passwords. I discovered most people simply increment a digit at the end of their password, so "thisismypassword1" becomes "thisismypassword2" then 3, 4, 5 etc.
Since all of these passwords are stored somewhere, I wondered if there was any weakness in the hashes themselves, for standard hashing algorithms used to store passwords like MD5. Could a hacker increase their chances of brute-forcing the password if they have a list of hashes of similar passwords?
With a good hash algorithm, similar passwords will get distributed across the hashes. So similar passwords will have very different hashes.
You can try this with MD5 and different strings.
"hello world" - 5eb63bbbe01eeed093cb22bb8f5acdc3
"hello world" - fd27fbb9872ba413320c606fdfb98db1
Do similar passwords have similar hashes?
No.
Any similarity, even a complex correlation, would be considered a weakness in the hash. Once discovered by the crypto community it would be published, and enough discovered weaknesses in the hash eventually add up to advice not to use that hash any more.
Of course there's no way to know whether a hash has undiscovered weaknesses, or weaknesses known to an attacker but not published, in which case most likely the attacker is a well-funded government organization. The NSA certainly is in possession of non-public theoretical attacks on some crypto components, but whether those attacks are usable is another matter. GCHQ probably is. I'd guess that a few other countries have secret crypto programs with enough mathematicians to have done original work: China would be my first guess. All you can do is act on the best available information. And if the best available information says that a hash is "good for crypto", then one of the things that means is no usable similarities of this kind.
Finally, some systems use weak hashes for passwords -- either due to ignorance by the implementer or legacy. All bets are off for the properties of a hashing scheme that either hasn't had public review, or else has been reviewed and found wanting, or else is old enough that significant weaknesses have eventually been found. MD5 is broken for some purposes (since there exist practical means to generate collisions) but not for all purposes. AFAIK it's OK for this, in the sense that there is no practical pre-image attack, and having a handful of hashes of related plaintexts is no better than having a handful of hashes of unrelated plaintexts. But for unrelated reasons you shouldn't really use a single application of any hash for password storage anyway, you should use multiple rounds.
Could a hacker increase their chances of brute-forcing the password if they have a list of hashes of similar passwords?
Indirectly, yes, knowing that those are your old passwords. Not because of any property of the hash, but suppose the attacker manages to (very slowly) brute-force one or more of your old passwords using those old hashes, and sees that in the past it has been "thisismypassword3" and "thisismypassword4".
Your password has since changed, to "thisismypassword5". Well done, by changing it before the attacker cracked it, you have successfully ensured that the attacker did not recover a valuable password! Victory! Except it does you no good, since the attacker has the means to guess the new one quickly anyway using the old password(s).
Even if the attacker only has one old password, and therefore cannot easily spot a trend, password crackers work by trying passwords which are similar to dictionary words and other values. To over-simplify a bit, it will try the dictionary words first, then strings consisting of a word with one extra character added, removed or changed, then strings with two changes, and so on.
By including your old password in the "other values", the attacker can ensure that strings very similar to it are checked early in the cracking process. So if your new password is similar to old ones, then having the old hashes does have some value to the attacker - reversing any one of them gives him a good seed to crack your current password.
So, incrementing your password regularly doesn't add much. Changing your password to something that's guessable from the old password puts your attacker in the same position as they'd be in if they knew nothing at all, but your password was guessable from nothing at all.
The main practical attacks on password systems these days are eavesdropping (via keyloggers and other malware) and phishing. Trying to reverse password hashes isn't a good percentage attack, although if an attacker has somehow got hold of an /etc/passwd file or equivalent, they will break some weak passwords that way on the average system.
It depends on the hashing algorithm. If it is any good, similar passwords should not have similar hashes.
The whole point of a cryptographic hash is that similar passwords would absolutely not create similar hashes.
More importantly, you would most likely salt the password so that even the same passwords do not produce the same hash.
It depends on the hash algorithm used. A good one will distribute similiar inputs to disparate outputs.
Different Inputs may result in the same Hash this is what is called a hash collision.
Check here:
http://en.wikipedia.org/wiki/Collision_%28computer_science%29
Hash colisions may be used to increase chances of a successfull brute force attack, see:
http://en.wikipedia.org/wiki/Birthday_attack
To expand on what others have said, a quick test shows that you get vastly different hashes with small changes made to the input.
I used the following code to run a quick test:
<?php
for($i=0;$i<5;$i++)
echo 'password' . $i . ' - ' .md5('password' . $i) . "<br />\n";
?>
and I got the following results:
password0 - 305e4f55ce823e111a46a9d500bcb86c
password1 - 7c6a180b36896a0a8c02787eeafb0e4c
password2 - 6cb75f652a9b52798eb6cf2201057c73
password3 - 819b0643d6b89dc9b579fdfc9094f28e
password4 - 34cc93ece0ba9e3f6f235d4af979b16c
Short answer, no!
The output of a hash function varies greatly even if one character is increased.
But this is only if you want to break the hashfunction itself.
Of course, it is bad practice since it makes bruteforcing easier.
No, if you check the password even slightly it produces completely new hash.
As a general rule, a "good hash" will not hash two similar (but unequal) strings to similar hashes. MD5 is good enough that this isn't a problem. However, there are "rainbow tables" (essentially password:hash pairs) for quite a few common passwords (and for some password hashes, the traditional DES-based unix passwords, for example) full rainbow tables exist.

How to store passwords *correctly*?

An article that I stumbled upon here in SO provided links to other articles which in turn provided links to even more articles etc.
And in the end I was left completely stumped - so what is the best way to store passwords in the DB? From what I can put together you should:
Use a long (at least 128 fully random bits) salt, which is stored in plaintext next to the password;
Use several iterations of SHA-256 (or even greater SHA level) on the salted password.
But... the more I read about cryptography the more I understand that I don't really understand anything, and that things I had thought to be true for years are actually are flat out wrong. Are there any experts on the subject here?
Added: Seems that some people are missing the point. I repeat the last link given above. That should clarify my concerns.
https://www.nccgroup.trust/us/about-us/newsroom-and-events/blog/2007/july/enough-with-the-rainbow-tables-what-you-need-to-know-about-secure-password-schemes/
You got it right. Only two suggestions:
If one day SHA1 becomes too weak and you want to use something else, it is impossible to unhash the old passwords and rehash them with the new scheme. For this reason, I suggest that attached to each password a "version" number that tells you what scheme you used (salt length, which hash, how many times). If one day you need to switch from SHA to something stronger, you can create new-style passwords while still having old-style passwords in the database and still tell them apart. Migrating users to the new scheme will be easier.
Passwords still go from user to system without encryption. Look at SRP if that's a problem. SRP is so new that you should be a little paranoid about implementing it, but so far it looks promising.
Edit: Turns out bcrypt beat me to it on idea number 1. The stored info is (cost, salt, hash), where cost is how many times the hashing has been done. Looks like bcrypt did something right. Increasing the number of times that you hash can be done without user intervention.
In truth it depends on what the passwords are for. You should take storing any password with care, but sometimes much greater care is needed than others. As a general rule all passwords should be hashed and each password should have a unique salt.
Really, salts don't need to be that complex, even small ones can cause a real nightmare for crackers trying to gain entry into the system. They are added to a password to prevent the use of Rainbow tables to hack multiple account's passwords. I wouldn't add a single letter of the alphabet to a password and call it a salt, but you don't need to make it a unique guid which is encrypted somewhere else in the database either.
One other thing concerning salts. The key to making a password + salt work when hashing is the complexity of the combination of the two. If you have a 12 character password and add a 1 character salt to it, the salt doesn't do much, but cracking the password is still a monumental feat. The reverse is also true.
Use:
Hashed password storage
A 128+ bit user-level salt, random, regenerated (i.e. you make new salts when you make new password hashes, you don't persistently keep the same salt for a given user)
A strong, computationally expensive hashing method
Methodology that is somewhat different (hash algorithm, how many hashing iterations you use, what order the salts are concatenated in, something) from both any 'standard implementation guides' like these and from any other password storage implementation you've written
I think there no extra iteration on the password needed, juste make sure there is a salt, and a complexe one ;)
I personnaly use SHA-1 combined with 2 salt keyphrases.
The length of the salt doesnt really matter, as long as it is unique to a user. The reason for a salt is so that a given generated attempt at a hash match is only useful for a single row of your users table in the DB.
Simply said, use a cryptographically secure hash algorithm and some salt for the passwords, that should be good enough for 99.99% of all use cases. The weak link will be the code that checks the password as well as the password input.

Should I impose a maximum length on passwords?

I can understand that imposing a minimum length on passwords makes a lot of sense (to save users from themselves), but my bank has a requirement that passwords are between 6 and 8 characters long, and I started wondering...
Wouldn't this just make it easier for brute force attacks? (Bad)
Does this imply that my password is being stored unencrypted? (Bad)
If someone with (hopefully) some good IT security professionals working for them are imposing a max password length, should I think about doing similar? What are the pros/cons of this?
Passwords are hashed to 32, 40, 128, whatever length. The only reason for a minimum length is to prevent easy to guess passwords. There is no purpose for a maximum length.
The obligatory XKCD explaining why you're doing your user a disservice if you impose a max length:
A maximum length specified on a password field should be read as a SECURITY WARNING. Any sensible, security conscious user must assume the worst and expect that this site is storing your password literally (i.e. not hashed, as explained by epochwolf).
In that that is the case:
Avoid using this site like the plague if possible. They obviously know nothing about security.
If you truly must use the site, make sure your password is unique - unlike any password you use elsewhere.
If you are developing a site that accepts passwords, do not put a silly password limit, unless you want to get tarred with the same brush.
[Internally, of course your code may treat only the first 256/1024/2k/4k/(whatever) bytes as "significant", in order to avoid crunching on mammoth passwords.]
Allowing for completely unbounded password length has one major drawback if you accept the password from untrusted sources.
The sender could try to give you such a long password that it results in a denial of service for other people. For example, if the password is 1GB of data and you spend all your time accept it until you run out of memory. Now suppose this person sends you this password as many times as you are willing to accept. If you're not careful about the other parameters involved this could lead to a DoS attack.
Setting the upper bound to something like 256 chars seems overly generous by today's standards.
First, do not assume that banks have good IT security professionals working for them. Plenty don't.
That said, maximum password length is worthless. It often requires users to create a new password (arguments about the value of using different passwords on every site aside for the moment), which increases the likelihood they will just write them down. It also greatly increases the susceptibility to attack, by any vector from brute force to social engineering.
Setting maximum password length less than 128 characters is now discouraged by OWASP Authentication Cheat Sheet
https://www.owasp.org/index.php/Authentication_Cheat_Sheet
Citing the whole paragraph:
Longer passwords provide a greater combination of characters and consequently make it more difficult for an attacker to guess.
Minimum length of the passwords should be enforced by the application.
Passwords shorter than 10 characters are considered to be weak ([1]).
While minimum length enforcement may cause problems with memorizing passwords among some users, applications should encourage them to set passphrases (sentences or combination of words) that can be much longer than typical passwords and yet much easier to remember.
Maximum password length should not be set too low, as it will prevent users from creating passphrases. Typical maximum length is 128 characters.
Passphrases shorter than 20 characters are usually considered weak if they only consist of lower case Latin characters. Every character counts!!
Make sure that every character the user types in is actually included in the password. We've seen systems that truncate the password at a length shorter than what the user provided (e.g., truncated at 15 characters when they entered 20).
This is usually handled by setting the length of ALL password input fields to be exactly the same length as the maximum length password. This is particularly important if your max password length is short, like 20-30 characters.
One reason I can imagine for enforcing a maximum password length is if the frontend must interface with many legacy system backends, one of which itself enforces a maximum password length.
Another thinking process might be that if a user is forced to go with a short password they're more likely to invent random gibberish than an easily guessed (by their friends/family) catch-phrase or nickname. This approach is of course only effective if the frontend enforces mixing numbers/letters and rejects passwords which have any dictionary words, including words written in l33t-speak.
One potentially valid reason to impose some maximum password length is that the process of hashing it (due to the use of a slow hashing function such as bcrypt) takes up too much time; something that could be abused in order to execute a DOS attack against the server.
Then again, servers should be configured to automatically drop request handlers that take too long. So I doubt this would be much of a problem.
I think you're very right on both bullet points. If they're storing the passwords hashed, as they should, then password length doesn't affect their DB schema whatsoever. Having an open-ended password length throws in one more variable that a brute-force attacker has to account for.
It's hard to see any excuse for limiting password length, besides bad design.
The only benefit I can see to a maximum password length would be to eliminate the risk of a buffer overflow attack caused by an overly long password, but there are much better ways to handle that situation.
Ignore the people saying not to validate long passwords. Owasp literally says that 128 chars should be enough. Just to give enough breath space you can give a bit more say 300, 250, 500 if you feel like it.
https://www.owasp.org/index.php/Authentication_Cheat_Sheet#Password_Length
Password Length Longer passwords provide a greater combination of
characters and consequently make it more difficult for an attacker to
guess.
...
Maximum password length should not be set too low, as it will prevent
users from creating passphrases. Typical maximum length is 128
characters. Passphrases shorter than 20 characters are usually
considered weak if they only consist of lower case Latin characters.
My bank does this too. It used to allow any password, and I had a 20 character one. One day I changed it, and lo and behold it gave me a maximum of 8, and had cut out non-alphanumeric characters which were in my old password. Didn't make any sense to me.
All the back-end systems at the bank worked before when I was using my 20 char password with non alpha-numerics, so legacy support can't have been the reason. And even if it was, they should still allow you to have arbitrary passwords, and then make a hash that fits the requirements of the legacy systems. Better still, they should fix the legacy systems.
A smart card solution would not go well with me. I already have too many cards as it is... I don't need another gimmick.
If you accept an arbitrary sized password then one assumes that it is getting truncated to a curtain length for performance reasons before it is hashed. The issue with truncation is that as your server performance increases over time you can't easily increase the length before truncation as its hash would clearly be different. Of course you could have a transition period where both lengths are hashed and checked but this uses more resources.
Try not to impose any limitation unless necessary. Be warned: it might and will be necessary in a lot of different cases. Dealing with legacy systems is one of these reasons. Make sure you test the case of very long passwords well (can your system deal with 10MB long passwords?). You can run into Denial of Service (DoS) problems because the Key Defivation Functions (KDF) you will be using (usually PBKDF2, bcrypt, scrypt) will take to much time and resources. Real life example: http://arstechnica.com/security/2013/09/long-passwords-are-good-but-too-much-length-can-be-bad-for-security/
In .net core 6 I use HashPasswordV3 method that it use HMACSHA512 with 1000 iterations. I tested some password length and it generate a 86 characters hash.
So I set the PasswordHash field in sql server for varchar(100).
https://stackoverflow.com/a/72429730/9875486
Storage is cheap, why limit the password length. Even if you're encrypting the password as opposed to just hashing it a 64 character string isn't going to take much more than a 6 character string to encrypt.
Chances are the bank system is overlaying an older system so they were only able to allow a certain amount of space for the password.
Should there be a maximum length? This is a curious topic in IT in that, longer passwords are typically harder to remember, and therefore more likely to get written down (a BIG no-no for obvious reasons). Longer passwords also tend to get forgotten more, which while not necessarily a security risk, can lead to administrative hassles, lost productivity, etc. Admins who believe that these issues are pressing are likely to impose maximum lengths on passwords.
I personally believe on this specific issue, to each user their own. If you think you can remember a 40 character password, then all the more power to you!
Having said that though, passwords are fast becoming an outdated mode of security, Smart Cards and certificate authentication prove very difficult to impossible to brute force as you stated is an issue, and only a public key need be stored on the server end with the private key on your card/computer at all times.
Longer passwords, or pass-phrases, are harder to crack simply based on length, and easier to remember than requiring a complex password.
Probably best to go for a fairly long (10+) minimum length, restricting the length useless.
Legacy systems (mentioned already) or interfacing outside vendor's systems might necessitate the 8 character cap. It could also be a misguided attempt to save the users from themselves. Limiting it in that fashion will result in too many pssw0rd1, pssw0rd2, etc. passwords in the system.
One reason passwords may not be hashed is the authentication algorithm used. For example, some digest algorithms require a plaintext version of the password at the server as the authentication mechanism involves both the client and the server performing the same maths on the entered password (which generally won't produce the same output each time as the password is combined with a randomly generated 'nonce', which is shared between the two machines).
Often this can be strengthened as the digest can be part computed in some cases, but not always. A better route is for the password to be stored with reversible encryption - this then means the application sources need to be protected as they'll contain the encryption key.
Digst auth is there to allow authentication over otherwise non-encrypted channels. If using SSL or some other full-channel encryption, then there's no need to use digest auth mechanisms, meaning passwords can be stored hashed instead (as passwords could be sent plaintext over the wire safely (for a given value of safe).
Microsoft publishes security recommendations for developers based on their internal data (you know, from running the biggest software enterprise in the history of computing) and you can find these PDFs online. Microsoft has said that not only is password cracking near the least of their security concerns but that:
“Criminals attempt to victimize our customers in various ways and
we’ve found the vast majority of attacks are through phishing, malware
infected machines, and the reuse of passwords on third-party
sites—none of which are helped by very long passwords." -Microsoft
Microsoft's own practice is that passwords can be no longer than 16 and no shorter than 8 characters.
https://arstechnica.com/information-technology/2013/04/why-your-password-cant-have-symbols-or-be-longer-than-16-characters/#:~:text=Microsoft%20imposes%20a%20length%20limit,no%20shorter%20than%20eight%20characters.
I found using the same characters for the first 72 bytes of a password gives a successful verification using password_hash() and password_verify() in PHP, no matter what random string comes after the first 72 bytes.
From PHP docs: https://www.php.net/manual/en/function.password-hash.php
Caution: Using the PASSWORD_BCRYPT as the algorithm, will result in the password parameter being truncated to a maximum length of 72 bytes.
Recent Updates from OWASP now recommend a max length:
https://cheatsheetseries.owasp.org/cheatsheets/Authentication_Cheat_Sheet.html
Maximum password length should not be set too low, as it will prevent users from creating passphrases. A common maximum length is 64 characters due to limitations in certain hashing algorithms, as discussed in the Password Storage Cheat Sheet. It is important to set a maximum password length to prevent long password Denial of Service attacks.
Just 8 char long passwords sound simply wrong. If there ought to be a limit, then atleast 20 char is better idea.
I think the only limit that should be applied is like a 2000 letter limit, or something else insainly high, but only to limit the database size if that is an issue

Resources