I am trying to stop spam accounts from being created on my website. I run a website that has approximately 50-80k pageviews per month. It's a social media website. Users sign up and communicate with one another for free. We've been battling with spam as of late even though we have implemented multiple security measures to counteract bots. I'd like to get any further suggestions of tips and tricks that I can try and also some help to see if I can identify if these are people coming from clickfarms, etc. (i.e. real people or computers)
Problem:
Signup form being completed and users posting spam in their profile information. Spammer signs up for the website by completing the signup form, activates their account via an email account, Logs into their account, and then completes their profile, putting spam in the description box with a link/url to their website they are advertising (everything from ##$%S enlargment to random blogs, to web developer websites, etc.) If there was one link they were posting we could detect it and ban them but they are not -- They are coming from multiple IP's, posting various links, using multiple email provider addresses for activating the accounts, registering with information from multiple countries, and creating about 10-30 accounts per day. Before implementing many security measures we were getting moreso around 100-200 fake accounts per day, but now we're down to 10-30 ... so we've seen some improvement, but the issue is still annoying me. So I'm half thinking now that the security measures are helping quite a bit, but that this is possibly humans still targeting our website and perhaps getting paid per signup they do or something similar to that. Even if so, is there any way I could confirm they are humans versus bots?
Security measures:
I won't get into all of the details here (for security reasons), but I'll just indicate what we've done to counteract the spambots:
Created honeypots at various areas of our website which automatically ban based on IP
IP banning - based on known botter/spammer ip addresses
Duration detection of signup form pageload to form submission -- if less than 5 seconds to complete our signup form, we're confirming you're a bot and then preventing the signup
Hidden checkbox in signup form -- there is a hidden checkbox in the signup form that is invisible to regular users (if a bot checks it we are automatically detecting and preventing the signup)
Google re-Captcha - We've enabled Google re-Captcha in our signup form as well
Email activation link - We send our users an activation email with a link that they have to click on to signup -- they are not able to sign into our website until they've activated their account.
Future actions include:
Detecting what users are posting in their descriptions in their profiles and banning based on that -- string detection for banned words, etc.
Any other suggestions or tips or tricks? In all honesty, if spam bots are getting through all of those security measures above --
do you think they are just that intelligent?
do you think we're being targeted?
Also, any way I can determine if they are bots or real humans? Suggestions?
This is a perennial problem; over the years I've found that as I add more anti-spam measures, the spammers continually get better at circumventing my measures.
I recommend doing an analysis of your spam to figure out how you can detect it. The spam itself contains the key to how to outsmart it. Look at the patterns, the structure, and decide what information is most useful and how the easiest way is to filter it out. Your spam detection doesn't need to be perfect, but generally, you want to get as much as possible, while getting as few false positives as possible.
Also, to answer your one question, you can make your bot-detection perfect, but there will always be humans submitting spam. And humans are tough to outsmart, and you may always need some manual attention to do it.
You are already implementing a lot of measures. Here are some more I would suggest:
When a signup form is generated, put a hidden field with a unique hash generated from the user's browser info, including the user's IP, HTTP user agent, and the date. Then, when the form is submitted, check the hash. This one method eliminated a surprising amount of spam.
If you want to take the previous method even farther, use a custom, time-sensitive hash in the URL of your contact form, and have the link to this form be dynamically generated. This way, if a spammer stores the form's URL, it won't work, but the link will work for every legitimate user of the site.
Make it so newly created, non-trusted users, cannot display any public profile information, such as URL's or text even. With a site as small as yours you could require manual approval of each user, and if your userbase got bigger, you could use an automated reputation system, a lot like Stack Overflow and the other Stack Exchange sites use. This removes the incentive for spam. Also, I found an overwhelming majority of spammers only ever logged onto the site once. If you wait to do the manual approval of users, until they have logged on twice, or even have returned to the site on another day, using a persistent cookie, you will filter out the vast majority of spammers and you will only have to do a small amount of manual approval work. Then have the system delete the unvalidated/inactive accounts after a certain amount of time.
Check for certain keywords or structure of info. I found an overwhelming majority of my spammers would use certain words or phrases that were never used by my legitimate users. Another one was entering a phone number in their profile, a common pattern in spammers, that no legitimate user ever did. Also look for signs of foul play like XSS attacks. A huge portion of spammers will, at some point, submit something that has a ton of HTML tags in it, you can either use the tags itself to filter them out, or you can do something like stripping the HTML tags and then comparing string length and banning them if it's more than a small amount (i.e. allow someone to do something simple like a few <em></em> or <strong></strong> tags.) Usually, if there are HTML tags in the entry, there's a ton of it. Also look for material with weird encodings or characters that don't make sense. This is often an attempt at sophisticated SQL injection attacks, XSS, or other types of hacking attempts.
Use external IP blacklists. AbuseIPDB is one example; it has an API that you can use to check new IP's before storing them in your temporary database. Their free plan allows checking of up to 1000 IP's a day and you can pay for more than that. It won't catch all the manual spam but I find they catch a ton of the automated spam.
Are they targeting you? Yes. They are targeting everyone. But any site with 50k+ pageviews a month is high enough volume to be an attractive target. The higher traffic you get, the more attractive of a target you will be. Even some of my tiny sites have been targeted with suprisingly sophisticated attacks these days. Everyone needs to be on guard.
Good luck. I wish this weren't so much of a problem, but it is.
I know this question is crazy - but my employers client is demanding that email verification be removed from the sign up process (they feel it is impeding sign up).
I wanted to garner feedback from the programming community at large as to their experience and opinions regarding sign up and email verification - and the possible consequences of removing this safeguard.
I'm on their side -- 95% of the time websites don't actually need an e-mail address, they just collect it because all the other web registrations they've seen collect one. If you're worried about spam, use a captcha; e-mails are a horrible way to stop automated registrations. With sites like Mailinator to give people instant throwaway e-mails and BugMeNot to save people the hassle of dealing with registrations like yours, you should avoid making your registration any harder than it needs to be. Stack Overflow is a great example -- you don't even need to register to ask/answer questions
My guess is that robots will not bother going through a registration process. Your average simple-minded robot simply spams into a form that requires no other action (authentication, identification) at all. The mere act of asking for one or more extra clicks will prevent most simple-minded "attacks." If you look at the blog site for Coding Horror, they use a captcha with a constant capture word.
On the other hand, while a few extra clicks will deter dumb robots, they will not deter human spammers, jokers, griefers, etc. But then again, throwaway email addresses are pretty easy to come by, so if someone truly wants to fill your site with junk they can.
My conclusion is this: I guess you will get about 10% to 20% more "junk" on your pages, and between 5% and 25% more "desired" accesses, depending on how badly it was bothering your potential customers. Thus, I don't see any big harm in removing the email barrier.
Email is important to identify the user, for instance, when they forget their password. If email verification is setup in such a way that users are not able to log on until they verify their email address then I also think that it is impeding. The application should allow the user to log on and use the application and set it up so that the user needs to verify their email address in a fixed number of days, for example, a week. If they do not their account is suspended.
On the other hand if we have to remove the email verification then I think we would need to add a feature similar to the major email services that allow the user to reset their forgotten passwords in the absence of a valid email address.
I have an online registry of professionals with about 300 members. These are smart people, but non technical. Currently, if somebody forgets their email address, the system resends it to the email address they registered with.
The problem is that people change their email addresses over time, then forget their password, and can't receive the reminder.
I need to come up with a simple authentication system that allows people to recover their passwords even if they have changed email address.
I'm struggling to come up with anything that is even moderately secure that doesn't require the users email address.
Can anyone suggest anything?
Keep their mobile numbers for SMSing-- those might change less often or at least not in tandem with email addresses.
Also consider handling this case via manual support if the user base is only 300; but if you do so, don't forget to be diligent in whatever your manual verification method is. :)
The most common practice would be to introduce additional questions with registered answers, that would allow a user to reset their email address and password. (Though only one at a time and the second only after verification of the first).
For instance
In What City did you grow up?
Where did you go to College.
Usually you would have a stack of questions, and let the user select 3 questions and you registered their answers. The key being not to ask the same 3 questions of everyone.
i'd like to prevent bots from hacking weak password-protected accounts. (e.g. this happend to ebay and other big sites)
So i'll set a (mem-) cached value with the ip, amount of tries and timestamp of last try (memcache-fall-out).
But what about bots trying to open any account with just one password. For example, the bot tries all 500.000 Useraccounts with the password "password123". Maybe 10 will open.
So my attempt was to just cache the ip with tries and set max-tries to ~50. The i would delete it after a successful login. So the good-bot would just login with a valid account every 49 tries to reset the lock.
Is there any way to do it right?
What do big platforms do about this?
What can i do to prevent idiots from blocking all users on a proxy with retrying 50 times?
If there is no best practice - does this mean any platform is brute-forceable? At least with a hint on when counters are resetted?
I think you can mix your solution with captchas:
Count the number of tries per IP
In case there are too many tries from a given IP address within a given time, add a captcha to your login form.
Some sites give you maybe two or three tries before they start making you enter a captcha along with your username/password. The captcha goes away once you successfully log in.
There was a relatively good article on Coding Horror a few days ago.
While the code is focused on Django there is some really good discussion on the best practice methods on Simon Willison’s blog. He uses memcached to track IPs and login failures.
You could use a password strength checker when a user sets their password to make sure they're not using an easily brute-forced password.
EDIT: Just to be clear, this shouldn't be seen as a complete solution to the problem you're trying to solve, but it should be considered in conjunction with some of the other answers.
You're never going to be able to prevent a group of bots from trying this from lots of different IP addresses.
From the same IP address: I would say if you see an example of "suspicious" behavior (invalid username, or several valid accounts with incorrect login attempts), just block the login for a few seconds. If it's a legitimate user, they won't mind waiting a few seconds. If it's a bot this will slow them down to the point of being impractical. If you continue to see the behavior from the IP address, just block them -- but leave an out-of-band door for legitimate users (call phone #x, or email this address).
PLEASE NOTE: IP addresses can be shared among THOUSANDS or even MILLIONS of users!!! For example, most/all AOL users appear as a very small set of IP addresses due to AOL's network architecture. Most ISPs map their large user bases to a small set of public IP addresses.
You cannot assume that an IP address belongs to only a single user.
You cannot assume that a single user will be using only a single IP address.
Check the following question discussing best practices against distibuted brute force and dictionary attacks:
What is the best Distributed Brute Force countermeasure?
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
From Wired magazine:
...the Palin hack didn't require any
real skill. Instead, the hacker simply
reset Palin's password using her
birthdate, ZIP code and information
about where she met her spouse -- the
security question on her Yahoo
account, which was answered (Wasilla
High) by a simple Google search.
We cannot trust such security questions to reset forgotten passwords.
How do you design a better system?
The insecurity of so-called "security questions" has been known for a long time. As Bruce Schneier puts it:
The result is the normal security protocol (passwords) falls back to a much less secure protocol (secret questions). And the security of the entire system suffers.
What can one do? My usual technique is to type a completely random answer -- I madly slap at my keyboard for a few seconds -- and then forget about it. This ensures that some attacker can't bypass my password and try to guess the answer to my secret question, but is pretty unpleasant if I forget my password. The one time this happened to me, I had to call the company to get my password and question reset. (Honestly, I don't remember how I authenticated myself to the customer service rep at the other end of the phone line.)
I think the better technique is to just send an e-mail with a link they can use to generate a new random password to the e-mail account the user originally used to register. If they didn't request a new password, they can just ignore it and keep using their old one. As others have pointed out, this wouldn't necessarily have helped Yahoo, since they were running an e-mail service, but for most other services e-mail is a decent authentication measure (in effect, you foist the authentication problem off on the user's e-mail provider).
Of course, you could just use OpenID.
Out-of-band communication is the way to go.
For instance, sending a temporary password in SMS may be acceptable (depending on the system). I've seen this implemented often by telecoms, where SMS is cheap/free/part of business, and the user's cellphone number is pre-registered...
Banks often require a phone call to/from a specific number, but I personally am not too crazy about that....
And of course, depending on the system, forcing the user to come in to the branch office to personally identify themselves can also work (just royally annoy the user).
Bottom line, DON'T create a weaker channel to bypass the strong password requirements.
Having seen a lot of posters suggest email, all I can suggest is DONT use email as your line of defense.
Compromising somebodys email account can be relatively easy. Many web based email services DONT provide any real security either, and even if they offer SSL, its often not default and you are still relying on the weakness of the email password to protect the user ( Which, in turn has a reset mechanism most the time ).
Email is one of the most insecure technologies, and there are good reasons why its a really bad idea to send information like credit card details over them. They're usually transmitted between servers in plaintext, and equally often, between server and desktop client equally unencrypted, and all it takes is a wire sniff to get the reset url and trigger it. ( Don't say I'm paranoid, because banks use SSL encryption for a good reason. How can you trust the 20-200 physical devices on the route have good intentions? )
Once you get the reset data, you can reset the password, and then change your(their) email address, and have permanent control of their account ( it happens all the time ).
And if they get your email account, all they have to do is have a browse through your inbox to find whom you're subscribed with, and then easily reset the password ON ALL OF THEM
So now, using the email based security, can lead to a propogative security weakness!. I'm sure thats beneficial!.
The question being asked Is one I figure is almost impossible to do with software alone. This is why we have 2-factor authentication with hardware dongles that respond to challenges with their own unique private key signature, and only if you lose that are you screwed, and you then have to deal with a human ( oh no ) to get a new one.
It 'depends' on the 'system'.
If you are a Bank or a credit card provider, you have already issued
some physical token to your customer that you can validate against and more.
If you are an ecommerce site, you ask for some recent transactions
-exact amounts, credit card number used et al..
If you are like Yahoo, an automated approach I would use is to send an
activation code via either a phone call or a text message to the cell
phone along with some other basic question and answers.
Jay
Do away with the (in)security questions completely. They're such an obvious security hole that I'm actually a bit surprised that it's taken this long for them to create a serious (well, highly-publicized) incident.
Until they disappear, I'm just going to keep on telling websites which use them that I went to "n4weu6vyeli4u5t" high school...
Have the user enter 3 questions and answers. When they request a reset present them with a drop down of 5 questions, one if which is a random one from the 3 they entered. Then send a confirmation email to actually reset the password.
Of course, nothing is going to be truly "hacker proof".
When users are involved (and mostly when not, too) there is no security; there is only the illusion of security. There's not a lot you can do about it. You could have 'less common' security questions but even they are prone to exploitation since some people put everything out in the public eye.
Secondary channels like email offer a reasonable solution to the problem. If the user requests a password reset you can email them a password reset token. Still not perfect, as others have said, but exploiting this would require the attacker to be somewhere in the line of sight between the website, its MTA and the users MUA. It's technically easy but I suggest that the reality is it's just too much work/risk for them to bother on anyone except very high profile individuals.
Requiring the user to supply SSL or GPG public keys at account creation time will help enormously, but clueless users won't know what those things are let-alone be able to keep their private keys secure and backed up so they don't lose them.
Asking the user to supply a second emergency password (kind of like PIN/PUK on mobile phone SIM cards) could help but it's likely the user would use the same password twice or forget the second password too.
Short answer, you're S.O.L unless you want to educate your users on security and then hit them with a cluestick until they realise that it is necessary to be secure and the slight amount of extra work is not simply there to be a pain in the arse.
Authenticating everything by sending emails is a reasonably effective solution. (although, that might not have been workable for Yahoo in this case :)).
Rather than messing about with security questions or other means to recover passwords, simply respond to password recover requests by sending an email to a predefined email account with an authorisation link. From there you can change passwords, or whatever you need to do (never SEND the password though - you should always store it as a salted hash anyway, always change it. Then if the email account has ben compromised, at least there's some indication to the user that their other services have been accessed)
The true answer is, there isn't a fool proof way to keep hackers out. I hate security questions, but if your going to use them, allow for user defined security questions. As a user, if I must have a security question on a site to set up an account, I really like having the ability to setup my own security question to allow me to ask something that only I know how to answer. It doesn't even have to be a real question in this case. But a users account is then as secure as the stupidity of the user, and the fact that many users will use something like "question?" and "answer!" or something equally dumb. You can't save users from their own stupidity.
Treating these security questions as something actually being two-factor authentication is totally misleading. From spurious items read before, when certain (banks) sites were required to have "two-factor authentication" they started implementing this as a cheap way to do it. Bruce Schneier talked about this a [while back][1].
Multiple factors are best things that are not-the-same. It should not be all things you "know" but something you know and something you have, etc. This is where the hardware authentication tokens, smart cards, and other such devices come into play.
[1]: http://www.schneier.com/blog/archives/2005/03/the_failure_of.html The Failure of Two-Factor Authentication
when its not an email system, email them a link to a secure page, with a hash that must come back in the query string to reset password.
Then if someone tried to reset your password, you would know, and they wouldn't be able to guess the hash potentially.
We use 2 guids multiplied together, represented as hex.
Well for one it should not directly reset the password but send an email with a link to reset the password. That way she would have got the email and known that it was not her who initiated the reset, and that her question / answer had been compromised.
In the case where the email address is no longer valid, it should wait for a timeout ( few days or a week ) before allowing a new email to be attached to an account.
Send a message to a different e-mail account, or text their cell phone, or call them, or send a snail-mail message. Anything that doesn't involve matters of public record or preferences that may change at any time.
Good security questions are a misnomer. They actually create a vulnerability into a system. We should call them in-secure questions. However, recognizing the risk and value they provide, "good" security questions should have 4 characteristics:
1. cannot be easily guessed or researched (safe),
2. doesn't change over time (stable),
3. is memorable,
4. is definitive or simple.
You can read more about this at http://www.goodsecurityquestions.com.
Here's a list of good, fair, and poor security questions.
IMO Secret questions should only be used as a very weak control with a time limit as part of a system.
Ex: Password reset system.
You are authenticated. Registrate your mobile phone number and your secret(not so secret) answer.
You forget your password.
You request to unlock it.
a) Your "Not so secret" question asks you for the "not so secret answer".
b) If correct, a text message is sent to the pre registrated phone.
This way, if your phone gets stolen and also, controls like pin/lock on the phone is not working. You still will have a measure of obfuscation for the attacker to get to reset the password until the time it is reported the phone is lost/stolen and can be disabled.
This usage is what i think the only purpose at all for the "not so secret" questions/answers.
So i would argue there is a place in this world for them and that usually a system needs to be the discussion.
Only provide questions that aren't on the public record.
always send the password reset to a registered email account (which is tricky for an email account) or send a PIN number to a registerd mobile phone, or a link to a IM address, etc - basically, capture some secondary contact information on registration and use it to send a 'password reset' link.
Never let anyone change their password directly, always make sure they go through an additional step.
I prefer to keep things simple and use an honor system approach. For example I'll present the user with something like,
Is this really you? Select: Yes or No.
How about requesting the users to enter their own security question and answer, and a secondary email (not the one where the password reset link is sent). Store the security question and answer hashed in the database for that extra step of security.
If the user forgets his/her password, send the password reset link to the user's primary email.
User then clicks on the link which redirects and asks for the security question and answer. If this step is successful then allow the user to reset his/her password. If the user forgets the security question/answer send a link to reset the security question/answer to the user's secondary email.
If the attacker gets access to one of the emails, it will still be useless without access to the other (very unlikely the attacker can get access to both). I know this process needs a lot of extra work on both the developers and users, but I think it is worth it. (Maybe we could give the users a recommended option to activate the security question/answer if they need this extra bit of security.)
Bottom line is that how strong or weak this system works will depends heavily on the user. The strength of the security question/answer and how well the two emails are "untied" (that is, there is no way of gaining access to one email through the other) will decide this systems strength.
I don't know if there are any problems with this way of doing it, but if any, I'd be happy if anyone could point those out :)
Generate a hash that contains the person's username and password and send it over Https to the user as a file. The user saves the file to disk. It is their responsibility to store this file in a secure location. Alternatively you can send it to their email address but this will result in less security. If the user forgets their login credentials they must then upload this file. Once the server verifies the username and password, they are then presented with a dialog to alter their password.
Due to the evolution of social media, security questions asked by websites are too easy to crack. Since most of the questions are personal information which is easily available on social media platforms one or another. One of the alternatives to avoid account hacking is to make password rules strict for login like adding special characters, numerical, capital letters etc. These kind of passwords are hard to decode and can enhance the security to a great extent.
But there are new alternative methods like multi-factor authentication, passwordless login, SMS authentication etc. SMS authentication is part of multi-factor authentication where a user is provided with an OTP on his/her cellphone which he/she need to enter in order to log in to a website. This is a secure way since the access of mobile is limited to the user only(mostly). Another multi-factor authentication method is sending a verification link to email to complete the signing in process. There is a very well written blog on this topic on Medium that explains this concept in a detailed manner.