This question already has answers here:
Is it worth obfuscating email addresses on the web these days?
(7 answers)
Closed 8 years ago.
I wish to display my email on my website, however I do not want scrappers to collect it. In other words I do not want to open the spam flood gates. Is there an effective way of leaving my email visually intact while thwarting email scrapping bots?
Several solutions:
Wrap it into an image. Make sure you do something with the image in order bots not to be able to read it using OCR.
Encode it using your custom encryption algorithm, store the encrypted text in the web page, then at page load time, use JavaScript to decode it. This might not be very secure as some bots may be able to interpret JavaScript.
One of the best options is to require the user to type in a CAPTCHA code and only then redirect him/her to a separate page with your e-mail address shown.
It is very convenient for visitors to have mailto:mail#example.com so they can click on it..
So consider other options as
- publishing email address as is but using address with Google's Gmail (provides very good spam filtering)
- placing "contact" form in addition to email address (but the form should be protected from spammers with captcha or some additional trick to prove as a human)
For a well thought out solution check out reCAPTCHA Mailhide. If you are willing to go with less security you could try writing your email address out like my name at domain dot com or something like that although spammers can easily write software to scrape these (and probably already have given the growing popularity of this approach).
Related
For legal reasons we can't send the email to HIBP in clear text.
Regarding "Domain Search" functionnality, there's no API (as far as I know).
It works by sending you multiple emails, no API, so hard to automate process around that.
I've also read one of Troy's old blog post about a Domain Search callback system he put in place for a specific case : https://www.troyhunt.com/have-i-been-pwned-goes-little-bit/ which looks good but no publicly available (that's my understanding).
So, if HIBP doesn't allow me to do that, I thought of other solutions based on it :
Firefox Monitor and 1pwd :
They use the k-anonymity principle (https://blog.mozilla.org/security/2018/06/25/scanning-breached-accounts-k-anonymity/) which consist to send only a hash of the searched email.
Unfortunately, that functionality is reserved to 1pwd & firefox monitor.
Am I missing a way to interact with HIBP without sending in clear the email ?
Thank you
Short answer: no.
Mozilla and 1Password use the k-anonymity model described here: https://www.troyhunt.com/were-baking-have-i-been-pwned-into-firefox-and-1password/
The reason I don't make that generally available is that every single k-anonymity search returns multiple results which would make it much easier for a nefarious party to abuse. Domain searches require verification of control at the time of search; an API that didn't require verification would also be open to misuse.
If you'd like to see new features, suggest them here (or vote on them if they already exist): https://haveibeenpwned.uservoice.com/
I am trying to stop spam accounts from being created on my website. I run a website that has approximately 50-80k pageviews per month. It's a social media website. Users sign up and communicate with one another for free. We've been battling with spam as of late even though we have implemented multiple security measures to counteract bots. I'd like to get any further suggestions of tips and tricks that I can try and also some help to see if I can identify if these are people coming from clickfarms, etc. (i.e. real people or computers)
Problem:
Signup form being completed and users posting spam in their profile information. Spammer signs up for the website by completing the signup form, activates their account via an email account, Logs into their account, and then completes their profile, putting spam in the description box with a link/url to their website they are advertising (everything from ##$%S enlargment to random blogs, to web developer websites, etc.) If there was one link they were posting we could detect it and ban them but they are not -- They are coming from multiple IP's, posting various links, using multiple email provider addresses for activating the accounts, registering with information from multiple countries, and creating about 10-30 accounts per day. Before implementing many security measures we were getting moreso around 100-200 fake accounts per day, but now we're down to 10-30 ... so we've seen some improvement, but the issue is still annoying me. So I'm half thinking now that the security measures are helping quite a bit, but that this is possibly humans still targeting our website and perhaps getting paid per signup they do or something similar to that. Even if so, is there any way I could confirm they are humans versus bots?
Security measures:
I won't get into all of the details here (for security reasons), but I'll just indicate what we've done to counteract the spambots:
Created honeypots at various areas of our website which automatically ban based on IP
IP banning - based on known botter/spammer ip addresses
Duration detection of signup form pageload to form submission -- if less than 5 seconds to complete our signup form, we're confirming you're a bot and then preventing the signup
Hidden checkbox in signup form -- there is a hidden checkbox in the signup form that is invisible to regular users (if a bot checks it we are automatically detecting and preventing the signup)
Google re-Captcha - We've enabled Google re-Captcha in our signup form as well
Email activation link - We send our users an activation email with a link that they have to click on to signup -- they are not able to sign into our website until they've activated their account.
Future actions include:
Detecting what users are posting in their descriptions in their profiles and banning based on that -- string detection for banned words, etc.
Any other suggestions or tips or tricks? In all honesty, if spam bots are getting through all of those security measures above --
do you think they are just that intelligent?
do you think we're being targeted?
Also, any way I can determine if they are bots or real humans? Suggestions?
This is a perennial problem; over the years I've found that as I add more anti-spam measures, the spammers continually get better at circumventing my measures.
I recommend doing an analysis of your spam to figure out how you can detect it. The spam itself contains the key to how to outsmart it. Look at the patterns, the structure, and decide what information is most useful and how the easiest way is to filter it out. Your spam detection doesn't need to be perfect, but generally, you want to get as much as possible, while getting as few false positives as possible.
Also, to answer your one question, you can make your bot-detection perfect, but there will always be humans submitting spam. And humans are tough to outsmart, and you may always need some manual attention to do it.
You are already implementing a lot of measures. Here are some more I would suggest:
When a signup form is generated, put a hidden field with a unique hash generated from the user's browser info, including the user's IP, HTTP user agent, and the date. Then, when the form is submitted, check the hash. This one method eliminated a surprising amount of spam.
If you want to take the previous method even farther, use a custom, time-sensitive hash in the URL of your contact form, and have the link to this form be dynamically generated. This way, if a spammer stores the form's URL, it won't work, but the link will work for every legitimate user of the site.
Make it so newly created, non-trusted users, cannot display any public profile information, such as URL's or text even. With a site as small as yours you could require manual approval of each user, and if your userbase got bigger, you could use an automated reputation system, a lot like Stack Overflow and the other Stack Exchange sites use. This removes the incentive for spam. Also, I found an overwhelming majority of spammers only ever logged onto the site once. If you wait to do the manual approval of users, until they have logged on twice, or even have returned to the site on another day, using a persistent cookie, you will filter out the vast majority of spammers and you will only have to do a small amount of manual approval work. Then have the system delete the unvalidated/inactive accounts after a certain amount of time.
Check for certain keywords or structure of info. I found an overwhelming majority of my spammers would use certain words or phrases that were never used by my legitimate users. Another one was entering a phone number in their profile, a common pattern in spammers, that no legitimate user ever did. Also look for signs of foul play like XSS attacks. A huge portion of spammers will, at some point, submit something that has a ton of HTML tags in it, you can either use the tags itself to filter them out, or you can do something like stripping the HTML tags and then comparing string length and banning them if it's more than a small amount (i.e. allow someone to do something simple like a few <em></em> or <strong></strong> tags.) Usually, if there are HTML tags in the entry, there's a ton of it. Also look for material with weird encodings or characters that don't make sense. This is often an attempt at sophisticated SQL injection attacks, XSS, or other types of hacking attempts.
Use external IP blacklists. AbuseIPDB is one example; it has an API that you can use to check new IP's before storing them in your temporary database. Their free plan allows checking of up to 1000 IP's a day and you can pay for more than that. It won't catch all the manual spam but I find they catch a ton of the automated spam.
Are they targeting you? Yes. They are targeting everyone. But any site with 50k+ pageviews a month is high enough volume to be an attractive target. The higher traffic you get, the more attractive of a target you will be. Even some of my tiny sites have been targeted with suprisingly sophisticated attacks these days. Everyone needs to be on guard.
Good luck. I wish this weren't so much of a problem, but it is.
I'm seeking a way where I can protect documents (Word/PDF) from being forwarded to another unauthorized users via email.
Our analysts write their opinion on a specific topic and then e-mail them to our clients (paid subscribers). Problem is that our clients can click FW to the entire universe and we lose money.
One way to circumvent this is by sending a link to a secured-site where they will be asked to use login w/pw but I wonder if there's an existing tool of some kind that does "digital Rights Management" so the file will not open unless you're subscribed.
A foolproof way does not exist, and cannot exist.
If a user can see it, nothing will stop him from capturing the content and sending it on. The next to impossible to defeat: a video camera capturing it ...
But you can make it harder:
Mark files
Mark the files to indicate who they are intended for, along with copyright notices etc. Makes it easier to both know who forwarded it, and makes it much scarier to forward it for the user as their details (e.g. email address etc. are in it). It also allows you to see recurring problem customers and take action against the individuals.
Password protect the file
Formats like PDF, word etc. have some (easily defeated) ways to protect the files from unauthorised opening. But it also greatly affects the legitimate users as they'll have to enter the password as well. Add to that that including the password in the email defeats the "forwarded email" problem.
Closed clients
Using an app on a system like the iPhone might be a different approach: you don't email it and only make the content available to a client on a platform that is rather hardened by the maker (most users will not jailbreak their iOS device and as such you get a bit more control over the apps on them). Have the client app send notifications, retrieve it from a server and display it.
Website/passwords
In you example of a login and password on a website: nothing is going to stop them there either to either share their password or capture the document/information and send that on ... You will still need far more.
Balance
In the end you must find a balance between what your legitimate users want and need (speedy access, no hurdles), with your needs for protecting it.
And that is a risk assessment you need to make: costs of protecting vs. costs of not protecting.
Personally: Considering what you seem to be doing, I'd go for marking PDF files and password protect them to prevent easy modification (but allow printing and viewing). I'd mark them with something like "For the personal use of first.last#example.com" or "For the internal use of Corporation, inc." or so on. Include that in low contrast in the margin of every page and then let your legal team draft letters to those who infringe on your rights...
Actually: add a unique serial number in the PDF and make it invisible in the output. Track it internally and you should even find out who's trying to remove their tracks.
I have my first website. One of the first tasks I have encountered is creating a registration page to register a new user. I have concerns about "safe" ways to do this. Essentially a registration page is a window to do database inserts into a user table. I'm concerned about script kiddies getting a hold of my registration form and mercilessly pounding the database with false inserts.
A couple things I've researched and struggled with:
Captchas: I really wanted to be able to create my site without these as from my research it sounds like they're about 20% effective at turning away bots while they are guaranteed to anger real human users. If at all possible I'd like to make captcha's be either non-existent on my site or dynamically appear if it seems I'm being scripted against.
IP Spoofing - I toyed with the idea of checking based on IP so that if I get a lot of successive form submissions from the same IP I could give them a captcha. However, it is my understanding that it is trivial to spoof IP addresses and that checking for repeat submissions from someone who is appropriately spoofing would be ineffective.
Registration Confirmation via Email Link - You see this a lot on forums, etc. After the user registers you send them a confirmation link with a unique token to verify they have a real email box and haven't put in a fake one (or perhaps genuinely mis-typed). While this may add some value around validating a user is "real" you have already inserted into your user table and thus script kiddies prevail at filling a database with useless information.
How do site developers prevent script kiddies from spamming their database with tons of useless users? If the assumptions I've made above are correct I don't see an effective way to prevent it. I have toyed with other ideas that after I think about them are all crap. The search terms I'm currently using aren't turning up many results so I apologize if this is an overplayed topic.
I'm not totally agreed with eliminating the captcha part, however you can trap some bots in a Honeypot. Make an input field which is invisible to the end-user, however still exists for bots. If the submitted form contains the fake-field value then ignore it, real users can't see invisible fields! :)
For example:
// jQuery
$("#username").hide();
// HTML
<input type="text" name="real-username">
<input type="text" name="username" id="username">
// PHP
if (!empty($_REQUEST['username']))
die('Oops!');
Just remember that you need to ignore the username field, your real username is in real-username.
I have found the Confirmation Emails, combined with a clean up task (that deletes all registrations over x number of days which are not confirmed) will help. You won't be able to prevent all spam registrations, but a little bit of work in the DB will help keep the table small.
I have a website with protected content and I've recently started getting a lot of email alerts about unsuccessful logins. It gets really annoying, about one attempt in a minute or two on average.
Because I host the website on a dedicated Windows box and I log the IPs, I realized that these requests are not coming from regular visitors to my site (because statcounter doesn't record the IPs), but from automated scripts from many different IP addresses (mostly from Ukraine, but mainly Reverse DNS cannot locate them).
I created a blocking rule in Windows Firewall and started to add all the addresses I found in the log file, but they are many. I already added probably 50 (5 times by 10 IPs) and this is stopping them only for few hours and then new IPs start coming around.
I am actually a software programmer and managing a real server is not my best side. Are there any tools that I can use to prevent these attacks?
You should implement a rate limiter in your code.
If you get more than (for example) 4 failed login requests from the same IP in 5 minutes, require a CAPTCHA for the next login.
Google Accounts login pages do exactly this.
It's not a tool but one practice that might be useful is to not include the word "Login" on your page and don't name your aspx page "Login". This might help to prevent crawlers that are looking for common keywords associated from finding your login page.
Doing something like replacing a Text link that contains "Login" with an image that looks just like the text and changing the name of your login form might go a long way to prevent crawlers from even finding your login page.
Example:
<img src="li.png" />