At my site, I present a form for visitor input. No login is required. I cannot require a login. So anyone browsing the site can submit the form. It also opens up the form to bots. I need to prevent the bots. I had asked the question on the following thread.
Unwanted garbage input from bots?
I did get some useful response. I read a few solutions to the this (captcha and non-captcha).
Mine is not a site where a I get significant traffic. My users are not terribly computer savvy. So I was thinking of doing something like this. I am not a very accomplished programmer and what I am saying here may be very stupid. But I am simply trying to learn, so please bear with me.
Every time I present the form, I generate a unique key (unix time + remote host IP). I store the key in a db table and I send out the form with the key being a hidden field on the form. When a form is submitted, I check to see if the value for the key is in the db table. If it is, I remove the key from the db table and I process the form. If the key is not in the db table, I discard the form and ask the user to do the operation again.
With every submission I also remove stale entries(where the users did not submit the form within a stipulated time). I will need to have some mechanism where I prevent the request for the form, from bots. Say for example, if I have n number of pending requests from a particular host, I ask people to request for the form after a few moments.
Will something like this work?
the bots will be able to request the hidden field and submit it anyway. try a non-re-captcha library so that your users don't get overwhelmed (recaptcha is overwhelming due to its extra goal of hijacking your users to do OCR of pretty illegible text).
however, since you ask for a non-captcha solution, i would propose that you measure the time between form request and form submission (with the hidden key). a bot would submit the form within a couple of seconds of request, but a human would not.
if you find that this simple approach does not work for your site then you can try something more complex.
You could also hide the form and then a user would have to click on a button to reveal it. Much like how twitter does it when you log in.
I wouldn't worry too much about bots submitting your form. It's not gonna happen. If you're terribly fearful then instead of a captcha ask a stupid question like "what is 1+1?" before a submission.
It all depends on how desperately the spammers want to submit junk to your form. Your method will work for the most stupid of bots, but as agks mehx pointed out it's trivial for a bot to load up the form and extract the field if someone bothers to take a minute or so to tweak their bot.
At the other end of the spectrum, there's little you can do to automatically stop the "pay people in certain countries the equivalent of 10¢/hr to spam every board they can find" tactic without locking things down to an extent that also prevents the general public from posting useful comments.
What about hashing form field names so the name is different each time? hash(Original field name + time stamp + secret salt) and the just pass the time stamp with the form, it will take ages for the bot to figure it out, especially if the salt is different per user and changes every couple of hours/days. Just an idea I had. Was wondering if you think it would stop bots?
Related
I'm trying to set up a google wallet payment process for users to purchase an entry into a tournament. In order to do this, the user has to fill out a bunch of information about themselves (name, league player number, contact phone number, etc), and there are some other pieces of data that are implicit, such as a unique identifier for the tournament they're entering.
It seems like there are two ways to accomplish this in Google Wallet, and I'm wondering if I'm missing another, better workflow and/or if one of these two ways is preferred.
Possibility #1
When the user clicks the wallet button, I serialize the form and submit it to my server using ajax. If the form is properly filled out, the server encodes everything about the form into the sellerData field of the JWT, and returns the JWT asynchronously. I then pass this JWT to wallet, expecting to receive it in my postback handler.
The postback handler then constructs the entry using the information from the JWT sellerData field and records it in the database.
This possibility is intuitive to me, and I've implemented it, but I'm running up against the 200 character limit for the sellerData field, since it contains multiple peoples' names, phone numbers, and various other form elements. There's just no room. I don't have a workaround for this, and would welcome thoughts.
This approach has the advantage that nothing is created in my database until payment is successful, but I don't know how to work around the difficulties with representing the entire form in the JWT to get it to the postback handler somehow.
Possibility #2
The user just submits the entry form using the normal web-form submission process, which creates something in the database. Database objects newly created in this way are marked as "unpaid", and are therefore incomplete.
Once the user successfully creates their entry in the database, they are then presented with a second page at which they can pay. This works better because I can now just put the database key for the object they just created into the sellerData field, and not worry about the size limit.
It does have the unfortunate side-effect of having these half-completed objects in the database, as well as running the risk of users not quite understanding the two-step register-and-then-pay process, and forgetting to pay. I'd have to be quite careful and proactive about making sure that users realize that A) it's okay to submit the form with no payment information, and B) that submitting the first form doesn't mean that they're done.
Thoughts?
I think Option 2 is a pretty standard buy flow. Step 1 enter in your information Step 2 confirm your information and pay with Wallet.
The onsuccess callback could then redirect the user to a purchase receipt page.
My consumer mind doesn't see any purchase flow red flags.
I ended up going with option 2, because it was simplest for me and I didn't think it would confuse users.
I missed a third option, though, which is that I could allow people to buy a blank entry form and then fill it out after purchase. I think this might have even been better; it matches the in-person buying experience better and would therefore be more familiar to purchasers, and it avoids the problem of people who think they've registered but somehow didn't realize that they had to pay.
I have a website where I have a job section. I allow applicants fill out job applications online. No login is required. The data input gets stored in a database.
I have NOT put any captcha or bot blocking mechanism in the HTML form. I understand that this is a dumb thing to do. But mine is a small website and I did not spend too much time programming this.
I see every once in a while garbage inputs into the application form fields like the following:
yRERRCEXEUOMCew
Some times the 'City' field in the data would have a valid input (such as New York)
I am trying to understand where does this input come from and what would anyone gain by doing this.
Thanks
It comes from spam bots and they just submit random information to check to see if it is a working form or can send email, etc. If you are looking for a non-intrusive method (i.e. no CAPTCHA or JavaScript) to prevent spam bots from submitting bogus data, I would highly recommend throttling form submissions. If you are using PHP, you could use code like this:
// Sessions needed to tie forms to specific users
session_start();
// Process form here
if ( isset($_POST['submit']) )
{
$now = time();
// See if the current time less the start time is less than or equal to 5 seconds
if ( ( $now - $_SESSION['start_time'] )
Note: this will not stop dedicated bots nor will it provide any real security. It will stop automatic flood bots though since they will not normally wait 5 seconds between submissions.
Hope this helps.
I am trying to understand where does this input come from ?
It can come from any where , a user can also input this as your app isn't validating this input.
and what would anyone gain by doing this.
No Idea
You can put captcha. or simply you can add two attribute
lets say
1,98 and one operation sign(for example , +) let the user perform and validate this thing # server.
Also See
Practical non-image based CAPTCHA approaches
I have a backend that generates gift codes, each with a certain number of uses. Give these to a blogger or whatever, and their readership can redeem the code for a promotional item.
I'm working on the best way to check a codes validity without having collisions/dupes, or anything like that. I need to 1) validate the code 2) collect shipping info
My first draft was
A) Check code via a form, if good, proceed to address input. When input is received, save code and address/name etc.
This fails because if there are 74 uses on a 75 use code, 25 people could "validate" but not enter their address yet, and we'd end up with more than 75 valid redemptions.
My current solution looks more like:
B) Just have the code as the first field in the information gathering form, and when a valid code is typed in, ajaxify that and live check it against the DB. If the code is valid, it then shows the rest of the form, and that entry of the code is "claimed" for half an hour or something. If no DB entry w/in half an hour, it's then released.
This seems pretty complex, and I'm wondering if I'd need to do throttling against the ajax attempts to make sure people don't brute force a valid code.
Is this method secure, and/or are there any other blatantly obvious patterns I'm missing for this type of application?
Let everyone enter their gift-code and address, and then submit
In the backend, verify the address and the gift-code.
If the gift-code is valid and not exhausted, congratulate the user. Else apologise to them and suggest they buy it instead anyway.
Does it have to be more complicated than that?
Why don't you just have one form with all the information (redemption code and shipping info)?
Then, when the user submits, atomically (using transactions on your database) check if it's valid and commit the user's information.
If the code is no longer valid, just show a message like "Sorry, the redemption code you used has been depleted and is no longer valid."
Just wanted to add, if you're worried about bruteforcing attempts, you can require a captcha or javascript based hashcash value to be submitted along with the gift code. If you want to be as unobtrusive as possible, you can only require this for subsequent attempts after the first failed one.
One thing you might consider, is after the user enters a gift code, create an intermediate page that has more details about the offer, shows the number of claims remaining, and has some information about what will be required to complete the offer (address, creditcard, whatever). If the user chooses to claim the offer, have a 10-15 minute countdown (updated via javascript) on the data entry page for the address and other personal information, so the user knows that the offer might expire if they don't enter their information immediately.
Another thing to consider is implementing a "cancel" button that indicates the user can make the offer available for another user, without waiting for the countdown to expire.
Your current solution looks like the proper one, although I think you left out the method by which you associate the user association with the code. Still, providing the functionality of "reserving" a redemption of the code for a user is a good solution.
Option B seems reasonable. Just use a captcha rather than trying to throttle it. Captchas aren't perfect but it's less obnoxious than say misreading the code three times and then being denied the ability to try another for 24 hours. This will work particularly well if you're already planning on doing it AJAXy.
So -
User will fill in code field and captcha.
You'll confirm the captcha, then confrim the code.
Once successful, the user will fill in the other info and submit.
Using this method you could also probably only lock the code for something more like 5 minutes (ticket agency style) and show a timer on the form somewhere notifying the user.
Your A method (Check code via a form, if good, proceed to address input) looks very reasonable. Just combine it with B's "code is "claimed" for half an hour or something", and everything should work as you expect.
That is:
Customer enters code
Check code – if valid, and not already used MAX+ times, add an extra entry in code use table, with a timestamp that expires after x minutes.
Collect other info
On submit, permanently mark the code as used (remove entry expiration)
If customer never makes the order, the timestamped entry is removed (or ignored) after time x, and released for others to use.
We do a low end encryption (RC4) with a checksum added for this type of thing. Because RC4 generates a problematic character set, we also converted it to HEX. The combination is relatively secure and self checking. The decrypted value is just a number that we can verify in the database. This works with both our eMail reminders and gift certificates.
How can I prevent that forms can be scanned with a sort of massive vulnerability scanners like XSSME, SQLinjectMe (those two are free Firefox add-ons), Accunetix Web Scanner and others?
These "web vulnerability scanners" work catching a copy of a form with all its fields and sending thousands of tests in minutes, introducing all kind of malicious strings in the fields.
Even if you sanitize very well your input, there is a speed response delay in the server, and sometimes if the form sends e-mail, you vill receive thousands of emails in the receiver mailbox. I know that one way to reduce this problem is the use of a CAPTCHA component, but sometimes this kind of component is too much for some types of forms and delays the user response (as an example a login/password form).
Any suggestion?
Thanks in advance and sorry for my English!
Hmm, if this is a major problem you could add a server-side submission-rate limiter. When someone submits a form, store some information in a database about their IP address and what time they submitted the form. Then whenever someone submits the form, check the database to see if it's been "long enough" since the last time that IP address submitted the form. Even a fairly short wait like 10 seconds would seriously slow down this sort of automated probing. This database could be automatically cleared out every day/hour/whatever, you don't need to keep the data around for long.
Of course someone with access to a botnet could avoid this limiter, but if your site is under attack by a large botnet you probably have larger problems than this.
On top the rate-limiting solutions that others have offered, you may also want to implement some logging or auditing on sensitive pages and forms to make sure that your rate limiting actually works. It could be something simple like just logging request counts per IP. Then you can send yourself an hourly or daily digest to keep an eye on things without having to repeatedly check your site.
Theres only so much you can do... "Where theres a will theres a way", anything that you want the user to do can be automated and abused. You need to find a median when developing, and toss in a few things that may make it harder for abuse.
One thing you can do is sign the form with a hash, for example if the form is there for sending a message to another user you can do this:
hash = md5(userid + action + salt)
then when you actually process the response you would do
if (hash == md5(userid + action + salt))
This prevents the abuser from injecting 1000's of user id's and easily spamming your system. Its just another loop for the attacker to jump through.
Id love to hear other peoples techniques. CAPTCHA's should be used on entry points like registration. And the method above should be used on actions to specific things (messaging, voting, ...).
also you could create a flagging system, and anything the user does X times in X amount of time that may look fishy would flag the user, and make them do a CAPTCHA (once they enter it they are no longer flagged).
This question is not exactly like the other questions about captchas but I think reading them if you haven't already would be worthwhile. "Honey Pot Captcha" sounds like it might work for you.
Practical non-image based CAPTCHA approaches?
What can be done to prevent spam in forum-like apps?
Reviewing all the answers I had made one solution customized for my case with a little bit of each one:
I checked again the behavior of the known vulnerability scanners. They load the page one time and with the information gathered they start to submit it changing the content of the fields with malicious scripts in order to verify certain types of vulnerabilities.
But: What if we sign the form? How? Creating a hidden field with a random content stored in the Session object. If the value is submitted more than n times we just create it again. We only have to check if it matches, and if it don't just take the actions we want.
But we can do it even better: Why instead to change the value of the field, we change the name of the field randomly? Yes changing the name of the field randomly and storing it in the session object is maybe a more tricky solution, because the form is always different, and the vulnerability scanners just load it once. If we don’t get input for a field with the stored name, simply we don't process the form.
I think this can save a lot of CPU cycles. I was doing some test with the vulnerability scanners mentioned in the question and it works perfectly!
Well, thanks a lot to all of you, as a said before this solution was made with a little bit of each answer.
What I can think of is pre-populating certain form input elements based on the user's geographical information.
What are other ways can you think of to speed up user input on long application forms?
Or at least keep them focus on completing the application form?
If you have a long form, try to prune it down. Don't ask them to fill in fields that you don't really need.
If the form spans several pages, give the user some feedback as to how many more pages there are. We users hate clicking on the continue button wondering if this will be the last page.
Never lose a field that they filled in, no matter what they do. This could have security implications if passwords are involved.
Use dropdowns to provide the user with options unless there are a lot of options that the user would have to scroll through or if the terms in the dropdown aren't widely accepted (e.g. dropdown filled with Systems Engineer, Solution Developer, IT Application... I just want Programmer.).
Provide help for fields that might be hard to fill in (or provide examples).
If it is possible in your case, just collect the bare minimum up front and then allow the user to use the basic features of your service.
For the user to upgrade to a better level of service, they will need to fill in the 2nd form with more detail.
How important it is to you to collect ALL that information up front ? It is worth losing customers by demanding too much from them ? Why not demand it later at a time more convenient to the user.
Creating a multi-step wizard offering only a small number of input fields per step. Ensure that they are aware of how far they have progressed in the sequence.
The psychology is that once a user is 'invested' in a task, they are more likely to continue. If you present the whole list of input fields at once, you scare them off.
Offering musings at each step (cartoon, humor, sayings etc) makes them move to the next step out of curiosity.
Users won't mind filling in long forms if and only if they feel that the questions that you ask are important: otherwise they will be discouraged, and become impatient with it.
Remember, in a web application people have very, very short attention spans. When the user starts feeling that you are asking too much, they're usually right.
Keep required information as few as possible: other info should only be optional, and you have to give something in return to the user to compel them to complete that information.
However you implement it, please please please use some kind of Ajax hearbeat to store their progress server side and repopulate it if it's lost. There is nothing more infuriating to a user that working through a long form and having a browser or network hiccup lose their entire submission.
Whenever it happens to me I generally never give it a second shot, because at that point recreating my submission isn't worth whatever I was signing up for.
Checklist:
Explain clearly the purpose of the form. (What's in it for them?)
Prune, prune, prune, and keep questions clearly relevant!
Give the user feedback on his/her progress (if the form is split over multiple pages)
Ask for as little as you can up-front and leave the rest for later.
Clearly mark required fields
Group fields logically.
Keep labels/headings brief and easy to understand.
Prefill as much as possible - but not too much.
Spread super long forms over multiple pages and allow backtracking.
Cleverly placed "Back", "Save" and "Cancel" buttons put people's minds at ease - even when redundant.
Provide friendly (but clear!) validation error messages, in a timely manner.
Allow the user to reclaim half-filled in forms - don't lose their data!
No matter what you do, do not include a reset button. :-)
Finally:
Explicitly tell the user when the process is finished. ("Thank you! Your application has been sent.")
Tell the user what will happen next. ("A confirmation e-mail has been sent to your e-mail address, and we'll process your application within two working days.")
use Ajax to populate and update the controls asynchronously.It will speedup the filling of long application forms.
Split it up into multiple pages - there's nothing quite so discouraging as seeing that you have another 100 questions to go.
Put validation on each input and check it onblur(). If they get to the end of the page and then it says "question #2 was incorrect", chances are they've forgotten what that one was anyway and it'll be more difficult to return to it. Plus, if they answer a series of similar inputs in a particular, incorrect way, you should let them know straight away (eg: entering dates as mm/dd/yyyy when you want dd/mm/yyy)
Split the form into several steps. It's like how someone is much more likely to read five 3-sentence paragraphs than one big 15-sentence paragraph of the same length.
I agree with tim; just let them fill in the bare minimum information and then leave the rest to profile updates. If any data is necessary for the service offered on your site, ask for it when they try to avail of the service (and no earlier).
That said, I wouldn't advocate the kind of forcing function that adam suggests. It pays to give your users the warm, fuzzy feeling that they are privileged and can use ALL of the services on your site. Although, if you look at it hard enough, adam's and my suggestions are pretty much the same.
If the application needs to include a lot of information, then make sure the user can save at any point, and log off, and log in later to complete the form. This would make more sense if some of the information is not necessarily easily available. Tax returns are an obvious example, where some of the data may need to be calculated, or the user must find the relevant documentation.
In some cases the user might use the same information in multiple applications. In that case it might make sense for the user to register their details (Name, Address, Telephone numbers, etc), which are automatically filled in on each application. For example, if you had a website for a recruitment agency, they may allow users to register their details, and then to apply for a particular job, they can just include a personal statement that applies to that job in particular.
As another consideration, if some information may be incorrect (particular if this is not always clear, such as a CAPTCHA, or a user name that must be unique), either separate it from the rest of the data, or otherwise make it so a mistake doesn't mean the rest of the information must be reentered.
These are basically ways of avoiding the user having to enter the same information twice.