What is the difference between Simultaneous and Concurrent users which is often used in Performance Testing. I see the explanations are contradictory in the internet.
Most probably they have the same meaning. In performance testing there should be only one term: virtual user which must represent real user, that's it. Well-behaved load test has to represent real user as close as possible, otherwise it will not make a lot of sense to run such a test. By "as close as possible" I mean all real user's stuff, i.e.:
Cookies
Headers
Cache
Handling embedded resources (images, scripts, styles, fonts included into HTML page)
Handling AJAX requests
Think time between operations (users don't hammer server non-stop, they need some time to "think" between operations)
Load test steps (i.e. open login page, perform login, navigate somewhere, etc.)
Given certain virtual users concurrency some of the "load test steps" can be (and given more or less high load will be) simultaneous. Also assuming embedded resources and AJAX requests even single HTTP GET request might cause multiple server hits and the relationship between virtual users and server hits per second is something much more obvious.
simultaneous = same action, same time. "Everyone push the enter button in 3, 2, 1, Now!" In Microfocus LoadRunner this is represented by a concept called rendezvous.
Concurrent = operating in same window of time, but each on an independent set of actions.
Related
How to find out my site is being scraped?
I've some points...
Network Bandwidth occupation, causing throughput problems (matches if proxy used).
When querting search engine for key words the new referrences appear to other similar resources with the same content (matches if proxy used).
Multiple requesting from the same IP.
High requests rate from a single IP. (by the way: What is a normal rate?)
Headless or weird user agent (matches if proxy used).
Requesting with predictable (equal) intervals from the same IP.
Certain support files are never requested, ex. favicon.ico, various CSS and javascript files (matches if proxy used).
The client's requests sequence. Ex. client access not directly accessible pages (matches if proxy used).
Would you add more to this list?
What points might fit/match if a scraper uses proxying?
As a first note; consider if its worthwhile to provide an API for bots for the future. If you are being crawled by another company/etc, if it is information you want to provide to them anyways it makes your website valuable to them. Creating an API would reduce your server load substantially and give you 100% clarity on people crawling you.
Second, coming from personal experience (I created web-crawls for quite a while), generally you can tell immediately by tracking what the browser was that accessed your website. If they are using one of the automated ones or one out of a development language it will be uniquely different from your average user. Not to mention tracking the log file and updating your .htaccess with banning them (if that's what you are looking to do).
Its usually other then that fairly easy to spot. Repeated, very consistent opening of pages.
Check out this other post for more information on how you might want to deal with them, also for some thoughts on how to identify them.
How to block bad unidentified bots crawling my website?
I would also add analysis of when the requests by the same people are made. For example if the same IP address requests the same data at the same time every day, it's likely the process is on an automated schedule. Hence is likely to be scraping...
Possible add analysis of how many pages each user session has impacted. For example if a particular user on a particular day has browsed to every page in your site and you deem this unusual, then perhaps its another indicator.
It feels like you need a range of indicators and need to score them and combine the score to show who is most likely scraping.
As a newbie to JMeter, I have created some scenarios like some number of users are logging in to the system, sending some HTTP Request, Requests are looped, etc.
I would like to know what are the real world scenarios implemented by Companies to Performance test their System using JMeter.
Consider a E-Commerce Website and what all scenarios they might consider to performance test their Website?
The whole idea of performance testing is generating a real life load to the system simulating real users as close as possible. In regards to E-commerce system it would be something like:
N users searching for some term
M users browsing and navigating
X users making purchases
To simulate different usage scenarios you can use different thread groups or set weight using Throughput Controller
To make your JMeter test looking more like a real browser add the following test elements to your test plan:
HTTP Cookie Manager - to represent browser cookies, simulate different unique sessions and deal with cookie-based authentication
HTTP Cache Manager - to simulate browser cache. Browsers download embedded resources like images, scripts, styles, etc. but to it only once. Cache Manager replicates this behavior and also respects cache control headers.
HTTP Header Manager - to represent browser headers like User-Agent, Accept-Language and so on.
Also according to How to make JMeter behave more like a real browser you need to "tell" JMeter to retrieve all embedded resources from pages and use concurrent thread pool from 3 to 5 threads for it. The best place to put this config in is HTTP Request Defaults.
Vaadin and Eclipse RAP/RWT are web application frameworks with - as far as I understand - similar architecture. My question is if an application built with Vaadin or RAP is prone to denial of service attacks? I am currently evaluating the frameworks and want to be sure that this is not a concern.
Here is my reasoning:
With Vaadin or RAP you don't build HTML output but instead construct a Widget tree similar to an Swing/SWT application (in case of RAP it is SWT). The frameworks renders the widget tree as HTML in the browser, and sends user interactions back to the server, where the application gets notified in terms of events. The events are delivered to listener objects, which have previously been registered on widget.
Therefore the widget tree must be kept in some sort of user session and of course consumes some memory.
If the application is public (i.e. does not sit behind a login page), then it seems that there is a possible denial of service attack for such an application:
Just fire requests to the apps landing page, and probably fake a first response. For any such request, the framework will build a new widget tree, which will live some time on the server, until the session expires. Therefore the server memory should be filled with tangling users sessions very soon.
Or did these frameworks invent protection against this scenario?
Thanks.
A framework only can not protect you from DoS attacks.
Vaadin has some features built in to prevent attacks, but of course these features depend on how you code your application.
There is a long webinar about vaadin security:
https://vaadin.com/de/blog/-/blogs/vaadin-application-security-webinar
Vaadin does some validation of the client<->server traffic
to prevent XSS and other attacks.
But when you do some special things, you can open doors for such attacks.
As for the scenario you described:
The initial vaadin session takes some memory on the server (Just as all other sessions on any server)
How large this memory footprint is, depends on the initial number of widgets and what you load in memory for this. (DB connections etc.)
Usually this is not a problem when you have a very lightweight login page
But if you display large tables and many other fancy things, then you will have to have enough memory for the number of requests. (The same apply to all othe http servers/applications, they also need memory for this)
If the number of requests surpasses the capacity of your server, any web service can be brought down in a DoS attack
EDIT:
Due to good points in Andrés answer I want to sharpen my question.
First of all, of course I agree, if you put an application behind a login-wall
then the DOS-threat is not that big. At least you can identify the attacking user. And the login page itself can
be lightweight or even must not be implemented with Vaadin/RAP.
And as Vaadin/RAP applications are most likely used in RIA-style intranet settings then the DOS scenario does not invalidate
their use in these settings.
But at least both frameworks itself expose demo pages without logins on the internet:
See http://www.eclipse.org/rap/demos/ and http://demo.vaadin.com/dashboard/
These are not simple pages, and probably use quite a bit of memory.
My concern is about such a scenario, a non access-restricted internet page:
Once these frameworks responded to a request, they must keep serverside memory for that request for quite some time
(say the classical 30 minutes of a HTTP session, at least in minute scale).
Or to express it differently, if the application retains memory per intial user request for some substantial time it will be
prone to DOS attacks.
Contrast this with an old-style roundtrip web application which does not require user identification.
All the information needed to decide what to return is contained in the request (path, parameters, http-method, ...),
so a stateless server is possible.
If the user interacts with such an application, the application is still free to store session-persistent data on the client
(shopping cart contents in a cookie, etc.).
Our team have built a web application using Ruby on Rails. It currently doesn't restrict users from making excessive login requests. We want to ignore a user's login requests for a while after she made several failed attempts mainly for the purpose of defending automated robots.
Here are my questions:
How to write a program or script that can make excessive requests to our website? I need it because it will help me to test our web application.
How to restrict a user who made some unsuccessful login attempts within a period? Does Ruby on Rails have built-in solutions for identifying a requester and tracking whether she made any recent requests? If not, is there a general way to identify a requester (not specific to Ruby on Rails) and keep track of the requester's activities? Can I identify a user by ip address or cookies or some other information I can gather from her machine? We also hope that we can distinguish normal users (who make infrequent requests) from automatic robots (who make requests frequently).
Thanks!
One trick I've seen is having form fields included on the login form that through css hacks make them invisible to the user.
Automated systems/bots will still see these fields and may attempt to fill them with data. If you see any data in that field you immediately know its not a legit user and ignore the request.
This is not a complete security solution but one trick that you can add to the arsenal.
In regards to #1, there are many automation tools out there that can simulate large-volume posting to a given url. Depending on your platform, something as simple as wget might suffice; or something as complex (relatively speaking) a script that asks a UserAgent to post a given request multiple times in succession (again, depending on platform, this can be simple; also depending on language of choice for task 1).
In regards to #2, considering first the lesser issue of someone just firing multiple attempts manually. Such instances usually share a session (that being the actual webserver session); you should be able to track failed logins based on these session IDs ang force an early failure if the volume of failed attempts breaks some threshold. I don't know of any plugins or gems that do this specifically, but even if there is not one, it should be simple enough to create a solution.
If session ID does not work, then a combination of IP and UserAgent is also a pretty safe means, although individuals who use a proxy may find themselves blocked unfairly by such a practice (whether that is an issue or not depends largely on your business needs).
If the attacker is malicious, you may need to look at using firewall rules to block their access, as they are likely going to: a) use a proxy (so IP rotation occurs), b) not use cookies during probing, and c) not play nice with UserAgent strings.
RoR provides means for testing your applications as described in A Guide to Testing Rails Applications. Simple solution is to write such a test containing a loop sending 10 (or whatever value you define as excessive) login request. The framework provides means for sending HTTP requests or fake them
Not many people will abuse your login system, so just remembering IP addresses of failed logins (for an hour or any period your think is sufficient) would be sufficient and not too much data to store. Unless some hacker has access to a great many amount of IP addresses... But in such situations you'd need more/serious security measurements I guess.
Obviously some sort of mechanism for limiting login attempts is a security requisite. While I like the concept of an exponentially increasing time between attempts, what I'm not sure of storing the information. I'm also interested in alternative solutions, preferrably not including captchas.
I'm guessing a cookie wouldn't work due to blocking cookies or clearing them automatically, but would sessions work? Or does it have to be stored in a database? Being unaware of what methods can/are being used so I simply don't know what's practical.
Use some columns in your users table 'failed_login_attempts' and 'failed_login_time'. The first one increments per failed login, and resets on successful login. The second one allows you to compare the current time with the last failed time.
Your code can use this data in the db to determine how long it waits to lock out users, time between allowed logins etc
Assuming google has done the necessary usability testing (not an unfair assumption) and decided to use captchas , I'd suggest going along with them.
Increasing timeouts is frustrating when I'm a genuine user and have forgotten my password (with so many websites and their associated passwords that happens a lot , especially to me)
Storing attempts in the database is the best solution IMHO since it gives you the auditing records of the security breach attempts. Depending on your application this may or may not be a legal requirement.
By recording all bad attempts you can also gather higher level information, such as if the requests are coming from one IP address (i.e. someone / thing is attempting a brute force attack) so you can block the IP address. This can be VERY usefull information.
Once you have determined a threshold, why not force them to request the email to be sent to their email address (i.e. similar to 'I have forgotten my password'), or you can go for the CAPCHA approach.
Answers in this post prioritize database centered solutions because they provide a structure of records that make auditing and lockout logic convenient.
While the answers here address guessing attacks on individual users, a major concern with this approach is that it leaves the system open to Denial of Service attacks. Any and every request from the world should not trigger database work.
An alternative (or additional) layer of security should be implemented earlier in the req/ res cycle to protect the application and database from performing lock out operations that can be expensive and are unnecessary.
Express-Brute is an excellent example that utilizes Redis caching to filter out malicious requests while allowing honest ones.
You know which userid is being hit, keep a flag and when it reaches a threshold value simply stop accepting anything for that user. But that means you store an extra data value for every user.
I like the concept of an exponentially increasing time between attempts, [...]
Instead of using exponentially increasing time, you could actually have a randomized lag between successive attempts.
Maybe if you explain what technology you are using people here will be able to help with more specific examples.
Lock out Policy is all well and good but there is a balance.
One consideration is to think about the consruction of usernames - guessable? Can they be enumerated at all?
I was on an External App Pen Test for a dotcom with an Employee Portal that served Outlook Web Access /Intranet Services, certain Apps. It was easy to enumerate users (the Exec /Managament Team on the web site itself, and through the likes of Google, Facebook, LinkedIn etc). Once you got the format of the username logon (firstname then surname entered as a single string) I had the capability to shut 100's of users out due to their 3 strikes and out policy.
Store the information server-side. This would allow you to also defend against distributed attacks (coming from multiple machines).
You may like to say block the login for some time say for example, 10 minutes after 3 failure attempts for example. Exponentially increasing time sounds good to me. And yes, store the information at the server side session or database. Database is better. No cookies business as it is easy to manipulate by the user.
You may also want to map such attempts against the client IP adrress as it is quite possible that valid user might get a blocked message while someone else is trying to guess valid user's password with failure attempts.