I'm playing around with Azure Load Test for my own app and I just realized that I can type any URL into the "Test Target" field. Does this mean that any Azure subscriber could "simulate" 1,000,000 concurrent users against my website, effectively DDoSing me?
You have just hit upon the license and ethical problem of performance testing tools. They are a necessarily violent and destructive class of tools: the tactical nuclear weapon of tests designed to find and exploit the weak parts of your architecture.
If you read the license agreements of almost every tool there is typically a clause about using the tools only against sites you own, manage, control or where you have explicit permission from that do hold those attributes. Also, most sites have user agreements which prohibit the use of automation against their sites outside of a defined supported API and also prohibitions against performance testing without coordination.
Does anything stop you from driving around the web neighborhood and tossing nuclear grenades at websites? No, not really. You have your license and user agreement constraints, assuming you choose to honor them - many don't. You have the threat of prosecution for engaging in a DDoS attack against a site - this is usually the greater curb against behavior.
Likewise, because of their destructive power these tools should not be placed in the hands of people without training and supervision for a period of time lest they start asking, "Hmmmmm, I wonder what happens when I use 10,000,000 fake email addresses for account signups in test....only to find that they have now destroyed the ability to transfer mail in and out of the enterprise because the SMTP relay has 10,000,000 confirmation emails queued up and waiting to timeout after 72 hours as undeliverable.
Related
I'm designing a game that makes queries to a database on the web. The database is fronted by a web service. For example, a request could look like this:
Endpoint: "server.com/user/UID/buygold"
POST:
amount: 100
The web service would make sure that userid has enough funds to purchase 100 gold, then would return a Boolean answer based on the success of the transaction.
However, I want to limit the amount of scripting someone could possibly do to automate gameplay. For example, they could figure out their userid and have automated tasks that buy gold for them while they are at work.
On the web services side, what are some sound security measures that I can put in place to decline all but real app traffic. Is there also a way to trump reverse engineers who will take the app apart and look for keys/certs?
I hope that this is not for a production environment, the security implications alone are mind boggling and certainly go beyond the scope of the allowed response, as it would require a rather lengthy and in depth list of requirements and recommendations, that are really contingent on many other factors that make up your web-service environment. For example, the network topology, authentication, session control and management, and various other variables all play important roles in fostering and implementing a sound cyber security counter measure.
However, assuming that you have all that taken care of, I will answer your main questions as follows:
For question:
"what are some sound security measures that I can put in place to decline all but real app traffic"
Answer:
This is one of many options out there that would address your particular concern, and that is to check for the client User-Agent header in the the request, which may appear something like this:
"Mozilla/5.0 (Macintosh; Intel Mac OS X 32.11; rv:49.0) Gecko/20122101 Firefox/42.0"
Depending on how the script is being run to automate gameplay, if it is in the form of a browser extension, then the User Agent really plays very poorly as a counter measure, if on the other hand, the script is being run directly from the client to your server (web-service), then, you can detect it right away, and there are ways to detect if someone spoofed a User Agent just to bypass this counter measure.
Another counter measure you can utilize is session management at the client level. So, this would require an architectural overview of how you implemented your particular project, but a general summary would follow a pattern like this:
Customer/GamePlayer would naturally be required to login (authentication of some sort)
The client system (which is the User Interface) that the game player is using, will have counter measures implemented in a front-end type of scripting language, i.e. Javascript or any framework that makes use of JS, such as jQuery, DoJo, etc.
Register event handlers that monitor actions, such as type of input, some Boolean logic that will follow something like this:
"if input is not from keyboard or mouse, then send flag with request"
The server/web-service will have logic to handle this request appropriately. This would be a way to catch/detect the game player after they commit the violation, used for legal reasons and such to profile evidence and such. If on the other hand you want to prevent that from happening, then, you could have some Boolean logic that goes something like this: "if input is not from the keyboard or mouse (or whatever permitted input device), then do not allow action (GamePlay), and still report back to web-server".
There are a dozen other ways, but this one seems to generally address your question, provided that you take into consideration that there are hundreds of other factors to think about, from networking level all the way to the application layer, and down to what pattern are you using for your web-service, such as whether it's a REST/API type of environment, does it follow an MVC pattern, and so on. There is no silver lining when it comes to cyber security, it's really a proactive and constant initiative on your end to ensure that all stakeholders' assets are protected, in this case, the asset is the web-service and gameplay, and the threat is the risk of gameplay tampering, that would affect the integrity of your game.
Now, regarding your second question:
" Is there also a way to trump reverse engineers who will take the app apart and look for keys/certs?"
When reverse engineers really put their mind to it, there is nothing you can really do, as whatever counter measure you may implement, they will find a workaround, that's why it's called reverse engineering, they will reverse engineer your "counter measure", so, not to be all cynical about it, you have to accept the reality that there is really no such thing as a "trump" counter measure when it comes to cyber security. You can however employ various mechanisms at both, the network layer and all the way to the application level, combined with proactive initiatives, intrusion detection, abnormal behavioral characteristics of the gameplay pattern, all will mitigate your risk; with all that said, your final frontier will be to ensure you have a good legal policy in place in your TOS (terms of service), and depending where you're hosting your web-service (geographically), you will be protected when users violate such terms, especially when you have verbiage that precludes users from attempting to reverse-engineer, tamper with gameplay scoreboards or currency, and so on.
Another good way, is to really connect with users, users are people, and people sometimes forget that they are also affecting others by their actions, so, once a user is aware of how his/her actions may negatively affect others, such as in the case of a user's actions to increase their 100 Gold, they may financially and emotionally affect others who may have put real time and effort into making this service even possible, so, a simple introductory welcome video upon signing up can do wonders for example; however, sometimes the user may not really know that they were prohibited from using auto-scripts for gameplay, or at least can raise an affirmative defense of that, so, having well published policies can really mitigate and potentially reduce altogether these types of risks. Despite the potentially optimistic outlook on users, you still have to exercise good programming practices and have security counter measures in place though.
I hope that I have given you some insight and direction to assist you with this matter, and that as you can see, it is really an involved and can be a very complicated type of process.
Good luck with your initiatives.
I am wondering whether Google App Engine is secure enough for financial applications? This would involve storing sensitive information, access to users' funds, etc. Are there any applications like that already running on App Engine?
Generally speaking, App Engine is more secure than most other options.
(1) Google has more people working on security than most companies can afford to work on their own servers or VMs.
When a new security threat is discovered, Google is very likely to fix it quickly, compared to dedicated servers/VMs, where you have to rely on your own sys admin to fix it in time.
(2) There are no OS, firewalls, etc., to configure, which reduces a possibility of a wrong configuration that exposes a security hole. Run times are also limited.
Ultimately, the vast majority of all security breaches happen for two reasons:
wrong application code/architecture
human factor (people storing their passwords in email messages, choosing weak passwords, doing harm on purpose, phishing, etc.)
Neither of these factors are any different on App Engine than on any other platform.
As for the second part of your question, Snapchat has a lot of, should I say, very sensitive information. It runs on App Engine.
Webfilings, a company that handles financial data for most of the Fortune 500, runs on Google App Engine. Their statement about it in the second link I've given:
Google App Engine gave us that speed we needed to grow, but as Murray stated, “being on the Google foundation gives us and our customers peace of mind.” WebFilings needed a platform with a strong approach to security that was very reliable because our customers’ security is a top priority. Because our customers rely on accurate and timely access to the cloud as they input pre-released financial information, security is top-of-mind at our company. As a result, we want our customers’ security to be in the best hands, in Google App Engine’s hands.
I am developing a website (basically a public facing site).
How can I simulate multiple users are surfing my site and doing various activities so that I can understand how my site will behave in a real time environment?
I am using Apache server and PHP.
As mentioned in previous posts, you will need a load test tool. The good news is that there are many tools and services in this field: open source load test ones like JMeter and Gatling, commercial one like Loadrunner and Neoload, the bad news is that you have to answer some questions and make some decisions.
One key decision you need to make is whether to test your application in the lab or in the cloud.
cloud based testing: Blazemeter, soasta, neustar ...
in-lab testing: JMeter, Gatling, Neoload, loadrunner, webperformer ....
In addition, you need to answer the following questioins:
how many virtual clients you want to emulate to stress the server
how much budget do you have
how complex is the web application
how much skills do the tester(s) have.
If you have high budget, complex web application and testers with good skills (like a developer), you can consider Loadrunner, NeoLoad.
If you have low budget but your tester(s) have good skills, you can consider Jmeter and Gating.
If you need to emulate lots of virtual clients (say 10000) to stress your complex web site and your tester(s) don't have the skill of a developer/programer, you may want to consider NetGend. There is a blog site where you can find out how complex performance testing can be (like filling HTML forms, extract values from JSON messages etc) and how easy it is on NetGend platform. By the way, you don't need high budget for NetGend.
Good lucky in your load/performance testing!
What you want is a load testing tool, there are several but id check out Neoload. You could also use Selenium and the various way to run selenium tests automatically
As this is your first time engaging in this task you would be well advised to find someone who has been there, done that and developed the battle scars from this activity. It is not a trivial effort to performance test a piece of software. If you listen to the traditional software vendors they will tell you that "any business analyst can use this tool and be effective" as if the tool is 85-95% of the skills you will need to be able to successfully performance test an application or a site. This is marketing foo to remove barriers to a sale.
In actuality the tool you select is anywhere from 5-15% of the total skill set you will need to be successful. Also, if the financial risk of failure is sufficiently high to warrant a performance test then it almost matters not which tool you pick, for the cost of the tool and the expertise will be dwarfed by your financial risk of not scaling.
If you don't have time to develop the skills or enough lead time to get a solid performance tester then you may want to consider some of the managed services offerings in the market, such as SOASTA, which can provide the expertise and the tool bundled within the deal. Here are some things you will want to look at in advance of any test (common issues)
Load Balancer misconfiguration resulting in distorted load to one node
Not appropriately managing your cache age for your static resources (.jpg, .css, ...) resulting in higher than expected load
All of your lookup queries to the database should be index optimized. Use a database profiler to check this
Holding onto resources too long. If your 95th percentile page to page request delay is five minutes then don't set your timeout at 30 minutes or 90 minutes for the HTTP session. This will hold onto resources far too long for the dead session. I use a rule of thumb of 95th percentile value times 1.5.
if this is a shopping site then don't hand out a default cart to everyone who shows up. Make sure they are on the revenue path before you hand them a cart, such as looking at the cart or placing something in the cart. Otherwise you have just built a 1:1 relationship with every customer and just about every piece of your architecture from web server to app server to the database server where the cart is created and managed
Also on the cart front, implement a 100x100 rule. If someone has a 100 items in the cart pick up the phone and call them to personalize the sale. If you have persistent carts which never expire then consider implementing a 100 day rule for evacuating from the cart items of that age or killing a cart altogether which hasn't been touched in that period. These people are clearly not on the revenue path.
Consider your design for ecommerce. Every step between add to cart and checkout is an opportunity to abandon the sale. The fewer the steps the greater the conversion rate: This is the genius behind the Amazon one click checkout. Minimize your number of steps and you will see a higher revenue flow as a result.
I would recommend gatling for load testings. It's scala based but a recorder is provided to generate workload test cases.
=> http://gatling-tool.org/
It's important to set some dimensions for test tool assessment to simulate user traffic:
SLA details (performance goals such as pages per second (PPS), http request per second (HPS), throughput, CPU usage etc.)
Virtual user size to simulate (You will need this info to decide number of
slave PCs/VMs, in other words that's called load generators, according to virtual user number
range)
Maintenance cost of scripts over changes
Effort to develop and execute test scripts
Number of test scenarios
Scheduling needs (You may want to schedule tool in regular basis or
execute test on-demand if needed)
Budget for test tool licenses and ROI (return of investment) calculations (Price, tool
expertise cost, utilizaton of test tool on other web applications etc)
Metrics provided by test tools
Monitoring requirements of network, servers and client
Integration with current test infrastructure (If HP ALM exist, you may be interested in Loadrunner)
If you are in hurry and don't have time to evaluate which tool to select, you may start with JMeter.
Selenium can be used for test automation of regression tests, I would like to highlight it's not effective for performance test due to its API. Sahi is another option for test automation.
I think that you are definitely looking for a load testing tool like Blazemeter. I recently discover this webinar which shows you how to do load testing on your application using a PaaS provider as a development and runtime environment, where you can deploy your application to run the load tests. They combine Blazemeter with a monitoring tool, New Relic in this case, to see the way in which you detect the new users surfing on your website. It is really cool and very interesting since you can know what is the performance of your application with a specific infrastructure.
Simple.
Set up server / application monitoring - New Relic is the easiest and most powerful. Free for 14 days.
Record a typical user's activity - Use JMeter to set up a proxy on your laptop and route web requests, mobile app usage etc through it. Sounds difficult but it's really easy. JMeter can act as the man-in-the-middle and capture all the requests sent by the browser/app to the server(s).
Now you "clone" the above user as many times as you need/could and blast the server. Initially you'd run the load test from your dev machine. Mine can take up to 80 concurrent users before cpu/ram runs out. Beyond this level explore BlazeMeter (free 50 concurrent users), jmeter-ec2 script (free), flood.io etc. Upload your script and blast away at your server. Ideally you should run incremental stress tests at your server. 10 users, 50 users, 100 users, 200 users etc.
Analyse, fix issues & ramp up the stress - In between each stress blast, go over your new relic. How is the application & server performing? What's failing? How are the alerts working?
If you are also searching for UI-Testing, you should check out Sahi Pro 6 automated testing tool, it also can be integrated with Jenkins.
==> http://sahipro.com/
It is really easy to record user actions with it on any browser and just playback the recorded scripts.
You can run scripts simultaneously on multiple browsers, thus simulate multiple users browsing your page:
https://sahipro.com/docs/using-sahi/playback-desktop.html#Distributed%20Runs%20-%20More%20Information
As part of a PCI-DSS audit we are looking into our improving our coding standards in the area of security, with a view to ensuring that all developers understand the importance of this area.
How do you approach this topic within your organisation?
As an aside we are writing public-facing web apps in .NET 3.5 that accept payment by credit/debit card.
There are so many different ways to break security. You can expect infinite attackers. You have to stop them all - even attacks that haven't been invented yet. It's hard. Some ideas:
Developers need to understand well known secure software development guidelines. Howard & Le Blanc "Writing Secure Code" is a good start.
But being good rule-followers is only half the point. It's just as important to be able to think like an attacker. In any situation (not only software-related), think about what the vulnerabilities are. You need to understand some of those weird ways that people can attack systems - monitoring power consumption, speed of calculation, random number weaknesses, protocol weaknesses, human system weaknesses, etc. Giving developers freedom and creative opportunities to explore these is important.
Use checklist approaches such as OWASP (http://www.owasp.org/index.php/Main_Page).
Use independent evaluation (eg. http://www.commoncriteriaportal.org/thecc.html). Even if such evaluation is too expensive, design & document as though you were going to use it.
Make sure your security argument is expressed clearly. The common criteria Security Target is a good format. For serious systems, a formal description can also be useful. Be clear about any assumptions or secrets you rely on. Monitor security trends, and frequently re-examine threats and countermeasures to make sure that they're up to date.
Examine the incentives around your software development people and processes. Make sure that the rewards are in the right place. Don't make it tempting for developers to hide problems.
Consider asking your QSA or ASV to provide some training to your developers.
Security basically falls into one or more of three domains:
1) Inside users
2) Network infrastructure
3) Client side scripting
That list is written in order of severity, which opposite the order to violation probability. Here are the proper management solutions form a very broad perspective:
The only solution to prevent violations from the inside user is to educate the user, enforce awareness of company policies, limit user freedoms, and monitor user activities. This is extremely important as this is where the most severe security violations always occur whether malicious or unintentional.
Network infrastructure is the traditional domain of information security. Two years ago security experts would not consider looking anywhere else for security management. Some basic strategies are to use NAT for all internal IP addresses, enable port security in your network switches, physically separate services onto separate hardware and carefully protect access to those services ever after everything is buried behind the firewall. Protect your database from code injection. Use IPSEC to reach all automation services behind the firewall and limit points of access to known points behind an IDS or IPS. Basically, limit access to everything, encrypt that access, and inherently trust every access request is potentially malicious.
Over 95% of reported security vulnerabilities are related to client side scripting from the web and about 70% of those target memory corruption, such as buffer overflows. Disable ActiveX and require administrator privileges to activate ActiveX. Patch all software that executes any sort of client side scripting in a test lab no later than 48 hours after the patches are released from the vendor. If the tests do not show interference to the companies authorized software configuration then deploy the patches immediately. The only solution for memory corruption vulnerabilities is to patch your software. This software may include: Java client software, Flash, Acrobat, all web browsers, all email clients, and so forth.
As far as ensuring your developers are compliant with PCI accreditation ensure they and their management are educated to understand the importance security. Most web servers, even large corporate client facing web servers, are never patched. Those that are patched may take months to be patched after they are discovered to be vulnerable. That is a technology problem, but even more important is that is a gross management failure. Web developers must be made to understand that client side scripting is inherently open to exploitation, even JavaScript. This problem is easily realized with the advance of AJAX since information can by dynamically injected to an anonymous third party in violation of the same origin policy and completely bypass the encryption provided by SSL. The bottom line is that Web 2.0 technologies are inherently insecure and those fundamental problems cannot be solved without defeating the benefits of the technology.
When all else fails hire some CISSP certified security managers who have the management experience to have the balls to speak directly to your company executives. If your leadership is not willing to take security seriously then your company will never meet PCI compliance.
I work at a small firm with little technical skill/knowledge.
One colleague had a hard drive die without any backup, and we recently had a virus come through and infect our test server (the gumblar.cn one) which we may or may not have transferred to a client's server.
After these two events, management danced around promoting good practices to avoid future occurences, for about a week.
Changing the company's culture to take this more seriously is one problem I'll try and deal with, but my question is...
What events should be planned for?
I suppose there are natural disasters, hardware failures, people quitting (bus factor?).
Here are some common things:
Shared Directories on a Fault Tolerant server to be used as a policy for user files & data (with appropriate security). Event=data loss limitation
Scheduled Backups of the Server. Event= data loss limitation
Firewall Proxy with logging and intrusion detection. Event=Data damage and theft
Enterprise Virus Software deployed on server and clients. Event=Virus Infection, Data theft, System Damage
Automated IT assets tracking software that reports on hardware and software changes happening on servers and clients. Event=Data and Hardware theft, unauthorised modification
Off Site storage of data. Event=Data Loss limitation
Firefighting Equipment & Automated firefighting mechanisms. Event=Fire
Internet Filtering Proxy such as WebMarshall. Event=Protection against "drive-by" infections and risks.
etc. etc. You should be able to find much more comprehensive strategies, measures etc. on the Internet.
Think for a while what equipment and services you use and how likely it is that they fail or become unavailable for a while. Build a list. Evaluate how likely each problem is to happen, how much it will cost you and how much a backup solution costs. Then you decide.