Does anyone know how to understand the security or penetration test coverage?
I found the traditional method for functional test coverage measurement is not quite useful for security test. Because for security test, actually, you don't need to cover every logic branch. If you cover the whole URLs and parameters, basically, you cover everything.
Any idea?
Thanks.
One possible metric for coverage of a web application security assessment is the range of issues tested for. At a bare minimum, the OWASP Top 10 issues should be tested for, but a high quality assessment will properly assess business logic and application specific issues. Also, the tester should have an understanding of any specific technologies used by the web application (e.g. Adobe Flash, Google Gears).
Penetration testing is a specialist activity, so get a trustworthy and respected company to perform the testing. In the UK, the CHECK scheme is highly respected, a list of certified companies can be found here: http://www.crest-approved.org/member_companies.php
Full disclosure: I work for Verizon Business who offer penetration testing services.
Related
We want to regularly test our web apps for vulnerabilities - we have a BurpSuite licence but would like to look into the possibility of using an online solution.
Benefits as we see them:
Little/no staff training required
It's always up to date (checking the latest known vuns)
It may be more cost-effective
Has anyone used such a solution before? How was the experience over manual testing with a local app such as BurpSuite?
Does your regular testing involve running purely automated scans or manual analysis using BurpSuite?
A cloud-based scanner isn't going to offer as much interactivity as running BurpSuite locally. Manual analysis is important for analyzing design issues, authorization problems, or logic errors. What a fully automated scanner will (or should) do is take care of tedious tests for vulns like SQL injection and XSS. (Think of it as finding implementation problems rather than design problems.)
A cloud-based scanner could be a drop-in replacement for BurpSuite scans you're running "out of the box" -- i.e. not using much configuration. But it should never the less complement, not replace, good manual testing. This can be a benefit cost-wise.
While a cloud-based scanner can be "always up to date", in reality there are very few new web vulns. For example, the OWASP Top 10 has only changed a little in the last 7 years. The updated vulns will help if you're mostly using open source or commercial web apps, but they won't have any bearing on the apps you build in house.
(FYI, my perspective comes from working on a cloud-based scanner.)
DISCLAIMER: At my place of work we are aware that, as none of us are security experts, we can't avoid hiring security consultants to get a true picture of our security status and remedial actions for vulnerabilities. This question is asked in the spirit of trying to be a little less dumb and a bit more aware of the issues.
In my place of work, a small business with a sum total of 7 employees, we need to do some work on reviewing our application for security flaw and vulnerabilities. We have identified two main requirements in a security tester:
They are competent, thorough and know their stuff.
They are able to leave us with a clear idea of the work we need to do to make our security better.
This process will be iterative so we will have a scan, do the remedial work and repeat. This will be a regular occurrence going forward.
The problem we have is: How do we know 1? And, even if we're reasonably sure of 1, how on earth do we proceed to 2?
Our first idea was to do some light security scanning on our code ourselves and see if we could identify any definite issues. Then, if the security consultants we choose identify those issues and a few more we're well on the way to 1 and 2. The only problem is that I've been trawling the interweb for days now looking at OWASP, Metasploit, w3af, burp, wikto, sectools (and Stack Overflow, natch)...
As far as I can tell security software seems to come in two flavours, complex open source security stuff for security experts and expensive complex proprietary security stuff for security experts.
I am not a security expert, I am an intermediate level business systems programmer looking for guidance. Is there no approachable scanner type software or similar which will give me an overview of the state of my codebase? Am I just going to have to take a part time degree in order to understand this stuff at a brass tacks level? Or am I missing something?
I read that you're first interested in hiring someone and knowing they're good. Well, you've got a few options, but the easiest is to talk to someone in the know. I've worked with a few companies, and can tell you that Neohapsis and Matasano are very good (though it'll cost you).
The second option you have is to research the company. Who have they worked with? Can they give you references? What do the references have to say? What vulns has the company published to the world? What was the community response (were they shouted down, was the vuln considered minor, or was it game changing, like the SSL MitM vuln)? Have any of the company's employees talked at a conference? Was it a respected conference? Was the talk considered good by the attendees?
Second, you're interested in understanding the vulnerabilities that are reported to you. A good testing company will (a) give you a document describing what they did and did not do, what vulnerabilities they found, how to reproduce the vulnerabilities, and how they know the vulnerability is valid, and (b) will meet with you (possibly teleconference) to review the vulnerabilities and explain how the vulns work, and (c) will have written into the contract that they will retest once after you fix the vulns to validate that they are truly fixed.
You can also get training for your developers (or hire someone who has a good reputation in the field) so they can understand what's what. SafeLight is a good company. SANS offers good training, too. You can use training tools like OWASP's webgoat, which walks you through common web app vulns. Or you can do some reading - NIST SP 800 is a freely downloadable fantastic intro to computer security concepts, and the Hacking Exposed series do a good job teaching how to do the very basic stuff. After that Microsoft Press offers a great set of books about security and security development lifecycle activities. SafeCode offers some good, short recommendations.
Hope this helps!
If you can afford to hire expert security consultants, then that may be your best bet given that your in-house security skills are low.
If not, there is not escaping the fact that you are going to need to understand more about security, how to identify threats, and how to write tests to test for common security exploits like XSS, SQL injection, CSRF, and so on.
Automated security vulnerability software (static code analysis and runtime vulnerability scanning) are useful, but they are only ever going to be one piece in your overall security approach. Automated tools do not identify all exploits, and they can leave you with a false sense of security, or a huge list of false positives. Without the ability to interpret the output of these tools, you might as well not have them.
One tool I would recommend for external vulnerability scanning is QualysGuard. They have a huge and up to date database of common exploits that they can scan for in public facing web applications, web servers, DNS servers, firewalls, VPN servers etc., and the output of the reports usually leaves you with a very clear idea of what is wrong, and what to do about it. But again, this would only be one part in your overall security approach.
If you want to take a holistic approach to security that covers not only the components in your network, applications, databases, and so on, but also the processes (eg. change management, data retention policy, patching) you may find the PCI-DSS specification to be a useful guide, even if you are not storing credit card numbers.
Wow. I wasn't really expecting this little activity.
I may have to alter this answer depending on my experiences but in continuing to wade through the acres of verbiage on my quest for something approachable I happened on a project which has been brought into the OWASP fold:
http://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project
It boasts, and I quote from the project documentation's introduction:
[ZAP] is designed to be used by people
with a wide range of security
experience and as such is ideal for
developers and functional testers who
a (sic) new to penetration testing.
EDIT: After having a swift play with ZAP this morning, although I couldn't directly switch on the attack mode on our site right away I can see that the proxy works in a manner very similar to OWASP's Web Scarab (Would link but lack of rep and anti-spam rules prevent this. Web Scarab is more technically oriented, it seems, looking over the feature list Scarab does more stuff, but it doesn't have a pen test vulnerability scanner. I'll update more once I've worked out how to have a go with the vulnerability scanner.
Anyone else who would like to pitch in and have a go would be welcome to do so and comment or answer as well below.
As far as I can see the offerings fall into two categories – scanning services such as McAfee, Comodo, etc. and tools such as Burp Proxy, HP’s WebInspect,CodeScan, etc.
In an ideal world, I’d use something that actively scanned a certain URL (the target being a LAMP stack) on a daily basis (or as required if it’s a standalone tool), but I’m a bit wary of standalone tools in terms of their coverage and frequency of updating. (The ‘remote’ scanners such as McAfee are presumably updated as required.)
I’ve also had issues with some standalone tools (can’t remember which one unfortunately) that managed to get themselves lost within our URL rewriting system (there’s a facetted search in play, so you can imagine things get fairly deep on the URL front).
As such, I’m just wondering what experiences people have had with the offerings out there and whether the standalone tools stack up against the scanning services.
(Incidentally, I'm aware of Penetration testing tools - I'm just wondering if the situation has changed since then)
I have done penetration testing and exploit development. I can tell you from first hand experience that hacking isn't just firing off some tool. Sometimes tools can make life easier, but if you don't know what you are doing then a tool isn't going to help.
If you want to KNOW that your system is secure then you need to higher a skilled hacker to break in. The PCI-DSS is a certification required for credit card processing which mandates that you have regular penetration testing conducted on your server. Conducting regular penetration testing is something that you should adopt if you want to have a very secure server.
A very good security measure for web servers is a Web Application Firewall (WAF). WAFs are also required by the PCI-DSS. Mod_security is free and open source WAF. Mod_Security can be used to prevent hundreds of different types of attacks. A WAF can be a nightmare for a penetration tester or would be hacker.
Anyone have suggestions for security auditing of an .NET Web Application?
I'm interested in all options. I'd like to be able to have something agnostically probe my application for security risks.
EDIT:
To clarify, the system has been designed with security in mind. The environment has been setup with security in mind. I want an independent measure of security, other than - 'yeah it's secure'... The cost of having someone audit 1M+ lines of code is probably more expensive than the development. It looks like there really isn't a good automated/inexpensive approach to this yet. Thanks for your suggestions.
The point of an audit would be to independently verify the security that was implemented by the team.
BTW - there are several automated hack/probe tools to probe applications/web servers, but i'm a bit concerned about whether they are worms or not...
Best Thing to do:
Hiring a security guy for source code analysis
Second best thing to do hiring a security guy / pentesting company for black-box analysis
Following tools will help :
Static Analysis Tools Fortify / Ounce Labs - Code Review
Consider solutions such as HP WebInspects's secure object (VS.NET addon)
Buying a blackbox application scanner such as Netsparker, Appscan, WebInspect, Hailstorm, Acunetix or free version of Netsparker
Hiring some security specialist is so much better idea (will cost more though) because they won't only find injection and technical issues where an automated tool might find, they will also find all logical issues as well.
Anyone in your situation has the following options available:
Code Review,
Static Analysis of the code base using a tool,
Dynamic Analysis of the application at run time.
Mitchel has already pointed out the use of Fortify. In fact, Fortify has two products to cover the areas of static and dynamic analysis - SCA (static analysis tool, to be used in development) and PTA (that performs analysis of the application as test cases are executed during testing).
However, no tool is perfect and you can end up with false positives (fragments of your code base although not vulnerable will be flagged) and false negatives. Only a code review could solve such problems. Code reviews are expensive - not everyone in your organization would be capable of reviewing code with the eyes of a security expert.
To begin, with one can start with OWASP. Understanding the principles behind security is highly recommended before studying the OWASP Development Guide (3.0 is in draft; 2.0 can be considered stable). Finally, you can prepare to perform the first scan of your code base.
One of the first things that I have started to do with our internal application is use a tool such as Fortify that does a security analysis of your code base.
Otherwise, you might consider enlisting the services of a third-party company that specializes in security to have them test your application
Testing and static analysis is a very poor way to find security vulnerabilities, and is really a method of last resort if you haven't thought of security throughout the design and implementation process.
The problem is that you are now trying to enumerate all of the ways your application could fail, and deny those (by patching), rather than trying to specify what your application should do, and prevent everything that isn't that (by defensive programming). Since your application probably has infinite ways to go wrong and only a few things that it is meant to do, you should take an approach of 'deny by default' and allow only the good stuff.
Put it another way, it's easier and more effective to build in controls to prevent whole classes of typical vulnerabilities (for examples, see OWASP as mentioned in other answers) no matter how they may arise, than it is to go looking for which specific screwup some version of your code has. You should be trying to evidence the presence of good controls (which can be done), rather than the absence of bad stuff (which can't).
If you get somebody to review your design and security requirements (what exactly are you trying to protect against?), with full access to code and all details, that will be more valuable than some kind of black box test. Because if your design is wrong then it won't matter how well you implemented it.
We have used Telus to conduct Pen Testing for us a few times and have been impressed with the results.
May I recommend you contact Artec Group, Security Compass and Veracode and check out their offerings...
What are the different types of Security Testing?
We have a fairly full list which is discussed over on Security Stack Exchange here and here.
Discovery
The purpose of this stage is to identify systems within scope and the services in use. It is not intended to discover vulnerabilities, but version detection may highlight deprecated versions of software / firmware and thus indicate potential vulnerabilities.
Vulnerability Scan
Following the discovery stage this looks for known security issues by using automated tools to match conditions with known vulnerabilities. The reported risk level is set automatically by the tool with no manual verification or interpretation by the test vendor. This can be supplemented with credential based scanning that looks to remove some common false positives by using supplied credentials to authenticate with a service (such as local windows accounts).
Vulnerability Assessment
This uses discovery and vulnerability scanning to identify security vulnerabilities and places the findings into the context of the environment under test. An example would be removing common false positives from the report and deciding risk levels that should be applied to each report finding to improve business understanding and context.
Security Assessment
Builds upon Vulnerability Assessment by adding manual verification to confirm exposure, but does not include the exploitation of vulnerabilities to gain further access. Verification could be in the form of authorised access to a system to confirm system settings and involve examining logs, system responses, error messages, codes, etc. A Security Assessment is looking to gain a broad coverage of the systems under test but not the depth of exposure that a specific vulnerability could lead to.
Penetration Test
Penetration testing simulates an attack by a malicious party. Building on the previous stages and involves exploitation of found vulnerabilities to gain further access. Using this approach will result in an understanding of the ability of an attacker to gain access to confidential information, affect data integrity or availability of a service and the respective impact. Each test is approached using a consistent and complete methodology in a way that allows the tester to use their problem solving abilities, the output from a range of tools and their own knowledge of networking and systems to find vulnerabilities that would/ could not be identified by automated tools. This approach looks at the depth of attack as compared to the Security Assessment approach that looks at the broader coverage.
Security Audit
Driven by an Audit / Risk function to look at a specific control or compliance issue. Characterised by a narrow scope, this type of engagement could make use of any of the earlier approaches discussed (vulnerability assessment, security assessment, penetration test).
Security Review
Verification that industry or internal security standards have been applied to system components or product. This is typically completed through gap analysis and utilises build / code reviews or by reviewing design documents and architecture diagrams. This activity does not utilise any of the earlier approaches (Vulnerability Assessment, Security Assessment, Penetration Test, Security Audit)
Risk assessment - creating a threat model and defining what will be tested.
Security auditing - using the threat model to probe the system design.
Vulnerability scanning - using software to probe the system inplementation.
Penetration testing - trying to hack into the system, either externally or internally.
Operational testing - some or all of the above after the system is in production.
Vulnerability Scanning - Typically an automated procedure to scan one or more systems against known vulnerability signatures.
Security Scanning - This is a vulnerability scan plus a manual verification of the findings to help remove false positives/ negatives.
Penetration Testing - A tester will attempt to gain access and prove access to the system owner.
Risk Assessment - involves a security analysis of interviews with employees compiled with business and industry justifications for risks discovered.
Security Auditing - Typically an in-depth auditing of software code and/or Operating Systems. This is often a very thorough line-by-line inspection of code.
Ethical Hacking - This is very similar to a penetration test, but it is usually many of them against a number of systems in order to discover as many attack vectors as possible.
Posture Assessment and Security Testing - This combines security scanning, ethical hacking and risk assessments to show the overall security posture of the organization.
Each of these security testing types can be further sub-categorized by different methodologies.
Penetration can be of different types, broadly categorized as follows:
Web parameter tampering: The user manipulates parameters exchanged between client and server and modifies application data such as user credentials, permissions, price or quantity of products, etc. for their benefit.
Database Tampering: compromising the databases that support the system and store data critical for business or running of the app
Cookie Stealing: A valid computer session is exploited to gain unauthorized access
Cross-site Scripting: An attacker injects malicious scripts on the client-side code to redirect the website link.
Cross-site Request Forgery: Also called one-click attack or session riding, unauthori
Privilege Escalation: To hack into a senior’s ID and misuse privileges.
Let’s break down security testing into its constituent parts by discussing the different types of security tests that you might perform.
Static code analysis
Static code analysis is perhaps the first type of security testing that comes to mind, its the oldest form also.
Static code analysis involves reviewing source code to identify problems that could lead to security breaches in an application (or in resources to which the application has access). Classic examples of vulnerabilities that you might be looking out for using this type of analysis are coding flaws that could enable buffer overflows or injection attacks.
It’s possible to perform some amount of static code analysis by hand, meaning that developers read through code manually to find security flaws. But that is often not practical to do on a large scale, given the size of many source code files; plus, humans can easily overlook flaws. That’s why using automated analysis tools to scan your source code is important.
Penetration testing
Penetration tests involve simulating attacks against an application or infrastructure in order to identify weak points. For example, you could use a tool like nmap to attempt to connect to all endpoints on a network from a non-trusted host and see if any endpoints accept the connection; if they do, you probably want to make them stop accepting connections from arbitrary hosts.
Some folks might argue that penetration testing should be broken down into subcategories, since there are different types of penetration tests. Some focus on the network, some on applications, some on authentication gateways, some on databases, and so on.
Compliance testing
Compliance tests (which are sometimes called conformance tests) are used to assess whether a configuration, architecture or process meets an organization’s predefined policies. Compliance testing is not strictly limited to the realm of security; you could conceivably use compliance tests to help maintain standards for application performance or response time, for example.
However, when it comes to security, compliance tests are an important resource for ensuring that a given application’s configuration or deployment architecture meets minimum standards set by your organization. Compliance tests typically work by comparing actual configurations with those that are deemed to be safe. When the tests identify incongruity, admins know that there may be a security issue or other problem.
Load testing
Load testing refers to tests that measure how an application or infrastructure performs under heavy demand. Load testing is not often thought of as a type of security test; it’s more commonly used to help optimize application performance and availability.
However, there is a reason why security admins might want to pay attention to load testing results, too. That reason is Distributed-Denial-of-Service, or DDoS, attacks, which aim to disrupt application availability by overwhelming an application or its host infrastructure with traffic or other requests.
Origin analysis testing
As the popularity of open source software has grown over the past decade, so has the importance of origin analysis testing. This type of testing helps developers and security admins determine where a given piece of source code originated.
In cases where some of your source code came from a third-party project or repository — which is very common these days, given the ease with which developers can incorporate upstream open source code into their applications — security admins will need to make sure that any known vulnerabilities in that code are addressed, and that the code conforms to internal security standards.