What are the different types of Security Testing?
We have a fairly full list which is discussed over on Security Stack Exchange here and here.
Discovery
The purpose of this stage is to identify systems within scope and the services in use. It is not intended to discover vulnerabilities, but version detection may highlight deprecated versions of software / firmware and thus indicate potential vulnerabilities.
Vulnerability Scan
Following the discovery stage this looks for known security issues by using automated tools to match conditions with known vulnerabilities. The reported risk level is set automatically by the tool with no manual verification or interpretation by the test vendor. This can be supplemented with credential based scanning that looks to remove some common false positives by using supplied credentials to authenticate with a service (such as local windows accounts).
Vulnerability Assessment
This uses discovery and vulnerability scanning to identify security vulnerabilities and places the findings into the context of the environment under test. An example would be removing common false positives from the report and deciding risk levels that should be applied to each report finding to improve business understanding and context.
Security Assessment
Builds upon Vulnerability Assessment by adding manual verification to confirm exposure, but does not include the exploitation of vulnerabilities to gain further access. Verification could be in the form of authorised access to a system to confirm system settings and involve examining logs, system responses, error messages, codes, etc. A Security Assessment is looking to gain a broad coverage of the systems under test but not the depth of exposure that a specific vulnerability could lead to.
Penetration Test
Penetration testing simulates an attack by a malicious party. Building on the previous stages and involves exploitation of found vulnerabilities to gain further access. Using this approach will result in an understanding of the ability of an attacker to gain access to confidential information, affect data integrity or availability of a service and the respective impact. Each test is approached using a consistent and complete methodology in a way that allows the tester to use their problem solving abilities, the output from a range of tools and their own knowledge of networking and systems to find vulnerabilities that would/ could not be identified by automated tools. This approach looks at the depth of attack as compared to the Security Assessment approach that looks at the broader coverage.
Security Audit
Driven by an Audit / Risk function to look at a specific control or compliance issue. Characterised by a narrow scope, this type of engagement could make use of any of the earlier approaches discussed (vulnerability assessment, security assessment, penetration test).
Security Review
Verification that industry or internal security standards have been applied to system components or product. This is typically completed through gap analysis and utilises build / code reviews or by reviewing design documents and architecture diagrams. This activity does not utilise any of the earlier approaches (Vulnerability Assessment, Security Assessment, Penetration Test, Security Audit)
Risk assessment - creating a threat model and defining what will be tested.
Security auditing - using the threat model to probe the system design.
Vulnerability scanning - using software to probe the system inplementation.
Penetration testing - trying to hack into the system, either externally or internally.
Operational testing - some or all of the above after the system is in production.
Vulnerability Scanning - Typically an automated procedure to scan one or more systems against known vulnerability signatures.
Security Scanning - This is a vulnerability scan plus a manual verification of the findings to help remove false positives/ negatives.
Penetration Testing - A tester will attempt to gain access and prove access to the system owner.
Risk Assessment - involves a security analysis of interviews with employees compiled with business and industry justifications for risks discovered.
Security Auditing - Typically an in-depth auditing of software code and/or Operating Systems. This is often a very thorough line-by-line inspection of code.
Ethical Hacking - This is very similar to a penetration test, but it is usually many of them against a number of systems in order to discover as many attack vectors as possible.
Posture Assessment and Security Testing - This combines security scanning, ethical hacking and risk assessments to show the overall security posture of the organization.
Each of these security testing types can be further sub-categorized by different methodologies.
Penetration can be of different types, broadly categorized as follows:
Web parameter tampering: The user manipulates parameters exchanged between client and server and modifies application data such as user credentials, permissions, price or quantity of products, etc. for their benefit.
Database Tampering: compromising the databases that support the system and store data critical for business or running of the app
Cookie Stealing: A valid computer session is exploited to gain unauthorized access
Cross-site Scripting: An attacker injects malicious scripts on the client-side code to redirect the website link.
Cross-site Request Forgery: Also called one-click attack or session riding, unauthori
Privilege Escalation: To hack into a senior’s ID and misuse privileges.
Let’s break down security testing into its constituent parts by discussing the different types of security tests that you might perform.
Static code analysis
Static code analysis is perhaps the first type of security testing that comes to mind, its the oldest form also.
Static code analysis involves reviewing source code to identify problems that could lead to security breaches in an application (or in resources to which the application has access). Classic examples of vulnerabilities that you might be looking out for using this type of analysis are coding flaws that could enable buffer overflows or injection attacks.
It’s possible to perform some amount of static code analysis by hand, meaning that developers read through code manually to find security flaws. But that is often not practical to do on a large scale, given the size of many source code files; plus, humans can easily overlook flaws. That’s why using automated analysis tools to scan your source code is important.
Penetration testing
Penetration tests involve simulating attacks against an application or infrastructure in order to identify weak points. For example, you could use a tool like nmap to attempt to connect to all endpoints on a network from a non-trusted host and see if any endpoints accept the connection; if they do, you probably want to make them stop accepting connections from arbitrary hosts.
Some folks might argue that penetration testing should be broken down into subcategories, since there are different types of penetration tests. Some focus on the network, some on applications, some on authentication gateways, some on databases, and so on.
Compliance testing
Compliance tests (which are sometimes called conformance tests) are used to assess whether a configuration, architecture or process meets an organization’s predefined policies. Compliance testing is not strictly limited to the realm of security; you could conceivably use compliance tests to help maintain standards for application performance or response time, for example.
However, when it comes to security, compliance tests are an important resource for ensuring that a given application’s configuration or deployment architecture meets minimum standards set by your organization. Compliance tests typically work by comparing actual configurations with those that are deemed to be safe. When the tests identify incongruity, admins know that there may be a security issue or other problem.
Load testing
Load testing refers to tests that measure how an application or infrastructure performs under heavy demand. Load testing is not often thought of as a type of security test; it’s more commonly used to help optimize application performance and availability.
However, there is a reason why security admins might want to pay attention to load testing results, too. That reason is Distributed-Denial-of-Service, or DDoS, attacks, which aim to disrupt application availability by overwhelming an application or its host infrastructure with traffic or other requests.
Origin analysis testing
As the popularity of open source software has grown over the past decade, so has the importance of origin analysis testing. This type of testing helps developers and security admins determine where a given piece of source code originated.
In cases where some of your source code came from a third-party project or repository — which is very common these days, given the ease with which developers can incorporate upstream open source code into their applications — security admins will need to make sure that any known vulnerabilities in that code are addressed, and that the code conforms to internal security standards.
Related
When reading blog posts about WAFs and Kubernetes, it seems 90+ % of the posts are written by WAF-providers, while the remaining posts seem to be sceptical. So I would like to hear what your experiences are with WAFs, do they make sense, and if so can you recommend any good open-source WAFs? We are currently not allowed to used American cloud providers, as we work with "person data", and the Schrems II judgement has indicated that unencrypted "person data" is not allowed on their platforms (even if on EU servers).
To my understanding WAF help with the following:
IP-whitelists/blacklists
Rate Limits
Scanning of HTTPS requests for SQLi and XSS
Cookie Poisoning and session-jacking
DDOS (requires a huge WAF cluster)
But I would also think that these problems can be handled elsewhere:
IP-whitelists/blacklists can be handled by the Loadbalancer or NetworkPolicies
Rate Limits can be configured in the Ingress
Handling of SQLi and XSS is done by input sanitization in the application
Server-side sessions bound to IPs can prevent poisoning and jacking
DDOS are hard to absorb, so I have no native solution here (but they are low risk?)
Sure, I can see the advantage in centralizing security at the access gate to the network, but from what I have read WAFs are hard to maintain, they have tons af false positives and most companies mainly use them to be compliant with ISO-standards, and mainly in "monitoring mode". Shouldn't it be secure enough to use SecurityPolicies, NetworkPolicies, Ingress Rules and Loadbalancer Rules rather than a WAF?
A WAF is not strictly necessary on Kubernetes — or on any other deployment platform. Honestly, even after consulting for dozens of companies, I've seldom encountered any site that used a WAF at all.
You're right that you could duplicate the functions of a WAF using other technology. But you're basically reinventing the wheel by doing so, and the programmers you assign to do it are not as expert in those security tasks than the developers of the WAF are. At least they are probably doing it as one of many other tasks they are working on, so they can't devote full-time to implementation and testing of the WAF.
There is also a valid argument that defense in depth in computing is a good thing. Even if you have other security measures in place, they might fail. It's worth creating redundant layers of security defense, to account for that possibility.
There's a tradeoff between implementing security (or any other feature) yourself versus paying someone else for their expert work. This is true for many areas of software development, not only a WAF.
For example, it has become popular to use a web application framework. Is it possible to develop your own framework? Of course it is, and sometimes it's necessary if you want the code to have very specific behavior. But most of the time you can use some third-party framework off the shelf. It saves you a lot of time, and you get the instant benefit from years of development and testing done by someone else.
A good waf does a lot more than that, and it is independent of the deployment model (kubernetes or else).
A waf can
Detect and prevent application level exploits far beyond sqli and xss. Sure, you can make a secure application... but can you actually make a secure application? (A team of sometimes changing developers usually cannot.)
Detect and prevent vulnerabilities in underlying layers, like nginx or the OS - or maybe even kubernetes itself.
Provide hotfixing known vulnerabilities until they are actually fixed in the code or patched in the underlying component (like for example preventing certain values for certain parameters you know are vulnerable and so on).
So in short, yes, a waf does make sense with k8s too, in fact it is not dependent on the deployment model. A waf is just a layer 7 firewall that understands http, and can look into traffic to find flaws and prevent exploits.
Update:
For example a recent vulnerability was log4shell, in log4j. In a request it was possible to run arbitrary stuff on servers due to a framework level (3rd party) vulnerability. A good, regularly updated waf would prevent that probably even before you read about the problem.
Spring4shell was a somewhat similar vulnerability in Spring, that can also be prevented by wafs. So could Heartbleed, a vulnerability in openssl.
There was a php vulnerability quite a while ago that involved a magic number, sent as any parameter.
Command injection vulnerabilities in any application or component follow specific patterns, and so on.
A waf also has more generic patterns for usual application vulnerabilities including (but not limited to) sql injection and xss. Sure, your application could be secure and not have these. But especially over time, it will for sure be vulnerable, even the best team cannot produce bug free code, and that applies to security bugs too.
As a web application is usually only accessible through http, ALL of that is available for capture for a component that understands http. All application layer attacks (and that's a lot) will come through http and a waf at least in theory is capable of preventing them. Surely it will not always recognize everything, it's not magic, and again, you could all implement it yourself. But it would be very difficult and time consuming. The same as you would not implement an API gateway or a network firewall, you would want to use a WAF to provide a layer of protection to your application and it's underlying components.
On the other hand, it's true that it takes some time to configure for your specific scenario and application. At first, it will probably produce false positives. Then you can decide how to manage those, you can disable entire rules, or remove certain pages or parameters from checks and so on. It does involve some work, maybe a lot for a very complex application. But when it's configured, it provides an additional layer of protection against threats you may not even have currently, but will in the future.
WAF suggestions:
If you are running managed kubernetes (AWS EKS, Azure AKS and the like) then probably your cloud provider's waf is the best choice due to easy setup and good integration (though I understand that might not be an option for you). I don't know of a good one apart from modsecurity if you are running your own. Naxsi would come to mind, and while I don't have experience with it, its functionality seems very limited compared to other options and what's described above.
WAF &/or API Gateway you may call, play a very vital role in a web application that many developers fail to understand initially.
First and foremost note that its another "out of process" component of your application that assumes all your attack surface
Least it can provide is to play the role as a "Circuit Breaker". For example your main kubernetes based deployment is down, for multiple reasons, this layer can provide some maintenance pages to your users
Further to that, it can provide caching of response, aggregation of responses from different microservices, buffering, prevention of injection types of attacks, centralized request logging, request analysis, TLS termination, Authentication decoupling, TLS translations, HTTP translations, OWASP protection and the list goes on. See this brief video for one reference implementation: link
There is a reason why a web application like Google Search and all other big similar ones rely on a WAF/API Gateway!
I have an Ubuntu droplet running a webserver. It's serving dynamic pages.
The backend is written in python with fastapi + uvicorn.
Generally speaking, security can often affect performance. As this paper points out
Network security and network performance are inversely related.
It goes on to say that a firewall indeed does have an negative impact on performance.
As seen from the result of the simulation, network performance is adversely affected when firewall is implemented.
I am concerned about speed. I want it to be lightning fast. That's why I've chosen ASGI.
Does it make sense to set up a firewall in this scenario? There is not input of user data (forms and the like) anywhere on the website.
No one can tell based on the information provided whether you should use a firewall or not. There are too many moving parts.
Have you done any risk assessment for your application? What kind of data is your application processing, and how sensitive is it? What are your requirements regarding Confidentiality, Integrity and Availability? Have you done any hardening already? Has someone evaluated the security of your application? Do you require authentication and access control? Which firewall would you like to have (normal, next gen, application firewall) and what for?
When you look on the definition of application security (application should offer Confidentiality, Integrity and Availability of the data at rest, in transfer and processed) you notice that Confidentiality and Integrity goes against the Availability principle. The very core of application security contains a contradiction. By improving security (putting a firewall) you might worsen the security (legitimate users may not access the site, everything gets slower).
Having said that - please do either a threat and risk analysis of your application or have a look on the OWASP Application Security Verification Standard and go over Level 1 requirements.
As part of a PCI-DSS audit we are looking into our improving our coding standards in the area of security, with a view to ensuring that all developers understand the importance of this area.
How do you approach this topic within your organisation?
As an aside we are writing public-facing web apps in .NET 3.5 that accept payment by credit/debit card.
There are so many different ways to break security. You can expect infinite attackers. You have to stop them all - even attacks that haven't been invented yet. It's hard. Some ideas:
Developers need to understand well known secure software development guidelines. Howard & Le Blanc "Writing Secure Code" is a good start.
But being good rule-followers is only half the point. It's just as important to be able to think like an attacker. In any situation (not only software-related), think about what the vulnerabilities are. You need to understand some of those weird ways that people can attack systems - monitoring power consumption, speed of calculation, random number weaknesses, protocol weaknesses, human system weaknesses, etc. Giving developers freedom and creative opportunities to explore these is important.
Use checklist approaches such as OWASP (http://www.owasp.org/index.php/Main_Page).
Use independent evaluation (eg. http://www.commoncriteriaportal.org/thecc.html). Even if such evaluation is too expensive, design & document as though you were going to use it.
Make sure your security argument is expressed clearly. The common criteria Security Target is a good format. For serious systems, a formal description can also be useful. Be clear about any assumptions or secrets you rely on. Monitor security trends, and frequently re-examine threats and countermeasures to make sure that they're up to date.
Examine the incentives around your software development people and processes. Make sure that the rewards are in the right place. Don't make it tempting for developers to hide problems.
Consider asking your QSA or ASV to provide some training to your developers.
Security basically falls into one or more of three domains:
1) Inside users
2) Network infrastructure
3) Client side scripting
That list is written in order of severity, which opposite the order to violation probability. Here are the proper management solutions form a very broad perspective:
The only solution to prevent violations from the inside user is to educate the user, enforce awareness of company policies, limit user freedoms, and monitor user activities. This is extremely important as this is where the most severe security violations always occur whether malicious or unintentional.
Network infrastructure is the traditional domain of information security. Two years ago security experts would not consider looking anywhere else for security management. Some basic strategies are to use NAT for all internal IP addresses, enable port security in your network switches, physically separate services onto separate hardware and carefully protect access to those services ever after everything is buried behind the firewall. Protect your database from code injection. Use IPSEC to reach all automation services behind the firewall and limit points of access to known points behind an IDS or IPS. Basically, limit access to everything, encrypt that access, and inherently trust every access request is potentially malicious.
Over 95% of reported security vulnerabilities are related to client side scripting from the web and about 70% of those target memory corruption, such as buffer overflows. Disable ActiveX and require administrator privileges to activate ActiveX. Patch all software that executes any sort of client side scripting in a test lab no later than 48 hours after the patches are released from the vendor. If the tests do not show interference to the companies authorized software configuration then deploy the patches immediately. The only solution for memory corruption vulnerabilities is to patch your software. This software may include: Java client software, Flash, Acrobat, all web browsers, all email clients, and so forth.
As far as ensuring your developers are compliant with PCI accreditation ensure they and their management are educated to understand the importance security. Most web servers, even large corporate client facing web servers, are never patched. Those that are patched may take months to be patched after they are discovered to be vulnerable. That is a technology problem, but even more important is that is a gross management failure. Web developers must be made to understand that client side scripting is inherently open to exploitation, even JavaScript. This problem is easily realized with the advance of AJAX since information can by dynamically injected to an anonymous third party in violation of the same origin policy and completely bypass the encryption provided by SSL. The bottom line is that Web 2.0 technologies are inherently insecure and those fundamental problems cannot be solved without defeating the benefits of the technology.
When all else fails hire some CISSP certified security managers who have the management experience to have the balls to speak directly to your company executives. If your leadership is not willing to take security seriously then your company will never meet PCI compliance.
As far as I can see the offerings fall into two categories – scanning services such as McAfee, Comodo, etc. and tools such as Burp Proxy, HP’s WebInspect,CodeScan, etc.
In an ideal world, I’d use something that actively scanned a certain URL (the target being a LAMP stack) on a daily basis (or as required if it’s a standalone tool), but I’m a bit wary of standalone tools in terms of their coverage and frequency of updating. (The ‘remote’ scanners such as McAfee are presumably updated as required.)
I’ve also had issues with some standalone tools (can’t remember which one unfortunately) that managed to get themselves lost within our URL rewriting system (there’s a facetted search in play, so you can imagine things get fairly deep on the URL front).
As such, I’m just wondering what experiences people have had with the offerings out there and whether the standalone tools stack up against the scanning services.
(Incidentally, I'm aware of Penetration testing tools - I'm just wondering if the situation has changed since then)
I have done penetration testing and exploit development. I can tell you from first hand experience that hacking isn't just firing off some tool. Sometimes tools can make life easier, but if you don't know what you are doing then a tool isn't going to help.
If you want to KNOW that your system is secure then you need to higher a skilled hacker to break in. The PCI-DSS is a certification required for credit card processing which mandates that you have regular penetration testing conducted on your server. Conducting regular penetration testing is something that you should adopt if you want to have a very secure server.
A very good security measure for web servers is a Web Application Firewall (WAF). WAFs are also required by the PCI-DSS. Mod_security is free and open source WAF. Mod_Security can be used to prevent hundreds of different types of attacks. A WAF can be a nightmare for a penetration tester or would be hacker.
Anyone have suggestions for security auditing of an .NET Web Application?
I'm interested in all options. I'd like to be able to have something agnostically probe my application for security risks.
EDIT:
To clarify, the system has been designed with security in mind. The environment has been setup with security in mind. I want an independent measure of security, other than - 'yeah it's secure'... The cost of having someone audit 1M+ lines of code is probably more expensive than the development. It looks like there really isn't a good automated/inexpensive approach to this yet. Thanks for your suggestions.
The point of an audit would be to independently verify the security that was implemented by the team.
BTW - there are several automated hack/probe tools to probe applications/web servers, but i'm a bit concerned about whether they are worms or not...
Best Thing to do:
Hiring a security guy for source code analysis
Second best thing to do hiring a security guy / pentesting company for black-box analysis
Following tools will help :
Static Analysis Tools Fortify / Ounce Labs - Code Review
Consider solutions such as HP WebInspects's secure object (VS.NET addon)
Buying a blackbox application scanner such as Netsparker, Appscan, WebInspect, Hailstorm, Acunetix or free version of Netsparker
Hiring some security specialist is so much better idea (will cost more though) because they won't only find injection and technical issues where an automated tool might find, they will also find all logical issues as well.
Anyone in your situation has the following options available:
Code Review,
Static Analysis of the code base using a tool,
Dynamic Analysis of the application at run time.
Mitchel has already pointed out the use of Fortify. In fact, Fortify has two products to cover the areas of static and dynamic analysis - SCA (static analysis tool, to be used in development) and PTA (that performs analysis of the application as test cases are executed during testing).
However, no tool is perfect and you can end up with false positives (fragments of your code base although not vulnerable will be flagged) and false negatives. Only a code review could solve such problems. Code reviews are expensive - not everyone in your organization would be capable of reviewing code with the eyes of a security expert.
To begin, with one can start with OWASP. Understanding the principles behind security is highly recommended before studying the OWASP Development Guide (3.0 is in draft; 2.0 can be considered stable). Finally, you can prepare to perform the first scan of your code base.
One of the first things that I have started to do with our internal application is use a tool such as Fortify that does a security analysis of your code base.
Otherwise, you might consider enlisting the services of a third-party company that specializes in security to have them test your application
Testing and static analysis is a very poor way to find security vulnerabilities, and is really a method of last resort if you haven't thought of security throughout the design and implementation process.
The problem is that you are now trying to enumerate all of the ways your application could fail, and deny those (by patching), rather than trying to specify what your application should do, and prevent everything that isn't that (by defensive programming). Since your application probably has infinite ways to go wrong and only a few things that it is meant to do, you should take an approach of 'deny by default' and allow only the good stuff.
Put it another way, it's easier and more effective to build in controls to prevent whole classes of typical vulnerabilities (for examples, see OWASP as mentioned in other answers) no matter how they may arise, than it is to go looking for which specific screwup some version of your code has. You should be trying to evidence the presence of good controls (which can be done), rather than the absence of bad stuff (which can't).
If you get somebody to review your design and security requirements (what exactly are you trying to protect against?), with full access to code and all details, that will be more valuable than some kind of black box test. Because if your design is wrong then it won't matter how well you implemented it.
We have used Telus to conduct Pen Testing for us a few times and have been impressed with the results.
May I recommend you contact Artec Group, Security Compass and Veracode and check out their offerings...