A must read interview for anyone who is interested in evaluating web vulnerability scanners. In this interview we discuss the process of choosing a web vulnerability scanner and underline several factors that should be taken into consideration in the decision-making process.
Which is the best web vulnerability scanner out there?
This question has been haunting the web application security field for quite some time and rest assured that no one will ever give you a definite answer. What works for Mr A does not work for Mr B. This is because every website, or web application – as we call them today – is different. There are some scanners that perform better than others on websites developed in PHP and others that might perform better on websites developed in .NET, and so on. Also, people have different needs. Some just need a scanner to generate a PCI DSS compliance report. Others use it for consulting services, to assist them during a penetration test, and therefore need a scanner that gives them as much information as possible about the target and one that includes a good set of tools for easing the lengthy process of manual penetration testing.
How can I find out which web vulnerability scanner best suits my needs?
The best way to find out which web vulnerability scanner suites your needs is to get your hands dirty and try them out yourself against a real life website that you will be securing. Most of the software companies developing web vulnerability scanners will willingly give you evaluation licenses. There is also a good number of test websites available on the Internet which you can use to evaluate a number of web vulnerability scanners, but such test websites can never beat the real thing, i.e. your own website.
You can also find a lot of information on the Internet about web vulnerability scanners and their performance. From time to time, a number of web security researches and universities test these scanners against their test scenarios, and publish their findings online in white papers and web security articles. Such white papers and technical articles can give you a broad idea of who is on top of the game, but don’t base your decision only them. Unfortunately, they can be very misleading. I am not saying that they are wrong, or they don’t do a good job, far from it. These people are doing a very useful job, and they are helping software companies improve their web vulnerability scanners, but as explained before, you should try out web vulnerability scanners on your own websites. You’ll be surprised how differently each scanner performs on different websites.
Any suggestions for what users should lookout for when testing/evaluating web vulnerability scanners?
To start off with, you should always run a web application security test scan against a test website. This is very important since you don’t know the scanner’s capabilities or the weaknesses of the target website. An out-of-the-box scan might be able to inject code that might disrupt the operations of the web application you are trying to scan.
But first, you must understand how these scanners work. The web vulnerability scanner crawls the website, to discover all the files and inputs present in that website, and then launches a number of security checks against those discovered objects. The crawling process is the most crucial part of the scan, so you should always make sure that the web vulnerability scanner is able to crawl all of the website’s objects and inputs. If it does not discover all of them, the security scan results will not be correct, because even if it discovered thousands of vulnerable objects, but missed one, that is the input or parameter which a hacker will exploit and use it to deface your website.
Secondly, you should also check how many actual vulnerabilities the web vulnerability scanner discovered. A common mistake that people make is to base their choice on the number of vulnerabilities the web vulnerability scanner discovered, without checking if some of them are false positives. You do not want a scanner to report a large number of false positives because then you have to check each one by yourself. In such cases, you might as well do the penetration test manually – a web vulnerability scanner’s purpose is to ease your job and help you be more productive – not to waste your time.
If a scanner reports a good number of false positives, most probably the problem is in the configuration. Web vulnerability scanners are complex software, and because they have to support a wide variety of web applications, they all have a considerable number of options/settings. An out-of-the-box scan might not return the desired results, but just ten minutes of tweaking the scanner, might return a 100% accurate scan result. So during the testing phase, make sure you also go through the settings, get familiar with them and use them.
Last but not least, you should check the efficiency of the company’s support department. A web vulnerability scanner is a complex piece of software, and discovering vulnerabilities in a website can be a difficult and long process. Once in a while you will need an efficient technical support engineer to assist you with your findings. If the support department takes too long to reply to your queries, or if it takes you quite some time to simply get in touch with its operator, it might be too late. An attacker can discover vulnerabilities much faster than you think!
This issue also leads us to another question – should we use an open source solution or not? Most “technical” people – especially those in the security field – tried using an open source solution at least once in their lifetime. It might be a solution that works, but when you encounter a problem that you don’t know the answer to, you must post it on forums or mailing lists. It often happens that you don’t get a response, or if you do, it takes a lot of time to get it right while you engage in an exhausting back-and-forth of mailing. In the meantime, your website is still vulnerable.
Is an automated web vulnerability scan enough to completely secure your website or web application?
I always emphasize that an automated scan should always be accompanied by a manual penetration test. A good scanner will definitely make your job easier, and will help you not to forget a particular object or input. But, there are some vulnerabilities that automated software cannot discover. Such vulnerabilities are called logical vulnerabilities. For example, if you manually set a parameter called ‘price’ to ‘free’ while testing an online shopping cart, then the customer gets the ordered product for free.
An automated web vulnerability scanner will definitely help you discover this parameter, understand how the web application works and uses such a parameter, but it will never discover flaw in it. This is another reason why I suggest going for a commercial scanner and not an open source one. While in the open source software repository you will find a number of different tools that can assist you with a manual penetration test, most of the commercial web vulnerability scanners out there are shipped with a number of pentesting tools, such as fuzzers, HTTP editors, sniffers, etc., that can assist you and ease the manual penetration test process. The advantage is having a single website security solution that supports data exporting and importing from one tool to another. When using a number of different tools, exporting and importing data from one to the other might prove troublesome.
Can web vulnerability scanners help you remediate the vulnerability?
When a vulnerability is discovered by web vulnerability scanners, a good amount of technical details are presented to the user to help him understand and fix the issue. This technical information typically includes:
- Detailed description of the vulnerability
- HTTP request and response headers
- The vulnerable parameter or object name
- The injection value
- Remediation suggestions.
Most of the time, the remediation suggestions will be generic for that vulnerability class – for example, fixing a cross-site scripting vulnerability by filtering the user input for that vulnerable object. My advice is to not depend only on what the scanner’ suggestion. You should read more about that vulnerability class and understand what is it and how to fix it. This also serves to the user or developer as a lesson to write more secure code next time around. Some of the commercial web vulnerability scanners out there also suggest a number of web links when a vulnerability is reported, where you can usually find all the information you need.
This interview was originally published on Help Net Security on the 22nd of November 2010.
Get the latest content on web security
in your inbox each week.
THE AUTHOR
