Penetration testing: Accurate or abused?

According to a recent Ponemon study, since 2010 cybercrime costs have climbed 78% and the time required to recover from a breach has increased 130%. On average, U.S. businesses fall victim to two successful attacks per week where their perimeter security defenses have been breached.

Penetration testing (pen testing), also known as “ethical hacking,’ is an important and key step in reducing the risks of a security breach because it helps provide IT staff with an accurate view of the information system from an attackers point of view.

The pen test process results in an active analysis of the system for any potential vulnerabilities that could result from poor or improper system configuration, from both known and unknown hardware or software flaws, or operational weaknesses in process or technical countermeasures. In other words, through pen testing, IT teams find the holes and vulnerabilities and quickly work to fix these areas to prevent attacks.

The one thing that separates a pen tester from an outside malicious attacker is permission to gain entry to the information system. The pen tester will have permission to “attack’ and is thereby responsible to provide a detailed report of results found. Examples of a successful penetration would be obtaining confidential documents, identity information, databases and other “protected” information – all without the need for passwords or other security measures.


Pen tests are a component of a full security audit. For example, the Payment Card Industry Data Security Standard (PCI DSS), and security and auditing standard, requires both annual and ongoing pen testing (after system changes).

Pen tests are valuable for several reasons, including:

• Determining the risk associated with a particular set of attack vectors
• Identifying higher-risk vulnerabilities that result from a combination of lower-risk vulnerabilities exploited in a particular sequence
• Identifying vulnerabilities that may be difficult or impossible to detect with automated network or application vulnerability scanning software
• Assessing the magnitude of potential business and operational impacts of successful attacks
• Testing the ability of network defenders to successfully detect and respond to the attacks
• Providing evidence to support increased investments in security personnel and technology.

Obviously, there are a variety of ways to secure databases, applications, and networks, as there are many layers and levels to be secured. But the only way to truly assess the security of an environment is through direct testing. A good pen tester can actually replicate the types of actions that a malicious attacker would take, giving IT a more accurate view of the vulnerabilities within a network at any given time. There are a number of high quality commercial tools available, that can be implemented to ensure that both testing parameters and results are high-quality and trustworthy, but nothing replaces a hands-on direct test.

Even so, the quality of pen testing can vary by the skill and thoroughness of the pen tester. Given the limited time available for testing it is impossible to exercise all aspects of an application with all possible attack vectors. This problem is compounded in environments where secure coding practices have started to take root. Often the first phase of secure coding often involves limiting failure feedback to the users to limit the information a hacker has to determine he has discovered a flaw.

Unfortunately these same limitation make the pen testers job more difficult as well. Unfortunately, this means it is highly unlikely that a pen tester will find all the security issues. To aid in finding these partially obscured vulnerabilities it is necessary to monitor the application from within. This insures that tests that breach the application but don’t create a response the pen tester can use will still be seen, as they are still vectors that could be exploited by a dedicated hacker.

Further, it’s important to note that a pen test is a snap shot in time and new vulnerabilities appear every day. Companies have to employ continuous monitoring throughout their information systems including in the database tier and be vigilant against attacks. For example, if a pen test was performed on a Monday, the organization may pass the pen test. But what if the next day, there’s an announcement of a new vulnerability in database servers that were previously considered secure? And the next week or next month another vulnerability is announced? This is a scenario that plays out on a regular basis.

Companies are constantly playing “catch up’ apply patches. Ongoing, regular pen testing is critical and has proven to be a highly accurate method in identifying information system vulnerabilities. To get the most out of a thorough pen test the system should be properly instrumented to log all activity at the network tier, web tier, and database tier. At the conclusion of the pen test the logs from these instruments can provide extremely valuable insight into the system vulnerabilities.

As with most policies and procedures however, there still may be issues that need resolving. Many organization feel that pen testing is an area open for “abuse’ – most likely due to the fact that there are no firmly adhered to rules for the pen testing procedure. It is possible for a pen tester to skirt the process.

The PCI DSS regulation has 12 mandatory requirements for stringently protective guidelines, built to preserve the safety and identity of cardholder data – and in particular, section 11.3 for example, gets to the heart of the pen test, which is quite different from the former sub-section requirements.

11.3 is technically not a new requirement. Previous versions of the PCI standard made assumptions merchants would always conduct legitimate pen tests. Unfortunately, 11.3 is an area of the PCI DSS regulation that has been excessively abused. Companies have previously cut corners on this requirement and many pen testers were know to conduct meaningless scans in place of real testing. The new 3.0 version of the PCI DSS regulation effectively ends this scenario and companies will be required to develop and adopt an official methodology for testing. However, some believe that V3.0 is still lacking with regards to the precise industry-accepted methodology for pen testing the merchant should implement.

The good news is that the PCI Council has continued to follow up on this issue and is forcing new measures be adopted by organizations around the world. PCI DSS 3.0 requires that organizations identify the scope of their card data environment and have a pen test conducted that proves the card data environment is truly segmented from the rest of their network and the open Internet.

With the new rules in place with V3.0, demand for pen testers is on the increase, which is probably a good thing. The new requirements should help stop the abuse, and foster policies for accurate pen testing. These new pen testing requirements are long overdue. Merchants need to take pen testing seriously and adopt the new requirements as soon as possible to ensure they’re prepared for their first PCI DSS 3.0 assessment this year.