By John Pulley
Feb. 26, 2007
Are there perils in penetration testing?
Yes, but calm heads can help you choose whether to outsource or do it
The New York State Office of Cyber Security and Critical Infrastructure
labored for several years to protect its networks and information
technology assets with layers of security. It tightened existing IT
policies, procedures and practices and instituted new ones, conducted
gap analyses and established plans for mitigating security breaches.
Since then, the state has conducted regular scans to detect IT
vulnerabilities the equivalent of looking for frayed threads that can
unravel and leave an agencys networks and systems exposed.
The state will soon throw another protective blanket onto the pile. For
the first time, it will try to hack into its own systems. For IT
professionals, penetration testing is the ultimate security measure.
Penetration testing goes beyond tapping at the door, said William
Pelgrin, chief cybersecurity officer of New Yorks Cyber Security and
Critical Infrastructure Coordination. Its breaking through the door.
New York officials havent decided who they will let try to break through
the door. Until recently, penetration testing was the exclusive purview
of highly skilled technicians who employed extensive toolkits of
specialized programs to probe and exploit system and network weaknesses.
A deep-dive penetration test could take weeks to complete and cost
hundreds of thousands of dollars.
But new automated tools promise to do the job more quickly and at less
expense. Among those is Core Security Technologies Core Impact, which
the company says can run a meaningful penetration test in a few hours.
The company charges an annual fee of $25,000 for an unlimited-use
license. State and local chief information officers and chief
information security officers say they must evaluate the pros and cons
of the new software options, including open-source applications such as
A growth industry
So when does it make sense to use an automated penetration test?
Advocates of the new tools say the applications give in-house security
professionals more control, including the ability to perform penetration
tests as often as they want. Critics of automated tools say they are a
poor substitute for a thorough and nuanced manual test that a skilled
practitioner performs. Most experts agree, however, that an automated
penetration test in the hands of an untrained novice could do more harm
A fool with a tool is still a fool, said Bill Harrod, a security
management consultant at CA, formerly Computer Associates.
Penetration testing and other types of IT security assessments are a
growing industry. In a world made increasingly unsafe by identity theft,
online rip-offs and other cybercrimes, IT security pros are under
pressure to fortify systems and networks and protect information assets.
A growing number of federal regulations require public- and
private-sector CIOs to harden their systems and networks against
external and internal threats.
Vulnerabilities are increasing exponentially. The number reached 5,990
in 2005, according to Carnegie Mellon Universitys Computer Emergency
Response Team Coordination Center, an increase from 171 vulnerabilities
reported a decade earlier.
First line of defense
Every vulnerability represents a potential security risk. Penetration
testing determines whether the risk can be exploited. If a systems
owners can break in, an unauthorized hacker can, too.
Either you, the owner, can find them, or the hacker can find them, but
they will be found, said John Carpenter, a product manager at DevPartner
SecurityChecker, an automated application security test Compuware
A vulnerability scan is a first line of defense. It detects missing
security patches and spots other vulnerabilities that could potentially
compromise networks or systems. But scans often produce reams of data
that show false positives and identify vulnerabilities that, for
technical or practical reasons, no one could exploit anyway.
Penetration tests pick up where vulnerability scans leave off. Both
manual and automated penetration tests try to exploit network
vulnerabilities to determine whether they afford an opportunity for
hackers to take over computer systems, gain access to private data or
Armed with a Web browser and a proxy device, Brad MacKenzie, director of
IBMs X Force Penetration Test Team, recalled hacking into the network of
a state prison system and gaining access to detailed records of current
and former prisoners.
It was as if we were sitting in a branch office of the states prison
system, MacKenzie said.
South Carolinas strategy
Allowing a consultant to hack into your system and see sensitive data in
the name of security makes no sense to James MacDougall, chief
information security officer of South Carolina, which uses Core Impact
for in-house penetration testing.
We have outsourced some penetration testing, but I didnt think it was
wise to outsource the testing of critical infrastructure to a vendor,
MacDougall said. We thought we should arm ourselves.
Government procurement rules requiring agencies to select low-bid
vendors can undermine the confidence of users, including law-enforcement
agencies that dont want their data outside on the street, MacDougall
But detractors of automated penetration testing say the applications can
give agencies a false sense of security. In reality, the tests are only
as effective as the people running them. The best tools are unable to
discern the intent of hackers or reliably prioritize vulnerabilities.
Officials must remember, MacKenzie said, that an automated tool is
always a step behind the elite attackers exploits.
But not far behind, said Max Caceres, director of product management at
Core Security Technologies, which develops new commercial-grade exploits
that mimic what the bad guys do. We continually update the product with
new attacks every week, Caceres said.
Delawares Department of Technology and Information has been using Core
Impact for about a year as part of an overall IT security strategy that
included a thorough assessment performed by an independent auditor.
The auditor completed technical scans and reviewed policies and physical
controls, including firewalls and routers. The auditor also scrutinized
application development practices.
The review also benchmarked the states security program, which complies
with International Organization for Standardization 17799, a detailed
international standard for managing information security.
Networks evolve constantly and automated penetration testing is a way to
do the maintenance in between the full-blown assessments, said Elayne
Starkey, Delaware Department of Technology and Informations chief
However, Starkey said there are limitations to automated penetration
tests. Dont think of a tool like this as a silver bullet, she said.
But as part of an overall security program, automated penetration tests
can add value if handled properly, said Yong-Gon Chon, senior vice
president of services at SecureInfo.
My biggest fear is that you end up with an untrained new security
engineer with a few grand in the security budget who buys a tool that
they run, and it has a massive impact throughout the infrastructure,
Subscribe to the InfoSec News RSS Feed