By Cara Garretson
WASHINGTON - Testing in-house and vendor-built software for security
holes should be an enterprise priority, said a group of vulnerability
research experts speaking on a panel at the Gartner IT Security Summit
held here this week. But Rich Mogull, the Gartner analyst who hosted the
panel, questioned how practical it would be for companies to dedicate
the dollars and resources required for this testing.
Thomas Ptacek, founder of application security consulting firm Matasano
Security, defined vulnerability research as analyzing software for holes
that attackers could take advantage of before the product is deployed,
using techniques such as reverse engineering and source-code auditing.
Software vendors and many enterprises have teams of engineers in house
to perform this testing, or rely on third parties such as the panelists
companies that specialize in finding vulnerabilities.
The benefit of this testing is being able to avoid the damage an
attacker could cause by fixing software problems before implementation.
If you dont find the problems, someone [else] will find the problems,
said Chris Wysopal, co-founder of Veracode. If you leave crumbs on the
floor the ants are going to show up. Thats a huge liability for your
For software built in-house, vulnerability testing should be part of the
software development life cycle, not an afterthought, Wysopal said.
Threat modeling to find out what are the weakest parts and easiest
attack vectors [of an application] is what people should do when
designing software; you find the weak points through threat modeling
then start reverse engineering, he said.
Simply using tools that scan for vulnerabilities is not enough, the
Scanning tools can reduce the amount of time you spend [analyzing the
code] manually, but if you care about the security of the application
you need to go deep and augment the scanner, says Ptacek. The place of
the scanner is to accelerate testing, but you cant rely on them.
Gartners Mogull did an electronic poll of the roughly 1,000 conference
attendees, asking what level of vulnerability testing they performed at
their organizations. The majority said their testing was limited to
using commercial scanning tools.
One reason enterprises may not be doing more intense vulnerability
testing is because the necessary skills are rare, Mogull suggested.
Its a huge skills issue, conceded Wysopal. It would be best to have an
expert researcher looking at every piece of code out there, but you just
cant find them.
Ptacek disagreed, saying services such as Web application penetration
testing are readily available.
Another panelist, Errata Security co-founder David Maynor, added that
any steps an organization can take to find vulnerabilities in software
are worth it.
Youre not wasting your money just because you dont find bugs, Maynor
Yet the process is still an expensive one, Mogull said, and enterprises
cant be expected to dedicate such time and money to extensively testing
You have to have the appropriate level of testing [correlate to] the
risk of the application, said Wysopal. Not every application requires
hired experts coming in, or training up a large team. Some applications
are not as risky as others.
Ptacek again disagreed, saying that every application offers entry into
Even innocuous applications deployed inside an organization can be
lethal if exploited, since just about every application contains or
connects to sensitive data. The one example the panel came up with of a
program that wouldnt pose a threat if compromised was an employee
Every piece of software is not a Boeing flight-control system where
everyone will die if theres a bug, countered Wysopal.
Mogull asked what percentage of the application development life cycle
budget should be devoted to vulnerability research. Wysopal answered at
least 5%, Ptacek said between 5% and 10% - adding that the cost for such
testing should actually come out of the quality and assurance budget --
but Maynor said closer to 25%.
Its easier to allocate dollars before deployment than fix [a
vulnerability] after its been deployed, he said.
Turning to commercial software, Mogull asked the audience for a show of
hands if they reverse engineer products they buy from vendors to look
for vulnerabilities. Outside of the panelists, few hands went up.
It would be ridiculous to test [to that level] every single product you
bring into your organization. What level of testing is OK? Mogull asked
Put one to two person weeks on any new product and youll find stuff,
said Ptacek. Use a sniffer and look at packets on the wire. Youll find
vulnerabilities, and youll gain control over how theyre going to be
All contents copyright 1995-2007 Network World, Inc.
Attend Black Hat USA, July 28-August 2 in Las Vegas,
the world's premier technical event for ICT security
experts. Featuring 30 hands-on training courses and
90 Briefings presentations with lots of new content
and new tools. Network with 4,000 delegates from
70 nations. Visit product displays by 30 top
sponsors in a relaxed setting. Rates increase on
June 1 so register today. http://www.blackhat.com