The importance of vulnerability research

The importance of vulnerability research
The importance of vulnerability research 

By Cara Garretson
Network World

WASHINGTON - Testing in-house and vendor-built software for security 
holes should be an enterprise priority, said a group of vulnerability 
research experts speaking on a panel at the Gartner IT Security Summit 
held here this week. But Rich Mogull, the Gartner analyst who hosted the 
panel, questioned how practical it would be for companies to dedicate 
the dollars and resources required for this testing.

Thomas Ptacek, founder of application security consulting firm Matasano 
Security, defined vulnerability research as analyzing software for holes 
that attackers could take advantage of before the product is deployed, 
using techniques such as reverse engineering and source-code auditing. 
Software vendors and many enterprises have teams of engineers in house 
to perform this testing, or rely on third parties such as the panelists 
companies that specialize in finding vulnerabilities.

The benefit of this testing is being able to avoid the damage an 
attacker could cause by fixing software problems before implementation.

If you dont find the problems, someone [else] will find the problems, 
said Chris Wysopal, co-founder of Veracode. If you leave crumbs on the 
floor the ants are going to show up. Thats a huge liability for your 

For software built in-house, vulnerability testing should be part of the 
software development life cycle, not an afterthought, Wysopal said.

Threat modeling to find out what are the weakest parts and easiest 
attack vectors [of an application] is what people should do when 
designing software; you find the weak points through threat modeling 
then start reverse engineering, he said.

Simply using tools that scan for vulnerabilities is not enough, the 
panelists agreed.

Scanning tools can reduce the amount of time you spend [analyzing the 
code] manually, but if you care about the security of the application 
you need to go deep and augment the scanner, says Ptacek. The place of 
the scanner is to accelerate testing, but you cant rely on them.

Gartners Mogull did an electronic poll of the roughly 1,000 conference 
attendees, asking what level of vulnerability testing they performed at 
their organizations. The majority said their testing was limited to 
using commercial scanning tools.

One reason enterprises may not be doing more intense vulnerability 
testing is because the necessary skills are rare, Mogull suggested.

Its a huge skills issue, conceded Wysopal. It would be best to have an 
expert researcher looking at every piece of code out there, but you just 
cant find them.

Ptacek disagreed, saying services such as Web application penetration 
testing are readily available.

Another panelist, Errata Security co-founder David Maynor, added that 
any steps an organization can take to find vulnerabilities in software 
are worth it.

Youre not wasting your money just because you dont find bugs, Maynor 

Yet the process is still an expensive one, Mogull said, and enterprises 
cant be expected to dedicate such time and money to extensively testing 
every application.

You have to have the appropriate level of testing [correlate to] the 
risk of the application, said Wysopal. Not every application requires 
hired experts coming in, or training up a large team. Some applications 
are not as risky as others.

Ptacek again disagreed, saying that every application offers entry into 
an organization.

Even innocuous applications deployed inside an organization can be 
lethal if exploited, since just about every application contains or 
connects to sensitive data. The one example the panel came up with of a 
program that wouldnt pose a threat if compromised was an employee 
lunch-ordering application.

Every piece of software is not a Boeing flight-control system where 
everyone will die if theres a bug, countered Wysopal.

Mogull asked what percentage of the application development life cycle 
budget should be devoted to vulnerability research. Wysopal answered at 
least 5%, Ptacek said between 5% and 10% - adding that the cost for such 
testing should actually come out of the quality and assurance budget -- 
but Maynor said closer to 25%.

Its easier to allocate dollars before deployment than fix [a 
vulnerability] after its been deployed, he said.

Turning to commercial software, Mogull asked the audience for a show of 
hands if they reverse engineer products they buy from vendors to look 
for vulnerabilities. Outside of the panelists, few hands went up.

It would be ridiculous to test [to that level] every single product you 
bring into your organization. What level of testing is OK? Mogull asked 
the panel.

Put one to two person weeks on any new product and youll find stuff, 
said Ptacek. Use a sniffer and look at packets on the wire. Youll find 
vulnerabilities, and youll gain control over how theyre going to be 

All contents copyright 1995-2007 Network World, Inc.

Attend Black Hat USA, July 28-August 2 in Las Vegas, 
the world's premier technical event for ICT security 
experts. Featuring 30 hands-on training courses and 
90 Briefings presentations with lots of new content 
and new tools. Network with 4,000 delegates from 
70 nations.   Visit product displays by 30 top
sponsors in a relaxed setting. Rates increase on 
June 1 so register today. 

Site design & layout copyright © 1986-2014 CodeGods