By Matt Hines
March 01, 2007
Security experts working on the CWE (Common Weakness Enumeration)
project claim that the initiative to create a central resource of
software vulnerabilities for developers is gaining momentum.
Sponsored by the Department of Homeland Security (DHS) and maintained by
a team of workers at nonprofit Mitre and other security professionals,
the ongoing effort is roughly four months shy of publishing a final
draft of its vulnerability encyclopedia, said leaders of the project.
Presenting at the ongoing Black Hat 2007 conference, CWE initiative
leaders said they are busy aggregating and organizing the mountains of
vulnerability data they have gathered and said they are working more
closely than ever with applications security testing companies to help
compare the abilities of various software scanning tools.
Launched in Dec. 2005, CWE seeks to establish a unified, measurable set
of software flaws to help developers improve the quality of their
products and drive out the types of vulnerabilities that have led to the
ongoing malware explosion.
By gathering input on commonly seen mistakes from developers,
researchers, and security vendors, the group believes it can create a
common language and standard procedures for handling the many different
types of loopholes that exist in programs' source code today.
Prior to the delivery of a final draft of its encyclopedia of flaws
later this year, CWE officials said they are preparing a sixth iteration
of the index to likely be published some time in April.
While much of the work the group has completed thus far has revolved
around the gathering of vulnerability formats and the various methods
used to identify and remediate the coding problems, the project has
recently involved a significant amount of testing of security scanning
tools to get a better idea of the capabilities and limitations of those
By gaining an understanding of the vulnerabilities that popular code
scanning engines can find -- and those they can't -- CWE can help
developers understand the types of issues they will need to look for on
their own, said Bob Martin, a CWE leader and the head of Mitre's related
CVE (Common Vulnerability Exposures) Compatibility effort.
"We wanted to evaluate what the tools claim to cover and what they are
most effective at finding," Martin said. "Right now, best test is to
throw tools at a big pile of code and see what tools find the most
vulnerabilities, but we're changing that paradigm into test cases where
we now look at the answers so we can evaluate what the tools found and
what kinds of complexities they can handle."
CWE's research will not list the names and performance results of the
products it is testing -- provided by over 20 firms, including Cenzic,
Fortify, SPI Dynamics, Veracode, and Watchfire -- but the work to
compile a resource that offers developers an idea of the types of
vulnerabilities missed by the tools should provide a great deal of
value, Martin said.
Officials with the project said that they have been pleasantly surprised
by the variety of methods employed by the commercial scanning tools and
the different types of flaws found by the various products. Going into
this phase of the research, the group expected to find that many
scanning systems identified the same types of issues, said Sean Barnum,
director of knowledge management at Cigital, a software quality
The tests also revealed that the products were looking for only 45
percent of the 600 common vulnerabilities that have already been entered
into the CWE index.
"We found that less than half of what we already have in CWE is covered
by these tools, so this helps prove that there are a lot of known issues
out there that aren't being addressed," Barnum said. "We also thought
that the tools would look for the same types of things, but they are
actually very different, and there's not a lot of overlap; that's
something that developers need to be aware of as they choose tools; you
want to right set for aggregated coverage."
Before each release of CWE, workers with the project spend much of their
time comparing all the vulnerability definitions and mitigation
taxonomies in the index, attempting to refine the language used in the
descriptions and add real-world examples of attacks that target the
That work is continuing and will remain the primary focus of CWE's
efforts going forward, officials said, including work to de-emphasize
nomenclature that describes common problems based on the attack methods
used to exploit them.
Barnum said that at this point, the encyclopedia isn't as polished as it
might be, but he observed that he and other project organizers believe
that taking an all-inclusive approach is the best strategy for expanding
the resource at this point.
"It's still a kitchen sink approach, but that's a smart approach,"
Visit the InfoSec News Security Bookstore