http://www.gcn.com/print/26_26/45166-1.html 

By William Jackson
GCN Home 
10/08/07 issue

THE COMMON CRITERIA Evaluation and Validation Scheme has been heavily 
criticized lately (GCN.com, Quickfind 850). Devised as an independent 
evaluation of security products against a set of standard criteria, 
Common Criteria has been faulted for being expensive and not providing a 
foolproof measure to increase security. Not everyone shares these views.

Mary Ann Davidson, chief security officer at Oracle, for one, feels 
Common Criteria has a number of strengths.

GCN: It seems as if the perceived value of a Common Criteria evaluation 
depends in large part on how a vendor approaches the process. Those that 
put the most into it get the best value from the investment. Is this 
true?

DAVIDSON: The value of assurance is the extent to which a vendor 
embraces it across its development processes. That said, since every 
vendor of [information technology] products claims, Our product is 
secure: trust us! having a third party validate the product against the 
Common Criteria is tremendously valuable to customers, who otherwise 
would have to rely on unproven security claims. Also, many vendors, 
including Oracle, view the Common Criteria as the starting point for 
assurance, not the ending point.


GCN: How do you use the Common Criteria evaluation to create a reliable, 
repeatable development process?

DAVIDSON: The Common Criteria allows vendors to start their evaluations 
with a lower Evaluation Assurance Level and improve their processes to 
meet a higher assurance level over time. The higher in assurance levels 
you go, the more aspects of your development process the evaluators 
validate, and thus you need more process to meet the requirements. This 
avoids an all-or-nothing benchmark that few vendors could meet and 
allows them to improve their assurance over time.


GCN: How helpful are automated vulnerability assessment tools in 
improving the quality of your products and in achieving evaluation?

DAVIDSON: Automated vulnerability assessment tools do not come into play 
in Common Criteria until you reach those Evaluation Assurance Levels 
that are higher than those mutually recognized under the Common Criteria 
Recognition Arrangement. The national schemes that use such tools do not 
release them to vendors, which means they are of no use in helping 
improve product security.

The main value of automated vulnerability assessment tools is finding 
and fixing problems during development, before products ship. Also, 
automated vulnerability assessment tools are just one component of a 
robust, comprehensive assurance program. Oracle uses multiple tools as 
part of its Software Security Assurance program.


GCN: What are the weaknesses of these tools, and why are they not 
required in Common Criteria?

DAVIDSON: There is no tool validation program. Anybody can create a tool 
and even if wildly inaccurate claim they found a problem, and the burden 
is on the product vendor to prove that there isnt a problem instead of 
on the tool vendor to prove its tool is accurate.

Automated tools can find, at best, half the common security defects in 
software, and they miss many design defects. Also, in a product with 
millions of lines of code, if the tool has a 90 percent false-positive 
rate, the vendor could spend thousands of hours in nonrecoverable time 
chasing false alarms instead of actually improving security. Finally, 
these tools do not validate whether a product has any useful security 
functionality.


GCN: How do you synchronize your development process with the National 
Information Assurance Partnership evaluation process so products are not 
outdated by the time they have been evaluated?

DAVIDSON: The National Information Assurance Partnership is merely one 
of the national schemes under which vendors can evaluate their products. 
Under the Common Criteria Recognition Arrangement, vendors can do an 
evaluation up to Evaluation Assurance Level four that is accepted in 
other venues. In Oracles experience, we can evaluate a large, complex 
product like our database in about six months. The length of the 
evaluation cycle has not been an impediment to customer adoption of the 
product. A vendor going through an evaluation for the first time or who 
does not have well-developed development processes may take longer to go 
through an evaluation.


GCN: How do you select an evaluation laboratory? And is it possible to 
shop for labs to get a favorable evaluation?

DAVIDSON: Labs do not have the final say on whether a product completes 
an evaluation successfully; the national schemes that certify the labs 
do. A lab doing substandard work would face scrutiny by the national 
scheme. Generally, vendors shop for labs based on expertise and cost. We 
do our evaluations primarily in the United Kingdom and in Germany 
because we found these labs to have a higher level of expertise in 
Oracle software and an acceptable cost versus labs in other countries.


GCN: A frequent complaint of Common Criteria is that it focuses on 
process rather than the product. Has Common Criteria helped improve your 
products? If so, how?

DAVIDSON: Oracle continues to invest significant resources in building 
market-leading new functionality and products that we evaluate under the 
Common Criteria so that our evaluation-aware customers will feel 
comfortable using the products. We also continue to improve development 
processes; for example, we have formal processes addressing security 
vulnerabilities that we have included in our evaluations under flaw 
remediation.


GCN: What are the strengths and weaknesses in Common Criteria as it now 
stands?

DAVIDSON: Common Criteria has a number of strengths. Common Criteria 
considers both threats and the technical remedies needed to counter 
those threats; a product must have actual security functionality and 
assurance proof points for it. Common Criteria evaluation assurance 
levels are graduated, so that vendors can improve their assurance level 
over time. Also, secure development processes need to be demonstrable 
and repeatable.

Common Criteria is flexible. Vendors can assert a security claim, such 
as the use of automated vulnerability tools, through the target of 
evaluation they choose. And because of the Common Criteria Recognition 
Arrangement, evaluations are cost-effective. A vendor can do one 
evaluation that is recognized and accepted in multiple venues. Many 
vendors who complain about Common Criteria did not experience the 
pre-Common Criteria days when vendors evaluated the exact same product 
in multiple countries.

As far as challenges, Common Criteria is a committee-led organization 
representing 24 countries and is thus often slow to change. There is a 
lack of transparency in its structure, its decision-making and its 
technical review processes that the Common Criteria Vendors Forum has 
raised. The organization is slowly starting to address these issues.


GCN: What changes would you like to see made in Common Criteria?

DAVIDSON: Any critical piece of software should be designed in 
consideration of what kinds of threats the product is likely to face and 
the appropriate technical remedy for those threats. As threats evolve, 
so should Common Criteria.

That said, the National Cyber Security Partnership recommended the use 
of automated vulnerability testing tools at lower assurance levels, 
which could be a useful change, provided that all the criteria are met. 
The tools themselves must be evaluated and validated for what they find 
and how well they find it. The vendor can only make claims about use of 
automated tools as part of an evaluation based only on the tools they 
actually use in development. The vendor must assure that the Common 
Criteria Recognition Arrangement still applies: The use of these tools 
is mutually accepted in multiple venues. And, finally, the vendor must 
continue to maintain control of its source code. That is, the company is 
protected by contractual relationships it has with its labs but is not 
required to give source to any other party.


GCN: How well is the National Information Assurance Partnership working 
with industry to make needed changes in the criteria and processes?

DAVIDSON: Since the National Information Assurance Partnership is just 
one of the national schemes under which vendors can do Common Criteria 
evaluations, vendors who want the Common Criteria improved should work 
within the Common Criteria Vendors Forum. Oracle also believes it is 
duplicative and wasteful for individual schemes to want country-specific 
variants that would require vendors to evaluate their products only 
under those schemes.

1996-2007 1105 Media, Inc. All Rights Reserved.


__________________________________________________________________      
CSI 2007 is the only conference that delivers a business-focused
overview of enterprise security. It will convene 1,500+ delegates,
80 exhibitors and features 100+ sessions/seminars providing a
roadmap for integrating policies and procedures with new tools
and techniques.  Register now for savings on conference fees   
and/or free exhibits admission. - www.csiannual.com