http://www.gcn.com/online/vol1_no1/43544-1.html 

By William Jackson
GCN 
04/23/07

The new federal computer security report card is out, and once again the 
grades are pretty bad. And once again it is hard to say just what they 
really mean.

The report card is issued each year by Virginia Rep. Tom Davis, ranking 
Republican on the House Government Oversight and Reform Committee. Davis 
gave the 24 executive branch agencies covered in the report an overall 
grade of C- for 2006, a grade he said showed slow but steady improvement 
from past years.

"Slow" is right, but I dont know how much improvement there is or how 
steady it has been. The grade had been stalled at D or D+ for the 
previous three years. Agencies receiving an F or an A this year are tied 
at eight each. Seven agencies improved their grades this year, six got 
worse and 10 remained the same. One major department, Veterans Affairs, 
didnt bother to provide a report for 2006 and so receives an incomplete.

But the biggest challenge is determining just what the grades are 
measuring. The report card bills them as federal computer security 
grades, but they are primarily based on compliance with the Federal 
Information Security Management Act. As I have said before, FISMA does 
not equal security. FISMA does not require secure IT systems; it 
requires a process for assessing, testing and managing IT security. 
Davis grades are based largely on how good a job an agency is doing at 
inventorying, testing, certifying and accrediting its IT systems. It 
would be possible to test, certify and accredit all your systems and get 
a splendid grade even if your systems failed the tests and you were 
accrediting them despite their vulnerabilities.

That is not to say that agencies are doing this. Good FISMA compliance 
should enable an IT shop to improve its security posture. But we really 
dont know from the report card whether or not it is helping. Is it 
really reasonable to believe that Housing and Urban Development improved 
its security from a D+ to an A+ in one year, or that Justice can go from 
a D to an A-? Or that NASA could drop from a B- to a D-? Thats what this 
years grades show, and I have a hard time believing it.

FISMA can be a powerful tool for improving federal IT security, and the 
annual report card has done a good job in helping to focus attention on 
this subject. But Im not sure just what the grades are measuring. I 
suspect it is not computer security and maybe not even FISMA 
performance.

A group made up of IT security vendors called the Merlin International 
Federal Research Consortium, surveyed federal chief information security 
officers about FISMA in advance of the report card. The results should 
probably be taken with a grain of salt only 30 of 117 CISOs 
participated, and 75 percent of the respondents said their FISMA grades 
were going to improve this year, so it probably wasnt a representative 
sample. But a couple of good ideas did come out of the report.

By a wide margin, the two greatest problems cited in FISMA compliance 
were funding and ambiguity in the way FISMA requirements are written. 
When asked for suggestions on how to improve the act, the CISOs didnt 
say anything about funding for security. They apparently do not think 
that is ever going to happen. But they did say that there should be 
better guidance to agencies for the yearly security controls tests and 
that FISMA guidelines should be clarified.

Maybe these two simple improvements could result in some real progress 
in both FISMA compliance and IT security.


__________________________
Subscribe to InfoSec News
http://www.infosecnews.org