http://www.wired.com/news/technology/0,72051-0.html 

By Kevin Poulsen
Nov, 02, 2006

A Morocco-born computer virus that crashed the Department of Homeland 
Security's US-VISIT border screening system last year first passed 
though the backbone network of the Immigrations and Customs Enforcement 
bureau, according to newly released documents on the incident.

The documents were released by court order, following a yearlong battle 
by Wired News to obtain the pages under the Freedom of Information Act. 
They provide the first official acknowledgement that DHS erred by 
deliberately leaving more than 1,300 sensitive US-VISIT workstations 
vulnerable to attack, even as it mounted an all-out effort to patch 
routine desktop computers against the virulent Zotob worm.

US-VISIT is a hodgepodge of older databases maintained by various 
government agencies, tied to a national network of workstations with 
biometric readers installed at airports and other U.S. points of entry. 
The $400 million program was launched in January 2004 in an effort to 
secure the border from terrorists by thoroughly screening visiting 
foreign nationals against scores of government watch lists.

While the idea of US-VISIT is universally lauded within government, the 
program's implementation has faced a steady barrage of criticism from 
congressional auditors concerned over management issues and 
cybersecurity problems. When Zotob began to spread last year, DHS' 
inspector general had just finished a six-month audit of US-VISIT's 
security; the resulting 42-page report, released in December, would 
conclude that the system suffered "security related issues (that) could 
compromise the confidentiality, integrity and availability of sensitive 
US-VISIT data if they are not remediated."

Zotob was destined to make those theoretical issues real.

The worm had its roots in a critical vulnerability in Windows 2000's 
plug-and-play feature that allowed attackers to take complete control of 
a computer over a network. Microsoft announced the hole Aug. 9, and it 
took only four days for a teenage virus writer in Morocco to launch 
Zotob, which spread through the security hole.

The workstations at the front end of US-VISIT run Windows 2000 
Professional, so they were vulnerable to attack. Those computers are 
administered by the DHS' Bureau of Customs and Border Protection, which 
learned of the plug-and-play vulnerability Aug. 11, according to the new 
documents. The agency's security team began testing Microsoft's patch 
Aug. 12, with an eye to installing it on more than 40,000 desktop 
computers in use in the agency.

But as CBP started pushing the patch to its internal desktop machines 
Aug. 17, it made the fateful decision not to patch the 1,313 US-VISIT 
workstations.

Because of the array of peripherals hanging off the US-VISIT computers 
-- fingerprint readers, digital cameras and passport scanners -- 
officials believed additional testing was needed to ensure the patch 
wouldn't cause more problems than it cured. The agency was testing the 
patch at a US-VISIT station at a border crossing with Mexico in Nogales, 
Arizona.

By that time, Zotob was already flooding DHS compartments like water 
filling a sinking battleship. Four CBP Border Patrol stations in Texas 
were "experiencing issues related to this worm," reads one report. More 
ominously, the virus had made itself at home on the network of an 
interconnected DHS agency -- the Immigrations and Customs Enforcement 
bureau, or ICE. The ICE network serves as the hub for traffic between 
the US-VISIT workstations and sensitive law enforcement and intelligence 
databases, and US-VISIT visibly slowed as traffic slogged over ICE's 
compromised backbone.

On Aug. 18, Zotob finally hit the US-VISIT workstations, rapidly 
spreading from one to another. Phone logs offer a glimpse of the mayhem 
that ensued. Calls flooded the CBP help desk, with callers complaining 
that their workstations were rebooting every five minutes. Most are 
explained in the "status" line of the log with the single word "zotob."

Though accounting for only 3 percent of its Windows 2000 machines, the 
US-VISIT computers quickly became "the largest impacted population 
within (the CBP) environment," reads a summary of the incident.

At international airports in Los Angeles, San Francisco, Miami and 
elsewhere, long lines formed while CBP screeners processed foreign 
visitors by hand, or in some cases used backup computers, according to 
press reports at the time. At CBP's data center in Newington, Virginia, 
officials scrambled overnight to distribute the tardy patch. By 8:30 
p.m. EST on Aug. 18, a third of the workstations had been fixed. By 1 
a.m., Aug. 19, 72 percent were patched. At 5 a.m., 220 US-VISIT machines 
were still vulnerable.

"In retrospect," reads an executive summary of the incident, "CBP should 
have proceeded with deploying the patch to the US-VISIT workstations 
during the initial push."

A spokeswoman for DHS' US-VISIT program office refused to comment on the 
incident this week. ICE declined to speak to the virus' infiltration of 
its backbone network, referring inquiries back to DHS.

While DHS and its agencies are taciturn about discussing security 
issues, they couldn't hide the travelers stranded on the wrong side of 
Customs at airports across the country. The day after the infection, DHS 
publicly acknowledged a worm was responsible. But by December, a 
different story emerged; a department spokesman speaking to CNET 
News.com claimed there was no evidence that a virus caused the August 
incident. Instead, the problem was merely one of the routine "computer 
glitches" one expects in any complex system, he said.

By then, Wired News had already filed a Freedom of Information Act 
request with CBP seeking documents about the incident. The request 
received a cool response. An agency representative phoned us and asked 
that we withdraw it, while refusing to answer any questions about the 
outage. When we declined, CBP misplaced the FOIA request. We refiled it, 
and it was officially denied, in total, a month later. After an 
administrative appeal went unanswered, we filed a federal lawsuit in 
U.S. District Court in San Francisco, represented by the Stanford Law 
School Cyberlaw Clinic.

After we sued, CBP released three internal documents, totaling five 
pages, and a copy of Microsoft's security bulletin on the plug-and-play 
vulnerability. Though heavily redacted, the documents were enough to 
establish that Zotob had infiltrated US-VISIT after CBP made the 
strategic decision to leave the workstations unpatched. Virtually every 
other detail was blacked out. In the ensuing court proceedings, CBP 
claimed the redactions were necessary to protect the security of its 
computers, and acknowledged it had an additional 12 documents, totaling 
hundreds of pages, which it withheld entirely on the same grounds.

U.S. District Judge Susan Illston reviewed all the documents in 
chambers, and ordered an additional four documents to be released last 
month. The court also directed DHS to reveal much of what it had 
previously hidden beneath thick black pen strokes in the original five 
pages.

"Although defendant repeatedly asserts that this information would 
render the CBP computer system vulnerable, defendant has not articulated 
how this general information would do so," Illston wrote in her ruling 
(emphasis is lllston's).

A before-and-after comparison of those documents offers little to 
support CBP's security claims. Most of the now-revealed redactions 
document errors officials made handling the vulnerability, and the 
severity of the consequences, with no technical information about CBP's 
systems. (Decide for yourself with our interactive un-redaction tool.)

That comes as no surprise to Steven Aftergood, who directs the 
Federation of American Scientists' Project on Government Secrecy. In the 
wake of Sept. 11, the Bush administration has been keen to expand its 
ability to withhold information from the public under the FOIA, and most 
commonly offers security concerns as the explanation.

"The Justice Department more or less explicitly told agencies to do so," 
says Aftergood. "Many requests yield greater disclosure on appeal, and 
time and again FOIA lawsuits succeed in shaking loose records that an 
agency wanted to withhold."

Despite the outward silence, it's clear Zotob left a lasting mark on 
DHS.

An inspector general report released a month after the US-VISIT outage 
recommended CBP reform its patch-management procedures; a scan found 
systems still vulnerable to security holes dating from 2003. And in the 
aftermath of the attack, CBP resolved to "(i)nitiate timely 
distributions of software and application elements for testing and 
pre-staging events," according to one of the internal documents.

Phone logs released under the court order show that Zotob lurked on 
CBP's networks as late as Oct. 6, 2005 -- nearly two months after 
Microsoft released its patch.

The call logs also show a lingering presence of Zotob in the agency's 
collective memory.

On Oct. 12, 2005, a user phoned the help desk to advise it of a new 
critical Microsoft vulnerability that had not been patched on the 
caller's machine. "The workarounds require administrator access," the 
caller is reported as saying. "I do not have admin rights."

"Please open a ticket to update my CBP laptop with the latest security 
patches from Microsoft," the caller says. "It is vulnerable, just like 
it was during the Zotob outbreak."


_________________________________
Visit the InfoSec News store!
http://www.shopinfosecnews.org