Constant patch releases force IT to adopt new processes

Constant patch releases force IT to adopt new processes
Constant patch releases force IT to adopt new processes 

By Jaikumar Vijayan
February 25, 2008 

The first Sunday after the second Tuesday of every month is a big day 
for the Arlington County, Va., IT unit's network operations team.

That's when the group gets to test and deploy the patches that Microsoft 
Corp. releases each month as part of its regularly scheduled security 
update process.

Some months, the team gets lucky and the vendor issues only a few 
security fixes.

On other occasions, such as this month, the county government's IT 
staffers aren't so fortunate. On Feb.12, Microsoft released fixes for 17 
vulnerabilities -- the company's biggest monthly patch release since 
February 2007.

Analysts and users said that such large releases can be overwhelming to 
some organizations, prompting IT staffers to look for ways to ease the 
patching process. Some shops, like Arlington County's, have created 
especially strong procedures for dealing with the problem.

Lou Michael, director of network and infrastructure services in 
Arlington County's department of technology services, said his 
organization began setting up formal processes for fixing software 
vulnerabilities after Microsoft moved to a monthly patch release 
schedule in October 2003.

Previously, Michael said, patch implementation was mostly handled on an 
ad hoc basis, and IT personnel were directed "not to touch the patches 
until there was some problem."

Microsoft's move to issuing patches monthly "has allowed us to plan for 
ourselves and to set expectations for our customers," Michael said. 
"We've added structure and some formality to our patching process. 
There's been a shift from being reactive [to threats] to having a plan" 
for addressing them.

The county now has a fairly mature process that enables it to assess, 
prioritize and automatically implement security fixes, Michael added.

"Folks are giving the entire patch life cycle more attention and higher 
priority," noted Pete Lindstrom, an analyst at Burton Group, an IT 
consulting firm in Midvale, Utah.

Big Workload

This month's "Patch Tuesday" release from Microsoft included fixes for 
widely used programs like the Windows operating system, Office 
applications, Internet Explorer and the Internet Information Services 
Web server. The list included five updates that were rated "critical" -- 
the highest rating in Microsoft's four-level threat-scoring system -- 
and 12 that were labeled "important," the second-highest rating.

"Overall, we [were] astounded with the quantity and size of the latest 
patches," said Matt Kesner, chief technology officer at Fenwick & West 
LLP, a law firm based in Mountain View, Calif. "This month's [patches] 
will cost us over 100 hours of IT time to test and apply. That seems 
excessive for a midsize enterprise like ours."

Jonathan Fan, senior director of product management at BigFix Inc., an 
Emeryville, Calif.-based vendor of vulnerability management products, 
noted that even companies that don't rely on Microsoft software are 
increasingly facing similar issues with products that run on non-Windows 
operating systems.

Several other major software vendors, including Apple, Oracle, Adobe 
Systems and Skype, issued fixes for corporate and consumer software just 
before Microsoft released its February patches, said Fan.

Setting Priorities

The increasing volume of patches has led some companies to create 
systems for prioritizing vulnerabilities to make sure the most critical 
ones are fixed first, said Matt Mosher, senior vice president of the 
Americas at Lumension Security Inc., a vulnerability assessment and 
patch management vendor in Scottsdale, Ariz.

Gone are the days when IT security personnel rushed to patch everything 
just for the sake of patching, he said. Companies must become more 
methodical and make sure that the most serious vulnerabilities are fixed 

"They are definitely trying to prioritize on the ones they feel pose the 
greatest risk," Mosher said. "They are trying to apply some risk 
assessment and risk scoring" to patching decisions.

Fenwick & West, for instance, prioritizes Microsoft patches, fixing 
critical vulnerabilities immediately and taking up to 30 days to fix the 
less important ones.

Regulatory and internal requirements have also helped push IT shops to 
adopt formal patch management practices, Mosher noted. Companies are 
increasingly required not only to securely patch their systems, but also 
to demonstrate auditable compliance with government and industry rules, 
he added.

"The issues have changed," Mosher said. "Companies have to apply more 
patches and prove that they are patching. It's a question of, 'How do I 
report on compliance?'"

Companies also need to ensure that vulnerabilities remain patched so 
that previously patched bugs don't reappear, Mosher added.

Fan noted that some companies have implemented multiple defenses, such 
as firewalls and intrusion-detection and -prevention systems, to try to 
reduce their dependence on patching. While such measures may have 
helped, they haven't eliminated the need for patching, he said.

Fenwick & West has "multiple layers of security," Kesner said. "We hope 
that gives us time to bring our systems up to date, but one never knows 
if that is true -- except in hindsight.

"The six layers of antivirus, antispam and anti-malware we run don't 
reduce the need to patch," Kesner added. "They just give us hope that we 
have breathing room."

According to Michael, Arlington County's approach is to guard against 
vulnerabilities as well as patch them. It's akin to wearing a "belt and 
suspenders," Michael said.

Automation Helps

The emergence and relative maturity of automated patch management tools 
from vendors like BigFix and Lumension have also been catalysts for 
corporate change.

BigFix's policy content modules for patching and Lumension's PatchLink 
Update tool automatically scan networks for disclosed vulnerabilities 
and check to see whether patches for them have been applied.

When new patches become available, the agent-based technologies from 
both companies inspect each endpoint to see if the installed patches are 
working. If necessary, the tools can automatically fix unpatched 
vulnerabilities, according to officials at both vendors.

The tools can also monitor a system to see if changes are made that 
could once again leave it vulnerable. In addition, such products enable 
companies to roll back patches in case they disrupt other applications 
or cause them to crash.

Fan noted that some companies are also looking to integrate patch 
management practices with broader configuration management and 
vulnerability assessment and remediation processes.

"People are interested in seeing a single view" of vulnerabilities, he 
said. "They are trying to understand their security posture and have 
more visibility and controls over all of the software" in heterogeneous 

"It's about security configuration management," Fan said. "What are the 
security standards for my desktops and servers? What are the 
configurations, and how do I make sure I don't drift? How do I know in 
real time if a patch that came out for a vulnerability is something I 

One of the challenges with something like Microsoft's Patch Tuesday, Fan 
said, is that "as an IT organization, you have 11 different issues that 
you need do deal with, so how do you buy time? We are seeing a movement 
toward understanding" such issues.

Subscribe to InfoSec News 

Site design & layout copyright © 1986-2015 CodeGods