OPEN GOVERNMENT — What good are public information access laws if the state is allowing its records—in electronic form—to simply disappear wholesale? Anthony Pignataro, reporting for CalWatchdog, says no one really has a handle on what's happening.

If there’s one thing our state government does exceptionally well,
it’s the production of records. Every month, dozens of state agencies
and departments — to say nothing of the Legislature itself — churns out
thousands of pages of reports, studies, analyses, papers and fact
sheets, as well laws, orders, directives and, of course, regulations.
Seriously, it’s a lot; anyone suffering from insomnia should check out
the California State Library’s monthly index of new government publications.

Given the way computers permeate government, the vast majority of
these records exist in digital form. In fact, State Library officials
estimate that “80 percent to 90 percent of all California state
publications are now issued on the Web. For many of these there is no
print counterpart.”
Posting documents online was meant to increase the public’s access
to the workings of government, but it’s actually having the opposite
effect. For years now, official government records created
electronically have been vanishing. And while a June 21, 2009
University of California press release makes mention of “the wholesale disappearance of information,” no one seems to be able to quantify the extent of losses.

“The problem is, I don’t think anybody has done a scientific
evaluation of exactly how many electronic-only documents of California
state government are disappearing,” David Cismowski, the State
Library’s bureau chief for library services, e-mailed on Jan. 7.
The state Office of the Chief Information Officer (OCIO)
also couldn’t say for certain what has been lost. “While we are not
aware of any critical documents that are lost or being destroyed, given
the exponential growth of digital media the OCIO has been focused on
the issue of records and document management,” OCIO spokesman Bill
Maile e-mailed on Jan. 11. “Whether it is a digital photograph, video
file, spreadsheet or word processing document, we are modernizing our
approach to organizing, storing and managing electronic records.”

Terry Francke, general counsel of Californians Aware
— a Carmichael-based non-profit organization that advocates for open
government — wasn’t surprised by the news that state e-records have
been disappearing for years.“I don’t believe that there’s any records retention law that applies
generally to documents of the executive branch,” he said. “Without some
kind of legal requirement, agencies are left to their own devices.
Without knowing what’s being lost or at what rate, it’s easy to over-
or under-estimate the importance of this. But I can imagine it’s like a
warehouse full of records burning down every six months.”

None of this is new. In fact, the deletion — accidental or
purposeful — of state e-records has been going on a long time. This is
made clear by examining two state reports, both released in August
2004. Ironically, despite the fact that they’re nearly six years old,
the reports represent the most recent studies of the loss of government
e-records.
The first is Managing and Sustaining A State Government Publications Program in California, written by Judith Cobb and Gayle Palmer of OCLC,
a library services consulting firm. It’s a thorough analysis of the
State Library’s function as a depository for government records that’s
uncompromising in its conclusions.

“Now that most state government publications are available only
through the World Wide Web, the [California State Depository Library]
program fails to fulfill its mission because there are few mechanisms
in place to preserve those digital publications and provide access to
them over time, or even to notify librarians and the public about their
existence,” Cobb and Palmer wrote, who estimate that the state
government spends about $2 billion every year on its “current
technological infrastructure.”

“The resultant loss of state government
information is untold… The disappearance of state government
information has present and future implications for ever Californian.
Preservation of, and permanent access to, this information is
imperative; the state’s historical, cultural, and intellectual record
is at stake.”
According to Cobb and Palmer’s report, “digitally published
documents are dynamic, volatile and uncontrolled.”

Software and
hardware changes may account for some losses. Sometimes agencies will
dump some records to make room for others. Still other records may
disappear through “link rot” — “information that has become
inaccessible because of an invalid link to a Web page, deletion or
removal of Web sites, and/or loss of access to information previously
published on Web pages.”
Then, especially in the case of e-mails, there’s this view from First Amendment Coalition’s Peter Scheer: “People feel compelled to delete things.”

Made public around the same time as Cobb and Palmer’s report, the California Performance Review
— Governor Arnold Schwarzenegger’s big “blow up the boxes” look at
state government that never really went anywhere — also identified the
disappearance of state records as a big problem. Recommendation GG 45,
“State Digital Records Vanishing,” stated that because “Many digital
documents are deleted or otherwise lost each year… the governor should
issue an executive order that requires all state agencies to alert the
State Library of publication of digital documents, Web sites or other
products that may be candidates for permanent public access through the
State Library.”

To the best of Cismowski’s knowledge, GG 45 “was not fully
implemented.” But Cismowski did say that libraries statewide are
working on preserving government e-records.
“[T]he State Library has worked with the California Digital Library (CDL) and State Archives to study ways of preserving these vital electronic publications,” Cismowski e-mailed on Jan. 6.
For instance, there’s the CDL’s Web Archiving Service,
which preserves access to a pretty random collection of old websites of
historical import from all over the world. There’s the 2003 California Recall Election pages, a group of sites from the Guantanamo Bay Detention Camp, a wide variety of sites dealing with labor unions, anarchism and even Trostkyism, as well as various local government Web sites from around the state. Then there’s the far more massive Online Archive of California that gives the public online access to libraries and research centers across the state.

How easy it is for researchers — to say nothing of members of the public — to access these multiple archives is another story.“Searchability is critical to access,” Scheer said. “If there’s no way to find something, then it might as well not exist.”