Digital Storage proves problematic

http://news.bbc.co.uk/2/hi/technology/6265976.stm

[quote]The National Archives, which holds 900 years of written material, has more than 580 terabytes of data - the equivalent of 580,000 encyclopaedias - in older file formats that are no longer commercially available.

Ms Ceeney said: "If you put paper on shelves, it's pretty certain it is going to be there in a hundred years.

"If you stored something on a floppy disc just three or four years ago, you'd have a hard time finding a modern computer capable of opening it."

"Digital information is in fact inherently far more ephemeral than paper," warned [chief executive Natalie] Ceeney.

She added: "The pace of software and hardware developments means we are living in the world of a ticking time bomb when it comes to digital preservation.

"We cannot afford to let digital assets being created today disappear. We need to make information created in the digital age to be as resilient as paper."

But Ms Ceeney said some digital documents held by the National Archives had already been lost forever because the programs which could read them no longer existed.

[/quote]

And this:

http://www.variety.com/article/VR1117963533.html

[quote][b]Digital proves problematic ;
Industry lacks method to store footage[/b]

By DAVID S. COHEN -- "Variety"

As far as movies are concerned, digital, like diamonds, was supposed to
be forever.

No more dyes to fade, no more film stocks to decay or catch fire. Just
pristine digital data, preserved for all time, and release prints as
clear and sharp as the images caught by the camera.

Just one problem: For long-term storage, digital is -- so far --
proving to be a time bomb, more permanent than sand painting but not much else.

Simply put, there's no generally accepted way to store digital
"footage" for more than a few months. After that the industry is using a hodgepodge of improvised solutions, some rather costly, others not very reliable.

That looked like a small problem when digital filmmaking was limited to low-budget indies, animation houses and tech pioneers like James Cameron and George Lucas.

Now, though, that small problem is growing geometrically as the major
studios shift away from film to digital capture. Such recent releases
as "300," "Apocalypto," "Zodiac" and "Superman Returns" were shot on
digital. Their digital masters could be seriously degraded if the problem isn't addressed quickly.

In fact, the problem is so severe that the Academy of Motion Picture
Arts & Sciences' Science and Technology Council warned in 2005 that within just a few years films shot with digital cameras could be lost.

Two years on, digital is going mainstream, but "The problem is still
there," says Phil Feiner, chairman of the Acad Sci-Tech Council's archiving committee. And those few years the council warned of are nearly up.

It's not that there's no way to store digital data. On the contrary,
there are dozens of ways to store it, most of which go obsolete in just a few years. Remember 5" floppies and Zip disks?

And the disks that have stuck around? Not so reliable.

Data tapes are balky and can fall apart. Data DVDs and CDs have a history of "rotting" and can't be counted on to last as long as their commercially pressed cousins.

Plus there's no reason to expect that the computers of 20 years from
now -- never mind 100 -- will be able to plug in to today's hard disks.Some private companies are jumping in as awareness of the problem grows, and Feiner's committee will be launching several initiatives over the months to come.

But the amount of digital footage that needs to be archived is growing
faster than ever.

More than one tech expert, including the Academy's Sci-Tech Council
director Andy Maltz, told Variety they had found archival tapes unreadable just 18 months after they were made.

Feiner, the former longtime prexy of Pacific Title, says when he worked
on studio feature films he found missing frames or corrupted data on 40% of the data tapes that came in from digital intermediate houses.

The tapes were only nine months old.

"On certain pictures we had to go into the DI negative and re-scan the
data," he says. "You couldn't retrieve it. Gone."

Milt Shefter, who is a team leader on Feiner's digital archiving
committee, warns that "Long term, it's possible that we're looking going back to the early days of motion pictures, where films are made, put out for a week or two, then thrown away."

With acetate or polyester film, the typical approach to archiving has
been summed up as "store and ignore."

Color film can be turned into black-and-white color separations on
polyester stock. Properly stored in cool vaults at low humidity, such film can last centuries. But there's no way to "store and ignore" digital.

Instead, digital data has to be copied, or "migrated," to new storage
every few years. Migration, however, takes computers, an IT staff, software and a lot of labor. In short: money.

While indies may lack the funds to do regular migration, studios are
plunging in.

Sony's VP of asset management and film restoration, Grover Crisp, says
the studio has put in a program of migrating every two to three years.

"The motion pictures and original material, those are primary assets of
the company," says Crisp. "We all want to do whatever we can to protect those assets."

Disney's VP of production technology Howard Lukk, says as the studios'
digital archives grow, migration becomes a bigger job.

"It's like painting the Golden Gate Bridge and it getting a foot longer
every year."

Not only are more films shot digitally now, but digital filmmaking
encourages directors to shoot more footage.

"The technological issues here are not going to be solved by the
entertainment industry," says Shefter. "It's going to take big
business, big science and maybe big government."

In the meantime, the Academy is stepping in to make the motion picture industry's voice heard in any big business initiative to solve the problem.

The digital archive project is the broadest initiative launched since
the Academy decided in 2003 to fund the current incarnation of the Science & Council.

Maltz expects a report that will pin down what the industry needs to do
to be released in a few months.

Meanwhile, private industry is attacking the problem of digital archiving, too, with at least one announcement in the field planned for NAB.

At NAB, Elektrofilm Digital Studios and Sun Microsystems announced a
service to manage and archive the vast amounts of video from feature film production.

Many tech experts expect the studios to eventually outsource all their
archiving and migration to companies like Elektrofilm rather than try
to do it themselves. Feiner says what is happening is, in effect, the birth of a new business: digital archiving.

He speaks from experience. Earlier this year, three companies received
Science & Technology Awards for their work on archiving. Feiner and his
Pacific Title team were among the winners.

[b]Their solution takes the data from a digital intermediate and turns it
into three-color separation negatives. In other words, they take the digital movie and turn it into good old-fashioned film.[/b][/quote]

"EustaceScrubb" has left the building