Data Formats

One of my dad’s hobbies was photography. Apart from regular photos, he did trick photos and even had a slide projector for slide shows (today’s equivalent technology for that is a Power Point presentation of digital pics projected onto a giant screen). One thing he didn’t do though was to create videos. Camcorders hadn’t come to the market at that stage, I guess.

Notice what’s the common theme in the above paragraph? Apart from creating memories, that is? The answer: obsolescence. Camera rolls, analog cameras, slide projectors, camcorders: all dead or dieing or very niche.

But nowadays even the format in which content is stored becomes obsolete at unbelievable speed. In the book, This is Not the End of the Book, Jean-Claude Carrière talks about the transition from video to CD-ROM to MP3/MP4’s to streaming (video on demand via the Net):

“It may seem as if I’m talking about things that changed over a very long time-span, a matter of centuries. But all this has taken place in barely twenty years.”

The speed of obsolescence of the format leads to an ironical situation:

“So we can still read a text printed five centuries ago. But you can no longer read, or rather watch, a video or CD-ROM that is only a few years old.”

Vincent Cerf, one of the pioneers of the World Wide Web, worries about the same problem (there’s even a technical term for it in the digital context: “bit-rot”):

“I am really worried right now, about the possibility of saving ‘bits’ but losing their meaning and ending up with bit-rot. This means, you have a bag of bits that you saved for a thousand years, but you don’t know what they mean, because the software that was needed to interpret them is no longer available, or it’s no longer executable, or you just don’t have a platform that will run it. This is a serious, serious problem and we have to solve that.”

The Net, in fact, was designed to prevent this problem: the format we know as HTML consciously avoids proprietary, twisted formats (like the ones used in Word docs or PDF’s) so that anyone can create a browser tomorrow to display the content.

But what about situations where we already have a legacy of proprietary formats? And commercial interests that want to continue to create proprietary formats? Funnily enough, the very same market forces may help solve that problem. For example, Google docs allows you to open and edit MS Word docs; Android and iOS have tools to migrate data from one format to the other to make it easy for you to switch from one phone to the other. And so the cause of the problem (market forces) may be also our best hope to avoid bit-rot in future!