Spreadsheets Everywhere

In October last year (long before the vaccines came along), there was an amusing-if-it-weren’t-so-serious incident where the UK’s COVID contact tracing system “lost” 16,000 cases. The risk this created was high, writes Tim Harford:

“These were 16,000 people who should have been warned they were infected and a danger to others, 16,000 cases contact tracers should have been running down to figure out where the infected went, who they met and who else might be at risk. None of which was happening.”

 

How had that happened? Because of a limitation in MS Excel: the older “xls” format supported only 64,000 rows. Enter any more and they wouldn’t get saved:

“This meant that during some automated process, cases had vanished off the bottom of the spreadsheet, and nobody had noticed.”

 

Digital spreadsheets have replaced the original paper spreadsheet, those “big sheets of paper spread across two pages of an accounting ledger”. It started with VisiCalc, then Lotus 1-2-3 until MS Excel took over. Suddenly:

“Any idiot could use it (spreadsheet). And goodness, we did.”

Spreadsheets are used for reasons totally unrelated to accounting. But since Excel had been designed for accounting, other errors can creep in when used for those other purposes. Like:

“Type an international phone number into Excel, for example, and the program strips off the leading zeroes, which are redundant in a mathematical integer but not in a phone number. If instead you type in a twenty digit serial number, Excel will decide those 20 digits are a huge quantity and round them off, turning the last few digits into zeroes.”

Or take genes with names like “Membrane Associated Ring-CH-Type Finger 1”, or March1 for short. Enter the shortened name into Excel, and it will convert it into a date! In all of the above cases, the data just got corrupted, without anyone noticing. All because a tool designed for accounting is now used for anything and everything.

 

Unfortunately, spreadsheets are not going away and solutions like what the geneticists came up with (rename the genes) aren’t possible in all cases. It’s simply not practical to have checks on data entry everywhere, nor are people ever going to double-check everything they enter. All of which is probably why Harford’s excellent article is titled, “The Tyranny of Spreadsheets”.

Comments

Post a Comment

Popular posts from this blog

Why we Deceive Ourselves

In their book, The Elephant in the Brain , the authors describe this weird aspect of deception that we practice: deceiving ourselves! That is so weird: “If our minds contain maps of our worlds, what good comes from having an inaccurate version of these maps?” The Sigmund Freud school of thought treats self-deception as a defense mechanism, “a way for the ego to protect itself”. The mind seeks to protect itself from anxiety and other negative emotions. Many object to this reasoning: since accurate information is so critical to our survival, surely distorting information would be dangerous. Wouldn’t the goal of protecting our self-esteem have been better achieved by making the brain’s self-esteem mechanism stronger instead? The new school of thought explains self-deception as “primarily outward-facing, manipulative, and ultimately self-serving”. This is based on Thomas Schelling’s work on game theory: “In a variety of scenarios, limiting or sabotaging yourself is the winn...
Read more

Europe #3 - Innsbruck

The last city we visited in Europe was Innsbruck in the Austrian Alps. After the cold and windy experience (at times) of Paris and Netherlands, Innsbruck was a wonderful change – it didn’t feel that cold, it was not windy all of which made it easy to walk around. Something absolutely necessary given how scenic the place is.   Since the hotel check-in time was much later in the day, we left the baggage and went to the Swarovski HQ-cum-museum about an hour by bus from Innsbruck. The crystal exhibits were brilliant – colorful, bright, shiny. Jewellery aside, it was a display of what one can do with crystals. While my wife and daughter loved the jewellery, I liked the superhero themed exhibits of Batman, Spiderman, the Hulk, and Star Wars. We didn’t buy anything there, though we did buy stuff in the Swarovski store in Innsbruck – jewellery, not the superhero stuff.   Innsbruck is a major tourist attraction during the skiing season but now in summer, we saw many groups of ...
Read more

The Thrill of the Chase

So many people complain about the difficulty in making decisions while shopping. Do you buy Brand X that gives 20% extra free? Or Brand Y that has 10% off? Or Brand Z that gives something else free with it? Do you buy that flat screen TV today or wait for the sale at Diwali? Now since the customer is king, shouldn’t companies be simplifying pricing choices? Like maybe offering consistently low prices all the time? Not so fast, as the US store, JC Penney discovered to its cost. Turns out that while people do complain about pricing choices, they are irrational. Also, they enjoy the thrill of the chase. Huh? The word “sale” acts like a siren call for many people: it lures them in to make that purchase. Consistently low prices don’t do the same. Besides, if a competitor lowers their prices just one time of the year, and Poof! Suddenly that low price claim becomes suspect in shoppers’ eyes. Then there are those sites that offer coupons: have lunch at Restaurant A on Wedne...
Read more