So Much Storage, So Little Need

An offer I received in the daily flood of spam email has had me pondering all week about computer storage and how it has changed over the last five decades.  Specifically, what started all of this was Newegg’s offer to sell me a terabyte solid-state hard drive for only $99.  And that included free delivery.  For those of you unsure, a terabyte is a thousand gigabytes or a million megabytes. (Or, as we used to say in Texas, a whole shit pot lot full!)

I distinctly remember that as a couple of other computer nerds and I were sitting around shooting the breeze many decades ago, we wistfully surmised that whoever was the first to have a terabyte of memory would rule the world—proving that any one of us was just a lab accident away from becoming the world’s first super villain.  (A theory since proven correct by Google). 

This amazing hard drive is just a little bigger than a deck of playing cards and due to its solid state (no moving parts) is faster than a stabbed rat—it can damn near give you your data before you ask for it.

While I have no need for the drive, I cannot stop mentally comparing it to earlier hard drives I have worked with.  In a very short time, technology has improved by a staggering amount. 

In 1979, I wrote a modestly small database program for a modestly small city’s police department.  Laughably simple by today’s standards, at the time it was the cutting edge of software for the emerging field of personal computers.  (Remember, this was several years before the IBM PC hit the market.)  My program kept track of parking tickets, allowing the operator to search for unpaid tickets by searching on any field.

At the time, no one was keeping track of the parking tickets.  If someone came to the city hall to pay a fine, the clerk had to search though a pile of thousands of outstanding tickets.  If a ticket was misfiled, it simple vanished, likely to never be paid or even located. 

When I offered the program to the police department, there was only one small hitch in the offer, it would require the police department to purchase a Winchester Drive.  Today, that term has fallen out of use, I’m not sure that many people even know it refers to, but in the Seventies, it was written WINCHESTER DRIVE, and hinted at the kind of incredible technology that could only be found at NASA.

Developed by IBM, a Winchester Drive was the first practical hard disk drive: a mechanical means of storing data on large rapidly spinning plates.  Sort of a bastard cross between a record player and a tape recorder, the first such commercially available drives were 24-inch platters spinning hundreds of revolutions per minute.  Since the first model held two 30 megabyte platers, the IBM project manager called the device a “Winchester Drive” after the venerable .30-30 rifle used by John Wayne and almost everyone else who has ever gone deer hunting. 

I should point out that 30 megabytes is—by today’s standards—almost nothing.  A letter to my grandkids uses up about half a megabyte.  Thirty megabytes would not store the email I receive in one day.  The smallest iPad sold today has over 4000 times that much memory.

Personal computers were rare at the time and the vast majority of the ones available used cassette tapes for storage.  Only a very few could afford the $500 it took to buy a floppy disk drive, and those available would store only about 75 kilobytes (roughly the size of a single email message today).

The police department was willing to buy the program from me, but only if I also sold them the required hard drive and installed the software on it—a condition I eagerly agreed to.  This was the first piece of computer hardware I ever sold, and eventually would lead to my opening up a couple of computer stores.

The hard drive I selected was 10 megabytes, 8.3 megabytes after the drive was formatted.  It was a 24-inch platter spinning at 300 RPM, housed in a yard square metal box about 10 inches tall.  It took a special table to handle the weight and soak up some of the vibrations.  It was a monster.

And it was expensive—$3900.  That doesn’t sound like much today, but it was the same price as a new Chevrolet Malibu.  And the Malibu would probably last a lot longer.  The expected service life of a hard drive was only 3-4 years.  In order to purchase it, I had to get a short-term bank loan.

Luckily for me, everything worked out well.  Collections on old parking tickets increased dramatically, the proceeds paid for the entire installation within two years.  The hard drive worked well, and five years later the program and the data were transferred to a larger and faster computer.  Eventually, I rewrote the software to work simultaneously on two computers—a baby network!—and the software was still being used a decade after it was originally written.

By today’s standards, the whole operation was a joke.  As a test, I recreated the application on my iPhone using HanDBase software.   It took less than 5 minutes, and my phone is faster and holds more storage. 

While I lust to own that new solid-state drive, I really don’t need it.  A terabyte is a hell of a lot of data storage.  Back in the nineties, Microsoft was looking for a way to publicly prove the scalability of its software, so it decided to put a database of over a terabyte online for public inspection.  The problem was finding a set of data that large.  (By comparison, the complete works of Shakespeare would take up about 1 millionth of a terabyte.  Add the Old and New Testaments, and we are up to 2 millionths.)

The original plan was to publish every single transaction of the New York Stock Exchange since inception, and make the data public and searchable.  Unfortunately, it turned out that such a massive database would only use up about half a terabyte.   Even if you threw in all the Chicago commodity trades, it was still far short of a terabyte.

Finally, in 1997, Microsoft put online its Terraserver, a database containing the satellite photos of everywhere on Earth, a database of 2.3 terabytes.  The physical server was a Compaq computer (basically the size of a one-story office building 25 feet wide and 45 feet long), briefly the largest computer in the world.  If we were to duplicate that server today, it would indistinguishable from any other desktop.

I can remember spending hours with my geographer friend, Jack Wright, staring at what previously had been unthinkable.

Yes, Microsoft invented what today we would call Google Earth, a free service that still uses an overlaying music interface that Microsoft invented.  Microsoft wasn’t really interested in providing data for the consumer, or even good service, so they dropped the ball and Google snapped up the market, swallowing it whole.

Personally, I am unlikely to ever need that much data storage.  I’ve been writing this blog slightly in excess of ten years and have calculated that at the current rate of accumulating copies of the files associated with this weekly screed, I would use up a terabyte sometime in May, 3815. 

But, I may buy that drive anyway.  My wife, The Doc, could fill it with cute cat photos in about a week.