
I really wanted to hit on this last week when I first read it, but held off because of the hiatus. The summary is that the EVSC lost a month’s worth of grades due to a computer malfunction. Students will now just have an 8 week grading period.
According to EVSC’s release, IBM engineers determined the loss of data was caused by “an unfortunate and very rare combination of hardware problems and backup configuration settings.”
Translation: “We fucked up really really bad.”
Apparently IBM and the EVSC have not learned the value of concurrent backup solutions. When you’re dealing with personal data, it’s good to have at least a single backup source. When you’re dealing with enterprise data (read: academic grades), there’s no excuse for not having data backed up in multiple locations and via multiple methods: RAID5, rsync or even basic database dumps. Take your pick.
It’s shocking that a school corporation the size of EVSC doesn’t seem to have an offsite backup plan of any sort. With the cost of bandwidth and storage dropping so drastically in the past five years this is both shocking and frightening at the same time.
I run a small business with a few gigabytes of mission critical data and store it on Amazon’s S3 service for under $10 a month. Every hard drive in my office could fail, and I’d still have my client data because its backed up to the cloud.
“We had multiple hard drive failures in the system,” Michael Russ, Director of Technology for the EVSC said. Since the system crash, Russ said the school has installed a separate back up system for assignments and grades in a new location.
The fact that Michael Russ is still employed amazes me. When you are the IT director for a school corporation and lose student grades, you’re not just fired. You are SO FIRED. Someone in power has to go down for such a catastrophic failure and I doubt it will be the new Superintendent.
Perhaps this debacle will force the EVSC to reevaluate their IT infrastructure and backup regimen. To start, maybe they should read my backup column from a few weeks ago?
In the words of Bugs Bunny, “What a bunch of maroons.”
Thanks to WFIE for some of the quotes.
photo courtesy of anonfx