Tests like these are why you buy the SCORM Engine.
“What is that?” you might ask. This is a dashboard widget that we maintain on our big screens in the two offices. Anytime you walk through the common space, you get a quick look at this dashboard. Each row here represents a fundamentally important automated test of the SCORM Engine. Green is good news; pink is bad. (Truth be told, I’ve been waiting several days to catch some portion of this screen pink. Things are very stable around here right now, and I thought an “all green” dashboard was a bit contrived. Further truth be told, catching this screen shot half pink might be retaliation for David eating my ice cream yesterday.)
Over the summer, Andrew, our intern, spent his time automating the SCORM test suites. SCORM test suites are anything but fun the first time, and going through them over and over is torture. We’ve done this plenty of times (as has Andrew) and it’s time-consuming enough that no one does it as often as they should. Our response? Automation.
The tests listed in the screen shot are run on different periods, but each confirms the current health of various versions of the SCORM Engine. 2008.1.x references are to the currently released product. “Trunk” references are to our forward-looking development. And you can also catch a glimpse here of the fact that we’re already testing against SCORM 2004 4th Edition, which is still in beta. Those who are perceptive will likely notice that last night, all of the Java tests failed. This is because we’re doing a bit of internal restructuring and we broke them. And you know what? That’s OK, because we immediately know that they’re broken, and David and crew will spend time today fixing them.
Would you or your organization ever take the time to create an automated testing platform like this? Maybe. Would you do so for the vagaries of the SCORM test suites? Would that be an effective use of your time? I doubt it. Most development shops would make a reasonable effort to test this stuff… once, when they create it, and hopefully again before they send it off for certification. Would they take the time, then, to test it monthly? Or before each release? As you see here, the SCORM Engine is being tested, in all of its flavors, every single day. This is not just a confirmation that it builds, either, this is a full fledged run through the entirety of the SCORM test suite.
Even at its best, SCORM is touchy. Things break, content can be bad, there are plenty of ways for things to go wrong. You need to know the quality of your delivery platform, whether you buy it from us or build it yourself. Tests like these are why you buy the SCORM Engine.
Update, 1/23/09: If you want to get developers in a tizzy, post a picture that shows their stuff is broken. 12 hours later, you get a completely green dashboard.
