The SCORM community is abuzz these days with talk about the security (or lack thereof) in SCORM. As an “alpha-scormmie”, I’d like to share some of my perspectives on the issue and try to put things in context.
The crux of the issue is that since SCORM communication uses JavaScript in a web browser it is inherently insecure and can be spoofed by any semi-competent web developer who knows a little bit about SCORM. This means that somebody who knows what they are doing can trick an LMS into thinking that a course was completed using some rather simple scripting. The fact that SCORM makes it easy for content to communicate with the LMS also makes it easy for a hacker to maliciously communicate with that LMS.
This issue is not new. It has been widely known for quite some time. Fellow alpha-scormmie Tom King has been talking about it a lot recently and even submitted a white paper to LETSI about it. The recent collective panic ensued after a blog post by Philip Hutchison which included a downloadable application that allows anybody to execute the cheat. Apparently, the cat is now out of the bag. Philip’s post caused major LMS vendor Plateau to send an email to its clients warning them of this “new” vulnerability. That, in turn, has led to a frenzy of discussion and a calls to address the problem now, to which ADL has responded with some modest attempts at stopgap solutions. Before we all start to panic, let’s take a step back and look at this with some perspective.
[And a note before I go much further…I fully agree that SCORM should be made more secure, please don’t doubt that. This post is designed to put security in perspective, calm the panic and ensure that whatever solution we adopt moving forward doesn’t sacrifice other important principles in the name of security.]
Defining “Secure” Online Training
Before we can start talking about how to secure online training, we need to understand what “security” means in this context.
There are two primary functions of online training:
- Delivering instructional content to the learner
- Assess the learner to ensure that delivered knowledge has been retained
That leads to the following definition of “secure online training”:
Secure online training ensures that the identified learner has “experienced” the intended instructional content and retained the knowledge conveyed therein.
Notice that there are three important parts to that definition:
- “the identified learner”
- “experienced the intended instructional content”
- “retained the knowledge”
Why Do We Want Secure Online Training?
If we want to secure something, there is usually a reason, i.e. something of value we want to protect. There must be something at stake. Online training is typically used in two contexts that provide valuable stakes:
- Delivering training to meet a compliance requirement. In the compliance context, we want to ensure that the learner “experienced the intended set of instructional content”.
- Instructing learners so that they are competent in new areas and ensuring that they are proficient enough to perform at a level suitable to their task. In this context, we want to ensure that the learner “retained the knowledge”. This retention may lead to a promotion or other reward based on his or her new skill.
These stakes are certainly high enough to merit attention and to tempt some to cheat the system. Of that there is no doubt.
SCORM insiders have often remarked that “SCORM is not intended for high stakes assessment”. However, with the proliferation of online training, the stakes are rising. The argument for increased security in SCORM asserts that the stakes are now high enough that this simple dismissal doesn’t suffice anymore. As Tom King said “it’s all low stakes until someone’s attorney gets involved”.
But, is this a SCORM issue?
Perspective – Security in Online Training
The “SCORM cheat” allows a learner to spoof two of the three aspects of secure online training. He can assert that he “experienced the desired delivery of instructional content” (i.e. completed the content) when he did not and he can assert that he “retained the knowledge” (i.e., passed the test) when he did not.
Let’s pretend for a moment that we implement Fort Knox level security into SCORM. Assume that there is absolutely no way for a malicious user to alter the communications between the content and the LMS. Will we then have achieved secure online training? Will we then have something that is good enough for “high stakes”? Not really. Not in any of the three areas required for secure online training.
Most fundamentally, how are we sure that “the identified learner” is the one actually taking the online training? How do we close the security vulnerability of “offering to buy my buddy a pizza” if he will click through my training while he is doing his anyway?
How do we ensure that the learner is really “experiencing the intended delivery of instructional content” and not just watching YouTube videos while mindlessly clicking through the content?
How do we ensure that the learner has really “retained the knowledge” and isn’t just looking up the answers on Google or asking his buddy in the next cubicle what the test answers are? Online training is an open book test.
These are gaping holes in the security of online training. Without the presence of a proctor, there is no way to ensure that the identified learner is actually doing anything of value. These problems are intrinsic to online training, no technical standard can overcome them. If “secure training” is an absolute requirement for an environment with stakes, then online training will never live up to the requirements.
Given these fundamental vulnerabilities, how much does the presence of a technical hack move the needle on the overall security of online training? Not very much.
As the technological experts and practitioners, we should make every effort to ensure that the solutions we provide are of the highest possible quality, but let’s not loose sight of the larger picture and start to panic.
The question becomes “at what level of stakes is the risk of cheating (both technical and non-technical) tolerable?” Certainly we wouldn’t certify somebody to fly a 747 or perform brain surgery based on the result of an online exam. I would venture to say, though, that despite being an OSHA-mandated compliance requirement, the possibility of cheating on annual refresher training about “Preventing Back Injury” is tolerable. Should promotions be based on the results of online training that could potentially be cheated? Not solely, but if potentially compromised training results contribute to an employee’s evaluation it certainly won’t be the only, or most damaging way for employees to game the system.
There has been some implication that the “public” is unaware of the SCORM security issue and won’t tolerate this vulnerability. My perception is different…although admittedly it is just my perception and certainly there are many people out there for whom this opinion is valid. I have two pieces of evidence to support my theory though. The first is the proliferation of secure assessment tools. Vendors like Questionmark specialize in offering these tools and most major LMS’s offer their own built-in secure assessment framework. I see the demand for these tools as acknowledgment that assessment provided by the content is not always valid. Secondly, I know that the public is aware of the non-technical potential cheats mentioned above. If the public judges those risks as being acceptable for their current stakes level, it seems to me that the possibility of a SCORM cheat would also fall into the level of acceptable risk.
[Again, I want to say that I do think we should make SCORM more secure. Simply because it is not strictly necessary doesn’t mean we should be complacent or do less than our best. We do need to consider security in its larger context and measure it’s worth against other important, and often competing, design aspects.]
An Analogy
There have been several comments to the effect of “if we can conduct very high stakes operations like finance electronically, then surely we can secure training”. It’s true, we can conduct very high stakes operations online, but that’s not to say that they are invulnerable. They are simply strong enough that the utility of their use significantly outweighs the pain caused by their misuse. This is the heart of security. There has to be enough security to prevent rampant misuse without overly interfering with the mainstream legitimate use.
SCORM is all about interoperability. It is about the seamless and easy transfer of data between different systems. It reduces the friction of transactions between parties and allows almost all learning systems to work together.
What is the “SCORM of the financial industry”? What is the de facto tool for easily and seamlessly transferring money between parties? What reduces the friction of everyday financial transactions and works virtually anywhere? It is the credit card.
Are credit cards completely secure? Nope, not by a long shot. In fact, I’d venture to say that SCORM is less vulnerable to misuse than a credit card is simply because of the technical knowledge required to perform the misuse.
Credit cards are central to the financial system, and money is by definition “high stakes”. There is huge temptation to cheat the system. Yet, all I need to cheat the credit card system is the numbers and name on the front of your card. Well, of course, for those really secure credit card systems, I also need the 3 numbers from the back of your card…and, of course for those really, really secure online systems, I also need to know your billing address (there’s no way the waiter who swipes your credit card at Chilli’s will ever be able to find that in the phone book, right?).
Of course credit cards aren’t completely secure. Credit card fraud happens all the time. In any system there are trade offs. One well known and intractable trade off is the balance between ease of use/convenience/low friction/interoperability and security. In the financial industry, the market has decided that the price of insecurity (fraud by a few) is worth the added convenience of credit cards. The utility of the credit card overcomes the cost of the fraud.
It is the same way with SCORM. The utility of SCORM’s ease of use, interoperability and portability outweigh the possibility that some people will cheat the system. Adding any kind of real security to SCORM would negatively affect the ease of use (i.e. development) and portability SCORM content. In light of the inherently insecure environment in which SCORM operates, this doesn’t seem like a bad design trade off.
What Should Be Done Today?
The SCORM vulnerability arises from the very essence of the standard. The ECMAScript API implementation is fundamentally exposed to the web browser and as such can not be made secure from a web developer armed with a tool as common as Firebug. Any change to the specifications that retains the ECMAScript API will only very marginally improve security while (potentially) inflicting large amounts of pain on implementers. There is no modification to the specifications that I am aware of or can imagine that will “move the needle” significantly enough to warrant any change of implementation. But there is something that we as a community can do to improve security.
All secure systems have two parts. The first prevents a security breach (the lock on the door). The second detects a security breach when it happens to enable corrective action (the burglar alarm). The solution to SCORM security in today’s world isn’t to add a more complex lock, it is to add a burglar alarm. Making the lock more secure makes it harder to get in and out of the door every day. A more secure lock decreases the usability of the system and doesn’t add significantly to security (especially when there’s an easily breakable window right next to it).
The solution to “SCORM fraud” (or more broadly “online training fraud”) is the same solution that the credit card industry uses, monitoring and detection. Credit cards could be made exceptionally secure, but their utility would go down significantly. Instead the industry has focused on detecting fraud, stopping it once it starts and punishing offenders. The training industry can do the same thing with SCORM; we just need to acknowledge that some fraud it is going to happen. Misuse is an inherent part of any system.
This monitoring can be done with data that is already tracked in LMS’s and for which many LMS’s even have existing reports. The appropriate security response from ADL is to issue a set of guidelines for LMS vendors that allow them to create “cheating alerts” that can be provided to concerned clients. Some simple yet powerful heuristics that can be easily monitored right now include:
- Comparing the session time reported by the SCO to the actual time that the learner spent in the SCO.
- Comparing the actual time it took the learner to complete a SCO against both the typical learning time defined in the metadata (if present) and the average time it has taken other learners to complete the same SCO.
- Comparing the run-time data reported by the SCO against the run-time data reported by other instances of learners taking the SCO. Specifically, the LMS could look at the number of interactions reported and the identifiers of these interactions to ensure they match expectations.
- Instances of SCOs that report both completion and satisfaction when other instances only report completion, or other abnormal combinations of data model element usage.
Furthermore, ADL could provide similar guidelines to content developers to encourage them to use all of the appropriate data model elements and metadata elements to enable LMS vendors to detect fraud.
These solutions certainly aren’t bulletproof. But they “move the needle” quite a bit without significant rework. Most importantly, for those who are happy with the status quo, no changes are needed, the specification remains the same. Changes like these allow those who need more protection to do a little work to achieve that protection, but do not adversely affect the masses of current adopters who are content.
ADL or the industry could even go a little bit farther and come up with a “More Secure SCORM” profile of SCORM. I wouldn’t suggest that this include the dramatic changes required for real security, but it could include some a simple change like a new metadata element that defines a secure token that the content will set the value of cmi.location to when it “really” is completed. This solution is still vulnerable, but it moves the needle a bit farther in the right direction.
What Can Be Done in the Future?
SCORM is due for an overhaul, that’s common knowledge and good people are working on the problem. The ECMAScript API is likely to be augmented with another communication scheme soon for many reason, including security. There are a lot of design aspects that must be considered when creating the next generation of SCORM; security is but one.
As a community, we need to decide how much of a priority to give security in relation to other conflicting design aspects. As I’ve already mentioned, given the inherent insecurity of online training, it’s not a bad idea to sacrifice technical security for big wins in other areas like interoperability. But that’s not to say that we can ignore it. On a security scale of 1-10 that I have just completely arbitrarily defined, I would arbitrarily give SCORM about a 3-4. If we were to bump it up to a 8-9, it would probably introduce too much friction in other areas to be considered a good design decision. Somewhere around a 5-6 is probably about right. We want a standard that allows for frictionless interoperability and we need to be willing to sacrifice some security to achieve it. Ideally an interoperability standard eliminates friction altogether, but enables those who are willing to put up with some friction in the name of security to do so.
Fundamentally, to be secure, we need to move communication out of the browser because anything in the browser sandbox is hackable. This move implies the existence of server-side code. Server-side code is an enormous barrier to portability. A security solution needs to enable those who are willing to sacrifice portability for security to do so, but not remove portability for those who are willing to sacrifice security.
A web services communication scheme that relies on a shared secret between the content and LMS is a very promising path forward and there is work being done in this arena already. This type of solution allows for hosted content to use server-side code to securely communicate while also allowing portable content that runs only in the browser to communicate (albeit in a less secure way).
