Key Points:
- Occasional requests for offline use, rare mobile requests
- Current data model is more than sufficient
- Collaboration isn’t popular with corporate users because of the liability need for moderation, and therefore the cost
- Interoperability problem with timing of calls
- Initialize() being called too late
- Waiting too long to set values and commit (risk of data loss)
- Adaptive learning – different versions for learning styles, doesn’t get used due to ROI
- Should be easier, not more complex
- Customers do not use sequencing
- Authoring tools do not support sequencing
- Simple documentation for simple use cases might help
- Should fix a couple bugs in 1.2, add test-out, call it “SCORM lite”
What is your role exactly?
I’m an implementation consultant for learning management systems.
What are your customers are asking for?
The understanding of SCORM in our customer base is extremely limited. If nonexistent. They do know what SCORM means — so, if I’m SCORM supported, if you offer SCORM support, my courses are supposed to work. And that’s basically where it ends, but we’re going into the different RTE flavors and things like that, you get a blank there. I’ve met over the past year maybe one e-learning developer at a large company that knew what I’m talking about. The advanced adaptation of SCORM we have with one customer and that customer is falling over and over again with the sequencing issues that the third edition has.
More on the periphery, not necessarily SCORM features they are asking for, but things they might ask for around SCORM in terms of your player — like, did they want mobile use, off-line use, that sort of thing?
We get the occasional off-line use, the very rare mobile use. Other than that, everyone wants it to work flawlessly and that just doesn’t fly. Most people are interested in asking for data elements, when you get into the discussion about data elements, most people are interested in — I’d say 90% of our customers want the core, the learner name and score, date and how long it took them, that’s it. Maybe 10% want to go into interaction details, once we’ve explained to them that it’s available. But it’s really a minimum.
What else do your customers want to track? Are they asking for anything that’s not even available in the data model?
No, our customer base, every single one of them that I have talked to, with the exception of one (the same one using the sequencing) is completely overwhelmed by what the data model offers. There is one thing I would personally like to see, and that is SCO progress. Let the SCO tell me how far you have progressed — fourth edition they do that, right? That’s one of my big pet peeves, especially of the single SCO stuff, by the way — that comes up to our next big thing: 90% of what we have across all of our customers are single SCOs. Because most of them use ready-box authoring tools like Captivate, etc., and they only do single SCO.
We’re trying push you a little bit in the direction we’re trying to think, are there other learning experiences that your system tracks, or has a need to track other than your traditional e-learning? What we’re tasked with here isn’t necessarily defining the next generation of SCORM as it currently exists, but re-imagining what a learning environment looks like in this century.
I haven’t even seen mobile e-learning yet, and everybody’s talking about mobile learning, right? It’s been the topic for the past couple of years. I haven’t seen a single but one customer, who wanted to do mobile learning. Of the hundreds I see every year, I’ve seen one who wanted it. You’re talking collaboration, what 2.0 stuff — as soon as I tell my customers, I give them — corporate customers, right? — Hey guys, collaboration rooms are cool, we got them, we can do that, we can do all this fancy stuff. That gets — you have to moderate it. Because at corporate you have liability, you can’t just have stuff posted all over the place that may or may not be appropriate. So you have to moderate it. And as soon as I say that word, everyone realizes what a can of worms they opened up and then they don’t want to do it. Because dollar for dollar, it doesn’t pay the bills.
We’ve been asking other people, what other systems do you want to tie your data in, how does that data flow work, but I imagine you’re saying it all flows into ours, everything under the sun.
Right. That is one of the strengths of our product, that we’re completely pre-integrated out of the box, you don’t have to do any integration at all. It’s a little bit more complex than maybe a stand-alone LMS, some of them are as simple as putting a CD into a server; that’s not happening here, it’s definitely a project to set up the learning solutions. But the advantage of it, of course, is that it’s completely integrated out of the box, too — as you said, everything under the sun. And the data goes with your employee, and we never ever delete it.
The other thing we do pretty well is scalability, we are running on and operating in learning environments where we have a quarter of a million learners, active learners.
My biggest pet peeve with SCORM is the timing of data. SCORM prescribes — Hey, you have to do this call, otherwise you aren’t conformant, but it doesn’t tell me when you need to do this call.
Expand on that a little more.
For example, let’s just use regular Adobe Captivate course, right, the way it comes out of Captivate 4 when you publish it right now. The CMI of the initialize doesn’t come until the Flash piece has actually loaded. That causes an issue, because our product waits, and it waits for about three seconds and if it doesn’t get the initialize after three seconds, after the course has launched, it assumes it is a minimally conformant SCO. And automatically sets it to complete. All the planning on how slow the network is, on the Captivate, how big the Captivate file is, the Flash file is, it can take 4, 5, 6, 10, 60 seconds before the Flash piece is fully loaded, and finally sends the CMI, to initialize. And that creates a problem. And we had to, for example, write a code to work around that.
The stupid mediator window you have in many courses, whether there is a little window open underneath your course content window — where all the “stuff,” the data, gets sent from the course and then stored in this mediator window, and then once you log out, it dumps it onto the LMS all at once. If we do talk to the developers, we discourage that behavior, we really want them in smaller chunks, as light and as real-time as we can get it, and there’s plenty of people around — and of course, SCORM doesn’t prohibit it. The drawback is, if your network connection goes down, or your user closes this mediator window in the middle of it, it’s lost. And then you get this call to the help desk. And that costs money.
So that sort of timing issue, at least it seems like could be addressed in a best practice guide, and that’s another project going on.
I would like to see that as a conformance guide. I would like to see — you know, the first thing a SCO must do is initialize, and the last thing it must do is terminate. It can’t do anything after terminate, things like that. Well, you can’t anyway, but it should be the last thing going out. If you do achieve a score, or report the CMI interactions, whatever else comes after that, then you must do that while you are at the item, not an hour later. That’s a big pet peeve, where we run into customer problems all the time.
My customers get tasked with — most of them are learning developers, a lot of them are instructors; We’ll go into educating them; you will be converting all your content to SCORM if it isn’t already. They say, “What is it? Oh, I’ve heard about that.” Then you dump the ADL guides on them and it’s a running joke, that ADL guide. Unless you’re really technical — you have to go to the internet, to SCORM.com, and read your SCORM primers, etc., before you even have a clue what’s going on.
It’s all about ROI, at the end of the day. I work for corporate customers; none of us are interested in cutting-edge research. We just want it to work and we want it to produce business results. For a lot of them, regulated industries, training, is a double-edged sword. On one hand, you want it to work for us, on the other hand, it’s really just to keep the regulatory agencies off our back and prove to them we are tight. If you really want to what percentage of courses these customers have that are pdf files that are uploaded — The only trackable interaction at the end is this button that says, “I hereby agree I have read and understood the content of this document; yea or nay.” That’s it; we’re talking in the thousands, especially in the pharmaceutical industry. That’s what the FDA regulates. There is really, we all know that knowledge telling ain’t training, that’s all telling there no showing. That’s the largest customer base I have; the request for really innovative e-learning on the corporate customers are very limited.
Do you think that 5 or 10 years down the road, what do you see about what your customers are asking for changing, if at all?
I think it’ll be the same kind of thing — but this is where the whole SCORM thing comes in, I would like to see it be easier rather than more complex. So SCORM 1.2 was solid, relatively, it had a certain degree of reliability. When we went to SCORM 2004, especially with sequencing, a whole kind of confusion got unleashed. Especially since the sequencing model didn’t work right from the beginning. It became so complex that no one understood it at all, none of my customers, at least. I have one customer using sequencing, one out of 1500 hundred or so. I don’t see it going anywhere; most customers, when you tell them about what is, the advantages, what they can do with it, they say stick with 1.2, don’t even think about 2004. They don’t think beyond that. They want it to integrate and want it to integrate easily.
Presumably if they are doing single SCO, they want all their transitions; they just want to control their content, right? They don’t want to worry about — we have ten SCOs and now we have to worry about how the LMS will sequence between them, to ourselves — push the learner forward where they are supposed to go?
Yes. What I do see from a future point is I do see the testing out of stuff, using adult learning methodology; that is a requirement quite often. But it comes with this can of worms that SCORM 2004 3rdedition with it, and people don’t want to do it anymore. There isn’t a single authoring tool out that does it, at least that I know of. There isn’t a single one that out of the box, does multi-SCO sequencing. We have our own that does it, but it’s not SCORM-conformant.
Does it do it using multiple SCOs, or is it one SCO, all within that SCO?
Our authoring tool — is technically speaking, not SCORM-conformant. But, we heavily rely on the SCORM idea, so we create learning objects as well, SCOs, and these learning objects we have another layer called an “instructional element,” which is an anchor to your content file. But that instructional element per learning object is also categorized by learning style, it’s adaptive, intuitive to — whether you have a graphic learning style, — so for the same learning object, you can line up to eight different instructional elements that serve the same content in different methods, and the learner gets to pick which method. All these multiple learning objects get tied into a learning net, which would be your course-instructor in SCORM, and we can assign pre-tests with in each learning object fulfills the qualification, and that’s how it ties together, very similarly — but it’s done in a graphical user interface layout, it works rather solidly and people understand all of them. A pretest for a certain learning objective, and then a hand that says learning objective to the content, and they realize they can test out. Because it’s all graphical; they can see the relationships.
At the end, we used to be non-SCORM-conformant, because we have our own tool, but with the latest release of our LMS, we put a SCORM wrapper around it and said it was SCORM compliant.
Your customers do want this test-out functionality, they do want adaptive learning that can have multiple takes on the content and it can adapt based on learning style. If there was a standard way to achieve that easily, would that be something you would want to use, or not?
Yes and no. Yes — if there’s a way to test out, I can get my people back on the factory floor much sooner. And they don’t look completely bored at the screen because they’ve been doing the job for 20 years, that person would go through that content anyway right, like good adult learning methodology. Obviously we want SCORM to support that.
As far as the multiple content, we offer this feature, we have a single instance where we have used it, one. That was the L3 government study that actually funded the development of the tool in Germany. The L3 Life-long Learning Project. Because none of our customers, every single time they have a different didactic learning style and you develop to support that didactic learning style, that’s money. To develop the same content up to eight different times, then to sell this to a customer and they’re going, “Yeah, right.” There are very few training departments in the American corporate world that understand how to prove ROI. Everyone knows Kirkpatrick, but how to do Kirkpatrick is a different story. My customers want simple and fast. With extremely few exceptions.
That’s an interesting challenge ahead of us we have been asked to “keep it simple”, but also for a lot of complex features.
I think the answer is right in front of you: SCORM lite, and SCORM advanced. Take the same root objects, the same SCORM core, those data values, and the interactions are fine too. Leave SCORM 1.2 alone, but add maybe the test-out capability, and fix a couple bugs in 1.2, and call it SCORM corporate or lite. And then, I don’t even want to go with SCORM 2004. It’s so complex the way they did that. Besides sequencing, the other thing that’s complex about it is the ADL CP mastery score. And that’s with 1.2 as well. I mean, if the SCO is in charge of it’s own life, why can I all of a sudden leave the judging of the SCO to the LMS by passing on that mastery score? If we’re true to our socks, we need to stay true to our socks. If it’s autonomous, than it is; let’s get rid of this mastery thing. It’s a little conflict of interest I think between the SCO and the LMS. I’m all for SCORM autonomy, it’s great. With SCORM 2004 now, where the SCO can now navigate to the next SCO, etc., I can see that as a desired behavior, but it gets confusing for the developers.
If you are looking into where most corporations are going, they are going away from a course-development perspective, away from a dedicated instructional designer, and going out to the SME. Here’s your power-point, make it pretty, do something with it. It’s terrible, I know, but that’s the reality. The engineer — the department who does the training, just using a rapid-authoring tool.
The issue in my mind is where does SCORM fail there; I don’t think it does in its technical implementation and the requirements it imposes, because it does allow you to do some bare minimum. It fails in that to understand what those requirements are, you need to read 1,000 pages and comprehend all of them. I think it’s less a challenge of technical structure and more of documentation.
I totally agree. I think our content-authoring tools do it a disservice. Because we know our courseware developers don’t understand SCORM and don’t want to, they just want it to work. It should be invisible, and it is. As soon as you select output to SCORM, and you have to give it the SCO ID and the manifest name ID, that’s a requirement, and you can write a note to it, a training note to it, that okay those are the four features you need to populate it, everything is in the background.
On the other hand, these guys at Reload, the visual manifest builders, they do a semi-effective approach to help visualize how all this works together. If we were to get those two together and come up with a way to take your content, and structure it visually and actually preview the way it works, etc., and have that as a product, and showing in the preview window how this manifest structures and looks like and how this course is put together, they would understand a little bit better.
So there’s your challenge Rustici: build us a visual SCORM authoring tool.
