Key Points:
- Should be able to track video-on-demand, not packaged as SCORM
- Reporting
- Item-analysis data should be available from the LMS, for trend analysis to determine quality of questions, content
- Union and diversity issues may result in mandate for analysis to ensure questions are fair
- Should be able to track branching
- Variables:
- Should be able to initialize variables for use in content based on learner profile (country, for example)
- Should be able to set learner profile variables, and read them in another course
- Should be able to see reports on these variables, identify if they are reportable
- Need to get rid of JavaScript API, use AJAX approach
- Should be more integration between LMS and HR systems
- The ability to find the course as a way to build competency or achieve a certain level was definitely required. But also things like previous licenses, certifications, language abilities.
- In the next 5-10 years
- Virtual world, alternate reality aspect will need to be considered
- Expect a return to a blended approach
What are you doing that’s new and innovative in the training you are creating now? I want to understand if there are ways that SCORM is letting you down in what you are trying to do that’s new or innovative, or that you’re planning on doing where the current technology isn’t working for you.
I guess they aren’t necessarily one and the same to me, but there’s some video content that within the training environment that I work in all videos, are really best delivered through video-on-demand service. So there isn’t an easy, direct way to have that be SCORM compliant. Without linking it through a course, and then you have window issues. So that would be one area where I think SCORM could be more adaptable to different environments.
So you just have a video, some other link you want to track, where someone has launched that video or some other stand-alone piece of content and you just want to assign it to people and see that they’ve gone and seen it, and not have to wrap it in a SCORM package as such?
Yes, because for a lot of companies, I think this would be true: they have a specific service of video-content delivery and video-on-demand systems that are packaged at a more corporate level. So you have to open it as a separate window and have to work around the fact that it’s in a different environment.
Another item, one of my main issues, and I saw some notes around this, is the difficulty of pulling item-analysis data for knowledge checks and pre-tests and post-tests. So down to the individual question level and to have that information available in a reportable and analytical way. Most of the LMSs I have had exposure to either can’t gather that data or gather that data and throw it into a binary logic black-hole or something, and that makes it very difficult to actually work with that data.
Okay, and then you want — how do you want ideally to get at that data and who needs to see it, what do they want to do with it or get out of it?
Well if you are doing any sort of rigorous review of your testing content and how it is actually being received by people, in terms of looking for trends and patterns, both so you can analyze the quality of the question as well as the quality of the learning content in relationship to the questions, and identify a pattern, you need your item-analysis information available to you.
I don’t know what kind of people you have been talking to, but I know I have worked in a couple environments that have union and diversity issues, and you need to not just assume that your quiz questions are bullet-proof in all those ways, so you need ways to get in and really dig at your data.
What else about that learning experience might you want to track?
Branching scenarios, if there was a way to track, if you have something that branched, and the branch had meaning to you, as an instructional designer or assessment person, a way to track pathways within a . Variables in general would be a challenge, I would think. Like I know I had a whole pre-SCORM course delivery system that relied on variables to set different regional settings and local settings and plugged in correct information to deal with the very large audience; they had unique information that drove part of the training or what would be delivered just part of the training and transactions in simulations, and there was virtually no way to integrate both needs into an LMS without introducing another whole layer and set of JavaScripts and such. However, if there was a way to have a kind of pool of variables available to assign as you need it, that would be applicable based on someone’s — I guess it depends what sort of employee interface or human resources database you are interfacing with.
Let’s say country, at a simple level, where you know you could adjust languages, role, monetary values, that sort of thing.
More environment of information coming in. But also when you are talking about variables, you are also talking about variables that can be set during the course or learning experience and used —
I would say both, variables that could be added based on the user’s information and also variables that would be generated through interaction with a learning item.
And presumably some that would only apply to the duration of one experience or course and others that you would apply in multiple different courses.
And again, part of that is the ability to know what those variables are that were selected in the end. If they’re reportable.
Right, get a list of the variables, maybe have a way to set, in addition to a name, assign a description for reporting and pull that information too, that sort of thing.
Let’s back up to your hope we would toss SCORM for something new; if you were to create something completely new to replace SCORM what would be the key properties that would have, in your mind?
Much easier and more user-friendly flow, window flow, and the biggest issues I continue to see with SCORM are just the amount and complexity of the windows and the passing are just crazy for the average user, it’s inevitable that things are not going to always report correctly, just because of the way windows open, and all of the browser variables. It seems most of the systems I see have a variety of layers and windows that aren’t well-coordinated, based on current web standards, which will, whatever. I find that incredibly frustrating and hard to justify to anyone who is having trouble with it. So, what I don’t get, I’ve done a lot of usability work and basic web-design, and AJAX, and I don’t understand why there can’t be an AJAX approach to a SCORM solution.
That’s definitely something we firmly have in mind, at least on the web side, it could be AJAX and then to support non-web technology, if you have web-service it can be accessed either via AJAX or a webpage and you can have courseware that isn’t even a webpage.
I have yet to see an LMS, and maybe there are some I should see, but most of the ones I have seen have these issues at one level or another, as opposed to a previous position where you just kind of have your own database for employees, and did it that way, and it worked perfectly fine, and you could do direct database calls or send, even without needing any pop-up windows, directly send course results, including item analysis or whatever it needed to send, back to that database. So I’m interested in that kind of interaction for SCORM.
So definitely no required extra windows or frames, that is something that I’m pretty confident is going to be addressed. Anything else on your list that you would want as a way to communicate —
From an industry perspective, I think it would be great to have more standardization of integration with human resource databases or core business systems.
What would that integration — what do you want out of that? Is that for context data, or would you want to feed back into that system, and what sort of data do you want to pull or push into it?
A standard for synchronizing with current employees — I know that’s a variety of applications and services that come into play. But I think what would be sweet — is a , literally — if you could seamlessly go have integration between a VirtualEdge sort of recruitment process flow and all the resume and employee information that comes in through that avenue, directly into the company’s current employee roster and whatever that interface or database system that is, and that there’s kind of a seamless integration from there into current learning and ongoing learning. So you have a simple and complete profile.
So an integration between the HR system and the LMS and not necessarily to the level of the courseware.
So you have one source for an employee’s current and previous skills, previous training, and degrees so you can step more into talent management more seamlessly, and also have employees automatically set up within your training paradigm.
And would those courses report back to the HR system of what’s been completed and scores and that sort of thing?
I don’t know that they’d have to report back to that system as much as there would be a way, I guess from any level to cross-reference. I don’t know on that — or which system, because I know everything gets kind of complicated, driven by the HR system?
I think more and more of the integration of competencies, talent management and the full package of exploring and resourcing your employee’s skills is going to be important and necessary.
So from that point of view, when you’re designing content, you design it to address particular competencies, and do you, within the LMS, do you decide if people have learned to raise their ability level within those competencies, or do you kind of assess right within the course and want to be able to specify within the course, what competencies are addressing and how people have done and report on that?
The ability to find the course as a way to build competency or achieve a certain level was definitely required. But also things like previous licenses, certifications, language abilities. From a training perspective, if you were able to easily source and see what pre-existed in your community, that would be a very valuable tool.
In order to be able to search content for competencies, you need a way to identify what competencies are addressed by what content.
Yes, which — you find a variety of ways to cross-reference or tag any one learning item so you can identify any competencies it addressed and at what level and catalog it several different ways basically.
How do you think, or what do you think about e-learning will change in the next five to ten years, and what do you think should change?
I think the whole virtual world/alternate reality aspect that we need to definitely consider, but that will be more informalized and utilized. Things like Second Life, etc. Do you hear that much?
Yes. Virtual worlds, serious games, all sorts of simulation, that comes up frequently. Anything else?
We’ve already talked about increased video learning. Just some other things that I’m not working on currently that are potentialities. I think there might be more of a return to a blended approach, which is what we’re currently in, something that’s happening, so more of a way to integrate tracking of diverse deliveries. So you build a curriculum. Maybe one piece is video, another is classroom, then e-learning, maybe one is a one- on-one with a manager. Sometimes they don’t fall into one learning path or curriculum, so you have to steer them — I’ve seen a lot of LMSs where you have to list things differently, or it’s just not always pretty to do those combinations.
