Score Rollup in SCORM 1.2: There’s no silver bullet

I got this question today (and yes, we really do mean you can ask us anything…):

We are making configuration settings for our new LMS ([redacted]) and one of the questions is:

1.3.5. How is the overall score for a SCORM course with more than one SCO computed? With the following choices:

  • MaximumScore. Highest score of all items (SCOs) in the course.
  • AverageScore. Average score of all items (SCOs) in the course.
  • FirstScore. The first score achieved by the user across all items (SCOs) in the course. Note: If this option is selected, ScormScoreUpdateOption should be set to “Never” and the user’s score will always remain the first score they achieved.
  • MostRecentScore. The most recent score achieved by the user across all items (SCOs) in the course.

What do you recommend?

My first reaction

Well, clearly we’re talking about an LMS that is really SCORM 1.2 centric. This ability to manage scores across attempts and SCOs is one of the things that SCORM 2004 actually does very well. Rollup rules allow the content author to specify these behaviors in great detail, including weighting the various elements and even excluding some. This provides the author with useful tools such as pre-assessment that doesn’t impact overall satisfaction.

My second reaction

There simply is not a good answer to this question. If these settings are global, as they appear to be, I can’t be forced to pick one in particular as it wouldn’t serve other situations adequately. The most common multi-SCO package structure, to my mind, is a course with several SCOs and a single post test (assuming we’re dealing with more than one score). I’m not sure that any of these options serve that package structure well at all.

My ultimate conclusion

An LMS simply can’t afford to make singular decisions about how to deal with content. It absolutely has to provide configurability on a package by package basis. Failing to acknowledge that the world of content is widely varied will lead to content that doesn’t work as desired. This, frankly, is why building an LMS with SCORM conformance is so difficult to get right.

How we handle it in SCORM Engine and SCORM Cloud implementations

Package properties provide our ability to address different pieces of content differently. (Take a look at the possibilities. There are more than 50 options for configuring a course. All are defaulted intelligently, but all are also the answer to a question that has to be answered for certain courses.)

packageproperty-1

Take a look at how we’ve approached the particular problem of rolling up scores in SCORM 1.2. We have options that aren’t unlike those offered in the question above. The differences are relevant, but only to a degree. Fundamentally important, though, is that you can set these properties for each package.

I hope this provides a bit of illustration as the level of detail required to really nail the SCORM problem. We suffer this minutiae so our customers don’t have to.

Note: Don’t worry, my recommendation for the person who sent the question isn’t as obtuse as, “Use the SCORM Engine”… We’re still chatting via email…


  • Tom

    In regards to the rollup configuration, I love the concept but for some reason our content flat refuses to rollup. We have several SCOs with one exam. All SCOs report a completion status of completed and the exam reports either a passed or failed. BUT, they do NOT roll up properly in the Rustici Test Track or the live LMS. The success status returns an Unkown.

    I made the SCOs send a “passed” value rather than a “completed” value and if the Exam is passed then the Success States of “Passed” is rolled up. However, if the exam is “Failed” then the rollup returns an “Unknown”.

    All SCORM calls are fine in the content so why doesn’t the rollup work?

    I then separate the content SCOs from the exam and tested them separately in two separate SCORM packages. Guess what, the content SCOs set to completeion rollup as completed and the exam rolls up as Passed. So, I am convinced the SCORM calls are fine, it appears to be the rollup logic in Test Track and the LMS.

    I am totally frustrated….

  • Tom.

    Lots of different possibilities here, frankly. If you’d like to submit the course to support@scorm.com, we’ll get a look at it as we’re able.

    Tim

  • Tom

    Tim,

    I am not able to upload the content as of right now as per client directions. I have opened tickets with detailed descritptions of what I am experiencing along with screen grabs of the results. So far I am not having any success (pun intended).

    Can you share with me on Test Track any SCORM 1.2 content examples that has multiple SCOs with one final exam that rolls up properly with a Pass when the exam is passed AND a fail if the exam is failed?

    Thanks,
    Tom

  • Thanks Tim. I’d been looking at this from the other direction: how to get my SCORM packages reporting scores sensibly. I’d got it all working lovely in SCORM 2004 before I realised our LMS only supports 1.2. D’oh!

    Looking at the options on SCORM Cloud, none of them fit my needs. This is not a fault with SCORM Cloud – the SCO simply doesn’t provide the information for SCORM Cloud to calculate what I want: the total score from the potential across all SCOs. Since the max score doesn’t get reported until the learner attempts each SCO, this is an impossible ask.

    I guess I’ll need to stick with the default: ‘Average Score of all Units with Scores’, and make sure I communicate to teaching staff and students how this works. Hopefully that’s an option with our LMS. Either way, I’ll try and focus them on the tracking data for the individual SCOs.

    I suppose the real solution is “use SCORM 2004”, or even better: Tin Can API!