4 Things Every SCORM Test Should Do When Reporting Interactions

A year ago, we talked about 5 things every piece of SCORM content should do.  Today, I wanted to mention 4 things every SCORM test should do… Keep in mind, SCORM tests are a subset of SCORM content, so they should be doing those same 5 things I mentioned a year ago.  Take that as a given.  This is just a further set of details.

1. Record your interactions with full detail

SCORM provides a way for content to record the learner’s answers (and the questions) to the LMS.  Put simply, do it.  We see content all the time that elects not to report these interactions, and it’s a waste.  Even if the LMS doesn’t report well on this information today, it could do so ultimately.  And from a learning/remediation perspective, it’s crucial that the administrators of the LMS be able to see how their learners are progressing.  Just do it.  Nike would be proud.

Now for the details.  Recording interactions can be done well or done poorly.  If you want to do it really well, and your LMS supports it, opt to use SCORM 2004, rather than SCORM 1.2.  SCORM 2004 allows content to be far more expressive in reporting interactions.

Next, understand the data model elements.  I’ll spare you all the details here, but here are some highlights.  For the purpose of examples, pretend like we’re working with this question:

What is my name?  
A. Tim Martin
B. Reggie Benes
C. Dan Stook
D. Keith Bolliger

  • Record the result (correct/incorrect) in cmi.interactions.n.result
  • Record the learner response and correct response using a human readable identifier (or collection of them).  Better to record “Tim_Martin” than “A” if the learner answered the question correctly.  This gives the LMS an opportunity to share that data with the administrator in a useful fashion.  And in SCORM 2004, “Tim_Martin” is now a valid response pattern.  (In SCORM 1.2, “A” was the best you could do.)
  • Use cmi.interactions.n.description.  Frankly, this is one of the best additions in SCORM 2004, allowing you to record that the question was, in fact, “What is my name?”  From a reporting perspective, this a vast improvement.
  • If you’re going to go this far, you might as well complete the data model and record the following:
    • cmi.interactions.n.type
    • cmi.interactions.n.weighting
    • cmi.interactions.n.latency
    • cmi.interactions.n.timestamp
2. Understand the difference between state and journaling

First things first… interactions are recorded in an array.  Take note of cmi.interactions.N.whatever.  That array is sequential, and each time a SCO wants to record something to it, it has to ask for the next available space (via cmi.interactions._count).  Separate from the N I’ve just mentioned, though, is the identifier of the interaction… cmi.interactions.n.ID.

If a piece of content wants to record a 10 question test and have a slot for each of the 10 questions, it can do that, even if they allow the user to update their answers.  It would do so by cycling through the existing interactions and examining their cmi.interactions.n.ID to see if it matches the one that needs to be updated.  This technique of updating a given interactions values by cycling through the array and resetting those values is called “state” or “stateful”.  The recorded interaction indicates the current state of those values.  It also eliminates any prior values that may have been recorded.  State is a valid approach to recording interactions.

On the other hand, the array allows for you to simply add another value to the interactions array rather than seeking out the old array location and overwriting it.  In this case, the content would simply request the cmi.interactions._count value and record the new interaction data in that slot of the array.  In using this journaling technique, all of the historical values for that interaction are maintained.  If the content wishes to retrieve those values, say on relaunch of the content, though, it has to be more intelligent about discerning which of the answers was most recently given.

Note, both journaling and state are valid option.  It’s crucial, though, that the content manage it’s concept of cmi.interactions.n.ID well though.  A piece of content that uses a new ID each time it reports and interaction is not properly journaling, because the association between multiple answers of the same question is lost.

3. Set completion status and success status

In SCORM 1.2, completion status and success status were rolled up into a single entity, cmi.core.lesson_status.  It had six potential values, including completed, incomplete, passed, failed.  In this world, it was impossible for the content to tell the LMS if a failed status meant that the user should be allowed to take the content again or not.  Was it failed because they hadn’t finished?  Who knew?

SCORM 2004, though, separates the concepts of passing and completing using two distinct data model elements:

  • cmi.complete_status (completed, incomplete, or unknown)
  • cmi.success_status (passed, failed, unknown)

This allows the content to be more expressive about whether a failure was final.  Each content vendor is welcome to their own interpretation here, but making use of both completion_status and success_status is important in SCORM 2004.

4. Post a score

Lastly, be sure to post a score. It’s such a simple thing to do, and it’s hugely useful to the LMS. Take note, in SCORM 2004, the posting of a score should look like this for a 10 question test on which you got 8 right.

  • Set cmi.score.raw to 8
  • Set cmi.score.min to 0
  • Set cmi.score.max to 10
  • Set cmi.score.scaled to 0.8

  • John

    Good Post.

    A follow-up post could be: “1 Thing Every SCORM LMS Should Do When Reporting Interactions” And the body would be simply “Actually provide a way to display/report the collected interactions”. You’d think that would be a no brainer, but …

    Also, hopefully, IEEE takes cmi.interactions another step and cleans it up even more for whatever is next with SCORM. There are still shortcomings.

  • Is there a minimum number of cmi.interactions.n.XXX parameters that are required for an interaction to be successfully recorded?

    I am writing the following successfully:
    * cmi.interactions.n.id
    * cmi.interactions.n.type
    * cmi.interactions.n.learner_response

    When I launch the SCORM package through http://cloud.scorm.com/ and submit the interaction, on subsequent SCO launch, cmi.interactions._count reports a larger than zero number.

    However, when I launch the exact same SCORM package through another LMS and submit the interaction, on subsequent SCO launch, cmi.interactions._count reports zero.

    What may be causing this issue?

  • Mark Statkus

    Tim thanks for clarifying this. I was having some issues testing this with QUnit on the scorm cloud. It takes cmi.interactions.0.x but when I add another interaction I get a 351 general set failure on cmi.interactions.1.x. Same with cmi.objectives. greater than 0.

  • Mark Statkus

    Never mind, I didn’t realize objective IDs had to remain unique. I guess from a journaling standpoint you couldnt keep a history of objectives without altering the ID.

  • brandon bradley

    This is a great list, and will be useful for explaining to others what sort of things they should really be supporting with their content.

    The score item in particular can be the source of a number of issues if not thought through carefully. For example the max score needs to be calculated up front, not accumulated as you go to each question, or the user that answers the first question correctly and times out receives 100%.

    There are also issues that can arise if your min points equal you max points (such as points automatically awarded upon load) or if your max points are 0 for something that the user has to open/complete, but doesn’t get points for, just credit (yeah I can’t give a good reason for doing that, but have seen it done).

  • lefthook77

    In SCORM 2004, is success always sent with completion? I’d like Success to be sent on user access, but can’t get it to work. Seems to be waiting on completion status in the reports, when I exit before finishing the quiz.

  • bsc_scorm

    Success isn’t tied to completion, but you’re probably running into this issue: http://scorm.com/blog/2008/10/resume-vs-rollup-the-eternal-struggle/

  • 台灣大樂透

    Amazing article, thanks for sharing !!

  • BARNALI MONDAL

    I’m trying to integrate one moodle site to a crm tool. How to get the score for individual users from the scorm and display it in the crm tool??

  • JCas

    I’m trying to record interaction data in Scorm 2004 in the Plateau LMS. We want the interaction data to be reported on each attempt, while user progress is retained. Is this impossible? It appears user progress is only retained with cmi.exit “suspend”, but then the interactions are not sent to the reports. Is there another way I could approach this? Many thanks

  • passerby

    Not sure, but I think your:

    cmi.complete_status

    is supposed to be:

    cmi.completion_status

  • Hi,
    This is really helpful information about Scorm Course.
    Can you tell me please, what ” n ” means,
    cmi.interactions.n.id…..

    what “n” represents.

    Thank you

  • Oliver

    Hey there!

    In my understanding “n” is a object/question number in a test.
    Example:

    cmi.interactions.6
    cmi.interactions.6.id: _8ABF9C2F-9CF5-4C80-9EA5-2FE5F0C335BC_
    cmi.interactions.6.type: FillIn
    cmi.interactions.6.timestamp: 2015-10-01T18:39:41.0Z
    cmi.interactions.6.correct_responses.0.pattern: equal to 4
    cmi.interactions.6.weighting: 1
    cmi.interactions.6.learner_response: 4
    cmi.interactions.6.result: correct
    cmi.interactions.6.latency: 0000:00:02
    cmi.interactions.6.description: 2+2=

    Instead of “n” you see “6″.
    This info can be found in detailed reports. This is for the 7th question of the test (“0″ is for the 1st question and so on).

  • Pingback: SCORM Quiz – Item Analysis issues & solution | EduTrends()

  • Peter

    Hi there,
    First of all thanks for the explanation, it helped me a lot updating a course from 1.2 to 2004.
    I might be a little late to the game but I have a question. Can I save score if the learner hasn’t finished the module yet? I have a game-like simulation where the player earn points as he / she goes along the game. I have a min and max score but the player’s score will change throughout the session and between sessions. Is score tied to the completion or success status in any way? I can see the score appearing in SCORM.Cloud when the game ends and I save a completed status, but the score undefined (I think) if the player exits the game before the end. I still set raw, min, max and scaled scores though. I can also see it in the session log- without errors.
    Thanks,
    Peter

  • Bineesh K Thomas

    Through our LMS – intented for primary kids, teachers can assign SCO as homework to learners. The issue in here is, most of the kids are completing the activity, but they – most of the time do not submit the activity. Is there a way once the deadline is passed, LMS can auto submit the SCO with the available response?