"Joe Smith answered Question 1 with B but the correct answer is C."

I got a great question from one of our SCORM Engine clients (Brian) first thing this morning, so straight to the blog it goes.

We are getting very involved with developing a custom SCORM report that utilizes and displays interaction data. As we are looking through the information sent back by a SCORM 1.2 published course (using Lectora), we see ID, LearnerResponse, CorrectResponse values being passed in. With this information, we can build a report that tells someone “Joe Smith answered Question 1 with B but the correct answer is C.” This may meet our need, but some of our clients are asking for a report that shows the actual text of the question and answer.
My research his showing that this is not a part of the SCORM standard, but do you have any recommendations or experience with capturing the actual text values associated with these variables?
First off, thanks to Brian for doing a bit of research before getting in touch with us. We’re happy to answer the basic questions, but a pointed question like this one allows us to jump straight to the details they’re seeking.

SCORM 1.2 has very limited capability in its data model for accepting details about interactions. Brian has this well understood… something to the effect of “Joe Smith answered Question 1 with B but the correct answer is C,” is about as far as one can be certain of going in SCORM 1.2.

SCORM 2004 enhances this a bit. The question element (cmi.interactions.n.) now has an .id that is a long_identifier_type in addition to a description element. This description element (cmi.interactions.n.description) allows the content to record, typically, the question itself as a localized_string_type. This is a vast improvement from my perspective. Answers, as well, have improved in SCORM 2004. Because the vocabulary varies for different types of interactions, it isn’t exactly straightforward. Taking multiple choice responses as an example, though, the content can at least record a collection of short_identifier_types.

So, what in the world did all that mean? Let’s go back to Brian’s example. In SCORM 1.2, he’s right, the best you can do is:

Joe Smith answered Question 1 with B but the correct answer is C.

In SCORM 2004, you would hope to see…

Joe Smith answered Question 1, ‘What is my name?’, incorrectly. His answer was ‘Bob’ instead of the correct answer, ‘Joe’.

This all sounds great, to this point, but now it’s time for some caveats.

  1. This is entirely dependent on the piece of content in question sending along the full set of data. If the piece of content elects not to send along the correct response, that is its right. If it elects to send along no interaction data whatsoever, it can still be conformant. An LMS is 100% beholden to the content it is playing. [Note to content vendors: Please don’t be lazy!]
  2. The Joe Smith question above shows the best side of the new answer functionality. Had the answers above been more sophisticated, it might have looked more like this:

Joe Smith answered Question 1, “Compare interactions in SCORM 1.2 and SCORM 2004,” incorrectly. His answer was “Interactions_are_perfect_now.__Lovely,” instead of “Interactions_are_somewhat_improved.”

So, don’t expect perfection in SCORM 2004, even if the content is behaving well. It is, in my book, something that might be worth tackling as the standard continues to evolve.

Later: I realized I didn’t fully answer Brian’s question here… The answer is, in SCORM 1.2 especially, reporting effectively on interactions is simply difficult. It might be possible, for content from a single vendor, to create a reporting mechanism that would allow you to establish an answer key of sorts… and then “join” that answer key to the answers from the interactions. Applying something like this broadly, across content vendors, is a problem we haven’t even solved yet!

  • Mike Rustici

    Expanding on Tim’s post a bit, SCORM 1.2 indicates that the identifiers provided for interaction responses should be in certain formats. For question types such as multiple choice, the specification requires that the response be formatted as a single alphanumeric character. That gives the content author limited options. The content can report that the user gave the answer “c” and the correct answer is “a”…but what should happen if the order of the presented questions is randomized for each user. Some content will report the first letter of the text of the answer, but that has obvious drawbacks when different responses start with the same letter. Other content simply ignores the single character requirement and reports a more descriptive value. SCORM 1.2 is a bit lax in its enforcement of these restrictions and many LMS’s will accept multiple character identifiers in order to provide good reporting.

    We recognize that this is a problem both on the content and LMS side. In the SCORM Driver, when content reports an interaction result, we ask for both a short and a long identifier. The Driver can then be configured to attempt to persist the long identifier if the LMS will allow it, but if not, it will fall back to persisting just the short identifier. This lets the content be both strictly conformant yet also provide useful information. Similarly on the LMS side, in the SCORM Engine there is a package property that controls whether or not the LMS will require strictly conformant SCORM 1.2 interaction results or if it will relax the validation and allow more descriptive values to be recorded.

  • Diseño web madrid

    great article, i’ve one question: how can I implement multiple choice? Iknow that simple choice is “QUESTION_TYPE_CHOICE”, and TF for true-false, can you tell me what’s the key for multiple?