Obama’s award and misplaced vitriol

The US Department of Labor just announced their solicitation for grant applications (SGA) and they called it this: “Employment and Training Administration Notice of Availability of Funds and Solicitation for Grant Applications for Trade Adjustment Assistance Community College and Career Training Grants Program“.  Whoa.

I’m no political pundit, so here’s my short version: The federal government has set up a large grant program that includes the creation of Open Educational Resources, and they’ve required (on page 8 ) that the output conform with SCORM 2004.

There’s been a lot of “reaction” to the inclusion of SCORM, and by reaction, I mean many people are pretty angry about its inclusion.  Most of that angst, though, originates from Rob Abel’s post on the IMS forums.

I will say this very plainly and directly: Rob’s post contains many inaccuracies and convenient explanations of the sort that you would see in a political campaign.  While it is tempting to break down Rob’s post on a line by line basis, my ever-so-brief analysis of political campaigns (OK, I watched The West Wing) indicates that helps no one.  I’ll limit my comments to a few:

  • SCORM is not based on “outdated technology” as Rob claims repeatedly. The fundamental technologies employed by SCORM are Javascript and XML, and both are absolutely core to today’s web.
  • “SCORM does not provide reliable interoperability or reuse.”  Our SCORM Engine alone supports millions of learners and their use of interoperable content every year.  Millions.
  • “SCORM has no concept of or support for assessment.”  False again.  Please see the SCORM books for details on cmi.interactions, which are used widely for the reporting of learner assessment.

Lest you think I’m one sided here, there are truths in Rob’s post as well.  SCORM is not well suited to “cohort-based” educational courses at this point, because it specifically governs single learner/host system communication.  SCORM also elects (intentionally) to remain silent on countless subjects such as wider IT infrastructure and security.

 

Setting aside the technical errors in Rob’s post, my primary issue is with his misplaced vitriol.  Rob has a vested interest in this debate. [Note: You could certainly argue that I do as well, given our domain name, but it's worth noting that we have equal support for AICC, and IMS CC has come up as a potential addition for us.  We are definitively not the standards body.]  As the leader of IMS, Rob has plenty of reasons to espouse the virtues of the standards they are creating.  Further, I think Rob would be justified in complaining about the exclusion of IMS CC as a potential approach to reuse as part of the grant program.  Michael Feldstein pointed this out in his balanced perspective on the issue.

SCORM and IMS Common Cartridge (the other main contender for a standard educational content interchange format) have substantially different affordances that are appropriate for substantially different use cases.

 

Michael Feldstein, in OER and Standards

My challenge to Rob and others in the conversation would be this.  Argue the things that merit argument and take far greater care when you lambast other solutions.

  • Does IMS CC provide some affordances that might be of use for a program such as this and should it be considered as a potential solution? I think it does.
  • Should a directive such as this specify a single standard for clarity and simplicity?  Or should other standards be options as well?  I have no idea.
  • Should SCORM, in its current state, be the only eLearning standard for the next 30 years?  No way.  Check out Project Tin Can and why SCORM needs to evolve, and tell us how it should evolve.

Ultimately, what’s the point here?

 

Elearning standards have a fundamental purpose: to remove the friction that separates learners from what they need to learn.

Rob has succeeded in inciting more than a few folks to criticize SCORM, when few of them have the background to determine the accuracy and reasonableness of his statements.  A vitriolic argument like this does nothing but set us back in the goal of helping learners reach the learning they need.

SCORM can absolutely increase the utility of the Open Educational Resources produced by this grant program.  IMS CC may well be able to as well.  Let’s move this discussion past politically motivated and inaccurate accusations to something that helps people get their learning.


  • http://pipwerks.com Philip Hutchison

    Amen. Let’s take the anger out of this and discuss like rational adults. No one thinks SCORM is perfect or the solution to every situation.

    The feeling I got from Rob’s post (which I read before reading this post) is that he is coming across as a jilted suitor. It’s very unbecoming for him and his organization. I agree that the OER should provide more flexibility and allow for IMS CC as well as SCORM, but I question the harsh tone of his post, which as you mentioned, seems to have set an incendiary, angry tone for the discussion in the industry.

  • http://www.learnnuggets.com Kevin Thorn

    Kudos Mr. Martin. No one ever ‘wins’ an argument when its one-sided view point fails to address every and all concerns.

    As we evolve in technologies and educational needs across a wide spectrum of industries, I don’t think there is ever going to be one single standard to suit them all. If one is good, then promote it to the industry(ies) that will benefit, but don’t bash another standard simply because it’s not your baby.

    Thank you for slapping some balance back into the SCORM ‘debate.’

  • http://www.pixelmaestro.com Lawrence Kevin

    I am “tired” of the complaint that SCORM is too difficult to implement. It really is not that difficult. I am sorry that it requires some technical knowledge to implement. It has been my experience that the technical specifications has caused administrators to make bad decisions on deciding course development tools. The reality that a SCORM compliant course can be developed with a text editor and winzip. Is SCORM the correct answer? It is one solution. I just don’t want the alternative to be simple because it requires the content developer to “think”.

  • http://www.dr-chuck.com Charles Severance

    Tim, this is a great discussion. I look at this issue with a lot of different hats on. First as part of the Sakai project and now participating in the development of systems like Moodle, D2L, ATutor, OLAT, and Blackboard, I am a technology builder, hoping to help the whole market better meet the needs of its customers. As a technology builder – the more standards these tools support – the better – period. Second, I am an IMS staff member helping build up our portfolio of standards with the goal of a more efficient marketplace. When I went into my rant about the demand that OER come out as SCORM, neither of these hats got flustered. Frankly, IMS and SCORM are talking and working and there are lots of win-win situations all around.

    The thing that gets me steamed about “must produce SCORM” is when I put on my teacher or student hat.

    I have had a situation where I was forced to take some required SCORM training as a student – by the time it as over, I almost hung myself with my mouse cord. The pro-SCORM folks will say “bad designers” – but there are some SCORM elements that instructional designers insist on using use because they work: (a) Lots of next buttons, (b) a left navigation menu with little checkboxes that appear as you progress through the material, (c) lots of questions where the content really does not care nor track your answers – they are just more screens that if you press Submit long enough – the software will let you get past the questions, and (d) the only permanent record of my training is that I found my way to some well-hidden trip-wire page where the lesson was marked “complete”. It was frustrating to be at 98% complete for two days because I could not find that last secret page no matter how hard I tried. After talking to HR training for two days, where they kept telling me what page I was supposed to find and sending me a sequence of screen shots for me to follow – and doing a view source on the page that I could not get past, I realized that there was some Internet Explorer specific code to call the SCORM runtime that was not running when I was clicking on the button in Firefox. I went and got a Windows box, navigated to the page, and viola! I was at 100%. And then once I hit 100% complete, I could never go back into the SCO because it was marked as a completed task in the learning system I had. Ah the joys of government-mandated HR training to avoid lawsuits.

    I am sure there are some sweet SCOs – but I bet those SCOs pretty much are just ZIP files with a lot of Flash, HTML, and Javascript – basically cool web pages that effectively use almost no SCORM at all.

    I am also a teacher and have a Phd. in Computer Science and actually I have built a compliant SCORM run-time in ASP.NET and Javascript in a previous job in 2001. So I know my way around SCORM, SCOs and the run-time API. Once I wanted to tweak a SCO in a way that would alter that left-side navigation and add a bit here and there for my own purposes. The SCO was so brittle that I ultimately wrote a script that went through the unzipped SCO and made specific patches to the common navigation on *every* page so I could do what I want. If you hand a SCO to a teacher – it is uneditable.

    So this is why I get a bit steamed (as a teacher and as a student) when the proposal is to pre-buy 2 Billion dollars of this stuff.

    Frankly, the thing that would take a bit of the wind out of my sails on this little rant of mine would be if some real-live teacher (not an instructional designer or standards wonk) came forward on this blog and made a comment like this: “I love SCOs! I got some Art History SCOs from a a teacher in Ireland and had the easiest time editing those SCOs and was easily able to extend and alter the SCOs in Blackboard to add my favorite paintings and then save new SCOs and send them to my colleagues across the state! They then used my altered SCOs to teach their Art History class in Moodle. They found some typos in my SCOs and sent my some SCOs back to me with all my typos fixed! Which I used this last semester. It was so easy!”.

    My rant is not abut SCORM as a standard versus IMS CC – SCORM is well-established and very mature and IMS CC is emerging and finding its real place in the marketplace. (sorry Rob) If it is a mistake to insist that all the content be in SCORM, it would be an *even bigger* mistake to insist that all the content be exclusively in the less mature IMS CC.

    The mistake is to pick only one format, and not allow alternate formats, and in particular pick a format that has never demonstrated it handles the edit/remix use case. The right way (as Michael Feldstein says in his blog) is to allow some diversity in approach and let the entire market work this out. Some experiments will work and others will fail – but we will have explored alternatives in a way to truly advance OER. I expect that both IMS CC and SCORM will have their significant roles to play. And I would be perfectly happy even if 90% of the OERs were SCORM.

  • http://www.imsglobal.org/ Rob Abel

    Tim-

    First, this is a passionate situation for the education industry. To be mandated to use a standard that does not fit what we want to do is a very big issue. IMS is an organization that represents 160 member organizations around the world primarily focused on education. Very bluntly – we are tired of having SCORM be mandated when it makes no sense. It’s been happening for years and needs to stop. We have no problem with SCORM being part of a solution and an option. But we have problems with it being force fed.

    Second, I was involved in AICC, IEEE, IMS, and all the rest in 1996-1999. When I came back to IMS in 2006 I was shocked and amazed at how little progress had occurred. SCORM is extremely outdated and way behind where the market is and needs to go. I have no apologies for that.

    Third, the funding of SCORM and the control of it is also a big problem. Who is the Wizard behind the curtain at SCORM? Who makes the decisions about SCORM mandates? There is no one there. There is no way to even work together. Rustici Software does not represent SCORM. Who does? Why does the U.S. government, using taxpayer dollars, feel it necessary to compete with industry standards groups doing essentially the same thing. This, I’m afraid is a problem that needs to be looked at.

    Fourth, the arguments I am making are fully endorsed by the IMS Board of Directors who are elected by the IMS members. The posting was vetted with the IMS Board before it was put up. You might want to take a look at who is on the IMS Board – Blackboard, Oracle, the University of California System, Microsoft, Desire2Learn, EDUCAUSE, SMART Technologies, SAFARI Montage, eXact Learning, Sungard, Cengage, Pearson, JISC, Penn State. So, if you think you are having a disagreement with me, you’re not. The disagreement is with the industry.

    So, I apologize if there is some passion in this situation. But, this is in fact a very important issue. Standards are extremely important to improving the education segment. SCORM might have a place, I don’t know. All I can tell you is that the movers and shakers are only seeing it as “something they are compelled to do”- not something that really helps them. That’s the honest truth.

    So, to see a grant RFP come out that mandates SCORM in the education segment is a problem – and we are going to continue to fight all such mandates. Sorry!

    -Rob Abel

  • http://scorm.com tim.martin

    Chuck…

    As you reference on your blog, you and I are mutually consenting adults, with no hard feelings…

    To be clear, had you and/or Rob (to whom I’ll respond in a minute) said simply, “This proposal should make allowances for other standards, specifically IMS CC, because it offers additional capability and choice is good,” I wouldn’t have even had a reason to blog. That’s a well reasoned statement.

    The frustrations you express with SCORM, though, are a bit misplaced. The standard itself makes no effort to prevent the use of editable assets. SCORM packages can be built entirely using open technologies OR they can include the necessary source files to allow for future editing. There is absolutely nothing in the standard that would prevent the content author or tool from offering up the ability to edit. Now, you can properly complain that few authors and fewer tools elect to do this, but that’s not an issue you have with the standard. If authors of IMSCC based content elect to use closed technologies (which I suspect some ultimately will), those assets, too, would be un-editable.

    You express further frustration with a piece of content that had an unclear path to completion. Truthfully, there are more pieces of SCORM content where this is the case than I wish there were. It happens. Again, though, this is a frustration with a piece of content, a bad one. There are piles of SCORM content that don’t have this problem, and the standard itself can’t be blamed for poor usage thereof.

    [Here's the part where I acknowledge my limited knowledge of IMSCC.] Does CC have a runtime protocol of any sort? I’m well aware that it does include both a QTI based testing piece and something akin to the asset or course structure. But does it venture into the difficult world of progress reporting today? I know (from a talk Dr. Chuck gave) that basic LTI includes the launch part of the equation as well, but I don’t believe it yet encompasses the return of progress data to the launching application. This reporting of progress, frankly, is a difficult beast. It leads to exactly the kind of challenges that Chuck refers to in the evil piece of SCORM content. What do you have to do to finish your content? How do you express that to the user? Is the reporting of progress even meaningful in this context?

    I come away from Chuck’s response with a couple of new thoughts:

    • We “SCORM people” need to pay more attention to the edit-ability of the packages we create. While SCORM’s interoperability is pretty strong, it’s reuse is being killed by black box packages.
    • We both (SCORM and IMSCC folks) need to understand more about each other’s standards, if we’re going to argue about their merits, because we have to speak accurately about their respective affordances.
    • The commonality of purpose between the standards is strong.
    • Any standard that allows its users to express completion to a system ought to also provide guidelines so that those users know what they need to do. What a terribly frustrating state it must be to not know how to finish what you started.

    Thanks, Chuck, for your candor in expressing the current state of adoption for the two standards. It lends clear credibility to your “rant”, in that it shows your willingness to depart from the party line.

  • http://scorm.com tim.martin

    Rob.

    Some context first:

    • I understand very well that IMS is a relevant organization with a board full of important organizations. I have a great deal of respect for the influence Ray Henderson has had at Blackboard and how he’s approached standards adoption. To move a large organization like that one forward so effectively is impressive to me.
    • I have no complaint about your desire to have IMS CC as an option here. As I understand it, IMS CC offers some things that SCORM may not in its current form. That’s great.
    • I, too, found SCORM’s evolution from 2003 – 2007 disappointing. I can think of a small subset of people who think that the addition of sequencing and navigation (from IMS SS) in SCORM 2004 was a massive success.
    • SCORM, or AICC, or IMS CC, or any other standard that wants to remain relevant does have to evolve in a considered fashion, in a way that incorporates the needs of the industry, and that’s a difficult process. It’s a process you’re trying to engage in by introducing new standards and evolving their issues (QTI and CC and LTI) and it’s a process that ADL is continuing to participate in through their Learning Experience work and our Project Tin Can research. May all these efforts fair well.

    You respond with passion, and I understand that, but you need to take a bit more care with it. Your experience with ADL is your own, and you should feel free to speak candidly about that. But please keep in mind the following facts:

    • The core technologies used by SCORM are no more or less antiquated than those used by IMS CC.
    • You are attacking a standard that is effectively used for interoperability on a massive scale. It succeeds in that purpose, and that alone makes it a valid inclusion in a proposal (not necessarily the exclusive inclusion, mind you.)
    • Dating back at least as far as 2003, ADL has been nothing but open to participation in the evolution of SCORM. We literally invited ourselves to participate in their meetings that far back, when we were two guys working in a spare bedroom, and we have been welcomed with open arms at all stages. At no point have we contributed anything other than our passion and intelligence, and there is no louder voice in SCORM’s evolution than ours.

    Ultimately, though, it boils down to this.

    While there may be issues between ADL and IMS on some political or legal level (you would know better than I), the standards themselves are not nor should they ever be at odds. SCORM and AICC and IMS CC would be best served to find commonalities, to harmonize where they can and enumerate their differences where appropriate. Elearning standards are about elearning and organizations working together, not ripping at each other.

    • Amongst your 14 listed board member companies, no fewer than 4 (29%) buy SCORM/AICC products from us, and 10 (71%) or more make some use of SCORM.
    • Amongst your 40 listed IMS CC/LTI/LIS adopters (either complete or in development), fully 22 (55%) have substantive SCORM support.

    Rather than railing against a machine which has clearly frustrated you, Rob, why don’t you change your focus to espousing the virtues of your standards? I would be happy for you to use this venue, even, to talk about why IMS CC is useful, why it has moved the industry forward, what affordances it provides that set it apart from other options.

    In the same way that you have the ear of your constituent organizations, I have the ear of our customers with regard to emerging standards. Further, SCORM Engine and SCORM Cloud mean we have in-place integrations with well over 100 LMSs that would allow us to deploy IMS CC support to millions of learners with little friction. In order to make a compelling case to me, and thereby to our customers, though, you need to raise the level of the discussion. Our readers, including me in particular, are your audience.

    Here’s your opportunity…

  • http://www.dr-chuck.com Charles Severance

    Tim, this rant has been great fun and is winding down nicely with some reflection and summary. SCORM is highly successful in highly scalable *training* situations – particularly where there is some legal or governmental forcing function and a solution is better than no solution and where is enough money to force fit the solution.

    SCORM is 10 years old now. As a teacher and as a student over the past decade, and watching my own children and their learning experiences and my daughter’s own experiences as she becomes a teacher, none of us in any of our roles as student or teacher have *EVER* seen, touched, or used a SCO during that 10 years.

    I have seen a SCO in my HR training and my wife works at Kohls and I think they use SCOs for their cash register training every day. The reviews on these learning experiences range from lukewarm to hostile.

    After a decade of SCORM, it is a bit of hand-waving to claim the shortcomings in the SCORM tools and SCOs those tools produce are not due to the nature of the spec. The SCORM spec as architected around a training use case and not a teaching use case. It does a great job in in the use-case for which it was designed.

    Since the motivating use case for IMS CC is teaching and allowing teachers to edit the IMS CC content and then re-save it, it is likely that IMS CC will be more suitable when teaching is involved.

    I don’t expect the government to back down on their “must be a CC-licensed SCO” statement. It is just too easy.

    It would be fun to talk about this with a whiteboard to delve into more of the technical reasons that each of these specs are strong for their respective use cases but blogs don’t have whiteboards.

    Lets meet back here in a few years when there is 2 billion dollars of free SCOs out there and see the extent the investment has had a real impact on teaching and learning. :)

    This kind of reminds me of the government’s significant investment in road construction to help “boost the economy”. I do like the better roads, but the finance impact really never trickled down past the people who own dump-trucks and road graders. I would bet that every road grader owner in the country now lives in a mansion :)

  • http://www.elml.org Joël Fisler

    I totally agree with the “The core technologies used by SCORM are no more or less antiquated than those used by IMS CC” part of your comment. Neither SCORM nor IMS CC really support the remixing part of OER since they are “just” containers for HTML pages (and in SCORM’s case containing lots of nasty JavaScript which makes it even worse :-) If you take two SCORM-based Math courses both created in their school’s design, how are you gonna remix this to a new consistent course without redesigning it? What if you are not remixing from two courses but want to take out the best or for your particular students most relevant content from 10 or 20 courses?

    The only solution in my opinion is an XML-based, format-independent approach like CNXML does it. This is the only way to have true remix-possibilities. Feel free to read my longer reply in the Chronicles comments:
    http://chronicle.com/blogs/wiredcampus/technology-leaders-balk-at-technical-guideline-in-federal-grant-program

  • Ethan

    I too would like to hear what benefits CC offers rather than an angry diatribe because CC was left off the guest list. I hope Rob can post here and present himself and the IMS in a professional manner this time around.

    In regards to being able to make changes I think that is an issue of the OER needing to mandate that source files be included. that is a valid and important point that I wish was a requirement. As for Chucks issue with editing the SCO-he needs to make sure that any scos he receives include the source files (assuming the license permits).

  • http://scorm.com tim.martin

    Chuck.

    You haven’t met the people who are giddy about SCORM? The guys who read the SCORM books repeatedly for fun? There are three of them, and one of them is my partner Mike. (Ooooh, wait, that was probably an uncalled for shot at a guy who’s on vacation.)

    I’ve opted to stay on the tech side of the SCORM fence, rather than getting into policy, thankfully. When it crosses into politics, it all starts going over my head… Well, except that I’m pretty sure I like smoked salmon.

    Tim

    PS The whiteboard stuff sounds like fun, actually… Someday.

  • http://www.dr-chuck.com Charles Severance

    Ethan – here is the essential difference. SCOs are self-contained learning objects with navigation, structure, assessment, etc all internally contained in the SCO. The SCO is launched from an LMS and then stores and retrieves data to/form the LMS while it plays – as it sees fit. The LMS launches the SCO and then responds to the API callbacks from the SCO. An IMS Common Cartridge it a hierarchy of learning content where the individual pieces of content and the relationships between sub-elements of the cartridge are all revealed to the LMS. The cartridge does not dictate navigation, or sequencing, nor does the cartridge handle tracking. The LMS presents the cartridge content in the most natural form in that LMS and lets the teach work with the individual items, moving them around, showing them, hiding them, adding new items or deleting items. Assessment and discussion materials are explicitly modeled so the LMS can present assessments in the same way as native content in the LMS is shown and the grade book integration is the same as natively authored content in the LMS.

    So the look and feel is familiar to students and teachers in the LMS as the LMS simply makes IMS CC content look like it was authored natively in the LMS. Think of how you create course content in either Blackboard or Moodle – IMS CC models that pattern perfectly. And since the content comes in to the LMS as natively authored content, if the content is altered and the LMS supports it the content can be easily re-exported back into an IMS CC to be imported into some other LMS.

    So an IMS-CC compliant LMS that supports export (as Blackboard and ATutor do) is effectively an IMS CC authoring tool – with no further training required on the part of the teachers. They are creating OERs without even being aware that they are doing it.

  • Ethan

    Charles, so how does CC handle complex animated simulations, 3d environments, Javascript executed Canvas animation based on mouse gestures etc? Plugins like Flash, Silverlight or Unity3d?

    To be honest it sounds like CC is very limited when it comes to anything I’d consider advanced experiences. IE: “assemble the HVAC components into the correct sequence for the new Audi A6″ (that was an actual assessment I had to create for them 6 years ago or so).

    It sounds very focused on the education side.

  • http://www.dr-chuck.com Charles Severance

    Ethan, CC does much the same as SCORM for these things – it includes Flash, HTML, MPEG, JavaScript, etc etc for those things. One thing that IMS CC does *not* specify is how those rich experiences store and retrieve content-controlled data to/from a server. Which interestingly enough – is exactly what SCORM does well :) .

  • Ethan

    So if you have a swf interaction placed in a CC pkg and don’t have the .fla you have the same level of editing that you would have i a scorm pk. Zero.If the course is fully rendered in flash then CC is pretty much useless. In your example of wanting to edit the left side TOC, CC would not help you in any way. You also could not edit my HVAC anime with CC (other than moving the placement on the page.) To me that toc issue had more to do with you not having the source material than a deficiency in the SCORM spec

    My point is not to smack CC but it seems your smacking SCORM for something that CC can’t consistently do depending on the tech used in the course.

    Also I have no idea what you mean by this:

    “I am sure there are some sweet SCOs – but I bet those SCOs pretty much are just ZIP files with a lot of Flash, HTML, and Javascript – basically cool web pages that effectively use almost no SCORM at all.”