Key Points:
- Our content is constantly updating, so we need to keep control of it on our servers. This doesn’t work well with SCORM.
- We provide the ability to personalize the content, mix and match assets as desired, quickly.
- For customers who do want SCORM tracking, we develop a way to send that data for each customer.
- Everyone always uses the latest version of the content.
- Recording informal learning — message boards, instant messages, is important for preserving knowledge.
- Looking into mobile, at least tablets.
- Providing “just in time” searchable access information, doesn’t have to be taken all at once as large courses.
- New API should make use of open source.
You were saying you do technical training?
Right, for techs, engineers, and communications companies. Our primary market is service providers: Verizon, AT&T, telephone companies. There are about 1100 telephone companies in the US. That’s our target market. The issues with SCORM are about accurate content. We looked into Scorm a couple of years ago, and we’ve run into clients every now and then and maybe even a potential partner now and then wit questions about SCORM support. Their question was, are our courses SCORM-compliant? and the answer is, no, they’re not. The reason they are not SCORM-compliant is because SCORM doesn’t work too well with online training. The reason that we like online training is because we are constantly in there, making the course better, doing enhancements and improvements to the course. We tell our clients that the course we’re selling today is better than the course we sold two months ago, and the course we sell six months from now will be better than this one. And the reason is, we are constantly getting feedback from students, from people who interact with our students, and people who have gone over the syllabus with us. So we’re constantly in there improving the quality of our training.
When you say “online training,” do you mean hosted training, as opposed to training that’s delivered to the LMS?
That’s right, we have a partner who hosts the website for us. It’s strictly online. And that’s real important, particularly in our industry, because things are changing so rapidly. Right now, there’s a big push within our industry to standardize Internet connectivity. It’s accomplished by the service provider and it is called Carrier Ethernet. In the SCORM world, when you develop a course and make it SCORM-compliant, you have to press a disc. Then you give it to your customer, when that happens, you lose control over what the customer is looking at. For that reason, we shy away from SCORM. Now if we have customers who have SCORM systems, then we can adapt our system to do the reporting that SCORM system requires. But as far as the course material itself, we kind of stay away from SCORM, because we don’t want to be pressing discs. The theory is that as soon as you ship that disc out the door, it becomes obsolete. And then you kind of lose control over what version people are looking at when they’re taking your courses. In our particular industry, communications, accuracy is very important and the information has to be constantly updated. So that’s why we kind of shy away from SCORM.
We’re intrigued that people are looking at something new or different. I’ve read the comments and I know there are a certain number of people who are saying that, well, you can’t always rely on having an internet connection. We’re saying, at least for our market, that’s a given. They have an internet connection, and a high speed one. We only need 200 Kb, but all of our customers have megabit connections. So that’s not really an issues with us. Now I can see that if you are doing courses on, for example, sales, or pole-climbing, or any kind of management training, HR training, etc., then that lends itself well to SCORM systems because you can press a disc, send it out, and it’s more or less “fire and forget it.” So what we’re doing is we’re talking with the technical people inside these companies, and that’s where, as I say, accuracy and timeliness of the material is crucial. And we’re actually developing a relationship — certainly with the customer, and particularly with the students that the customer will send to us.
We like to initiate conversations, and there’s feedback, and that helps us, again, enhance the quality of the course. Basically that’s the message I want to get across to you; I’m not sure whether this new learning platform is something that’s applicable to a company like us. We’d like to think it is.
It sounds like your customers do sometimes have LMSs that they want results reported back into; have you been able to do that step via SCORM? How does that work?
When we first started this out, we didn’t know anything about SCORM, so we’ve created our own LMS. We use software and databases to present our material. When you look at any of our screens on our courses, what you’re looking at is a template on the screen, and then we populate that template with various files. There’s an audio file, there’s image files, there’s text files; so we dynamically create all of our pages when the user requests them. It’s transparent to the user. We use this capability, this flexibility, because we do something that we haven’t seen anyone else do. We’ve developed 13 text-book courses, and that means we have probably about 100 different technology chapters. And those chapters are made up of about 350 technical modules. We enable the customer to go into what we call the library, and they can mix and match any of those technology puzzle-pieces together and create their own course on the fly, in real time. So you can create a course in less than a minute, by just selecting what you want the course to contain. That’s why we do it the way that we do; to have that flexibility so that the customer can mix and match pieces of the courses in real time. We think that’s important. And that certainly doesn’t lend itself to SCORM. But when we get to a customer who has SCORM, then they’ll say, well we need to know the reporting — did this person complete the course, etc. But we can make our system talk to their system with that information. Because we’re collecting the information and we send it, we would normally send it to them anyway. It wouldn’t be in a SCORM format, but as far as changing it, we can do that.
So you just do that on the back-end, just figure out how their system wants it and negotiate a way to send it for each customer?
Yes. Another thing that is very important to us, is the quality of what we put out there. We have looked at a couple of companies who are making SCORM courses, and when I look at their course, and look at what we put out, it’s night and day. I’ll give an example: we talked with a company over in Europe who does SCORM courses. And we looked at their course and in our mind, it was just no comparison. To give you an example, I asked them how long it took them to develop their course. Their course, they said, took about three weeks to develop a SCORM course. Now, to contrast that, we have a new course on carrier ethernet. Took us 856 hours to develop the technical part, which is basically an instructor-led course, the course work and all the graphics for the course. Then, in order to adapt that course to put it online, there was an additional 1,020 hours. That’s engineer hours. So our quality has to be there. If the person gets in front of the screen, it has to work. First time, every time, no hiccups, nothing. And if something happens, they just throw up their hands and walk away. So we have to make sure that everything is spot-on. We have one place for images, and when the image changed, the audio lecture describing the image changes with it. It’s timed and the quality has to be there for us. I haven’t seen that in a lot of the SCORM courses that I’ve looked at. Another example is that in the SCORM courses I’ve looked at, they’ll have the instructor or some other person doing the audio lecture for the course. What we do is we hire professional voice-over people to do the lectures. The difference in quality of the audio is important to us as well.
That may not be anything that’s really a limitation of SCORM Also, you have the mix and match — it’s really the fact that you’re delivering your own content from your own system, is what enables you to do that.
But I think it’s important for a company to be able to personalize. A company that was making SCORM courses wants to have this on the back-end. If a customer comes to them and says, Hey, can you develop a course with just this, this, and this, then they can do it in like two minutes. We think that’s an important component of what we do, because a lot of our customers like the online stuff, but the fact that they can then tailor it to be exactly what they want. For years in our business, we were saying, What’s the killer app? And we think that’s personalization. Have it your way. So rather than have one size fits all, we like to give the customer the capability to design it how they see it. So that’s an important aspect as well. I guess what I’m saying is that any kind of program that has you press a disc, and publish a CD, well it’s hard to maintain accuracy of the content in a changing environment. I think that SCORM certainly applies to a lot of generic kinds of training, but I don’t think it applies that well to technology training, because things are just changing so rapidly. It’s hard to draw a line in the sand.
As things are changing rapidly, what do you find are your versioning needs? Do you find there are some changes you make and people immediately see, or are there others you need to hold back if someone has already started a course? Any sort of split like that?
What we do in addition to providing the course online, each has a student workbook associated with it. So the student workbook is a pdf document, and the inside of the document ranges from some 60 pages up to 360 pages, depending on the course. Whenever we make changes in the content of the course, then we have to make that corresponding change to the document itself. So you’re right; we do look at windows when there are people in the courses, and if we do make a change, we’ll notify those people at the time, that there has been a change and they may want to take another download of the student workbook. But it’s important for us to make sure that the student workbook agrees with what they’re seeing on the screen, online. Typically what people do is they download the book and then they take the course with the book sitting on their lap. The reason it’s a pdf document is because let’s say I sell you a course now, and you can’t take it until the fifteenth of May. When you actually go in to the course, that’s when you download the book. So we can be sure the book agrees with what’s online.
It sounds like sometimes you notify people that there are changes, but it sounds like you never run into a situation where you’ve made a change, but you want to hold it back from the people who are taking it currently, just to avoid rocking the boat, and only propagate that change to the next group of people taking that course?
Yes, and we have people coming and going all the time. Because one of the good things about this is they can stop the course anytime. We automatically bookmark the page. I was just talking to a CISCO engineer who was taking one of our courses. This person can access the course from your work location, from home, while you are on the road if you’re in a hotel. As long as you have Internet access, you can access the course material.
Have you been looking at mobile at all?
You can take the courses from a Netbook. We are thinking of putting up student work-binders on iPads. We’ve already done one of our student workbooks to be viewed on an iPad. Not the course itself. But that’s easily done, because the big thing we have to work with is the screen size, and the iPad actually has a larger screen than the Netbook does, so that would be no big issue. We have an iPad to and we had to work around a couple of issues but it works great. But I think it would probably take us a few days to adapt the full course for the iPad. The hard part about selling it in the mobile environment is to get your price for the course. The prices of our courses range from $300 all the way up to $1300 a course.
And people are used to 99 cent downloads in the app store?
That’s right. So one of the things we are thinking about doing is selling it online but a chapter at a time. Here’s Chapter 1, here’s Chapter 2.
Typically, the way we engage with a lot of clients is what we call a site license. A site license gives the company access to all of the material we’ve developed. A site license duration is a year, and in that year, that includes any new material we bring online during that time. Customers pay based on the number of users that are going to access the system, so a site license of 10, 20, 50, 100, 200, 300. Like that. That’s typically how they engage with us. When we engage on a site license we establish a point of contact for their company, someone in their company who is going to be the administrator. It takes us about five minutes to train the administrator: Here’s your log-in information, here’s the page where you can look at the reports. You can go in and look at each student, find out where they are, what course they’re working on, how well they’re doing, etc. Which courses have been completed, all. We collect that information already for our administrator, so it’s just a matter of them building a bridge to make it adaptable for the SCORM system.
If you require any information about the people who are taking the courses, then the administrator feeds that in as well? And do you need to know their name, title, what department, etc?
Sure. We use their e-mail address as the user name, and we come up with a password. Our system randomly develops generic passwords, or we can assign our own. That makes it easy for them, but at the conclusion of every course, in order to get credit for that course, you have to complete the course survey. And the course survey has places for name, address, etc.
The other thing we do for our students is we build a profile page for every student. When the student logs in to the system, there’ll be a page where they can upload a picture of themselves, put their name, title, company, what their area of expertise is, and then we offer a bunch of web 2.0 services in conjunction with that profile page. There is a section we call the “switchboard,” which is industry information that the user might want to read. There’s another section which is “conversations” and that’s an internal blog for techs and engineers to talk about technical issues, or ask questions of each other, etc.
And you scope that down to the customer level, or is that just a blog where —
Depends on the size of the customer. If we have a small customer, than the blog will be shared by a number of small companies, but if it’s a large customer, we’ll do it internally. Whatever they’re comfortable with. We do that so they have access to those two areas. We also have an internal message system, and here’s the cool thing about the message system. We tell our customers that, this is not like the old days where you get out of high school and you go to a company and you work there forever. The way that it is these days, a lot of people, especially in our business, are project-oriented. They’re engineers. They want to work on this, on that, etc. They want to build their resume. So if you’ve got something interesting that they want to work on, that’s a cinch — they’ll work there. When the project is up, then they’re going to take a hard look at what else you’ve got to do, or maybe they’ll go over and work on another project somewhere else. We’re telling our customers that — in the lifetime of your company, there’s going to be some really smart people transition through your company. They work for two, three, four years, and then they’re off to something else. We have created is an internal message system. We encourage the customer to have technology discussions using the message system, which is almost like an instant message system. With this message system, they can have those internal discussions, and we record the discussions and archive that information. Th archive is then searchable by other people inside the organization. An example would be, you have a company in New York, and they’re having a discussion on how they’re going to support voice-over IP. They may have that discussion between people at the same location or a different locations. And then maybe, let’s say, six weeks later, other people in San Francisco want to have a discussion about voice-over IP, they can go in to this message system, go into the archives and do a search on voice-over IP. They can recall this conversation that happened six weeks earlier in New York, and see what was the result of those conversations. We’re trying not to just deliver training courses, but to implement learning environments. The courses raise the level of discussion between the people who take them, and then we want to be involved in recording those discussions, or facilitating those discussions that helps people learn from one another.
If you go to any classroom, there’s a lot of learning that takes place outside the particular instruction — on a break, at lunch, etc. You might learn something that’s real important. So we want to tap into that, and facilitate these discussions as well. I don’t know how that would fit into an advanced learning system, but my encouragement is to develop something that not only deals with the actual instruction itself, but recreate that whole learning environment. Sort of like that unstructured learning. We think that’s important.
A lot of what I’ve been looking at is how to integrate various systems that provide learning experiences with systems that track it, or other systems that might offer an environment for collaboration, but it seems like you have the whole system. Whereas, sometimes, people think of the LMS —
And that’s part of our history. What we have right now, we were actually doing, probably 80% of this back in 1998, but it didn’t work because the access technology wasn’t there. You could download a page, but while the page was downloading, you had go get a cup of coffee. And a lot of the functions that we have now, we had to develop back in those days. We had to develop the capabilities ourselves. There were no software packages that did it. Now, there are software packages with all kinds of things to help you publish this stuff online, but we’re sort of taking the other route, where we’ve developed a lot of this stuff.
Do you have — it sounds like — would you say most customers use your system as it is and get reports out of that, or do more of them have other systems they want to — that you wind up integrating with?
No. I can count on one hand the number of customers we’ve had who wanted us to adapt our system to report to their SCORM system. Most of our customers are interested in the actual learning itself, and what they’re looking for is accurate and timely data. That’s what they want. And so, if this is an easy solution for them. The only thing they have to do is maintain a website or a browser. It’s very easy for them, it’s a turn-key solution for them. A lot of our customers don’t have big training organizations. Even so, I’ll still work with large companies that probably have these internal systems, but we get engineers and people calling us because they want access to the content that we have. And it’s easy for the companies to say, Yeah, go ahead, do it. Because all they have to do is get on the internet and they get access to the course. So we do the credit card process — they call us with a credit card, we take the card, we give them access: Bang! It’s done. They go in, get access, take the course — they get a certificate when they finish, if the boss calls up and wants to know how they did, we’ll that out for them. So, it’s — we’re trying to make it as easy as we can for the companies we work with, and the customer. So we do all the heavy lifting on our end.
A lot of them don’t have a big learning, tracking system they want to keep track of all this in, necessarily.
Sure, and typically what we do for those customers is we just generate a monthly report. It’s automatically generated. They go in in the first day of the following month, and they can just print out the report that’s sitting, waiting there. And it just shows who did what during the month.
Another thing we do is interactive exercises in our courses. So our courses are broken down into chapters, and there are several modules that make up a chapter. Every chapter has objectives, the course content, and then at the end of the chapter we have a summary that summarizes everything. And then we have interactive exercises, or multiple choice exercise questions. So the questions range from 16 up to 36 or 40 questions, depending on the content of the chapter. The student can go in and they’ll answer all of the questions in the exercise, and we make them answer all those questions. There’s a button where students can correct the exercise question. Let’s assume someone takes the test and they get 5 or 6 questions wrong, They may be 75% first time they take the exercise. We’re not interested in giving anyone an A or B or C; we want to make sure they understand the content. We allow the student to reset the exercise, go back into the material, look up the answers they got wrong, find out what the right answers are, come back to the exercise, answer those questions with the right answer. So the first time they take the test, they get a 75%; second time they answer the other questions, they get 100%. As far as we’re concerned, that’s 100%. We haven’t run across anyone who is really hung up on whether someone gets an 80 or 90 on a course, or whatever. The only thing is, if you have people who aren’t motivated, who you have to maybe coerce to take the course it doesn’t work. We have had several instances where they were made someone take the course, they went in and took the test one time and said, forget it, and moved on from there. We had a couple actually failed a course. That’s really hard to do. But that happened. Another thing we run across is we had a couple of companies where we look at the logs of people going in and taking the course and the exercises. We saw some people go in and take the exercises and complete it in a minute. And that told us that someone had gone in and taken the exercise, and written down the answers.
Now, in every exercise, we randomize the questions. The questions you have in one person’s test will be arranged in a different way for someone else. And even in some courses, we have multiple pools of questions. In one of our courses we have 400 questions, and what we do is randomly select a number of questions from that. It’s just making the system deal with real life. There people who are going to cheat. And that’s important. I tell all of my clients, “If you have to coerce people, if you have to bring them over and sit them down in front of us, save your time. There’s a lot of people out there looking for a job. Get somebody in there who is interested and wants to do this. It’s somebody who understands the gravity of the moment. If you aren’t trained, if you don’t know what you want, then you’re going to be out of a job. Your standard of living is going to drop like a rock. And so it’s no good to force people to do that.”
These features you’re mentioning — the question pools, the randomization, etc. — that’s all things you are perfectly happy with how they are working the way you’ve done it, so you aren’t looking for standards to help support that in the future? Are there things that if there were a standard that incorporated features around this that you might want to switch to?
There are capabilities in things that we have right now, that we thought were important to the development of our products. A lot of other people might not want to do that, but I want to mention that these are things that we’ve developed to separate us from a lot of other companies.
I’ve had some companies say to me, they are afraid to do training, because if they train their employees they will be more appealing to other companies, and they’ll go off and get another job. And my comeback to them is that there’s nothing you can do about that. You’re going to train people and there are some people who are going to leave. But the thing you should be thinking about is all of those people who aren’t trained but you’re still paying them. So what are you getting from them? Right now, another thing we are building in that we’re about two weeks away from having online chat. So that the persons taking a course can do an instant messaging with the person who wrote the course. Right now we do e-mail, but I think IM would be better.
This is what you’re doing now, but in the next 5 – 10 years, how do you think e-learning will change, or how will you need to change what you’re doing with your training?
Well, obviously — I’m an old guy. I can’t imagine taking this on an iPhone. But that’s something we’ll develop this year, to be able to offer this on a mobile device, certainly on an iPad. This will probably be done before the summer. As far as doing this on a mobile device, that’s another thing I think we’ll do. What we’re trying to do is create a technical partnership with our clients, so if they have any technology issues, they can talk to us, and we’ll give it to them in plain English. What are you looking for? For example we already have a search engine, where they can type in a keyword, and we’ll tell them every place in the Watershed library that the term appears. In real time we want the person to get the information right when they need it, from wherever they happen to be. If someone is working on an issue and they have a problem with Internet Group Message protocol, they can go on the system, type in IGMP, and in five minutes, we’ll give them — not just — a rundown on what it is, but not just text but graphics, animations, to explain it to them much quicker and in a format that’s easy to listen to. Rather than try to reach something that’s five to ten pages long, now you can just open this up and it’ll talk to you. It’s kind of like audio-books for engineers, with animations and texts and graphics. I think that’s going to be, that’s important. To have that information right at your fingertips. Whenever you need it.
So just “in-time” information, being more of a reference than just going and taking a big course and delving into that for information?
That’s part of our marketing. When we market the course, we’re saying, “If you’re going to put in voice-over IP in June, no sense in taking the course now. Wait until maybe the second or third week in May, take the course then. When you start to implement, start putting this new network in, and the information is in short term memory and you can apply it and that’ll really make it stick. It’s all just-in-time training. A couple of other good things about the way we do it, is, if you’re a company and you need to train lots of people quickly, we can do that in a week. And, those 150 people don’t need to be in the same location. They can all be in different locations, doesn’t matter to us.
Any other changes you can foresee coming up?
We feel like we’re hamsters on a wheel. We’re trying to keep up with what’s going on technology-wise, so that’s difficult by itself. What we’re doing is implementing a program where we’re going out to the industry and looking for industry experts who will write courses for us, we’ll put it on our system, and we’ll give them a share of the revenue.
Is there anything you didn’t get a chance to say that you wanted to?
No, I think that’s probably it. That’s a rundown on what we do, and that’s what I wanted to do was give you some input as to how we look at this and how it works with us. Like I say, the big problem we had with SCORM is the need to keep up to date with what’s going on in the industry. And I would say that not only applies to my industry, which is communication, but it probably applies to a lot of the sciences as well.
Supporting distributed content is one of the goals of this project.
I’m a strong proponent of using open source. We use PHP, AJAX, Linux, all open source software packages in our products. I think if you buy something that’s open source, that lets other people in the industry get in there and take a look — as long as you have several people that know what they’re doing to be the editors of changes you can make, just open it up and let people who are in the business go in there and look at it, maybe make adjustments to it, and changes. I think that’s the direction that everything is going. It’s probably a good idea to open it up and let other people get in there and collaborate and you’ll find out how much stuff you can get done for free by doing that. Whereas, the other point of view is to then hire some software company to develop this. I’m a proponent of opening it up and let other people — take all these people who have SCORM courses, SCORM systems, open them up and let’s see how you can adjust them, make changes and enhance them.
It doesn’t seem like it’s at cross-purposes to try to define an open standard, whether they’re open source or not, but that seems like it would be helpful both to open and closed source systems.
The thing is that I’ve been involved in standards for a number of years, ANSI standards, ITI, IETF, and IEEE standards, etc. It’s been my experience that the bulk of the implementation of these standards is complete before the final drafts are written. When the standards are finally complete all the leading-edge companies are off on something else. For product development, once you get a framework designed and people have an idea, okay this is pretty much how it’s supposed to work, then you have a number of companies who will develop their solutions, what they think is the right solution, and the market will decide which one is spot on.
