February 27, 2005

CI5336 - Case Study 12 Reflection

Case Study 12 showed how a perfectly good idea could get into trouble without a proper evaluation strategy.

For Sam Gonzales, the idea of using video clips and multiple choice questions, all delivered by computer, was a perfectly good way to assess training. It would be easier and faster than paper-based short answer, multiple choice tests. The data from one pilot test of the assessment seemed to show that it was more accurate than the existing method in predicting which students would have actual on the job performance problems, based on their test scores.

However the more that test scores are tied to evaluation and pay in a business, the more political the assessment process can become. When two students and a trainer objected to low scoring assessments, and the students wanted to take the regular paper-based assessment, Sam was not in a good spot. He had no evaluation results that could demonstrate that the protesting students and trainer had no basis for their complaints.

By evaluating and testing his assessment idea in its define and design stages, he might have discovered:
1) cultural/political objections to the new method that he could have possibly dealt with
2) computer-based skill issues that might have made the assessment system unfair or unreliable for a significant number of individuals.

With a proper evaluation strategy, Sam might have found ways to improve his idea and increased acceptance of it before releasing it in a real-world application.

Posted by burt0177 at 4:26 PM

February 20, 2005

CI5336 - 5 questions for instructional designer

I am interested in teaching non-credit classes about various computer skills in the community college/technical college system in Minnesota, and maybe elsewhere. This is part of a long-term strategy to find part-time work to supplement my full-time job now and my retirement income later.

So, I intend to interview at least one person who is teaching non-credit computer skill classes. I regard these individuals as instructional designers because they usually have to design and propose their own classes. The college may have a topic it wants to offer a non-credit course on, but it is up to the instructor to actually design and deliver the learning experience.

I would like to ask this person:

1) How do you gather information about the learners you expect to be in the class?

2) How detailed are the learner outcomes for the class? Does the college have to review and sign-off on them?

3) Do you have to come up with all learner activities? Do you design them from scratch or find existing materials, like a book or tutorial?

4) Do you have to test the learner's at the end of the course? Even if the college does not require it, do you go through some kind of assessment so the learners can prove to their employers (current or prospective) that their skills have improved?

5) How do you evaluate your course? A survey handed out to the learners at the end of the course? How do you use the evaluation to change the course?

I intend to conduct a face-to-face interview with the instructor, taking written notes and probably audio-taping the interview. I will transcribe the interview and prepare it for publication, probably doing some editing to clean up the grammar and improve the flow of ideas. I will make the interview available on my blog, probably as a pdf file attachment to a blog entry.

I may supplement the interview with an instructor by talking to one or more college staff members who organize and promote these non-credit courses.

Posted by burt0177 at 4:16 PM

CI5336 - Chapter 5 reflection

Key points of Chapter 5 in Real World Instructional Design

Most of chapter 5 covers how learning activities are delivered.

  • Learning activities are always delivered within a context. In many cases, the delivery context is a given, determined the client. Delivery context includes:
    • The instructor/instructional setting context, such as
      • location of instruction
      • equipment available for training
      • time of instruction
      • curriculum within which the learning takes place
    • The learner context. In a classroom setting, the learner's context is the same as the instructor's context. But in distance or asynchronous activities, learners may have:
      • their own hardware, software, and support issues
      • schedule constraints
      • size of the learner group
  • There are delivery options to be considered when designing learning activities
    • Synchronous vs Asynchronous
      • In synchronous learning, like a classroom setting, the instructor and learners are all present in the same place and time. The major issue to deal with in synchronous learning is that different learners bring different amounts and types of prior learning to the class and are able to learn the material at different paces.
      • In asynchronous learning, such as correspondence courses or online courses, learners proceed at their own pace, but more design attention must be paid to helping learners monitor their progress and stay on track
    • Distance vs Face-to-Face
      • In distance learning, instructors and learners are separated by space. The learning activities need to be designed to deal with the sense of isolation that may develop. The chief advantage of distance learning is eliminating or reducing the amount of travel needed to bring instructors and learners together, as well as reducing the physical infrastructure (classrooms, buildings) needed to deliver the learning activity.
      • In face-to-face learning, instructors can expect learners to participate in monitored, group activities such as discussion and role-playing. Social status and personality can play a larger role in determining how face-to-face activities proceed.
    • Formal vs Informal
      • Formal instruction occurs when specific learning outcomes are of primary importance and may be related to a learner's career path or current occupation. Formal instruction is often scheduled, rigorous, tested and has a tuition cost.
      • Informal instruction is often a byproduct of some other activity. A museum exhibit may teach while it entertains. A job aid may offer learning that has both near and far transfer, even though it was used by a learner to quickly solve a problem.
  • There are delivery modes to be considered when designing learning activities, such as
    • Classroom - the oldest, most popular, best-understood mode of delivering learning activities
    • Telecourses and teleconferences - used for distance learning but in synchronous mode, when instructor is in one location and learners are separated by great distances. May want to add learning activities that the local group can participate in
    • Live Web - very similar to telecourse, using internet technology to deliver voice, visuals and data, and may involve two-way interactivity with participants manipulating a common visual space, like a whiteboard or computer desktop space
    • Print - read a book, do a worksheet, take a written test
    • Videos - add sound, visuals from different locations, compression of time, special effects, animations
    • Networked (online) - delivery of multimedia modules for learning, as well as group interactivity
    • Computer media (CD-ROM, DVD-ROM) - allows high-bandwidth multimedia presentation as well as vast amounts of text and data, enables rapid searching for information, but unless it is part of a networked system, it is a one-way mode of delivery
    • Performance Support Systems - just in time learning, help systems, job aids
    • Of course, these modes can be blended. A DVD-ROM could hold animations and a large text-base for searching, while being part of a networked learning activity

Chapter 5 also presented the components of a learning package, including:

  • Learners materials
  • Instructors materials

Many learning activities will include a variety of media assets. Selecting media assets includes the following concerns:

  • choosing the right type of media asset to support a learning outcome
  • choosing the specific asset to best support a given learning outcome
  • making sure the media asset is licensed properly for the given use (and possible re-purpoing uses)
  • making sure the learning outcome can still be attained by individuals with a variety of accessibility needs

Finally, chapter 5 gets a little into structuring interactive programs. Although this is a major topic that can fill a book in itself, the main concerns listed in chapter 5 include:

  • how the material is organized and presented in menus
  • how the navigation system lets the user move around through the material
  • how individual screens are design for learning success

In terms of my prior knowledge, Chapter 5 covers things I have had to learn and do in the past. I have designed and taught classroom courses in scriptwriting. I have designed learning materials delivered by video, web and CD-ROM. I have had to be concerned with how interactivity supports learning outcomes. However, I continue to be impressed with how well Cennamo and Kalk cover the details of being a successful instructional designer and I will be keeping my copy of the book to refer to when I prepare to do future projects.

Posted by burt0177 at 2:25 PM

February 14, 2005

CI5336 - Case Study 7: Iris Daniels

This case study illustrates some of the challenges of trying to produce learning materials that work across different cultures. This is important for two reasons. A growing number of companies and organizations are multinational. Also, any company or organization within the U.S. will probably be working with a diverse group of employes and/or customers.

The case study itself seems to imply that Iris and Jim (the American developers working with Hill Industries, an American member of a multinational consortium of companies using Lapin software) had adjusted well to the challenge of adopting American ID models and practices to European expectations.

However, in our class, Janine pointed out that the prototype Iris and Jim were proposing was still riddled with American slang and idioms and might not be received well by the European developers and members of the consortium.

That brought home the point to me that if you want a learning product to work well across cultures and nations, you will have to go over the design again and again, and be willing to listen and challenge your own assumptions.

Posted by burt0177 at 11:42 AM

February 6, 2005

CI5336 - Case Study 4: Ross Caslon

In this case study, Ross Caslon has a challenging task - come up with plan to implement a web-based, course-management tool at a large university. Unfortunately, there's no strong sponsorship, no clear leadership, no strong desire by faculty to use the tool, no willingness from the technical staff to support it, and to top it all off the tool has many irritating technical deficiencies and problems.

The case study strikes me as an example of setting yourself up for failure by expecting too much to happen too soon. Ross did a test by having 7 faculty members use the tool after a one-day training session. Even that test showed some promising results, but not as promising as Ross had hoped.

Ross needs to be realistic about what he can achieve with the resources he has in hand and he needs to build on the successes in his test case. He can't realistically use direct instruction to change the behavior of faculty and support staff. He can identify a few specific successful uses of the course-management tool and keep working to refine those uses. He can identify uses that were not successful and evaluate them to determine if specific support and instruction would have made them successful or if other factors - like software problems - were the cause.

This case study reminds me of the role I play (as webmaster) in my organization as a change agent. I don't have the resources - technical, financial, political - to impose large-scale changes. But I can propose and deliver projects that improve the organization's performance. I can promote ideas and respond as quickly as possible when those ideas take root in the minds of others and they decide to act.

In the big picture, Ross is on the "winning" side by being an advocate of web-based learning management. He just needs to pick his goals carefully.

Posted by burt0177 at 8:24 PM