Bob Pike and Ken Blanchard had a teleseminar recently (download here) in which they talked about trainers as change agents. This powerful blurb is from that discussion:
Training is about creating… CHANGE.
CHANGE… in the way people do things.
CHANGE… in what people know.
CHANGE… in the results people get.
As I've said before in this blog (and over and over the workshops I give), the needs assessment step in the design process is absolutely critical. One of the most challenging parts of this step is uncovering what the real need is under the request. Those of you who work at the reference desk know what I'm talking about. How many times have you had patrons come up to the desk and ask for something that they didn't really mean to ask for? After a deft reference interview you uncover the actual question and the patron leaves satisfied. Same thing with the initial request that we sometimes get for a workshop.
One way to conduct an effective "instructional interview" (similar to a reference interview) is to use a technique called IWWMW that I'm borrowing from the business industry.
IWWMW stands for "In what ways might we" which is the core of the technique.
The process uses these 3 questions or statements:
Question - Why do we want to ….?
Answer – So that….
Redefinition – In what ways might we ….?
You would run through these 3 questions until you were satisfied that you had identified the crux of the problem that you will be addressing.
So, let's say you're talking to your client (the person who wants you to do a workshop). In this case they're a professor. What they ask you is to teach their students a particular database - let's say Pubmed.
Your job is to ask "Why do you want your students to learn Pubmed?"
Perhaps they respond, "So that my students learn to use peer-reviewed articles in their research."
Next you ask yourself if that answer really satisfies you (and them)? Is that the real problem that you will be "solving" in your workshop: "In what ways might students learn to use peer-reviewed articles in their research?"
Or do you want to go deeper?
Try another round:
"Why do you want your students to use peer-reviewed articles in their research?"
They respond, "So that...."
Does that satisfy you? If not, try for another round.
What might happen is that during this process both you and your client realize that you need to overhaul the whole assignment, or maybe even the syllabus. Maybe you need to have a pre or post-workshop assignment to address your bigger "problem." In other words, this discussion could open up enormous opportunities for you and your client to truly tackle some big stuff. And in the least, it can help you really focus on what's essential in the workshop.
Give it a try the next time you conduct a needs assessment!
Continue process until you’re satisfied you’ve identified the real problem.
I just came from a writing class whose professor allowed us to come in to evaluate a series of e-learning pilots we're developing in the Libraries. It was quick and easy and similar to the process we've used for our face-to-face workshop pilots. I encourage you to give this a try! Here's what we do:
We offer our pilot workshop to library supervisors who want their student workers and new staff to learn the content we teach in the workshop. They require that their workers attend. We teach the workshop as it is designed, but at the end we turn the group into a focus group. We review each section of the workshop, stop, and facilitate a discussion:
What are some things you did not know about until you took this part of the workshop?
What did you like about this part of the workshop?
Did anything about this part of the workshop bother you?
How could this part be improved?
Other comments?
Amazing the kind of useful, thoughtful feedback we have gotten.
We've also done this with "real" workshops and "real" users where we've asked participants in a workshop to stay afterwards (and we pay them!) and conduct the same kind of focus group discussion then.
It's pretty easy to throw this kind of evaluation together but yet you'll get tons of value out of it. Give it a try!
After a bit of a hiatus (in other words, having a baby), I gave a half-day workshop at Cornell University Library on Friday. Here is a list of individuals' workshop take-aways that many of them gave me at the end of the session:
Needs Assessment
- Take 1-2 stakeholders to lunch to get feedback on course
- Have a conversation with faculty
- Find the right questions
- Ask why the students want to attend the workshop
- What is desired outcome? This is workshop title or description
- Frame as deliverables
- Engage client
- Refine the interview, process with faculty
- Ask “Why? What’s the point?”
- Get from the client what/how does success look like.
- Concept of “delivering” on x, y, z outcomes - you’re giving the client something
Brainstorming Content
- Group brings ideas and their points of view
- Make this a group activity
- Great way to cover everything. I missed a lot on my own
- Not to analyze, just say it
- I like brainstorming in a circle (taking turns). Seems to prompt responses
- Work with co-workers to discuss. Can this be done with faculty? [Answer: Yes, if they know the content fairly well.]
- Be open to all ideas! Facilitate process.
Filtering Content
- Don’t overpack the session with information
- Look at what I plan to present and divide by need-to-know and nice-to-know
- Importance of limiting/focusing on goal -what I can deliver.
- Focus on need-to-know because nice-to-know will probably fall by the wayside
- Get to what’s important
- Be strict. More B’s (nice-to-knows) than A’s (need-to-knows)
- Breaking topics into different categories
- Focus on need-to-know, not nice-to-know
- Be ruthless with A’s and B’s
- Classify and eliminate
- Space facilitated discussion (from the brainstorming) to avoid anger over Boolean cut (as an example)
Task Analysis
- Delineating steps focuses us to ask whether the students have the necessary background knowledge
- Look at my existing exercises and break down tasks more; have a student do them beforehand to test
- It’s hard to figure out what your actual goal is
- Painful process to figure out what the learner has to do to be successful
- Change level of task based on what learner already knows
- Have realistic expectations, actions, and steps
- Try to imagine 3 modules per 50 minute class
Others
- Emphasis on moving from lecture to users teaching each other
- Design content for pre-class and post-class handouts/worksheets/web pages
- Content presentation can be done by students
- Give handouts with gaps. Reveal answers later (mystery!).
- Learners can present content
- Renewed enthusiasm for integrating information competency into curriculum
- Awareness of the need to discuss this more within the system as a whole and the need to advocate for a change in the model system-wide
Obviously there are many steps we didn't cover in a half-day session. I focused on some of the most difficult steps which often can have the biggest impact on the final library workshop design.
The Spring/Summer 2007 issue of Educators' Spotlight Digest (ESD), the
free online publication of S.O.S. for Information Literacy, has just come out and has a great article called "Librarian Challenge: Reaching College Freshmen." Particularly take a look at the interview segment with librarian, Michael Pasqualoni.
Here's a summary snippet taken from the article:
Freshman Teaching Tips
Here are a few excellent tips garnered from Pasqualoni and Galloway:
* Chunk lessons into a beginning, middle and ending
* Use mixed media and approaches to keep interest (e.g., instructor lecture, video clips & student interaction)
* Experiment and try new techniques (e.g., "Radical Syllabus" exercise )
* Ask students "what interests you?"
* Distribute helpful handouts
* Be "Presentation-Proofed" by carrying material in several formats
* Plan a “Library Lock-In” event and create excitement
Last entry in the "do's and don'ts" for good design I mentioned a "do" that said, "Take responsibility for your learners meeting your objectives." It seems to me that this one, in particular, merits more discussion. As teachers we all are faced with a decision - how much of our content are we going to take responsibility for our learners to actually learn it, and how much will we depend on the learner to take responsibility for learning on their own?
To answer this question, ask yourself who you are being in a certain situation. Are you a teacher, are you a lecturer, or are you an orientation leader?
The last two roles – lecturer and orientation leader – probably give the learner more responsibility to handle the content on their own. But the first role – teacher – places you with a greater responsibility to ensure that learning is taking place.
When I give my train the library teacher workshops to other librarians, this topic usually comes up after the design teams have brainstormed pages and pages of content and are now attempting to figure out what content is really crucial and what is less so. Inevitably, someone in the groups will say, "Well, we're just going to mention that item, we're not really going to teach it."
Now remember – we’re not designing orientations and we’re not designing lectures – we’re designing workshops. So, I have a pretty strong reaction to this. What I remind folks is that “mentioning” content comes at a cost to the learner. Learners only have so much cognitive ability to absorb and make sense of your content. In fact, when it comes to short term memory, the rule of thumb is that a person can only take in about 5 to 9 pieces of information before their capacity to take in information is reached. At that point the learner has to do something with that knowledge – apply it, synthesize it, process it – in order for it move into long term memory. So every time you mention something (“Oh, and you can order this through Interlibrary Loan. That’s a service where you… blah blah blah”) you’re tipping the learner towards or over the 5 to 9 mark. The problem is that when you do get to something that you actually want them to learn, you’ve used up your ration, and have to move directly into the “doing something” part of your lesson plan instead of covering more content/teaching points. But chances are, you wouldn't have designed this fast detour into "doing something" and so your handouts and exercises wouldn't work. If you're not fast on your feet, guess what - it probably means you’ve just messed up your essential need-to-know content with a bunch of "nice to know," yet less essential information. Yikes!
There’s a fabulous book out there called Telling Ain’t Training. This should be essential reading for all of us librarian enamored with our content and information. The single most important thing we can probably do is to lose our infatuation with sharing EVERYTHING that should be known and focus solely on the most essential items that must be known.
A library school student recently interviewed me over email about the instructional design process. One of the things she wanted was a short do's and don't list.
Thought I'd share this with you all:
Do's
> Identify a client with enough clout to strongly recommend or assign learners to your workshop
> Keep in communication with that client throughout the design process
> Conduct a thorough needs assessment
> Take responsibility for your learners meeting your objectives
> Keep content to a minimum and focus on application and feedback
Don'ts
> Drown your learners in content
> Blow off the needs assessment
> Think if you say something that your learners have learned something. (Saying does not equal teaching.)
> Design an entire workshop by yourself - stretch yourself by collaborating with others.
> Assume just because you designed the workshop that you have to teach it. There may be others who might be better suited to deliver your workshop.
> Assume just because you're good in front of the classroom that you're a good instructional designer. Sometimes being a good performer can get in the way of a good workshop design!
Hope this list helps!
We had a great instructional design workshop as an ACRL preconference at the Midwinter meeting of the Annual Library Conference in Seattle. We even had a guest teacher from the Minnesota Historical Society who taught a segment of a lesson plan that I have included in the Creating the One-Shot Library Workshop book. If you've got the book, check it out on page 115.
A heads-up to you all is that for the Seattle workshop I redid the worksheets.
For those of you who haven't gone through one of my workshops, the worksheets can be used as a guide when designing your own workshops. They can be used with the book, or on their own. The link for the worksheets are near the top of this page.
For those of you who have taken my workshop prior to the Seattle one, the main changes are the addition of a checklist after each step that I pulled from theCreating the One-Shot Library Workshop book. I also added in much more detailed information in the evaluation step since I don't usually cover that enough during the workshop.
ILI-L is full of inquiries about info lit tests, pre and post tests, and end-of-workshop quizzes. For the most part it seems like we we all need are examples – examples of tested quiz questions that are clear, that ask what is really intended to be asked, that don’t use confusing terminology and that have good, clear distracters (which are the incorrect responses in a multiple choice question).
Clearly, writing great questions is a challenge. Over the years in our “Unravel the Library” workshops at the U of MN-Twin Cities we try each semester to improve our questions. I talk about this quite a bit in the Creating the One-Shot Library Workshop book, but here, wanted to point out a resource that I recommend you track down if you're interested in writing better quiz questions:
Assessment in College Library Instruction Programs
There are lots of great examples there that can help keep us from reinventing the wheel.. again.
[Also, related to this topic is another entry in this blog about the relational model of design that links assessment to goals and activities.]
Games can sometimes be the perfect solution for an orientation or instruction session.
Often times at the University of Minnesota Libraries the goal for our 20 or 40 minutes as part of first year orientation isn’t to cram as much information as we can into those precious minutes – instead, our goal is to leave the students thinking that the library sounds like a great place to be and a great resource to have. If we get that far, we're pretty happy. What better way to do that than a game?
One of the training companies best known for developing instructional games is the Thiagi Group. They have some useful instructional design tips on the web that may be helpful to you. I've included library examples where it makes sense.
First, decide to which of these three domains your learning content belongs:
1. Facts and Information
A fact is a simple bit of knowledge or information. It is usually in the form of a statement that specifies a relationship between two or more objects, actions, or events:
- AND is a Boolean operator.
- An article index is where you can look to find articles on your topic.
Facts are mostly related to other facts and other types of knowledge.
2. Concepts
A concept is a set of objects, events, actions, characteristics, or ideas that share critical attributes and belong to a specific class that is identified by a common label.
- Controlled Vocabulary
- Citation Styles
For every concept, you should be able to provide multiple examples. (If you cannot, you are probably dealing with a fact.)
3. Procedures and Processes
A procedure is a step-by-step skill that is performed essentially the same way each time.
- Finding a book with a known title
- Identifying who is citing a seminal article in your field
A process is a sequential flow of events.
- Responding to a digital reference question
- Designing workshop handouts
What kind of game might you design for each type of learning content?
1. Information and Facts
- CHOICES with multiple choice questions
- TIC TAC with questions resulted to different subtopics
- HANGMAN (multiple question) with fact recall questions
- CATEGORIZE (True or False) with statements to be classified as true or false
2. Concepts
- CATEGORIZE (categories) with different related concept labels and examples to be classified.
- HANGMAN (single question) that requires recall of examples
- TIC TAC with questions classified according to different concepts.
- CHOICES with questions related to critical and irrelevant features of a concept.
3. Procedures and Processes
- SEQUENCE for arranging steps, stages, or phases in the right order.
- CATEGORIZES (categories) to classify items (inputs, activities, standards, outputs) according to the step (or stage or phase) associated with them.
- HANGMAN (single question) for recalling items associated with different steps (or stages or phases)
- TIC TAC with questions related to different steps (or stages or phases)
The best train-the-trainer training I've experienced was done by a company called Langevin Learning Services. I hope you'll find their "tips for instructional designers" to be helpful:
Performance-Based Focus
1. Base the content on the learners’ job tasks.
2. Break tasks down into step-by-step “how to” instructions.
3. Minimize “nice-to-know” information.
4. Target content to the experience level of the learners.
5. Design exercises that simulate the job tasks.
6. Design activities that will help learners transfer the skills learned to their job.
7. Design course materials to be job aids.
8. Build principles of adult learning into the course.
9. Structure the course content according to how the job is performed.
10. Spend about 1/3 of the course time on the presentation of content.
11. Allow about 2/3 of the course time for application (i.e. practice) and feedback.
12. Validate the course with a representative sample of the learner population.
If you're out there building one-shot stand-alone workshops that no one is coming to - perhaps one of the problems is a workshop title that falls flat and doesn't entice participation. I discuss workshop titles and descriptions in Step 1: Needs Assessment (page 27), but found a really good blog entry by my favorite marketing guru, Robbert Middleton, that I think is worth visiting.
Middleton is actually talking about the title of a program he offers, and not a one-shot workshop - but the idea is the same. His key advice is to make sure your title is benefit-oriented, that it addresses the problem and that it gives your learners what they want. Read the whole entry on "The More Clients Blog."
The idea is to use a great title and description to attract learners to your workshops that wouldn't ordinarily come. Give it a try and see if it makes a difference!
One of the major myths library folks (and others of course) carry around is that the librarian with the subject expertise is in the best position to deliver the instruction. This is not always the case. In fact, I think this myth is one of the major problems with library instruction programs – experts often struggle to translate their complex knowledge in ways that non-librarians and novices can understand. I’ve seen this difficulty in translation over and over again during our team design process at the U of MN Libraries.
Regardless of who designs and delivers the instruction session – the expert or a generalist instructional staff - the two tools that can make the most difference are the task analysis (step 6) and the articulation of teaching points (step 7). These tools help the expert/designer to translate a task into its discrete parts and to articulate a perfectly understandable sentence or couple of sentence that communicates a key point. It can mean the difference between a confusing workshop and a clear one.
I go into these steps in 2 separate chapters, but found a very readable article by a school librarian about her discovery and use of a task analysis that I think is worth tracking down. Here’s the citation:
Rankin, Virginia, “Task analysis or, relief for the major discomforts of research assignments.” School Library Journal, Nov92, Vol. 38, Issue 11
Here’s a paragraph from her conclusion to entice you to track this article down:
Task analysis can be helpful to many of us as we try to make our vision of information literacy more concrete. Other things that now seem crucial to me-like the idea of slowing down, and stopping, or going into reverse to make sure one comprehends information-had been absent from my instruction. I now feel I am in a better position to systematically help my students to be better engagers/ interrogators/ interpreters/ processors of information.
Take a look!
Reaction evaluations (often called "smile sheets") are the least informative of the four evaluation types I discuss in Step 9 of Creating the One-Shot Library Workshop. But let's face it - when you're whipping out the one-shots, they're often the only kind there’s time to create. If that’s the case for you, here are some tips for making your smile sheets more valuable:
Get beyond “how’d I do?”
I’ve seen a lot of smile sheets that are all about the instructor. Instructor-centered design is “so yesterday,” – remember? Try focusing the questions on the learner, not the instructor, to get to what needs to be changed to make the learning better. The following point takes this tip a step further:
Probe for the perceived impact
There is research that shows that people’s rating of relevance has a higher correlation with learning than their rating of learning.
Neil Rackman, in an article in Training Magazine has some interesting things to say about this:
“If people are having a good time, they will very often perceive their learning to be more than it actually is. But high enjoyment is not necessarily related to high learning. If you have a trainer who tells 100 war stories and is very entertaining, that instructor can end up getting tremendous ratings for 'perceived learning,' but two hours later, trainees can't remember a single thing that came out of the session. The danger is that we have some great entertainers in the training ranks who may rate very high on enjoyment but are so busy entertaining the class in order to get good ratings that they're delivering pitiful learning."
The take-away? Ask how relevant the learners thought the workshop was by using a scale from “not at all relevant to me and my [job/course/or other]” to “very relevant.” Skip the questions about how much the learners thought they learned.
Get specific
If you use the same smile sheet for all the workshops your library gives watch out for the time when instructors and designers start to get dismissive - or even just plain bored - with the form and start ignoring the results. If this starts to happen, spend a little time customizing the smile sheet for each workshop. Go back to your Needs Assessment (Step 1) and Learner Analysis (Step 2) to see if there are any potential evaluation questions that would be more specific for your particular workshop. This should help reinvigorate you and others to get back to taking your feedback to heart.
Encourage participation
How often do you practically have to beg to get your learners to stay and fill out the evaluation? Probably often. Jim Kirtpatrick suggests that you take some time to tell the learners why the questions where chosen and what you’ve done with past feedback to improve your workshop. Encourage them to be totally honest with you so that you can provide future learners with a far better workshop.
And finally, recognizing that level 1 “smile sheets” are a limited way to get good feedback, you might want to step it up a notch and use this last tip:
Try a mini focus group
If you have the time – and can gather some $$ or other incentives – see if you can entice some of the learners to stay after the workshop to have a short discussion about the session. When I’ve done this in the past I’ve had great luck by going through each module of the workshop, asking what worked and what didn’t work for each. Usually this type of quick-and-dirty evaluation is worth many times the results I get on my smile sheets.
I finally took some time to populate the part of my book site called "Other Resources."
I tried to be reductive and hold back from adding everything that might be useful so this is just a slim "need to know" list. Please take a look and feel free to add other resources in the comments area.