« Minneapolis Institute of Art - Grand Reopening Today | Main | More Blog-Mediated Serendipity »
June 13, 2006
What's Your Number?
I had a conversation with a graduate student last week during which I found myself constructing a scale of expertise in quantitative methods. I've thought about it some more and think it has some interesting ramifications for how we teach methods and statistics, how we train graduate students to be prepared for the job market, and how we select consultants for grants, all of which I do. I'm sharing it here as a work in progress and would be interested in comments and refinements.
In the quantitative spirit, I think of this type of expertise on a quasi-interval scale (more than ordinal but less than interval) from 0 - 5.
A person rated as a 5 on this scale is a methodological and/or statistical innovator. He or she thoroughly understands the math and statistics behind the computer programs and may indeed develop new methods and techniques for solving problems. He or she may also write software to make these techniques available. Here, I'm thinking of a person like Bengt Muthen from UCLA who is a statistician par excellence, develops computer software to make the statistics available and also understands the substantive needs in the field. Or Dave Kenny, who for years has pioneered in developing techniques for analyzing data at the level of the couple and the family.
A person rated as a 4 has strong statistical and methodological skills, but isn't involved in developing new methods or approaches. This person's interests may be more methodological than substantive. (I am not making a value judgment about which is better, since both are essential to progress in the field.) This person may regularly read journals like Psychological Methods or may contribute to special issues of journals that focus on methodology. This is the kind of person who can make strong methodological contributions to a research team as a stats consultant.
A person rated as a 3 has strong understanding of statistics and methods, but is more comfortable with techniques that are tried and true - he/she isn't innovating and isn't choosing to stretch by constantly learning new techniques. However, this person's knowledge is solid and he/she understands key issues and controversies in the field (e.g., data imputation, advantages of latent variable techniques vs. more traditional methods, issues involved in working with couple and/or family-level data, etc.) This person can write syntax for programs such as SPSS, SAS, and STATA and understands what the software does behind the "clicky-boxes." His/her interests are probably more about the substantive issues in the field than about the methodological ones; the methods are a means to an end.
A person rated as a 2 has some understanding of statistics, but generally feels that they are a "black box" - in other words, how they work is mostly a mystery. There is an emerging understanding of how the different statistical methods are related to each other. He/she may be comfortable using drop-down menus to generate analyses in SPSS, for example, but may not be able to generate the syntax that would correspond to the analyses. This person may be quite comfortable with a very limited range of approaches. Once out of his/her comfort zone, this person may feel quite insecure. The substantive issues in his/her field are the primary interest.
A person rated as a 1 on this scale may be able to generate an analysis but probably doesn't understand what the computer software is actually doing and is vulnerable to making mistakes in terms of assumptions, input, and data interpretation. When asked to explain basic statistical concepts, there may be some major points of confusion.
A person rated as a 0 on this scale has a layperson's understanding of statistics and methodology - no specialized training in these fields.
How might this quasi-interval scale apply to the preparation of graduate students? My assumption is that most students entering a masters program in family science have had some undergraduate preparation in methods and statistics, even though they may have learned things by rote and don't remember much. They would probably be at the 1 to 1.5 level. The goal for master's training would be approximately 2.3, where students are definitely comfortable with a range (albeit limited) of techniques. The goal of doctoral training would be to move them as close to a "3" as possible, and possibly beyond. My assumption is that entry level Assistant Professor positions in Research I institutions would be looking for about a 3.5 in terms of expertise. An ideal department would have several faculty comfortably at the 4 level, and a very fortunate department would have someone at the 5 level either in the department itself or psychologically nearby.
So what's your number?
Apart from your self-evaluation, I'd be interested in your thoughts about this scale. I will be teaching the master's level quantitative methods course again next spring and may use this in the class to give students a sense of the range of expertise in the field and help them identify their personal goals for developing quantitative expertise.
Posted by hgroteva at June 13, 2006 6:01 PM | Social Science | Teaching
Comments
I'm #1 I'm #1
OK, that is not such a flattering chant on this scale and I'm probably #2. Being one who will take that course, I find this construct very useful. Saying "stats" is kind of like saying, "stuff." It's way too big. Providing some sense of self-measure would, I think, provide benchmarks whereby a sense of accomplishment might be felt.
Some people need that and some don't, but the sense I get from the people I took my only stats course with was that 9 of 10 were afraid of the course, the topic, and the field in general.
From another perspective, some kind of standardized measure would perhaps provide some kind of quality check when utilizing a stats consultant. "Hi, I'm Chris Gonzalez M.MFT, LMFT Stat 3.5"
Credentialling this way might be going too far, but it might be useful.
So far as I am concerned, anything to help me wrap my mind around the number, the methods, and utility is helpful.
Posted by: Fajtia at June 13, 2006 8:14 PM
Chris, Thanks for the feedback. I'm glad it was useful. And hey - #1 can be a good thing!! I look forward to having you in the class.
Posted by: Hal at June 13, 2006 9:08 PM
I am a doctoral candidate in Geography, and unfortunately I'd rate myself a 0.5. Our program focuses more on qualitative methods in our required so-called methods course, Geog 8002. I have just been wondering if my education is really complete without some understanding of statistics, and was thinking that it would be beneficial to sit in on a course.
Interesting scale you've developed.
Posted by: Sno Cones at June 15, 2006 10:56 AM
Great scale! I was wondering where "being a good teacher" might fit in your scheme. It would seem, for example, that many people who are stellar #5s may be awful at communicating/explaining to others.
For example, in many stats classes I have taken over the years, I have learned more from grad student TAs (who might rate, say 3 or 4) than from star innovators who'd be high 4s and 5s...
Posted by: Yvette at June 15, 2006 1:14 PM