October 5, 2004

Quantitative Methods

These are the small group discussion notes for the QUANTITATIVE METHODS PANEL

Table A:
• People say what they think a researcher wants them to say.... perceptions can be different than behavior. It is important to perform several types of research.
• Both papers used a combination of studies of perceptions and behavior in quantitative ways.
• The user control-perception study was experimental.. In an experiment, the researcher has control over everything, and the methodologies are probably clear at the start. This might not be the case outside of experimental research. It seems here that the subject reporting of perceptions is more accurate than it was in the cell phone research. What are some of the reasons for this?
• Design could also be a factor for this type of experimental research.
• People with experience have a better intuitive sense of how things work. The inexperienced user may not know what to do.... this has an impact on design. How do you divide the two groups? What type of object could you present to them? People could have a perception of control, but does that mean they are able to master the technologies?
• Phone study... an example of behavior and what they say they do. So, again it’s important to do both types of methods. What do you do when they are contradictory? The cell phone studies... the real time data... the automated measures seem to be very valid, and tell the story in a very convincing manner. What was the use of the self reported uses?

Table B: • Anxiety - vcr metaphor, bring baggage to online contexts. How is a vcr interface in between? tv, vcr, vcr plus .what is more taxing?
• Unresolved questions - philosophical context. Brian is interested in the link to memory and retention & extension to online content. Brian talked of plans to look at science news project - large scale survey work. Interested in the overlay with qualitative methods to access some of the users´ perceptions to facilitate our wider understandings of user control. Is one variable the construction of the content? This taps into Brian’s interest in multi-level research.. This leads to challenges of exploring the relationship between user comprehension and the nature of their experience.
• The web diversifies user ‘reading’ experience.
• Some uncertainties about the application of such a multi-level model. The relationship between type´ of content, its construction and the individual is significant? Laboratory methods and environment are limited.
• Paper 1 - reminder of the range of information available from the diversity of new media. Do we take advantage of this wealth of data?
• Discussions about instant flow data collection. The subject of Akiba´s paper was fascinating - how do people respond under such conditions and the locality based issues. Method is innovative but what is its potential and its application to other technologies, eg. vehicle tracking devices. He had privileged access to normally ‘hidden’ data. How can those without baby-sitting connections access corporate information vaults.
• Some services better at using their data analytically than others (eg. Amazon.) Building on existing credibility helps and businesses need to see benefit for themselves. Many companies don´t want their data to be public. Are there business sector differences in willingness to share data?

Table C:
• Southwell paper: Alternative methods (brain activity sensing of control perception). Wiring people up - how scary can it be? Can people recall their own experiences? Are people self - aware?
• Psychology/ Management studies: Why do people think that (psychology) students are representative for the human race? Use talk aloud method to complement it? or interview on the basis of a videotaping of the activity(so why did you really do this?)
• Are all people good judges of their own behaviour? - examples of scientists not being aware of using email behaviour
• Cohen paper: segmentation of geographical areas was a good idea/ how did it come out? Company didn´t really get much out of it. Info gods out of control room. First study was more informative for them. Using Mobile from home is a trend in other countries as well.
• Companies giving access to scientists through connections - small countries are a good example - Should we start doing some baby sitting? Good news for Mobile companies - are we rejecting fixed line?

Table D
• Access is clearly a function of relationship and opportunity
• There was an expressed lack of trust in numerical data, does it require a FAITH statement?
• The combination of approaches seems to be a primary explanation of the positive responses in the unpresented paper. The combined approach brings the possibility of making online, email or web, work better. Is this truly positive for online or for integrated approaches?
• Need to be sure to have a function-purpose for the respondents in this type of experiment
• Levels of incentive might change the outcome as higher stakes mean different needs and desires
• How important are the alternatives in control to the task of the user and their level of comfort or their perceived usefulness or their use of the technology or their enjoyment
• As the technologies and studies become more complex how can we account or allow for emergent rather than specified uses?

Table E:
• Sophisticated data gathering requires the collaboration of the companies: can this limit or bias researcher's work?
• Access to technology is key to raw data analysis. If you don't have the tech, you can't do nothing!
• Comparing raw data and personal explanations (interviews, journals) can add an interesting research question: why don't they do the same as they say they do? The answer would be in self-perceptions. You may focus interviews on attitudes if you have the raw data (e.g. number of calls). It's worthy to be transparent towards subjects (if you are interviewing people after retrieving real time data about them, show them the data). Lab experiments take too little a part of the real situation. I may be taking you to misleading results, out of context. Lab products should resemble real situations as closely as possible.
• We do not see comparability among TV and multimedia products in Southwell's research. It is more than a matter of user control, they are different media. Why not have 3 levels of control in multimedia products? Where users overwhelmed or just lacking experience?

Table F:
1. Issues and questions about nonlinear relationships are important.
2. Do researchers devote enough attention to the assumptions of the paradigms employed?
3. Wonder if Southwell, et al. study was a realistic operationalization of user control as experienced on the web? Wonder about the relationship of the finding to the linear narratives of typical video documentaries.
4. User experiences with linear media formats raise important questions about reactions to user control, user satisfaction with choice controls.
5. Cohen study is important new work -- explorations of the relationship between actual and perceived uses and utility need to continue.

Posted by npaul at October 5, 2004 4:33 PM