View Single Post
  #1  
Old 14/03/2006, 04:24 PM
Isha Isha is offline
Junior Member
 
Join Date: Mar 2006
Posts: 5
Default Distortions in Self-Report Data

We all know this is a big problem with testing, so I thought maybe we should collaborate and put our insights and ideas into one place for easy access if anyone wants to develop a new test or improve on an old one, etc.

Here's a list of some common distortions (lists and definitions are incredibly useful things).

Social-desirability bias
- a tendency to give "socially approved" answers
- subjects try to create a favourable impression

Misunderstanding
- respondents misunderstand questions
- the way questions are worded can shape participants' responses

Memory errors
(pretty much self-explanatory)

Tendency to agree
- respondent tends to agree with nearly all items

Tendency to disagree
- respondent tends to disagree with nearly all items

There's probably not a great deal one can do about the last three distortions (although a large scale like rmcnew's might help somewhat for the last two).

Social-desirability bias is likely responsible for the "ENTp bias" that SG illustrated here. Maybe test-developers need to make sure that they don't make any functions sound better than others. (I don't have any answer's, I'm just speculating for the moment).

The distortion I'm particularly interested in is the misunderstanding. How does a test developer minimise misunderstanding? Maybe rather than have 1 statement per item, there should be 2 statements which illustrate the same concept in different wordings? Or maybe pairs of positive and negative statements in each item?

Anyway, ideas are wonderful! This is an area that deserves our focus.
Reply With Quote