Summary of the book Constructing Questions for Interviews and Questionnaires: Theory and practice in social research by William

Foddy Cambridge University Press, 1993 written by Walter Wood, M.Eng. Telecommunications Technology Management 2000
Chap 1 An initial statement of the problem .............................................................................. 3 principal causes of error in the gathering of data through survey procedures: ........................ 3 10 examples of problems illustrating the inadequacy of many of the questions that have been used in social research in the past: ........................................................................................... 3 Chap 2 A theoretical framework............................................................................................... 3 two broad approaches to collecting verbal data: ...................................................................... 3 principal assumptions that have defined the general orientation adopted by survey researchers in the past: ............................................................................................................. 3 4 steps in a successful Q-A sequence (The Q-A Comm Cycle):.......................................... 4 5 sources of response variability:............................................................................................. 4 Chap 3 Defining topics properly............................................................................................... 5 The researcher has clearly defined the required information (Assumption 1)......................... 5 Respondents have the required information (Assumption 2)................................................... 5 Respondents are capable of verbalizing the information the researcher wants under the conditions of the research situation (Assumption 3)................................................................ 5 Chap 4 Formulating intelligible requests for information......................................................... 5 The meanings of individual words........................................................................................... 5 Structural complexities and the requirement that questions should be interpreted as intended ................................................................................................................................................. 6 Chap 5 Contextual influences on respondents interpretations of questions ............................ 6 Clues afforded by either the question itself or its components ................................................ 6 Clues contained in accompanying sets of response options..................................................... 6 The impact of preceding questions (& answers)...................................................................... 6 The impact of previous answers on the respondent ................................................................. 7 The impact of the definition of the overall situation................................................................ 7 Chap 6 The need to provide response frameworks ................................................................... 7 Descriptive accounts ................................................................................................................ 7 Levels of social generality of responses .................................................................................. 7 Explanations............................................................................................................................. 8 Evaluations............................................................................................................................... 8 Chap 7 The limitations of human memory ............................................................................... 9 Problems associated with the recall of information in long-term memory .............................. 9

Problems associated with short-term memory ....................................................................... 10 Chap 8 Filters: establishing the relevance of questions to respondents .................................. 10 What kind of no opinion filter should be used? .................................................................. 10 The positioning of filters........................................................................................................ 11 Should a middle category be offered at all?........................................................................... 11 Chap 9 Reducing question threat ............................................................................................ 11 The definition of question threat............................................................................................ 11 Chap 10 The open vs closed questions debate........................................................................ 12 The principal advantages and disadvantages associated with the two formats...................... 12 An evaluation of the assumptions underlying the use of open questions .............................. 13 An evaluation of the assumptions underlying the use of closed questions ............................ 14 Problems associated with developing response options for closed questions........................ 14 Problems associated with recording responses to closed questions....................................... 14 Problems associated with interpreting responses to closed questions ................................... 14 An evaluation of three additional uses claimed for open questions....................................... 14 Chap 11 Measuring attitudes .................................................................................................. 15 Single rating scales ................................................................................................................ 15 Problems associated with batteries of rating scales (i.e. summated scales)........................... 16 A possible alternative to summated scales............................................................................. 17 Chap 12 Checks to ensure that questions work as intended.................................................... 17 Editting rules.......................................................................................................................... 17 Piloting questions................................................................................................................... 18 Question testing ..................................................................................................................... 18 Chap 13 Concluding comments .............................................................................................. 18 The TAP paradigm for constructing questions .......................................................................... 18 Tap ......................................................................................................................................... 18 Applicability .......................................................................................................................... 18 Perspective ............................................................................................................................. 18

Chap 1 An initial statement of the problem principal causes of error in the gathering of data through survey procedures:
respondents failure to understand questions as intended a lack of effort, or interest, on the part of respondents respondents unwillingness to admit to certain attitudes or behaviours the failure of respondents memory or comprehension processes in the stressed conditions of the interview interviewer failures of various kinds (e.g. the tendency to change wording, failures in presentation procedures and the adoption of faulty recording procedures)

10 examples of problems illustrating the inadequacy of many of the questions that have been used in social research in the past:
1) factual questions sometimes elicit invalid answers 2) the relationship between what respondents say they do and what the actually do is not always very strong 3) respondents attitudes, beliefs, opinions, habits, interests often seem to be extraordinarily unstable 4) small changes in wording sometimes produce major changes in the distribution of responses 5) respondents commonly misinterpret questions 6) answers to earlier questions can affect respondents answers to later questions 7) changing the order in which response options are presented sometimes affects respondents answers 8) respondents answers are sometimes affected by the question format per se 9) respondents often answer questions even when it appears that they know very little about the topic 10) the cultural context in which a question is presented often has an impact on the way respondents interpret and answer questions

Chap 2 A theoretical framework two broad approaches to collecting verbal data:

1) objective, positivistic orientation use of forced choice (closed) questions 2) qualitative field researchers phenomenological or subjectivist point of view non-directive, open questions

principal assumptions that have defined the general orientation adopted by survey researchers in the past:
1) researcher has clearly defined the topic about which information is required 2) respondents have the information that the researcher requires 3) respondents are able to access the required information under the conditions of the research situation 4) respondents can understand each question as the researcher intends it to be understood 5) respondents are willing (or at least can be motivated) to give the required information to the researcher 6) the answers that respondents give to a particular question are more valid if they have not been told why the researcher is asking the question (** refuted below **)

7) the answers that respondents give to a particular question are more valid if the researcher has not suggested them to the respondents 8) the research situation per se does not influence the nature of the answers given by respondents 9) the process of answering questions per se does not change the respondents beliefs, opinions, habits, etc 10) the answers that different repondents give to a particular question can be meaningfully compared with one another

4 steps in a successful Q-A sequence (The Q-A Comm Cycle):

1) the researcher must be clear about the nature of the information required and encode a request for this information 2) the respondent must decode this request in the way the researcher intends it to be decoded 3) the respondent must encode an answer that contains the information the researcher has requested 4) the researcher must decode the answer as the respondent intended it to be decoded

5 sources of response variability:

1) not telling respondents the reason for asking a particular question (causing respondents to frame their own context)

2) because any topic is multidimensional, any topic can be defined in terms of either a single dimension or a combination of dimensions 3) utterances (including Qs and As) can be made on different levels of generality 4) utterances about a topic can also be made on a number of theoretical levels 5) utterances are always framed within a descriptive, explanatory or evaluative frame of reference

Chap 3 Defining topics properly The researcher has clearly defined the required information (Assumption 1)
1) the researcher has clearly defined the topic at least 3 ways the multidimensional nature of any topic bears on Q-A behaviour: a) respondents answers in terms of specific dimensions will not always be congruent with their global judgements b) when respondents are required to answer in terms of specific dimensions, the dimensions that they are to focus on must be properly defined c) how a topic is defined can dramatically affect the way the responses are distributed 2) the researcher has a clear idea of the kind of information that is required about the topic 4 rules to ensure that the info collected satisfies the reason for conducting the research a) do not formulate specific questions until you have thought through the research question b) write the research question down and keep it handy while formulating specific questions c) keep asking why do I want to know this? d) prepare dummy tables showing the relationships that are anticipated write the avowed purpose beside each question in the first drafts of questionnaires or interview schedules (i.e. to write beside each question what information they think it will elict and what they will do with that information)

Respondents have the required information (Assumption 2)

check relevance of questions to respondents (dont assume respondents have the information sought) use filter questions

Respondents are capable of verbalizing the information the researcher wants under the conditions of the research situation (Assumption 3)
common pitfalls: proceeding too fast; failure to stimulate recall; questions too complex; questions too long

Chap 4 Formulating intelligible requests for information The meanings of individual words
4 factors in failure of respondents to give the same meaning to individual words: 1) the evolution of context-specific nuances of meaning cultural context, use of slang, choice of words to stress 2) their relative difficulty (e.g. number of syllables, frequency in common use) keep words/questions simple 3) lack of clear empirical referents 5

abstract words lacking clear empirical referents: frequency terms regularly, usually, often, frequently, seldom, rarely evaluative terms good, bad, approve, disapprove, agree, disagree, like, dislike the words used by the researcher when formulating questions should be as specific and as concrete as possible defined in empirical terms: frequency questions should be worded in terms of numeric referents per specified time period clarify concepts with short vignettes (word pictures) 4) the operation of related nuances of apparently similar words e.g. negative response options are perceived as being stronger than the countervailing positive response options

Structural complexities and the requirement that questions should be interpreted as intended
sheer number of words used more words = greater likelihood that the question will be interpreted incorrectly grammatical complexities avoid asking two or more questions at once avoid adding qualifying clauses, phrases, and instructions avoid the use of negatives

Chap 5 Contextual influences on respondents interpretations of questions

4 kinds of clues that respondents can use to interpret questions: 1) the content of the question itself 2) the accompanying sets of response options 3) preceding questions 4) attempts to take into account the researchers purpose when answering

Clues afforded by either the question itself or its components

clues associated with the question per se clues provided by components of the question

Clues contained in accompanying sets of response options

failure to give equal weight to all response options ordering of options prompting answers incompleteness of response arrays imbalance of options other ways to influence answers: pre-set response options as a source of memory cues information contained in the range covered by the options response biases associated with the number of options

The impact of preceding questions (& answers)

1) influences associated with prior specific questions a) most general question first, then increase specificity:

quintamensional plan: i. a general question to establish whether respondents have the required information concerning the topic ii. an open question to get at respondents general perceptions or feelings abut the topic iii. a dichotomous question to elicit perceptions or feelings about a specific aspect of the topic iv. an open question to get at reasons for responses toward the aspect of the topic specified in step 3 v. a rating question to allow respondents to indicate the strength of their responses towatd the aspect of the topic specified in step 3 the funnel sequence motivating full communication

b) priming / anti-priming (or discounting) sequence 2) the psychological need to be consistent convergence of attitude concerning a particular topic 3) even-handedness

The impact of previous answers on the respondent

convergence of attitude concerning a particular topic

The impact of the definition of the overall situation

it makes more sense to tell respondents about: the overall purpose of a study and the specific purpose of each question respondents willingness to impart information goal convergence? must play the role of respondent (vice some other agenda)

Chap 6 The need to provide response frameworks

answers are always formulated within response frameworks different respondents different perspectives different answers same respondent can consider different perspectives different answers

Descriptive accounts
respondents can look at any situation and describe it in many different ways if respondents adopt different perspectives when answering a question, it can be argued that they are not answering the same question; if this happens, the answers that they give cannot be meaningfully compared the researcher can assume the responsibility of clearly specifying the perspective that respondents should have in mind

Levels of social generality of responses

the perspectives respondents adopt can vary according to their level of social generality own point of view or a broader group or community point of view

the word you is prone to being interpreted collectively when used in conjunction with behaviour which the respondent does with others other problematic/ambiguous pronouns: one & who the researcher must be clear/specific about the level of social generality

Explanations
respondents can give any number of explanations for the same behaviour 5 different kinds of answers to the question: why did you do X? (a non-exhaustive list) 1) causal antecedents what caused the respondent to do X? 2) goal antecedents the respondents purpose for doing X? 3) enablement factors how it was possible for the respondent to do X? 4) causal consequences what happened after the respondent had done X? 5) the researchers expectations why the respondent had done X when the researcher had expected the respondent to do Y?

Evaluations
evaluative judgements are always relative; they are never absolute statements; evaluations always involve comparisons with standards

1) evaluative standards that are external to the question personal answers are likely to be affected by reference group effects when: they have been evoked by previous questions the researcher has neglected to clearly specify the standards that should be used reference group effect dead give-away words (all, always, each, every, never, nobody, only, none, etc) the problem with dead give-away words is that people learn that there are exceptions to most rules and consequently avoid selecting answers that include them 2) evaluative standards that are internal to the question usually either implicit or poorly defined the common rating scale epitomises the failure to use properly defined standards of comparison in social research lack consensual meaning simply because they lack empirical referents the implied evaluative standards of many rating tasks are often biased or imbalanced

Chap 7 The limitations of human memory

respondents are often not aware of many of the immediate influences on their behaviour sensible to ask people only about their past intentional behaviour

Problems associated with the recall of information in long-term memory

3 dimensions appear to define the salience of events: 1) unusualness (e.g. one off purchases, illness, etc) 2) associated high economic and social costs or rewards (e.g. buying a house) 3) continuing consequences (e.g. a resulting disability) forgetting is related to elapsed time, salience, and the number of events that compete woth the particular event(s) that the respondent is asked to remember generally, memory for salient events has been found to be satisfactory for up to one year; for low salience events, periods of less than one month appear to be more appropriate for many kinds of behaviour the optimum time period is between two weeks and a month respondents recall pleasant events more easily than negative ones

researchers should get as familiar as possible with the events that they are investigating so that they can refer to key events in the same terms that the respoondents would have used when they were encoded get respondents to work in chronological order either forwards or backwards (usually better to have them work backwards because of recency effects) the researcher should ask a series of cross-cutting questions to provide multiple frames of reference or recall cues 3 ways to encourage more accurate recall: 1) respondents explicitly instructed to recall accurately and told to take their time in answering 2) respondents given encouragement 3) get an explicit agreement to respond accurately and completely

Problems associated with short-term memory

if questions are to be comprehended properly they must be encompassed by the span of the respondents short term memory respondents better assimilate structurally simple questions when they are able to listen to them when questions are complex in structure, it appears that it is better if respondents can read, and perhaps reread, them for themselves helpful hint: let respondents read a second copy of the questionnaire while the interviewer reads the questions aloud (for even better recall?)

Chap 8 Filters: establishing the relevance of questions to respondents

respondents typically do their best to answer every question put to them even questions that they have difficulty understanding or relating to filter a question / question component either to establish the relevance of the question or to emphasize that it is acceptable for the respondent to not answer the question wide variety: dont know, cant recall, no opinion, undecided, not sure, it depends, neutral, or sets of questions to establish extent to which an opinion or belief is based on experience of or knowledge about the topic non-attitudes essentially random responses which respondents had arrived at hastily to satisfy the demands of the interview situation typically up to 20% of respondents will give a non-substantive response if they are allowed to but give a substantive response if a non-substantive option is not offered the provision of a filter can completely change the distribution of responses to a question typically between one and two-thirds of the respondents who endorse an undecided option are really ambivalent, while most of the remainder have no opinion it appears that many respondents give substantive answers, if they have to, that are not directly based on experience the sensible course of action is to include two filters: a no opinion filter plus an Im not sure or it all depends filter

What kind of no opinion filter should be used?

a filter which asks respondents how interested they are in an issue, or how much they have thought or read about it, will generally screen out more people than one which asks simply whether they have an opinion

10

The positioning of filters

it appears that respondents used the position of the non-substantive option as a clue to its interpretation the conservative decision would be to always place the middle option (e.g. or continued at its present level) at the end

Should a middle category be offered at all?

either a sureness measure or an importance measure is to be preferred over an intensity measure

Chap 9 Reducing question threat

social researchers tend to take it for granted that respondents are, or can be, motivated to give them the information they require

The definition of question threat

areas in which research is more likely to be threatening than others include questions that: intrude into private spheres and personal experiences concern deviance and social control raise fears of identification impinge on the vested interests of the powerful raise the fear of coercion or domination concern sacred issues which respondents do not want to profane

1) question threat generated by factors that are idiosyncratic to particular respondents might generate purely personal fears or feelings of guilt 2) characteristics of the questions per se that threaten respondents difficult vocabulary excessive question length grammatical complexity difficult instructions key words that have negative stereotypic or cultural connotations 3) threat associated with the nature of the relationship which exists between the researcher and the respondent a) fears of negative sanctions being either: socially rejected or thought less of by the researcher respondents will be under psychological pressure to give answers that they think are normatively acceptable to the interviewer when they perceive the interviewer to be socially similar to themselves strategies to reduce fears of social rejection: increase the psychological distance between the interviewers and respondents ensuring the respondents answers are anonymous ensuring that the questions do not contain clues about what the researcher thinks is normal b) materially sanctioned or physically punished in some way

11

inteviewers who have higher social status than their respondents can pose either a political or an economic threat for the respondents strategies: procedures that maximise trust on the part of the respondents anything that researchers can do to indicate a lack of naivety on their part may help to override the effects of the respondents fears stressing the fact that the respondents views are too important to be incorrectly recorded

4) a complicating fourth factor: the respondents definition of the situation any topic might generate two or even all three types of threat the researcher can decide whether to concentrate on the problem of minimising the threat that is most likely to be dominant in the situation or employ a number of threat-reducing strategies at the same time Types of questions Questions that concern normative issues, e.g.: matters of hygiene, social morality, social responsibility, aesthetic judgement, group loyalty Questions that concern either political or economic interests Types of threat generated Fear of being socially rejected by the interviewer Appropriate threat-reducing strategies allow respondents to answer anonymously decrease the likelihood that respondents will see the interviewer as a social peer use inteviewing procedures that increase either the physical or psychological distance between the respondents and the inteviewer increase respondents level of trust stress confidentiality establish lack of inteviewer gullibility emphasize social significance of respondents answers

Fear that the interviewer will impose either economic or political sanctions on the respondent

Chap 10 The open vs closed questions debate The principal advantages and disadvantages associated with the two formats
Open questions a) allow respondents to express themselves in their own words b) do not suggest answers indicate respondents level of information indicate what is salient in the respondents mind indicate strength of respondents feelings c) avoid format effects d) allow complex motivational influences and frames of reference to be identified Closed questions a) allow respondents to answer the same question so that answers can be meaningfully compared b) produce less variable answers c) present a recognition, as opposed to a recall, task to respondents and for this reason respondents find them much easier to answer d) produce answers that are much easier to computerize and analyze

12

e) are a necessary prerequisite for the proper development of sets of response options for closed questions f) aid in the interpretation of deviant responses to closed questions * Note: there are problems associated with each of these as pointed out in the following sections

An evaluation of the assumptions underlying the use of open questions

1) open questions do not suggest answers to respondents not necessarily valid 2) answers to open questions indicate respondents levels of knowledge about the topic but, respondents can forget appropriate answers in the heat of the moment respondents dont necessarily mention the things that are most important to them first people often repress, or refuse to disclose, psychologically or socially sensitive concerns 3) answers to open questions indicate the salience of the topic in the respondents minds but, little reason to believe that the first items mentioned by respondents in response to an open question ate among the most important to them some issues may be so threatening that respondents either repress them or avoid talking about them altogether other issues may be so salient that respondents do not bother to mention them simply because thay think that they are too obvious to mention respondents may also either mention or not mention items because the test situation either accentuates or does not accentuate them respondents answers to an open question indicate the strength of their feelings about the topic little, if any, published evidence to substantiate this

4) open questions avoid the format effects that have been associated with closed questions little basis for this assertion 5) answers to open questions allow complex motivational influences and frames of reference to be identified an answer to an open question indicates the way in which a respondent has intepreted it good reasons for rejecting this hypothesis an answer to an open question indicates the motivation(s) that have influenced the respondents orientation to the topic good reasons for rejecting this hypothesis the answer to an open question indicates the frame of reference used by the respondent theoretically nonsensical 6) problems associated with probing inadequate answers to open questions a) interviewers should clearly understand the researchersgoals so that they can probe respondents minds to clarify obscure answers ( crucial that probes be non-directive) b) interviewers should know the required sorts of answers c) interviewers should attempt to get adequate answers 13

7) problems associated with coding responses to open questions a question should be put to respondents in such a way that it elicits complete answers of the kind required in the absence of clear guidelines about what sorts of answers are required, respondents can, and will, focus upon such a variety of aspects of the topic that their answers are neither comparable nor codable

An evaluation of the assumptions underlying the use of closed questions

1) because all respondents answer the question in the same way, the answers can be meaningfully compared not unless the researcher has gone a long way towards ensuring that respondents are able to adopt the same perspective 2) respondents find closed questions easier to answer easily supported 3) answers to closed questions are more easily analyzed almost true by definition, but no guarantee that this is always the case

Problems associated with developing response options for closed questions

the alternatives presented should be properly developed through pilot work with open questions the widespread practice of including an other category does not seem to suffice all too easy to formulate inappropriate categories for use with closed questions also very easy to fail to properly list a complete set of appropriate response options there is a greater tendency for respondents to endorse a response option when it is presented by itself in a yes-no format than when it is presented with fully explicated contrasting alternatives

Problems associated with recording responses to closed questions

there are two kinds of mistakes that interviewers can make: 1) wrongly record answers 2) neglect to read all of the response options can program interviewers with standardized explanations to teach respondents to use the response options that are offered the problem with instructions like these is that they clearly pressure respondents into answering in terms of the alternatives, whether or not these are appropriate for them

Problems associated with interpreting responses to closed questions

respondents must have a clear understanding of what the question is about and are told what perspective to adopt when framing an answer

An evaluation of three additional uses claimed for open questions

1) open questions may be more motivating for respondents as long as the meaning of an open question is not obscure that is, if the respondents are told what sorts of answers they should give

14

2) open questions are useful when a set of meaningful alternatives is too large and complex to present to respondents 3) it may be necessary to use open questions when the researcher has reason to suspect that rapidly shifting external events will affect answers

Chap 11 Measuring attitudes

working with non-dichotomous variables is a step toward greater precision which in turn allows the formulation and testing of more complex hypotheses using sophisticated statistical procedures a great many question devices have been invented to measure respondents attitudes but, there has been little work directed toward comparing the relative strengths and weaknesses of the different procedures

Single rating scales

allow respondents to indicate the strength of their attitude toward a specified topic should satisfy three basic considerations: 1) the topic of focus should be clearly defined there has been a persistent tendency to confuse extremity of judgements with the importance of topics for respondents and with the certainty or sureness of their responses explicitly instruct respondents as to what kind of responses they should make that is, either to ask respondents to indicate, for example, how strongly they either agree or disagree with each statement or to ask them to indicate how much approvalor-disapproval each statement implies 2) the relevance (applicability) of the topic to the respondents should be established 3) respondents should all give the same kinds of answers respondents need to know what sort of an answer they should give the level of generality of responses an initial instruction should be given to increase the probability that respondents will give a personal answer to each item the specification of the standards of comparison that should be used the meaning of the ordered response options that constitute the heart of any attitudinal rating scale must be shared if a researcher is to be able to properly interpret and compare respondentsanswers the strength of the negative pole generally far outweighs the strength of the positive pole the consequence of the failure to properly specify comparison standards format effects a) stimulus centred effects the number of categories that should be included in rating scales two to five categories is not enough use at least seven the anchoring effects of category labels responses to rating scales are usually biased toward the largest numeric anchors and toward the most positive verbal anchors

15

respondents are less influenced by intermediate or middle labels than they are by the labels in the end positions labelling of all categories produces data of poorer quality than labelling only the end categories; if poorly defined categories generate confusions, the more poorly defined categories there are, the greater the number of confusions that will occur

b) respondent centred effects rating scales suffer from a number of respondent-centred biases, such as the tendency for some respondents to choose central response options rather that extreme ones, to agree rather than to disagree, and to be positive rather than negative respondent-centred response biases are inversely dependent upon the degree to which the topic and the categories have been properly defined, and directly influenced by the number of categories

Problems associated with batteries of rating scales (i.e. summated scales)

Suggestions for improving summated (Likert) rating scales a) An appropriate filter should be appended to establish the relevance of each item to the respondent b) The rating scale should: 1. include,at least, 6 substantive rating categories not counting middle and filter categories 2. either: include labels for the end categories and include, as middle categories, both a neutral category and either an undecided or an ambivalent category: neutral ____ ____ strongly agree ____ ____ ____ undecided or ambivalent ____ ____ ____ strongly disagree either: dont know ____ havent had a chance to form an opinion about this topic ____

or, include labelled end categories and an appended measure either of the importance of the topic or of the respondents sureness as a substitute for the middle categories

3. be attached to statements that are of the same level of generality as the behaviour that is to be predicted c) Make responses meaningfully comparable by: 1. explicitly instructing respondents that they are to make respondent centred responses 2. allowing respondents to look over the whole range of items before responding to each one

16

A possible alternative to summated scales

magnitude scaling (described by Foddy pp. 171-179)

Chap 12 Checks to ensure that questions work as intended

when a respondent finds it difficult to answer a question, he is likely to modify it in such a way as to be able to answer it more easily questions that are particularly vulnerable to modification are questions that: a) call for mental calculations b) do not apply to the respondent c) are very general questions d) are questions with conflicting parts which are prone to being partially answered besides the traditional use of editing rules and piloting procedures, there are a number of courses of action that the researcher can adopt these include: a) having respondents rephrase questions in their own words b) directly questioning respondents about their interpretations of key words c) having respondents think aloud as they formulate answers

Editting rules
standard methodological advice in the past has been to formulate questions in accordance with a list of principles of good practice which have been codified from past experience, and then to pilot or trial these questions on a small sample of respondents (i.e. twenty to thirty) drawn from the target population

Editting rules to aid the formulation of questions a) Make sure that the topic has been clearly defined. b) Be clear both about the information that is required about the topic and the reason for wanting this information. c) Make sure that the topic has been defined properly for the respondents by: avoiding the use of 'blab' words (i.e. words that are so abstract or general that they lack specific empirical referents) avoiding words that are unlikely to be understood by all respondents either because they are rarely used in everyday life, or are specialist (i.e. jargon) words d) Make sure that the question is relevant to respondents by: using an appropriate filter avoiding asking for information respondents are likely to have forgotten avoiding hypothetical issues e) Make sure that the question is not biased by: ensuring balance in the introduction to the question (e.g. some people like X, and some people dislike X. - Do you like X or dislike X?) ensuring that sets of response options are complete ensuring that sets of response options are balanced avoiding using words that are likely to invoke stereotypical reactions

17

f) Eliminate complexities that prevent respondents from easily assimilating the meaning of the question by: avoiding asking 2 or more questions at once avoiding the use of words that have several meanings checking whether the question has been worded as simply as possible avoiding the use of too many 'meaningful' words in the one question avoiding the use of qualifying clauses and phrases and the addition of complicating instructions which cause respondents to start to answer, before they have been exposed to the whole question if qualifying clauses and phrases have to be used, they should be placed at the beginning rather than at the end of the question making sure that the question is as short as possible avoiding the use of both negatives and double negatives g) Ensure that respondents understand what kind of answer is required by: setting the question in context informing respondents why the question is being asked informing respondents what will be done with the information they give specifying the perspective that respondents should adopt

Piloting questions
more useful for uncovering aspects of questions that will cause interviewers to have difficulty than for discovering whether or not the respondents interpret the questions as intended interviewers should work in pairs so that one can concentrate fully on conducting the interview while the other is free to listen to and observe both the interviewers and the respondents behaviour

Question testing
three techniques (these usually limit attention to the in-depth testing of no more than six or seven questions at a time): 1) asking respondents to rephrase questions in their own words 2) the double interview 3) asking respondents to think aloud as they answer each question

Chap 13 Concluding comments The TAP paradigm for constructing questions Tap
properly define questions so that each respondent clearly understands what is being talked about

Applicability
establish the applicability of the question to each respondent to ensure they have the information being asked

Perspective
specify the perspective that respondents should adopt when answering the question so that each respondent gives the same kind of answer 18