European Survey Research AssociationEuropean Survey Research Association
 
Home About us Membership Conferences Journal Courses Minutes Contact

Login to your account:

Sign up | Reset password

Conferences

Conferences


ESRA2009: Conference main page | Overview of sessions | Time table

Warsaw 2009: Presentations and short courses


Combining qualitative and quantitative methods in survey development: An example from the Life Opportunities Survey

Session: Can multiple question testing and evaluation methods improve survey questions or predict sources of measurement error?

Author:

  • Joanna Bulman; Office for National Statistics, United Kingdom

Abstract:

In Great Britain there has been a growing need for information about the dynamics of disability over time and the factors associated with transitions and onsets of disability. A new survey due to be launched by the Office for National Statistics will monitor the experiences and barriers that disabled people face from early childhood through their life course, how these relate to the type of impairment and how they change over time.

In recognising the conceptual shift from a medical to a social model definition of disability, the survey called for new measures of disability to be developed. These new measures, which focus on how social and environmental barriers interact with impairments to construct participation restrictions, were developed and tested through a combination of quantitative and qualitative methodologies.

The qualitative work began with a deliberative consultation exercise with disabled people, who were meaningfully involved from the start of the development process. A reference network was convened, comprised of 60 people with a range of impairments who took part in focus groups, conferences and online discussions. Findings from this consultation fed directly into the design of the questionnaire, which was qualitatively tested using cognitive interviewing techniques. Key aims of this phase of testing were to explore how respondents reacted to the proposed questions, how they understood them and if they were able and willing to answer them.

The initial version of the questionnaire went through major changes as a direct result of the qualitative testing. This revised version was then tested by means of traditional quantitative testing, however since the questionnaire had been thoroughly cognitively tested, the quantitative phase was less concerned with testing the wording of the questions, and more about reviewing field systems and practices. A feasibility trial was carried out which involved interviews at 118 addresses, purposively sampled to include a high proportion of disabled people. This was followed by a larger scale pilot, which focused on providing an indicative response rate and interview length, as well as a final opportunity to test the questionnaire and field processes.