A key stage to writing your PhD proposal, after coming up with your main research questions and hypotheses, is deciding how exactly you’re going to measure these factors. It may seem like the obvious choice is to pick the most widely used, validated questionnaire, but it’s important to spend some time considering whether that questionnaire will be relevant to your population, able to capture the unique experiences of your patient-group, and whether it fits in with the wider theory. Using a pre-validated generic questionnaire can save time but may lead to important idiosyncratic experiences of your target population being missed.
In this article, we briefly describe how we modified the Revised Illness Perception Questionnaire (IPQ-R; Moss-Morris et al., 2002) in two different patient populations: 1) patients with atrial fibrillation (AF), an irregular heart rhythm predisposing patients to a five-fold increased risk of stroke (Taylor, O’Neill, Hughes & Moss-Morris, 2017) and breast-cancer survivors (BCS; Moon, Moss-Morris, Hunter & Hughes, 2017), who have completed active treatment for breast cancer but who may require continued therapy and monitoring.
A brief background of the questionnaire
The IPQ-R measures patients’ cognitive representations of illness (illness representations) which are developed based on abstract (disease labels) and concrete (symptom-based) information sources. Illness representations consist of a number of different components including: identity (symptoms association with the illness), causes, consequences, timeline (acute, chronic or cyclic), controllability (treatment control/cure and personal control of illness) and illness coherence (whether the illness makes sense) (Moss-Morris et al., 2002). Emotional representations about illness (e.g. fear) are processed alongside these cognitive representations (Moss-Morris et al., 2002). Cognitive and emotional representations of illness can be understood within the context of the Common Sense Model (CSM; Leventhal, Meyer & Nerenz, 1980) which proposes that patients’ cognitive and emotional representations of illness guide coping behaviours, quality of life (QoL) and clinical outcomes, as supported by a wide range of research (Hagger, Koch, Chatzisarantis & Orbell, 2017). Whilst the scale was developed as a generic measure to assess illness representations across conditions, the authors recommend that it is modified to suit the specific needs of each illness population (Moss-Morris et al., 2002).
Step 1: Qualitative interviews
To understand the target population, we carried out interviews with our patient groups. Questions specifically related to components of the IPQ-R, outlined above. For instance, the following question related to the personal control component; ‘Is there anything you can do to control/prevent the risk of recurrence?’. Transcripts were analysed using deductive thematic analysis and key themes were identified which informed the modification of the IPQ-R.
In the AF study 30 participants were interviewed with a range of demographic characteristics including different treatment types and pre/post treatment status. Key themes included unpredictability of AF and a struggle to gain control of symptoms. Patients reported engaging in targeted behaviours such as avoidance to try to control symptoms. Patients also believed certain events or behaviours could trigger AF symptoms. Patients expressed different concerns relating to treatment-type. (For separate qualitative study see Taylor, O’Neill, Hughes, Carroll & Moss-Morris, 2017).
In BCS, 18 women prescribed tamoxifen were interviewed about their perceptions of cancer and suvivorship. A key theme was that the majority of patients did not identify as currently having breast cancer. Instead, when asked about control, consequences and causes, patients tended to discuss their risk of recurrence. Patients attributed symptoms to tamoxifen, an adjuvant treatment, rather than to their breast cancer. Specific causes of recurrence and symptoms were also elicited during the interviews.
Step 2: Modification of questionnaire
Interviews led to questionnaire modification through 1) retention of items, 2) minor revisions 3) development of new items.
Examples of minor revisions included wording changes such as replacing ‘my illness’ with ‘my AF/BCS’ and inclusion of population-specific symptoms onto the identity scale. For instance, in AF patients this included items such as heart palpitations and in BCS this included items such as hot flushes and night sweats.
Development of new items in AF patients related to the unpredictability theme. Three new items related to personal control behaviours (slowing down and avoidance): ‘Avoiding certain activities will control my AF’, ‘Resting will prevent me from having symptoms’, ‘By doing less and slowing down I can control whether I have AF symptoms’. In addition, AF patients seldom mentioned the original cause of AF but reported triggering AF symptoms instead. This led to changing the causes scale into a triggers scale to reflect factors which patients believed triggered AF. Due to different treatments, the treatment control component was re-worded to relate to pharmacological (antiarrhythmic and anticoagulant) and procedural (cardioversion, catheter ablation and AV-node ablation) treatments.
The modification of items in BCS focussed mainly on replacing references to ‘my breast cancer’ with ‘risk of recurrence’. For example, treatment control items were modified to assess the extent to which patients felt their treatment could reduce their risk of recurrence. The timeline scales were amended to reflect the fact that patients did not have symptoms which come and go, and instead are at increased risk of a recurrence. The identity scale was modified to identify symptoms attributed to tamoxifen as well as breast cancer.
Step 3: Think-aloud
Think aloud techniques enable the researcher to establish whether items on the questionnaire are interpreted as intended (i.e. face validity). In our studies, a small subset of patients were given the modified questionnaire and asked to read each item aloud during telephone interviews and to verbalise their thought process on how they would answer questions (Ericsson & Simon, 1998).
Changes were made in both studies to improve the clarity of questions. For instance, in BCS items were revised to improve applicability of questions to all participants. Some items were deleted where possible to reduce repetitiveness. In AF patients, some items were expanded upon to provide further context and improve interpretation.
Step 4: Factor analysis
Confirmatory factor analysis (CFA) is used when testing a hypothesised model and to ensure the original factor structure is still relevant for the modified questionnaire. Exploratory factor analysis (EFA) may be used when more significant changes have been made to the questionnaire and when the factor-structure is not pre-specified. Both analyses were used in Moon et al. (2017) and Taylor et al. (2017).
In both studies a CFA was conducted in MPlus (version 7) on the main scale of the IPQ-R (i.e. timeline (chronic/cyclic), consequences, control (personal and treatment), illness coherence, emotional representations). Items were specified to load on these hypothesised components using syntax to test model fit. It is recommended that CFA is run with at least 200 participants (Brown, 2015). While there are various methods of assessing model fit, both studies used Comparative Fit Index (CFI), Tucker Lewis Index (TLI) and Root Mean Square Error of Approximation (RMSEA), as recommended by Jackson, Gillaspy & Purc-Stephenson (2009). RMSEA values of less than 0.08 indicate reasonable fit and CFI/TLI values of greater than 0.95 suggest acceptable model fit (Hu & Bentler, 1999). For further useful instructions on conducting CFA see Brown (2014).
EFA was conducted on the causal attribution scale in both studies, as recommended by Moss-Morris et al. (2002) and because substantial changes were made to both this scale in both studies. In AF patients, Taylor et al (2017) conducted the EFA in SPSS (V22) using maximum-likelihood extraction and oblique rotation as factors were expected to correlate. For a useful paper on conducting EFA in SPSS see Yong & Pearce (2013). In BCS, Moon et al. (2017) used SPSS with R-menu for ordinal factor analysis based on polychoric correlations, which has shown some benefits over using SPSS alone (Basto & Pereira, 2012). Preliminary interpretation of the data should examine any ceiling effects, which can be tested by examining/removing item frequencies in which >80% of participants disagreed with any specified items. You should also examine whether there is a patterned relationship amongst variables to see if any items should initially be removed. Kaiser-Meyer-Olkin Measure (KMO) of >0.50 will also indicate that the data is suitable for an EFA. The number of factors to be extracted are indicated by using Kaiser’s criterion (eigenvalues of 1.0 cut-off), visual inspection of scree plots or parallel analysis. Items which do not load onto any factors or cross-load onto multiple factors can be removed. Factors should be labelled based on the items contained. For example, in the EFA for AF patients, a factor was labelled as ‘emotional triggers’ which contained items related to stress, mental attitude and emotional state.
Step 5: Testing the psychometric properties of the modified questionnaire
Internal reliability measures how well a set of items measure a particular concept. High internal reliability suggests that all items are measuring the same concept. Cronbach’s alpha can be examined using SPSS. Acceptable alpha values range from 0.70 to 0.95, with 0.95 indicating excellent reliability, (Tavakol & Dennick, 2011) and values higher than this indicating redundancy across items
Test-retest validity examines whether patients’ responses are consistent over time, and the stability of the questionnaire. Participants were asked to complete the questionnaire at baseline and at two-weeks. Intra-class correlations (ICC) can vary between 0 and 1.0 whereby 1 indicates perfect reliability. To examine any potential outliers, Bland-Altman plots can be used (Bland & Altman, 1986).
Construct validity examines the extent to which a test measures what it claims to be measuring, and can be tested by looking at the relationships between the subscales of the modified questionnaire and other theoretically-related questionnaires. Correlations range from 0 to 1.0, with scores closer to 1.0 indicating high correlation.
In the AF study, the modified IPQ-R was examined with a measure of treatment beliefs; the Beliefs about Medicines Questionnaire (BMQ, Horne, Weinman & Hankin, 1999), and an AF-specific QoL measure (AFEQT; Spertus et al., 2010). The BCS study examined the modified IPQ-R with the BMQ and Hospital Anxiety and Depression Scale (HADS; Zigmond & Snaith, 1983).
This blog post by Elaina Taylor and Zoe Moon has outlined how we modified and validated the IPQ-R specific to the patient populations we are studying. If you have any questions, please feel free to get in contact with us via the Healthily Psyched blog page. You can also check out the papers that we’ve written on this topic which are linked up to our profiles. In the meantime, we hope we’ve left you healthily psyched for more.
Basto, M., & Pereira, J. (2012). An SPSS R-menu for ordinal factor analysis. Journal of Statistical Software, 46, 1–29. https://doi.org/10.18637/jss.v046.i04
Bland, J.M., & Altman, D. G. (1986). Statistical methods for assessing agreement between two methods of clinical measurement. Lancet, 327(8476), 307-310. https://doi.org/10.1016/S0140-6736(86)90837-8
Brown, T.A. (2014). Confirmatory factor analysis for applied research. New York: Guilford Publications.
Ericsson, K. A., & Simon, H. A. (1998). How to study thinking aloud in everyday life: Contrasting think-aloud protocols with descriptions and explanations of thinking. Mind, Culture, and Activity, 5, 178-186. https://doi.org/10.1207/s15327884mca0503_3
Hu, L.T, & Bentler, P.M. (1999). Cut-off criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modelling: A Multidisciplinary Journal, 6(1), 1-55. https://doi.org/10.1080/10705519909540118
Hagger, S., Koch, S., Chatzisarantis, N. L. D., & Orbell, S. (2017). The common-sense model of self-regulation: Meta-analysis and test of a process model. Psychological Bulletin. https://doi.org/10.1037/bul0000118
Horne, R., Weinman, J., & Hankin, M. (1999). The beliefs about medicines questionnaire: The development and evaluation of a new method for assessing the cognitive representation of medication. Psychology and Health, 14, 1-24. https://doi.org/10.1080/08870449908407311.
Jackson, D.L., Gillaspy, J.A., & Purc-Stephenson, R. (2009). Reporting practices in confirmatory factor analysis: an overview and some recommendations. Psychological Methods, 14(1), 6-23. https://doi.org/10.1037/a0014694
Leventhal, H., Meyer, D., & Nerenz, D. (1980). The common sense representation of illness danger. Contributions to Medical Psychology, 2, 7–30.
Moon, Z., Moss-Morris, R., Hunter, M.S., & Hughes, L. D. (2017). Measuring illness representations in breast cancer survivors (BCS) prescribed tamoxifen: Modification and validation of the Revised Illness Perceptions Questionnaire (IPQ-BCS). Psychology & Health, 1, 1-20. https://doi.org/10.1080/08870446.2016.1275629
Moss-Morris, R., Weinman, J., Petrie, K., Horne, R., Cameron, L., & Buick, D. (2002). The revised illness perception questionnaire (IPQ-R). Psychology and Health, 17(1), 1-16. https://doi.org/10.1080/08870440290001494
Spertus, J., Dorian, P., Bubien, R., Lewis, S., Godejohn, D., Reynolds. M. R., …& Burk, C. (2010). Development and validation of the atrial fibrillation effect on quality-of-life (AFEQT) questionnaire in patients with atrial fibrillation. Circulation: Arrhythmia and Electrophysiology, 10(5),15-25. https://doi.org/10.1161/CIRCEP.110.958033
Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach’s alpha. International Journal of Medical Education, 2, 53-55. https://doi.org/10.5116/ijme.4dfb.8dfd
Taylor, E. C., O’Neill, M., Hughes, L., & Moss-Morris, R. (2017). An illness-specific version of the Revised Illness Perception Questionnaire in patients with atrial fibrillation (AF-IPQ-R): Unpacking beliefs about treatment control, personal control and symptom triggers. Psychology and Health, 32, 1-19. http://dx.doi.org/10.1080/08870446.2017.1373113
Taylor, E. C., O’Neill, M., Hughes, L. D., Carroll, S., & Moss-Morris, R. (2017). ‘It’s like a frog leaping about in your chest': Illness and treatment perceptions of patients with persistent atrial fibrillation. British Journal of Health Psychology, 22, 1-19. https://doi.org/10.1111/bjhp.12267
Yong, A. G., & Pearce, S. (2013). A beginner’s guide to factor analysis: Focusing on exploratory factor analysis. Tutorials in Quantitative Methods for Psychology, 9(2), 79-94. https://doi.org/10.20982/tqmp.09.2.p079
Zigmond, A. S., & Snaith, R. P. (1983). The Hospital Anxiety and Depression Scale. Acta Psychiatrica Scandinavica, 67, 361–370. https://doi.org/10.1111/j.1600-0447.1983.tb09716.x