Transformations and relocations of evidence in health research synthesis studies
Affiliated staffSupervisors: Prof John Dupré and Dr Susan Kelly
Tel: 01392 269140Fax: 01392 264676
BackgroundIf you have used GP or hospital care or heard a health recommendation in the last 15 or so years, you have certainly been face-to-face with the outcomes of “research synthesis” studies. If this has been in the last few years in the UK or any other country with a strong evidence-based medicine culture, you have been completely enveloped by them. Research synthesis studies are at the heart of evidence-based practice. The systematic review inclusive of meta-analysis is the most popular research synthesis method, but recent review articles have identified over 40 other synthesis methods. In all cases these involve the identification of relevant primary studies through highly systematic literature searches and the integration of relevant findings into various combinations of a literature review and more ‘synthetic’ outcomes (e.g. summary statistics, middle-range theories, practice recommendations, etc.).
In my dissertation I suggest that current synthesis methods and the directions in which novel ones are being developed are ‘locked’: 1) in history – of early successful examplars and the set of theoretical problems, concerns and solutions associated with them; 2) in form and media – in unchallenged vocabulary, rhetoric and reporting practices and in outdated technological capacities; and 3) in the invisibility of the repetitive, automatic and unavoidable. The practices and debates around research synthesis are solidifying so quickly and effectively that it will soon become impossible to see the many paths not taken.
My main aim is to examine if we could do at least some things in the field in radically different and more varied ways, with a view to substantially enhancing methodological rigour and exploratory creativity. I also want to offer a perspective of what research, in particular health research synthesis, is like that goes beyond the picture given of it in specialised methodological texts and public communication representations.
The approach taken is in itself, by being an unusual combination of starting points and methods, a meta-comment on synthesis studies. It combines elements of theoretical-descriptive, theoretical-critical and ‘empirical’ research.
The theoretical component is constituted primarily by ideas from the philosophy and sociology of science and theoretico-methodological literature on health research synthesis concerning 1) the nature of ‘data’, ‘evidence’ and ‘findings’; 2) their relationship with various ‘matrices’ (such as theories, methods, operationalisations, fundamental worldview assumptions, socio-cultural context, etc.) within which those are generated and acquire their meaning; and 3) the processes of de-contextualisation and ‘travelling’ of data, evidence and findings across contexts and uses. Whether these ideas support descriptive-theoretical or critical-theoretical aims, or are themselves challenged, varies. This depends on the particular relationship in which they are found to stand relative to claims in methodological texts on health research synthesis; my personal or shared community experiences of carrying out primary and secondary research in the health sciences; and, most importantly, relative to findings from the ‘empirical’ work underlying this study.
This ‘empirical’ work is carried out on 30 publications on cancer from the last three years, randomly selected out of a pool of publications retrieved from MEDLINE via a combination of search strategies aiming to capture psychological, psychosocial and bio-psycho-social research on the topic. It involves 1) standard ‘data extraction’, 2) non-standard data extraction and 3) meta-data-collection.
Data extraction is the process of identifying information from the primary studies that is relevant to the synthesis question and de-contextualising it in preparation for further processing on the way to the final synthesis outcome. (Usually this takes the form of reading primary studies and either recording relevant information in tables or ‘coding’ findings onto a copy of the primary study itself).
Standard data extraction refers to processes of extracting data that are more or less typical for current synthesis studies.
Non-standard data extraction refers to processes and techniques implemented in this work which aim to explore the feasibility of alternative ways of data extraction. These include approaches that are typical in other areas of health research but not used in research synthesis; typical of other stages of the research synthesis process but not of data extraction; reflective of modern thinking in the meta-scientific fields (e.g. philosophy and sociology of science) but rarely, if ever, transformed into concrete scientific methods and techniques. For instance, I carry out ‘extreme data extraction’ where I make no judgement of what is relevant to the synthesis question but extract ‘everything’ the paper seems to offer; base the sampling of publications on principles of randomness and maximum breadth, in a field that is very careful in setting up delimiting criteria and strict selection mechanisms very early in the synthesis processes; and perform multiple coding and classification of the same unit of information in a field where the default approach is one of finding the single best description and class.
Meta-data-collection refers to the process of collecting data about the very process of data extraction (both standard and non-standard), such as introspective observations on difficulties encountered and on the nature of source studies. These will be used to describe some of the processes under way in research synthesis work and discuss its promise and boundaries.
Vail L, Bosley S, Petrova M, Dale J. 'Health care assistants in general practice: a qualitative study of their experiences'. Primary Health Care Research & Development, 2001; 12(1): 29-41. doi:10.1017/S1463423610000204
Petrova M, Vail L, Bosley S, Dale J. ‘Benefits and challenges of employing health care assistants in general practice: A qualitative study of GPs’ and practice nurses’ perspectives’. Family Practice, 2010; 27(3): 303-311. http://fampra.oxfordjournals.org/content/27/3/303.abstract
Petrova M, Dale J, Munday D, Koistinen J, Agarwal S and Lall R. (2009) ‘The role and impact of facilitators in primary care: findings from the implementation of the Gold Standards Framework for palliative care’. Family Practice, 2010; 27 (1): 38-47. http://fampra.oxfordjournals.org/content/27/1/38.abstract
Munday D, Petrova M, Dale J. ‘Exploring preferences for place of death with terminally ill patients: a qualitative study of the experiences of GPs and community nurses’. British Medical Journal, 2009, 339:b2391
Dale J, Petrova M, Munday D, Koistinen-Harris J, Lall R, Thomas K. 'A national facilitation project to improve primary palliative care: the impact of the Gold Standards Framework on process and self-ratings of quality'. Quality and Safety in Health Care, 2009; 18 (3): 174-180
Petrova M, Dale J, Fulford KWM. 'Values-based practice in primary care: easing the tensions between individual values, ethical principles and best evidence’. British Journal of General Practice, 2006; 56 (530): 703-709
Petrova M, Sutcliffe P, Fulford KWM, Dale J. ‘Search terms and a validated brief search filter to retrieve publications on health-related values in MEDLINE: a word frequency analysis study’
Petrova M, Sutcliffe P, Dale J, Fulford KWM. VaST (Values Search Tools): A manual for searching electronic databases for publications on health-related values