Realist review — a new method of systematic review designed for complex policy interventions
KEY MESSAGES
- A realist review provides managers and policy makers with a rich and detailed understanding of a particular issue. The approach helps determine whether findings might work, in what settings, when, how and why. For example, hospital report cards give individual institutions a grade, but a realist review can offer decision makers the context behind why that grade was given.
- This contextualized approach is gaining in popularity among researchers and decision makers committed to evidence-informed decisions.
- Decision makers are involved in nearly every step of the process: framing the initial question; making sure unexpected directions in the review remain useful to them; and reporting on and implementing the results.
The following is a summary of a journal article by Ray Pawson, Trisha Greenhalgh, Gill Harvey and Kieran Walshe, published in the Journal of Health Services Research & Policy in 2005.
A realist review is an approach that’s growing more popular with researchers, policy makers and managers looking to make good decisions. It is a synthesis of the different findings related to a particular research question. Those who adopt it will not scan the literature to tell readers exactly how to move forward in a certain area. A realist review is not a step-by-step prescription, nor does it promise the same results for everyone. Instead, it presents a package of information meant to improve the thinking that goes into decisions on how services should be built and delivered.
Put simply, realist review allows users to take context into consideration when making decisions and sharing knowledge. The process looks at how something is supposed to work, with the goal of finding out what strategies work for which people, in what circumstances, and how.
The research process
A realist review starts with conversation, probably more than one.
The research team begins by talking to the people who are looking for answers — a group of decision makers faced with questions about how to make improvements to the system. They may be searching for ways to improve an existing program or set up a new one, or they may want to know under what conditions a particular initiative works best, or even how to measure its success.
Ideas and issues are tossed around as researchers become familiar with the reasons why decision makers want a review done and how they intend to use the results. Ideally, researchers end up with a question regarding how a specific intervention is supposed to work — for example, how to develop a new healthcare performance report card.
Digging through the literature
Doing a realist review could be described as “feeling your way” through the available literature to find out how to do something a hundred different ways, depending on the complex and changing social systems that surround a particular question. The research team isn’t confined to published research reports. Nor is the team limited to what has been written in a specific field. The search for what works in different situations is broad and can cross many disciplines.
Naturally, the amount of reading material can seem overwhelming. It’s up to researchers to draw on their skills and experience in deciding what to keep in the review and what to put aside. These decisions are not made lightly — people reading the review should be able to see exactly why the reviewers took the path they did and how decisions were made about the value of what they were finding along the way.
Before the reading goes too far, researchers draw up a rough framework of what they intend to cover. They then gather evidence systematically to test and refine their theory. Findings are compared with each other, accepted or rejected, and those accepted are placed within the framework. Different theories of what might work (and what will likely fail) should then become apparent, depending on context.
The search can take on a life of its own — researchers are open to new directions, new areas of interest crop up and the focus can sometimes change. Rather than fight these developments, changes in direction are discussed with decision makers to ensure the findings are still of interest to them. Ideally, this digging for new paths continues until no other significant findings can be found.
Getting the most out of the findings
A realist review is an approach that’s growing more popular with researchers, policy makers and managers looking to make good decisions. It is a synthesis of the different findings related to a particular research question. Those who adopt it will not scan the literature to tell readers exactly how to move forward in a certain area. A realist review is not a step-by-step prescription, nor does it promise the same results for everyone. Instead, it presents a package of information meant to improve the thinking that goes into decisions on how services should be built and delivered.When putting together the findings, the same group of decision makers works with the researchers to wrap things up. Working together, both parties ensure that the results are useful to as many decision makers as possible and can be applied in various contexts. But it doesn’t end there — researchers then work with decision makers to try to apply some of the findings and then evaluate their usefulness in different settings.
Whatever the results, they aren’t meant to drive immediate change. The review can suggest what may be tried in different settings, what to watch out for, and what probably won’t work. But more important than these is the review’s overall ability to illuminate the inner workings of a management or policy area for the people who are running it.
The authors state that the research process in a realist review provides policy makers with a roadmap, alerting decision makers to the problems they might confront along the way, and some of the safest measures to deal with these issues. Researchers are an important part of the process because they are the guides in knowledge transfer. By providing knowledge in the navigation process, researchers are helping decision makers understand what is happening around them and develop long-term strategies and ways of thinking that incorporate not only research results, but also different kinds of influences, ideologies and values.
