Developing a search strategy for systematic reviews of textual evidence
Challenges and recommendations
A recent publication in JBI Evidence Synthesis addresses the challenges inherent in developing search strategies for systematic reviews of textual evidence, given that this type of evidence is often produced and disseminated outside traditional academic publishing channels.
Textual evidence systematic reviews series
The publication, titled Textual evidence systematic reviews series paper 2: challenges and strategies in developing a search strategy for systematic reviews of textual evidence, is the second in a series of three papers. The series was launched with the publication of the first paper in the special issue of JBI Evidence Synthesis to mark JBI Methodology Month in March.
The series underscores the growing recognition and methodological advancements in the synthesis of non-research evidence, particularly textual evidence, to inform healthcare policy and practice.
The value of textual evidence
In the latest paper in the series, the authors highlight the increasing acknowledgement of the value of textual evidence, encompassing opinions, narratives, and policies, for informing healthcare practices. Textual evidence can play a vital role, especially when formal research is limited or absent, offering insights from clinical experience, expert consensus, and policy documents.
“…. narrative accounts, expert opinion, procedural reports, and policy documents are forms of textual evidence that have a crucial role in evidence-based decision-making. This is particularly true in cases where research studies are not available or when timely data and contextually derived evidence are better suited to address specific research questions. When a problem or question can be answered only through expert opinion or direct experience and agreement among experts in a field, whether they are clinicians or members of the public, this information becomes the most reliable evidence available. Subsequently, practitioners and policymakers may use this information, often referred to as expert evidence, to guide their decision-making.”
Synthesis of textual sources
While systematic review methodologies for quantitative and qualitative research are well-established, the synthesis of textual evidence, often referred to as grey literature, remains a developing area within evidence-based practice. This paper builds on JBI's ongoing efforts to provide a transparent and systematic approach to incorporating diverse knowledge sources into decision-making.
“Systematic evaluation and synthesis of textual sources such as opinions, narratives, and policies can be a challenging task, particularly when the embedded data reveal conflicting recommendations. As such, adopting a clear and systematic approach to identify the most reliable textual evidence can offer practical guidance to both practitioners and policymakers.”
Searching for textual evidence
Applying the same rigorous search methodologies used for peer-reviewed literature can be considerably more complex. To overcome these challenges, the authors outline a structured 3-phase approach to searching for textual evidence within a JBI review. The authors emphasise the importance of researchers collaborating with information specialists and experts to refine and strengthen their search techniques.
They also recommend exploring a variety of textual evidence sources beyond traditional databases, such as conference proceedings, theses, dissertations, media reports, and targeted searches of grey literature databases and websites. Advanced Google searching techniques can also be valuable for locating online documents, although challenges related to the volume of results and replicability need to be addressed through strategies such as using incognito browsers and involving a second reviewer.
The paper stresses the necessity of systematically documenting all search processes to ensure transparency and reproducibility in the review. This includes recording the databases and websites searched, the search terms used, the dates of the searches, and the number of records retrieved. This documentation is vital for the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram.
Resources for textual evidence synthesis methodology
The full series of papers will be published in JBI Evidence Synthesis, and promises to offer supplementary support for conducting high-quality systematic reviews of textual evidence. Comprehensive, step-by-step- guidance for conducting systematic reviews of textual evidence is available in the JBI Manual for Evidence Synthesis.
Edwards, Deborah; Cooper, Adam; McArthur, Alexa; Barber, Brittany V.; Gregg, Emily; Weeks, Lori E.; Jordan, Zoe
JBI Evidence Synthesis. 23(4):566-575, April 2025.