Errors in search strategies used in systematic reviews and their effects on information retrieval

Salvador-Oliván, José Antonio, Marco-Cuenca, Gonzalo and Arquero-Avilés, Rosario Errors in search strategies used in systematic reviews and their effects on information retrieval. Journal of the Medical Library Association, 2019, vol. 107, n. 2, pp. 210-221. [Journal article (Paginated)]

[thumbnail of 567-5170-1-PB.pdf]
Preview
 Text
567-5170-1-PB.pdf
Download (444kB) | Preview
[thumbnail of 567-5168-1-SPapp.pdf]
Preview
 Text
567-5168-1-SPapp.pdf
Download (139kB) | Preview

English abstract

Objectives: Errors in search strategies negatively affect the quality and validity of systematic reviews. The primary objective of this study was to evaluate searches performed in MEDLINE/PubMed to identify errors and determine their effects on information retrieval. Methods: A PubMed search was conducted using the systematic review filter to identify articles that were published in January of 2018. Systematic reviews or meta-analyses were selected from a systematic search for literature containing reproducible and explicit search strategies in MEDLINE/PubMed. Data were extracted from these studies related to ten types of errors and to the terms and phrases search modes. Results: The study included 137 systematic reviews in which the number of search strategies containing some type of error was very high (92.7%). Errors that affected recall were the most frequent (78.1%), and the most common search errors involved missing terms in both natural language and controlled language and those related to Medical Subject Headings (MeSH) search terms and the non-retrieval of their more specific terms. Conclusions: To improve the quality of searches and avoid errors, it is essential to plan the search strategy carefully, which includes consulting the MeSH database to identify the concepts and choose all appropriate terms, both descriptors and synonyms, and combining search techniques in the free-text and controlled-language fields, truncating the terms appropriately to retrieve all their variants.

Item type: Journal article (Paginated)
Keywords: search strategies; systematic reviews
Subjects: A. Theoretical and general aspects of libraries and information. > AA. Library and information science as a field.
G. Industry, profession and education. > GI. Training.
H. Information sources, supports, channels.
H. Information sources, supports, channels. > HL. Databases and database Networking.
Depositing user: Jose Antonio Salvador-Oliván
Date deposited: 17 Apr 2019 13:23
Last modified: 17 Apr 2019 13:23
URI: http://hdl.handle.net/10760/34374

References

1. Eden J, Levit L, Berg A, Morton S, editors. Finding what works in health care: Standards for systematic reviews. Washington, DC: National Academies Press; 2011. 340 p. DOI: http://dx.doi.org/10.17226/13059.

2. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol. 2009;62(10):e1–34.

3. Lefevre C, Manheimer E, Glanville J. Searching for studies. In: Higgins J, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions. Version 510 [Internet]. West Sussex, England: John Wiley & Sons; 2011. p. 95–150. [cited 29 May 2018]. <http://handbook-5-1.cochrane.org/>.

4. Buckland M, Gey F. The relationship between recall and precision. J Am Soc Inf Sci. 1994;45(1):12–9.

5. McGowan J, Sampson M. Systematic reviews need systematic searchers. J Med Libr Assoc. 2005;93(1):74–80.

6. Lane D, Dykeman J, Ferri M, Goldsmith CH, Stelfox HT. Capture-mark-recapture as a tool for estimating the number of articles available for systematic reviews in critical care medicine. J Crit Care. 2013;28(4):469–75.

7. Wieland LS, Robinson KA, Dickersin K. Understanding why evidence from randomised clinical trials may not be retrieved from Medline: Comparison of indexed and non-indexed records. BMJ. 2012;344(7838):1–10.

8. Delaney A, Tamás PA. Searching for evidence or approval?. A commentary on database search in systematic reviews and alternative information retrieval methodologies. Res Synth Methods. 2017;9(1):124–31.

9. Sampson M, McGowan J. Errors in search strategies were identified by type and frequency. J Clin Epidemiol. 2006;59(10):1057–63.

10. Goossen K, Tenckhoff S, Probst P, Grummich K, Mihaljevic AL. Optimal literature search for systematic reviews in surgery. Langenbecks Arch Surg. 2018;403(1):119-129.

11. Harter SP. Online information retrieval. Concepts, principles and techniques. San Diego, CA: Academic Press; 1986.

12. Lancaster FW, Warner AJ. Information retrieval today. Arlington, VA: Information Resources Press; 1993.

13. Meadow CT, Boyde BR, Kraft DoH. Text information retrieval systems. Second edition. San Diego, CA: Academic Press; 2002.

14. Walker G, Janes J. Online retrieval: A dialogue of theory and practice. Second edition. Englewood, CO: Libraries Unlimited; 1999.

15. Kugley S, Wade A, Thomas J, Mahood Q, Jørgensen AMK, Hammerstrøm K SN. Searching for studies: a guide to information retrieval for Campbell systematic reviews [Internet]. Oslo: The Campbell Collaboration; 2017. [cited 29 May 2018]. <https://campbellcollaboration.org/library/searching-for-studies-information-retrieval-guide-campbell-reviews.html>.

16. Li L, Tian J, Tian H, Moher D, Liang F, Jiang T, et al. Network meta-analyses could be improved by searching more sources and by involving a librarian. J Clin Epidemiol. 2014;67(9):1001–7.

17. Higgins J, Green S. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 [Internet]. The Cochrane Collaboration; 2011. [cited 29 May 2018]. <http://www.cochrane-handbook.org>.

18. Rethlefsen ML, Murad MH, Livingston EH. Engaging medical librarians to improve the quality of review articles. JAMA. 2014 Sep 10;312(10):999. DOI: http://dx.doi.org/ 10.1001/jama.2014.9263.

19. Spencer AJ, Eldredge JD. Roles for librarians in systematic reviews: a scoping review. J Med Libr Assoc. 2018 Jan;106(1):46–56.

20. Rethlefsen ML, Farrell AM, Osterhaus Trzasko LC, Brigham TJ. Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews. J Clin Epidemiol. 2015;68(6):617–26.

21. Koffel JB. Use of recommended search strategies in systematic reviews and the impact of librarian involvement: A cross-sectional survey of recent authors. PLoS One. 2015;10(5):1–14.

22. Zhang L, Sampson M, McGowan J. Reporting of the role of the expert searcher in Cochrane Reviews. Evid Based Libr Inf Pract. 2006;1(4):3–16.

23. Sampson M, McGowan J, Cogo E, Grimshaw J, Moher D, Lefebvre C. An evidence-based practice guideline for the peer review of electronic search strategies. J Clin Epidemiol 2009;62(9):944–52.

24. McGowan J, Sampson M, Lefebvre C. An evidence based checklist for the Peer Review of Electronic Search Strategies (PRESS EBC). Evid Based Libr Inf Pract. 2010;5(1):149.

25. McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS Peer Review of Electronic Search Strategies: 2015 guideline statement. J Clin Epidemiol. 2016;75:40–6.

26. Yoshii A, Plaut DA, McGraw KA, Anderson MJ, Wellik KE. Analysis of the reporting of search strategies in Cochrane systematic reviews. J Med Libr Assoc. 2009;97(1):21–9.

27. Sampson M, McGowan J, Tetzlaff J, Cogo E, Moher D. No consensus exists on search reporting methods for systematic reviews. J Clin Epidemiol. 2008;61(8):748–54.

28. Koffel JB, Rethlefsen ML. Reproducibility of search strategies is poor in systematic reviews published in high-impact pediatrics, cardiology and surgery journals: A cross-sectional study. PLoS One. 2016;11(9):1–16.

29. Layton D. A critical review of search strategies used in recent systematic reviews published in selected prosthodontic and implant-related journals: Are systematic reviews actually systematic?. Int J Prosthodont. 2017;30(1):13–21.

30. Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG. Epidemiology and reporting characteristics of systematic reviews. PLoS Med. 2007;4(3):447–55.

31. Toews LC. Compliance of systematic reviews in veterinary journals with Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) literature search reporting guidelines. J Med Libr Assoc. 2017;105(3):233–9.

32. Page MJ, Shamseer L, Altman DG, Tetzlaff J, Sampson M, Tricco AC, et al. Epidemiology and teporting characteristics of systematic reviews of biomedical research: A cross-sectional study. PLoS Med. 2016;13(5):1–30.

33. Faggion CM, Huivin R, Aranda L, Pandis N, Alarcon M. The search and selection for primary studies in systematic reviews published in dental journals indexed in Medline was not fully reproducible. J Clin Epidemiol. 2018;98:53–61.

34. Yaylali IE, Alaçam T. Critical assessment of search strategies in systematic seviews in endodontics. J Endod. 2016 Jun; 42(6):854–60.

35. Russell-Rose T, Chamberlain J. Expert search strategies: The information retrieval practices of healthcare information professionals. JMIR Med Informatics. 2017;5(4):e33. DOI: http://dx.doi.org/10.2196/medinform.7680.

36. Puljak L. If there is only one author or only one database was searched, a study should not be called a systematic review. J Clin Epidemiol. 2017;91:4–5.

37. Lam MT, McDiarmid M. Increasing number of databases searched in systematic reviews and meta-analyses between 1994 and 2014. J Med Libr Assoc. 2016;104(4):284–9.

38. Katchamart W, Faulkner A, Feldman B, Tomlinson G, Bombardier C. PubMed had a higher sensitivity than Ovid-Medline in the search for systematic reviews. J Clin Epidemiol. 2011;64(7):805–7.

39. U.S. National Library of Medicine. Search strategy used to create the systematic reviews subset on PubMed. [Internet]. 2002. [cited 29 May 2018]. <https://www.nlm.nih.gov/bsd/pubmed_subsets/sysreviews_strategy.html>

40. Al-Rifai RH, Ali N, Barigye ET, Al Haddad AHI, Loney T, Al-Maskari F, et al. Maternal and birth cohort studies in the Gulf Cooperation Council countries: protocol for a systematic review and narrative evaluation. BMJ Open. 2018 Jan;8(1):e019843. DOI: http://dx.doi.org/ 10.1136/bmjopen-2017-019843.

41. Marcolino MS, Oliveira JAQ, D’Agostino M, Ribeiro AL, Alkmim MBM, Novillo-Ortiz D. The Impact of mHealth Interventions: Systematic review of systematic reviews. JMIR MHealth UHealth. 2018 Jan;6(1):e23. DOI: http://dx.doi.org/ 10.2196/mhealth.8873.

42. Eddens L, van Someren K, Howatson G. The role of intra-session exercise sequence in the interference effect: A systematic review with meta-analysis. Sports Med. 2018 Jan;48(1):177–88.

43. Ezeamama AE, Bustinduy AL, Nkwata AK, Martinez L, Pabalan N, Boivin MJ, et al. Cognitive deficits and educational loss in children with schistosome infection. A systematic review and meta-analysis. PLoS Negl Trop Dis. 2018 Jan;12(1):e0005524. DOI: http://dx.doi.org/ 10.1371/journal.pntd.0005524.

44. Haniffa R, Isaam I, De Silva AP, Dondorp AM, De Keizer NF. Performance of critical care prognostic scoring systems in low and middle-income countries: a systematic review. Crit Care. 2018 Jan;22(1):18. DOI: http://dx.doi.org/ 10.1186/s13054-017-1930-8.

45. Johannsen M, Frederiksen Y, Jensen AB, Zachariae R. Psychosocial predictors of posttreatment pain after nonmetastatic breast cancer treatment: a systematic review and meta-analysis of prospective studies. J Pain Res. 2018;11:23–36.

46. Cabrini L, Landoni G, Baiardo Radaelli M, Saleh O, Votta CD, Fominskiy E, et al. Tracheal intubation in critically ill patients: a comprehensive systematic review of randomized trials. Crit Care. 2018 Jan;22(1):6. DOI: http://dx.doi.org/ 10.1186/s13054-017-1927-3.

47. Britt-Spells AM, Slebodnik M, Sands LP, Rollock D. Effects of perceived discrimination on depressive symptoms among black men residing in the United States: A meta-analysis. Am J Mens Health. 2018 Jan;12(1):52–63.

48. Gadalla MA, Huang S, Wang R, Norman RJ, Abdullah SA, El Saman AM, et al. Effect of clomiphene citrate on endometrial thickness, ovulation, pregnancy and live birth in anovulatory women: systematic review and meta-analysis. Ultrasound Obstet Gynecol. 2018 Jan;51(1):64–76.

49. Babu GR, Murthy GVS, Ana Y, Patel P, Deepa R, Neelon SEB, et al. Association of obesity with hypertension and type 2 diabetes mellitus in India: A meta-analysis of observational studies. World J Diabetes. 2018 Jan;9(1):40–52.

50. Jiang Y-Q, Tian Y, Zeng L-J, He S-N, Zheng Z-T, Shi L, et al. The safety and efficacy of hybrid ablation for the treatment of atrial fibrillation: A meta-analysis. PLoS One. 2018;13(1):e0190170. DOI: http://dx.doi.org/ 10.1371/journal.pone.0190170.

51. Buczinski S, Gicquel E, Fecteau G, Takwoingi Y, Chigerwe M, Vandeweerd JM. Systematic review and meta-analysis of diagnostic accuracy of serum refractometry and brix refractometry for the diagnosis of inadequate transfer of passive immunity in calves. J Vet Intern Med. 2018 Jan;32(1):474–83.

52. Nakhjavan-Shahraki B, Yousefifard M, Rahimi-Movaghar V, Baikpour M, Nasirinezhad F, Safari S, et al. Transplantation of olfactory ensheathing cells on functional recovery and neuropathic pain after spinal cord injury: systematic review and meta-analysis. Sci Rep. 2018 Jan;8(1):325. DOI: http://dx.doi.org/10.1038/s41598-017-18754-4.

53. Chiarito M, Pagnesi M, Martino EA, Pighi M, Scotti A, Biondi-Zoccai G, et al. Outcome after percutaneous edge-to-edge mitral repair for functional and degenerative mitral regurgitation: a systematic review and meta-analysis. Heart. 2018 Feb;104(4):306–12.

54. Hummel P, Saxena A, Klingler C. Rapid qualitative review of ethical issues surrounding healthcare for pregnant women or women of reproductive age in epidemic outbreaks. Epidemiol Health. 2018;40:e2018003. DOI: http://dx.doi.org/ 10.4178/epih.e2018003.

55. Bos M, Nikkels PGJ, Cohen D, Schoones JW, Bloemenkamp KWM, Bruijn JA, et al. Towards standardized criteria for diagnosing chronic intervillositis of unknown etiology: A systematic review. Placenta. 2018 Jan;61:80–8.

56. De Boer J, Prikken M, Lei WU, Begemann M, Sommer I. The effect of raloxifene augmentation in men and women with a schizophrenia spectrum disorder: A systematic review and meta-analysis. NPJ Schizophr. 2018 Jan;4(1):1. DOI: http://dx.doi.org/ 10.1038/s41537-017-0043-3.

57. De Groot AF, Appelman-Dijkstra NM, van der Burg SH, Kroep JR. The anti-tumor effect of RANKL inhibition in malignant solid tumors. A systematic review. Cancer Treat Rev. 2018 Jan;62:18–28.

58. Adam S, Feller A, Rohrmann S, Arndt V. Health-related quality of life among long-term (>/=5 years) prostate cancer survivors by primary intervention: a systematic review. Health Qual Life Outcomes. 2018 Jan;16(1):22. DOI: http://dx.doi.org/ 10.1186/s12955-017-0836-0.

59. Boatin AA, Cullinane F, Torloni MR, Betran AP. Audit and feedback using the Robson classification to reduce caesarean section rates: a systematic review. BJOG. 2018 Jan;125(1):36–42.

60. Chen H, Chai Y, Dong L, Niu W, Zhang P. Effectiveness and appropriateness of mHealth interventions for maternal and child health: Systematic review. JMIR MHealth UHealth. 2018 Jan;6(1):e7. DOI: http://dx.doi.org/ 10.2196/mhealth.8998.

61. Ho GJ, Liew SM, Ng CJ, Hisham Shunmugam R, Glasziou P. Development of a search strategy for an evidence based retrieval service. PLoS One. 2016 Dec 9;11(12):e0167170. DOI: http://dx.doi.org/ 10.1371/journal.pone.0167170.

62. Volpato E de SN, Betini M, Puga ME, Agarwal A, Cataneo AJM, Oliveira LD de, et al. Strategies to optimize Medline and Embase search strategies for anesthesiology systematic reviews. An experimental study. Sao Paulo Med J [Internet]. 2018;(0):1–6.


Downloads

Downloads per month over past year

Actions (login required)

View Item View Item
Contact us
Send email Sponsors
Sponsor: AIMS Sponsor: D&P Sponsor: UniNa CAB Get connected
Facebook Facebook Twitter Twitter Linkedin LinkedIn RSS RSS