The quality of reporting of systematic reviews in periodontology journals: a cross sectional survey

BACKGROUND: Systematic reviews (SR) with or without simultaneous meta-analyses intend to synthesize the vast amount of published primary research to answer a focused research question. Stringent research criteria must be followed to publish a high quality SR in order to attain clinically relevant answer to the focused research question. Validated instruments have been developed to assess the quality of SR such as the Assessment of Multiple Systematic Reviews (AMSTAR) and a checklist by Glenny and colleagues. Despite the best efforts of journal editors and publishers, the quality of reporting of SR remains below par impacting the evidence-based clinical decision-making process. ❧ PURPOSE: To assess the quality of reporting of Systematic Reviews with and without meta-analyses in two widely read periodontology journals, namely, Journal of Periodontology (JoP) and Journal or Clinical Periodontology (JCP) published in the years 2010 and 2013. ❧ MATERIAL AND METHODS: Electronic search using National Library of Medicine’s Medline database supplemented by manual hand search of JoP and JCP was done independently by two reviewers. AMSTAR statement and Glenny’s checklist were used to evaluate the reporting quality of the selected Systematic Reviews from both journals. Each item on these instruments was assigned an appropriate score (1=Yes, 0=No or N/A=Not applicable). The percentage of the number of articles fulfilling each item in each checklist for the years of 2010 and 2013 was calculated, excluding any non-applicable items from such calculation. Descriptive statistics were used to analyze the data. Trends in improvement of reporting were assessed between the two years in both the journals. The percentage of SR adhering to three different predetermined percentages for each checklist (i.e.; ≥50%, ≥75% and ≥90%) was calculated for each journal. ❧ RESULTS: Evaluation of JoP 2010 SR using the AMSTAR statement revealed low scores of (0%, 14.2% and 14.2%) in items 11, 4 and 10 respectively and high scores (100%) in items 1, 2, 3 and 6. Evaluation of JoP 2010 SR using the Glenny checklist revealed low scores (14.2%, 28.5% and 50%) in items D, E, and J respectively and high scores (100%) in items A,C,G,I, and M. ❧ Evaluation of JoP 2013 SR using the AMSTAR statement revealed low scores of (0%, 7% and 35.7%) in items 11, 4 and 10 respectively and high scores (100%) in items 1, 6 and 9. Evaluation of JoP 2013 SR using the Glenny checklist revealed low scores of (7%, 14%, 38.4%) in items D, E, and J respectively and high scores (100%) in items A, B, I, M and O. ❧ Evaluation of JCP 2010 SR using the AMSTAR statement revealed low scores of (0%, 14.2% and 42.8%) in items 11, 10 and 4 respectively and high scores (100%) in items 1, 3 & 6. Evaluation of JCP 2010 SR using the Glenny checklist revealed low scores of (28.5%, 40% and 42.8%) in items E, J and D respectively and high scores (100%) in items A, B, C, M and O. ❧ Evaluation of JCP 2013 SR using the AMSTAR statement revealed low scores of (0%, 11.1% and 35.2%) in items 11, 4 and 10 respectively and high score (100%) in item 1. Evaluation of JCP 2013 SR using the Glenny checklist revealed low scores of (11.1%, 20% and 50%) in items D, and J and E respectively and high scores (100%) in items A, B, I, M and O. ❧ For both journals, AMSTAR item #4 (Was the status of publication, i.e., gray literature, used as an inclusion criteria?), #10 (Was the likelihood of publication bias assessed?) and #11 (was the conflict of interest included) were consistently scored low in both the years of assessment. For both journals, Glenny item #D (search for published and un-published literature), #E (were all languages considered) and #J (was it stated that the quality assessment was carried out by at least 2 reviewers) were consistently scored low in both the years of assessment. ❧ For both journals, AMSTAR items #1 (Was an a priori design provided?) and #6 (were the characteristics of the included studies provided?) were consistently scored high in both the years of assessment. ❧ For both journals, Glenny item #A (Did review addressed a focused question), #B (did author looked for appropriate papers), #I (did the reviewers include the attempt to assess the quality of the included studies in the analysis) #M (are the results clearly displayed), and #O (were results of review interpreted appropriately) were consistently scored high in both the years of assessment. ❧ In JoP, the overall mean AMSTAR score/ Systematic Review for all of the SR ranged between 36-81% in 2010 and 18%-90% in 2013. The corresponding overall score for Glenny checklist ranged between 45-92% in 2010 and 38%-92% in 2013. ❧ In JCP, the overall mean AMSTAR score/Systematic Review for all of the SR ranged between 36-81% in 2010 and 45-81% in 2013. The corresponding overall score for Glenny checklist ranged between 57-92% in 2010 and 54-92% in 2013. ❧ The higher the predetermined percentage of adherence, the lower the total percentage of SR adhering to such percentage in each Journal. ❧ CONCLUSION: The quality of reporting of SR in JoP and JCP according to AMSTAR statement and Glenny’s checklist can be improved in the areas identified through this study. ❧ FUNDING: No research funding was obtained.