Irreproducibility in searches of scientific literature: a comparative analysis

1. Repeatability is the cornerstone of science and it is particularly important for systematic reviews. However, little is known on how researchers’ choice of database and search platform influence the repeatability of systematic reviews. Here, we aim to unveil how the computer environment and the location where the search was initiated from influence hit results.

2. We present a comparative analysis of time-synchronized searches at different institutional locations in the world, and evaluate the consistency of hits obtained within each of the search terms using different search platforms.

3. We revealed a large variation among search platforms and showed that PubMed and Scopus returned consistent results to identical search strings from different locations. Google Scholar and Web of Science’s Core Collection varied substantially both in the number of returned hits and in the list of individual articles depending on the search location and computing environment. Inconsistency in W...

1. 可重复性是科学研究的基石，对于系统综述（systematic reviews）而言尤为关键。然而，目前学界对研究者选择数据库（database）与搜索平台（search platform）如何影响系统综述的可重复性仍知之甚少。本研究旨在揭示计算机环境与搜索发起位置，如何对检索命中结果（hit results）产生影响。
2. 本研究针对全球不同研究机构所在地开展时间同步检索，并对比分析了不同搜索平台下，各搜索词对应命中结果的一致性。
3. 本研究揭示了不同搜索平台间存在较大差异：PubMed与Scopus在不同位置发起的相同检索式（search strings），可返回一致的命中结果；而谷歌学术（Google Scholar）与Web of Science核心合集（Web of Science’s Core Collection）的返回命中数量与单篇文献列表，均会随搜索发起位置与计算机环境的不同而发生显著变化。W[...]方面存在不一致性。