Are the Cochrane group registers comprehensive? A case study of Japanese psychiatry trials

BACKGROUND: Language bias is a form of publication bias and constitutes a serious threat to meta-analyses. The Cochrane Controlled Trials Register is one attempt to remedy this and now contains more than 300,000 citations. However we are still unsure if it provides comprehensive coverage, particularly for non-English trials. METHODS: We have recently established a comprehensive register of Japanese trials of psychotropic drugs through extensive personal contacts, electronic searches and handsearches. We examined two Cochrane psychiatry group registers against this Japanese database. RESULTS: The Japanese register contained 56 reports of randomized controlled trials (RCTs) of antidepressants for depression but the Cochrane Depression, Anxiety and Neurosis group register contained 18, with an overlap of only nine. The Japanese register contained 61 reports of RCTs of neuroleptics for schizophrenia and the Cochrane Schizophrenia group register contained 36, with an overlap of only six. Taking account of some duplicate publications, only a quarter to a third of all relevant Japanese RCTs were retrievable from the Cochrane group registers. CONCLUSIONS: Similar, or worse, yields may be expected with RCTs conducted in other East Asian countries, and in other fields of medicine. What evidence there is suggests that this situation may lead to a systematic over estimate of treatment effect.