Assessment of EBM

Evidence-based medicine: assessment of

knowledge of basic epidemiological and research methods among medical doctors

Submitted to Venera ma'am by Roshan Shinde Group 32

EVIDENCE BASED MEDICINE is the main source of new knowledge for doctors in this era. The main objectives of EBM are as follows,

To evaluate the knowledge of basic research methods and data analysis among medical doctors. To assess factors such as the country of the medical school graduation profession. 

Importance of Research Competence: 

1. The study emphasizes that a solid understanding of epidemiology and biostatistics is essential for doctors to critically appraise medical literature and make informed clinical decisions.

2. Previous Findings: Prior studies indicated that many doctors lack proficiency in research methods, with significant gaps in understanding key concepts of evidence-based medicine (EBM).

Materials and Methods 

Data collection and study population 

The study involved 40 departments and employed around 500 doctors.

A random selection of 15 departments was made for participant recruitment.

Data collection 

A supervised, self-administered questionnaire was distributed during morning staff meetings.

The questionnaire consisted of 10 multiple-choice questions focused on basic epidemiology and statistics, along with demographic data.

Participants were divided into two groups based on their country of medical school graduation: those from the former Soviet Union (Eastern education) and those from other countries (Western education).

The questionnaire was completed anonymously, and all participants were efficient in Hebrew.

Questionnaire 

1. Sections of the Questionnaire:

Personal Details: This section collected demographic information about the doctors, including:

• Country of graduation

• Year of graduation from medical school

• Professional status (whether they are specialists or residents)

• Reading and writing habits related to medical literature.

Knowledge Assessment: This section consisted of 10 multiple-choice questions focused on basic research methods and statistics, divided as follows:

Statistics: 5 questions

Epidemiology: 5 questions

2. Basis for Statistical Questions:

The questions on statistics were derived from a list of commonly used statistical methods identified by Emerson and Colditz in 1983. This list was previously utilized for quality evaluations of articles published in the New England Journal of Medicine and referenced in a similar study by Horton and Switzee. This approach ensures that the questions are relevant and grounded in established research practices.

3. Scoring Methodology:

• Any missing answers to questions on epidemiological and statistical methods were considered incorrect. This scoring method emphasizes the importance of attempting to answer all questions and reflects a strict approach to assessing knowledge.

• The decision to mark unanswered questions as incorrect may encourage participants to engage more thoughtfully with the questionnaire, although it could also discourage some from attempting to answer if they are unsure

To ensure validity of the questionnaire, the 10 questions assessing knowledge were given to 15 members of the Epidemiology Department, Ben‐Gurion University. All of them correctly answered all the questions.

Results: 

Response Rate: Out of 260 eligible doctors, 219 completed the questionnaire (84.2% response rate).

Statistical methods 

1. Comparison of Categorical Variables:

Chi-Squared Test (x²): This test was used to examine differences between categorical variables. It assesses whether the observed frequencies in each category differ from what would be expected under the null hypothesis.

Fisher's Exact Test: This test was employed when sample sizes were small or when the assumptions of the chi-squared test were not met. It is particularly useful for 2×2 contingency tables.

2. Comparison of Ordinal Variables:

Mann-Whitney U Test: This non-parametric test was used to compare ordinal variables with multiple values, such as the scores obtained from the questionnaire. It assesses whether the distributions of two independent samples differ.

3. Paired Comparisons:

Wilcoxon's Signed Rank Test: This non-parametric test was used for paired comparisons of scores. It evaluates whether the median of the differences between paired observations is significantly different from zero.

 4. Correlation Analysis:

Spearman's Rank Correlation Coefficient: This test was used to estimate the correlation between continuous variables. It assesses how well the relationship between two variables can be described using a monotonic function.

5. Multivariable Analysis:

Linear Regression: This method was used to explain the final score based on multiple variables. The analysis adjusted for all variables that were found to be related in the univariable analysis with a p-value of less than 0.1. This approach helps to identify the independent effects of each variable on the outcome.

  6. Significance Level:

A p-value of 0.05 was considered statistically significant, indicating that there is less than a 5% probability that the observed results occurred by chance.

 7. Data Presentation:

Normally distributed variables were expressed as mean (standard deviation, SD), while non-normally distributed variables were presented as median and interquartile range (IQR). This distinction is important for accurately representing the data's distribution.

Table 2 depicts doctors' professional characteristics according to the country of medical school graduation. Of 219 participants, 84 (38.4%) graduated from the former Soviet republics. The remaining 135 doctors were distributed by the country of graduation as follows: Israel, 100 (45.7%); West and Central Europe, 22 (10.0%); Italy, 8; Germany, 3; Czech Republic, 3; Hungary, 3; Netherlands, 1; Romania, 4; South America, 10 (4.6%); Argentina, 5; Cuba, 3; Uruguay, 1; Brazil, 1; and North America, 3 (1.4%).

Time Elapsed Since Graduation:

 • Doctors from Israel and other countries had a shorter time since graduation compared to those from the former Soviet Union:

 • Foreign Graduates: 8 years

(Interquartile Range (IQR) 4-19)

Former Soviet Union Graduates: 10 years (IQR 6-19)

 • The difference was statistically significant (p = 0.02), indicating that foreign graduates tended to have graduated more recently.

Professional Status:

There were fewer specialists among foreign graduates compared to those who graduated from Israel

Foreign Graduates: 32.8% were specialists

Israeli Graduates: 48.0% were specialists

This difference was also statistically significant (p = 0.02).

Choice of Residency:

There were notable differences in the choice of residency between the two groups:

Domestic Graduates: 29.3% chose pediatrics or obstetrics and gynecology

Conclusion

The analysis of doctors' professional characteristics based on their country of medical school graduation reveals important insights into the diversity of medical training backgrounds and their implications for specialization and residency choices. These findings underscore the need for ongoing evaluation of medical education and training systems to ensure that all graduates, regardless of their background, are adequately prepared to meet the healthcare needs of the population

Table 3 describes the reading and publishing habits of the participants. A total of 96% of the participants reported reading at least one article per week, whereas 35.2% usually read at least three articles. Specialists read significantly more articles per week—52.3% of them read at least three articles, compared with only 23.8% of the residents; p<0.001. Most of the doctors, 63.6%, participated in the writing of ⩽5 articles. Similar to the reading pattern, only 21.1% of the residents wrote ⩾6 articles compared with 44.0% of the specialists; p<0.001. The Spearman correlation value between reading and writing variables was 0.35; p<0.001

Conclusion

The analysis of reading and publishing habits among the study participants reveals important insights into the professional engagement of doctors with medical literature. The differences between specialists and residents, along with the positive correlation between reading and writing, underscore the need for targeted educational initiatives to enhance research literacy and foster a culture of inquiry within the medical community. Encouraging both reading and writing can contribute to the overall quality of medical practice and the advancement of evidence-based medicine.

Figure 1 

The figure describes the average of correct answers to 10 questions in understanding different aspects of basic research methods. Two populations of doctors are compared: those who graduated in the former Soviet Union (Eastern type of education) and those who graduated in Israel, USA, Western and Central Europe, and South America (Western type of education). RCT, randomised controlled trial; CI, confidence interval; ANOVA, analysis of variance.

Highest correct answer rate: 84.9% for questions on randomized control trials.

Lowest correct answer rate: 24.7% for questions requiring knowledge of analysis of variance methods.

The median total score for the test was 4 out of 10 (Interquartile Range (IQR) 2–6), indicating a generally low level of knowledge among participants.

Scores on the five questions related to epidemiological methods: Median: 3 (IQR 2–4)

Scores on the five questions related to statistical methods: Median: 2 (IQR 1–3)

Graduates from non-Soviet schools had a higher total score:

Non-Soviet graduates: Median 3 (IQR 2–4)

Soviet graduates: Median 2 (IQR 1–3)

This difference was statistically significant (p < 0.001).

Similarly, reading the discussion section was associated with higher scores:

•Median score for readers: 5 (IQR 3–6)

•Median score for non-readers: 4 (IQR 2–5)

This difference was statistically significant (p = 0.03)

Conclusion

The overall median score of 4 out of 10 indicates a significant gap in knowledge regarding epidemiological and statistical methods among the participants. This suggests a need for enhanced training and education in these areas.

The better performance on epidemiological questions compared to statistical questions may reflect differences in the training empha

sis or familiarity with these topics among the participants. This could inform curriculum development to address specific weaknesses in statistical knowledge.