Marlies Schillings - PhD project data for study 3

<i>Title</i>: Peer-to-peer dialogue about teachers’ written feedback enhances students’ understanding on how to improve writing skills<br><br>

<i>A short description of the study set-up</i>: Second-year university students (N=84) participated in a mixed-method study that included questionnaires and focus groups. The intervention comprised face-to-face dialogue in small groups about the participants’ written peer feedback on a draft report.

<br><br><i>Instruments</i><br><br>
<i>Questionnaires</i><br>
A pre-intervention questionnaire before the start of the face-to-face dialogue measured students’ beliefs about written peer feedback (part 1). For this purpose, a validated questionnaire by Huisman, Saab, van Driel, et al. (2019) was used to measure four components: 1) degree to which peer feedback is perceived as meaningful and useful (3 items), 2) the degree to which peer feedback is considered an important skill (3 items), 3) confidence in quality of provided peer feedback (2 items) and 4) confidence in quality of received peer feedback (2 items). A five-point Likert scale was employed, ranging from 1 (=‘Completely disagree’ or ‘Completely not applicable to me’) to 5 (=‘Completely agree’ or ‘Completely applicable to me’). In part 2, students rated the presence of written peer feedback in terms of feed-up, feed-back and feed-forward information for which an adjusted version of a validated questionnaire by De Kleijn, Bronkhorst, Meijer, Pilot, and Brekelmans (2016) was used. This part of the questionnaire was also on a five-point Likert scale, ranging from 1 (=‘Agree not at all’) to 5 (=‘Agree a lot’) and contained four items about Feed up, six items about Feed back and five items about Feed forward. The pre-intervention questionnaire also measured the overall instructiveness of the written feedback on a 10-point scale (1=lowest, 10=highest). A post-intervention questionnaire measured students’ perception of improved understanding of the written feedback through face-to-face peer dialogue and the quality of this dialogue in terms of overall instructiveness, which was measured on a 10-point scale. The post-intervention questionnaire also contained items about Feed up (4 items), Feed back (6 items) and Feed forward (5 items). As in the pre-intervention questionnaire, these items were answered on a five-point Likert scale. A pilot study was conducted to test clarity of both pre- and post-intervention questionnaires items.<br><br>

<i>Focus group</i><br>
Students were invited to participate in a focus group, which resulted in two groups of volunteers: N=9 (3 males, 6 females) and N=7 (4 males, 3 females). The participants all originated from different dialogue groups. Semi-structured, post-measurement interviews were conducted to search for explanations as to why dialogue improved students’ understanding and to distinguish important conditions for better understanding. The focus group sessions lasted one hour and were guided by a moderator (first author) while a second member of the research team (fourth author) acted as observer. The moderator and observer did not know the focus group members. Both interviews were audiotaped.<br><br>

<i>Analysis</i><br><br>
<i>Quantitative analysis</i><br>
Reliability analysis was performed for each subscale of ‘student beliefs’, as well as for Feed up, Feed back and Feed forward. The reliability of the subscales varied from 0.72 to 0.85, which was considered acceptable (Tavakol & Dennick, 2011).<br>
For all pre- and post-intervention variables, the median (Mdn) and interquartile range (IQR) was calculated, besides mean (M) and standard deviation (SD). The authors considered a median score equal or above 4.0 (scale 1–5) or 8.0 (scale 1–10) as very positive. A median score equal or below 3.0 (scale 1–5) or 6.0 (scale 1–10) was considered insufficient, while all the other scores were considered to be positive. A non-parametric Wilcoxon signed-rank test was performed to compare scores on ‘Instructiveness of written feedback’ and ‘Instructiveness of face-to-face dialogue’. Non-parametric Wilcoxon signed-rank tests were also performed to compare scores on pre- and post-intervention subscales of Feed up, Feed back and Feed forward. All tests were performed on the 5% level of significance.<br><br>
<i>Qualitative analysis</i><br>
Both focus groups sessions were transcribed verbatim and two authors (moderator and observer) first analysed the transcripts in a theoretically thematic way (Braun & Clarke, 2006). This method involves deductive or top-down analysis, led by the research questions. Making an inventory of phrases related to the explanations and conditions for an improved understanding by dialogue led the analysis of the transcripts. To this end, in the first phase of the analysis and in an iterative process of three separate rounds, both authors formulated a set of themes comprising explanations and conditions. In the next phase, all authors discussed the formulated themes and reached consensus through discussion.<br><br>

<i>Explanation of the data files: what data is stored in what file?</i><br><br>
The data files contain 84 anonymized pdf’s of the original questionnaires filled in by the participants; Data file FINAL.xlsx<br><br>
Instruction focus group interview; 2x audio files of focus groups (total 6 files); transcripts of both focus groups; <br><br>

<i>In case of quantitative data: meaning and ranges or codings of all columns: meaning and ranges or codings of all columns</i><br><br>
84 Original questionnaires (pdf’s)<br><br>
Data file FINAL (in Excel) + quantitative analysis study 3 (in Word)
<br><br>
<i>In case of qualitative data: description of the structure of the data files.</i><br><br>
Opzet focus groep gesprek (in pdf);<br><br>
3x FG1 voice recorder (A B C) (MP3), 3x FG2 voice recorder (A B C) (in MP3);<br><br>
2 transcripts of focus groups (in Word);