PSCR+ interrater agreement testing scores

This Readme for PSCR+ data.txt file was generated on 2024-01-05 by Bryce Hughes. GENERAL INFORMATION 1. Title of Dataset: PSCR+ Interrater Agreement Testing Scores 2. Author Information A. Principal Investigator Contact Information Shannon Willoughby Montana State University B. Associate or Co-investigator Contact Information Bryce Hughes Montana State University 3. Date of data collection: 2022-02-15 4. Geographic location of data collection: Utah Valley University, Orem, UT 5. This material is based upon work supported by the National Science Foundation under Grant No. 1735124. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. SHARING/ACCESS INFORMATION 1. This data is not subject to any license requirements or other restrictions. 2. Was data derived from another source? no 3. Recommended citation for this dataset Willoughby, S., Hughes, B. E., Blevins, M., & Sterman, L. (2023). PSCR+ Interrater Agreement Testing Scores. Dryad. DATA & FILE OVERVIEW 1. File List Readme for PSCR+ data.txt PSCR+ dataset for analysis.csv 2. Are there multiple versions of the dataset? no METHODOLOGICAL INFORMATION 1. Description of methods used for collection/generation of data The PSCR+ rubric was designed through a review of the literature and expert review. The original PSCR did not have many items for assessing nonverbal communication, so the PSCR+ was designed to add these items for assessment. The authors then tested the rubric by viewing four example talks produced in an introductory public speaking course at Utah Valley University. They used the PSCR+ to assess these talks. The four raters' scores were incorporated into a single dataset to analyze interrater agreement for each talk viewed. 2. Methods for processing the data The only processing of the raw data was arranging the data in Excel in a "long" format so that each row reflected each rubric item, nested by rater, which was needed to run the interrater agreement code in Stata. In Stata, some data needed to be converted from string to numeric format, and the ira program does not handle 0 as a rating option. For each rating, 1 was added to the score to shift the scale from 0-4 to 1-5. 3. Instrument- or software-specific information needed to interpret the data Data were analyzed using Stata version 15.1 with ira package installed. Code: ssc install ira Example code for analysis: ira rater talk_1, item(item_num) options(5) 4. Environmental/experimental conditions Student talks given in an introductory public speaking course. Students had signed waivers to film and make their talks publicly available online, and these online recordings were viewed by the raters for the purpose of testing the rubric. 5. People involved with sample collection, processing, analysis and/or submission Beyond the people named above, Brock LaMeres and Leila Sterman, at Montana State University, and Maria Blevins, Utah Valley University, served as raters. Shannon Willoughby was the fourth rater, and Bryce Hughes analyzed the dataset. DATA-SPECIFIC INFORMATION FOR PSCR+ dataset for analysis.csv 1. Number of variables: 7 2. Number of rows: 80 3. Variable List itemdesc: String variable; a brief description of each rubric criterion item_num: Numeric variable; a number corresponding to each rubric criterion rater: String variable; a letter corresponding to the rater talk_n: Value assigned to speech n viewed for the rubric criterion by the rater 0 - deficient 1 - minimal 2 - basic 3 - proficient 4 - advanced There are 4 talks total that were rated (talk_1 through talk_4) 4. Missing data codes There are no missing data for this dataset. CREDITS Based on a template by Cornell University Research Data Management Service Group: