Research Software at the University of Illinois Urbana-Champaign: A Mixed Methods Survey Dataset

Description The research employed a mixed methods online survey to understand better the meaning, use, and development of academic research software at the University of Illinois Urbana-Champaign. Other objectives include understanding academic research software support and training needs to make projects successful at Illinois, as well as investigating the use of generative AI tools in using and creating research software. At the beginning of the survey, all participants gave informed consent. The University of Illinois Urbana-Champaign Institutional Review Board (IRB Protocol no.: Project IRB24-0989) reviewed the study and gave it an exempt determination. Data collection took place from August 2024 to October 2024. Prior to data analysis, identifiable respondent details were removed during the data cleaning process. Not Applicable and Unsure style responses were used for descriptive statistics, but these responses were excluded for inferential statistics. Survey design At the beginning of the online survey, a consent form was provided based on guidelines from the University of Illinois Institutional Review Board to the respondents stating the aims of the study, its benefits and risks, ethical guidelines, being a voluntary survey for participation and withdrawal, privacy and confidentiality, data security, estimated time for survey completion, and contact information of researchers for asking questions. Respondents clicked to indicate their consent. Survey questions were divided into four parts: demographic information, using software for research, creating software for research, and the protocol of citing software for research. The survey had to stop points, whereby not all questions applied to respondents, which led to different sample sizes at the stop points. At the opening of the survey, the number of respondents was 251 with the funding demographic question being answered by all respondents, while other demographic questions had between 225 and 228 respondents answering them. For the first stop question, using research software in their research, the total respondents was 212, and at the last stop question, respondents considering themselves to be research developers, the total number of respondents was 74. The last question of the survey was answered by 71 respondents. Respondents may also have left the survey for other reasons. The questions were primarily closed-type questions with single choice, multiple choice, or Likert scale, as well as a few open-ended questions. Likert scale responses were created utilizing validated scales from Vagias' (2006) Likert Type Scale Response Anchors. Sampling Survey Respondents’ Demographics While most respondents were Tenure Track Faculty (34.7%, f=227), other key categories included Principal Investigator (22.4%, f=227) and Research Scientist (12.1%, f=227). Computer Science, Information Science, Mathematics, and Engineering fields combined for 16% (f=228) of the respondents surveyed, but it should be noted the remaining respondents were from various academic fields across campus from various arts, humanities, and social science fields (25%, f=228) to agriculture (10%, f=228), education (5%, f=228), economics (3%, f=228), medical sciences (4%, f=228), and politics and policy/law (1%, f=228). Most respondents were likely to receive funding from various government agencies. A more detailed breakdown of the demographic information can be found in the supplemental figures. Of the 74 respondents who answered whether they were a research software developer, most respondents did not consider themselves a research software developer, with respondents stating Not at All (39%, n=74) and Slightly (22%, n=74). In addition, open-ended questions asked for further detail about research software titles used in research, research software developer challenges, how generative AI assisted in creating research software, and how research software is preserved (e.g., reproducibility). Table 1: Survey Respondents’ Demographics Characteristics Respondent (%) Age 18-24 25-34 35-44 45-54 55-64 Over 64 Preferred Not Answer 3% 14% 33% 27% 14% 7% 2% Gender Woman Man Non-binary / non-conforming Prefer not to answer 49% 44% 2% 4% Race Asian Black or African American Hispanic or Latino Middle Eastern or North African (MENA; new) White Prefer not to answer Other 12% 5% 6% 1% 67% 8% 1% Highest Degree Bachelors Masters Professional degree (e.g., J.D.) Doctorate 6% 19% 5% 70% Professional Title Tenure Track Faculty Principal Investigator Research Scientist Staff Research Faculty Other Teaching Faculty Postdoc Research Assistant Research Software Engineer 35% 22% 12% 8% 7% 4% 4% 4% 2% 2% Academic Field Biological Sciences Other Agriculture Engineering Psychology Earth Sciences Physical Sciences Education Medical & Health Sciences Computer Science Library Chemical Sciences Human Society Economics Information Science Environment Veterinary Mathematical Sciences History Architecture Politics and Policy Law 18% 10% 10% 9% 8% 6% 6% 5% 4%3% 3% 3% 3% 3% 2% 2% 2% 2% 1% 1% 1% 0% Years Since Last Degree Less than 1 Year 1-2 Years 3-5 Years 6-9 Years 10-15 Years More than 15 Years 4% 8% 11% 14% 24% 40% Receive Funding Yes No 73% 27% Funders for Research Other National Science Foundation (NSF) United States Department of Agriculture (USDA) National Institute of Health (NIH) Department of Energy (DOE) Department of Defense (DOD) Environmental Protection Agency (EPA) National Aeronautics and Space Administration (NASA) Bill and Melinda Gates Foundation Advanced Research Projects Agency - Energy (ARPA-E) Institute of Education Sciences Alfred P. Sloan Foundation W.M. Keck Foundation Simons Foundation Gordon and Betty Moore Foundation Department of Justice (DOJ) National Endowment for the Humanities (NEH) Congressionally Directed Medical Research Programs (CDMRP) Andrew W. Mellon Foundation 22% 18% 18% 11% 9% 5% 4% 4% 2% 2% 1% 1% 1% 1% 1% 1% 0% 0% 0% Table 2: Survey Codebook QuestionID Variable Variable Label Survey Item Response Options 1 age Respondent’s Age Section Header: Demographics Thank you for your participation in this survey today! Before you begin to answer questions about academic research software, please answer a few demographic questions to better contextualize your responses to other survey questions. What is your age? Select one choice. Years 1-Under 18 2-18-24 3-25-34 4-35-44 5-45-54 6-55-64 7-Over 64 8-Prefer not to answer 2 gender Respondent’s Gender What is your gender? Select one choice. 1-Female 2-Male 3-Transgender 4-Non-binary / non-conforming 5-Prefer not to answer 6-Other: 3 race Respondent’s Race What is your race? Select one choice. 1-American Indian or Alaska Native 2-Asian 3-Black or African American 4-Hispanic or Latino 5-Middle Eastern or North African (MENA; new) 6-Native Hawaiian or Pacific Islander 7-White 8-Prefer not to answer 9-Other: 4 highest_degree Respondent’s Highest Degree What is the highest degree you have completed? Select one choice. 1-None 2-High school 3-Associate 4-Bachelor's 5-Master's 6-Professional degree (e.g., J.D.) 7-Doctorate 8-Other: 5 professional_title Respondent’s Professional Title What is your professional title? Select all that apply. 1-professional_title___1 Principal Investigator 2-professional_title___2 Tenure Track Faculty 3-professional_title___3 Teaching Faculty 4-professional_title___4 Research Faculty 5-professional_title___5 Research Scientist 6-professional_title___6 Research Software Engineer 7-professional_title___7 Staff 8-professional_title___8 Postdoc 9-professional_title___9 Research Assistant 10-professional_title___10 Other: 6 academic_field Respondent’s most strongly identified Academic Field What is the academic field or discipline you most strongly identify with (e.g., Psychology, Computer Science)? Select one choice. 1-Chemical sciences 2-Biological sciences 3-Medical & health sciences 4-Physical sciences 5-Mathematical sciences 6-Earth sciences 7-Agriculture 8-Veterinary 9-Environment 10-Psychology 11-Law 12-Philosophy 13-Economics 14-Human society 15-Journalism 16-Library 17-Education 18-Art & Design Management 19-Engineering 20-Language 21-History 22-Politics and policy 23-Architecture 24-Computer Science 25-Information science 26-Other: 7 years_since_last_degree Number of years since last respondent’s last degree How many years since the award of your last completed degree? Select one choice. 1-Less than 1 year 2-1-2 years 3-3-5 years 4-6-9 years 5-10-15 years 6-More than 15 years 8 receive_funding_for_research Whether respondent received funding for research Do you receive funding for your research? 1-Yes 0-No 9 funders_for_research Respondent’s funding sources if they answered yes in Question 8 Who funds your research or work (e.g., NIH, Gates Foundation)? Select all that apply. 1-funders_for_research___1 United States Department of Agriculture (USDA) 2-funders_for_research___2 Department of Energy (DOE) 3-funders_for_research___3 National Science Foundation(NSF) 4-funders_for_research___4 National Aeronautics and Space Administration (NASA) 5-funders_for_research___5 National Institutes of Health(NIH) 6-funders_for_research___6 Department of Justice (DOJ) 7-funders_for_research___7 Environmental Protection Agency (EPA) 8-funders_for_research___8 Department of Defense(DOD) 9-funders_for_research___9 Institute of EducationSciences 10-funders_for_research___10 Congressionally DirectedMedical Research Programs(CDMRP) 11-funders_for_research___11 Advanced Research ProjectsAgency - Energy (ARPA-E) 12-funders_for_research___12 National Endowment for theArts (NEA) 13-funders_for_research___13 National Endowment for theHumanities (NEH) 14-funders_for_research___14 Social Science ResearchCouncil (SSRC) 15-funders_for_research___15 Alfred P. Sloan Foundation 16-funders_for_research___16 Gordon and Betty MooreFoundation 17-funders_for_research___17 Andrew W. MellonFoundation 18-funders_for_research___18 Bill & Melinda GatesFoundation 19-funders_for_research___19 Simons Foundation 20-funders_for_research___20 W. M. Keck Foundation 21-funders_for_research___21 Other: 10 how_software_is_used How respondent uses software in research Section Header: Section 1: Research Software Definition How do you use software in your research? Select all that apply. 1-how_software_is_used___1 I do not use software in my research 2-how_software_is_used___2 To automate part of the research process 3-how_software_is_used___3 To collect data 4-how_software_is_used___4 To generate data 5-how_software_is_used___5 To simulate/model something 6-how_software_is_used___6 To clean/organize data 7-how_software_is_used___7 To analyze data 8-how_software_is_used___8 To present research results(e.g., visualize data) 9-how_software_is_used___9 Other: 11 use_research_software Respondent Uses Research Software Based Upon Definition Provided "Research Software includes source code files, algorithms, scripts,computational workflows and executables that were created during the research process or for a research purpose. Software components(e.g., operating systems, libraries, dependencies, packages, scripts,etc.) that are used for research but were not created during or with a clear research intent should be considered software in research and not Research Software. This differentiation may vary between disciplines. The minimal requirement for achieving computational reproducibility is that all the computational components (ResearchSoftware, software used in research, documentation and hardware)used during the research are identified, described, and made accessible to the extent that is possible." With this definition in mind, do you use research software? 1-Yes 0-No 2-Unsure Stop actions on 0 12 software_importanc Research Software Importance How important is research software to your research? Select one choice. 1-Not at all, 2-Slightly, 3-Somewhat, 4-Moderately, 5-Extremely, 6-Not Applicable 13 research_roles Respondent’s Role in Research Which of the following roles apply to you? Select all that apply. 1-research_roles___1 I conduct my own research 2-research_roles___2 I support someone else's research 3-research_roles___3 I manage/supervise researchers or research projects 4-research_roles___4 I develop software for researchers 5-research_roles___5 Other: 14 research_software_titles Research Software Used by Respondents in Research Please provide the name(s) of the main research software you use(up to 5). Separated by comma. Open-ended Response 15 gui Respondent’s use of graphical user interface in research How often is software that has a graphical user interface (GUI)critical to successfully complete your research/engineering/etc.project? "A graphical user interface (GUI) is a digital interface in which a user interacts with graphical components such as icons, buttons, and menus. In a GUI, the visuals displayed in the user interface convey information relevant to the user, as well as actions that they can take." (Juliver, 2023) Select one choice. 1-Never 2-Rarely 3-Sometimes 4-Often 5-Always 16 operating_system_used Operating System Used by Respondent for Research What operating system(s) do you use for your research? Select one or more of the following responses. If other, type the operating system(s). 1-operating_system_used___1 Microsoft Windows 2-operating_system_used___2 Mac OS X 3-operating_system_used___3 Linux 4-operating_system_used___4 Other: 17 if_no_research_software How would respondent complete research without research software What would happen if you could no longer use research software? Select one choice. 1-I would no longer be able to conduct my research 2-My research would require more effort, but it would still be possible 3-It would make no significant difference to my research 18 used_gen_ai_tool Does respondent use GenAI in Research Have you used a generative AI tool for your research? Select one choice. 1-Yes 0-No 19 how_genai_used_in_research Ways in which Respondent uses GenAI in Research How have you used generative AI tools in your research? Select all that apply. 1-how_genai_used_in_research___1 To automate part of the research process 2-how_genai_used_in_research___2 To collect data 3-how_genai_used_in_research___3 To analyze data 4-how_genai_used_in_research___4 To clean/organize data 5-how_genai_used_in_research___5 To visualize data 6-how_genai_used_in_research___6 To improve text 7-how_genai_used_in_research___7 To write new text 8-how_genai_used_in_research___8 To write code 9- how_genai_used_in_research___9 To determine research questions 10-how_genai_used_in_research___10 To gather relevant information 11-how_genai_used_in_research___11 Other: 20 how_to_locate_software_used Ways in which Respondent locates research software How do you locate software to use for your research? Select all that apply. 1-how_to_locate_software_used___1 Conversations with collaborators 2-how_to_locate_software_used___2 Reading the scholarly literature 3-how_to_locate_software_used___3 Using a search engine (e.g. Google) 4-how_to_locate_software_used___4 Using social media(e.g. Twitter/X) 5-how_to_locate_software_used___5 Asking a generativeAI tool 6-how_to_locate_software_used___6 Searching online communities (e.g.Stack Overfl ow) 7-how_to_locate_software_used___7 Searching code hosting sites (e.g.GitHub) 8-how_to_locate_software_used___8 Writing it myself 9-how_to_locate_software_used___9 Other: 10-how_to_locate_software_used___10 Not applicable 21 contribute_to_exist_software Whether respondent contributes to existing software Do you contribute to existing software (e.g., by reporting bugs at least)? Select one choice. 1-Yes 0-No 22 cost_software_development Whether respondent included software development costs in research proposal Have you included costs for software development in a funding proposal? Select all that apply. 1-cost_software_development___1 Yes 2-cost_software_development___2 No (I'm not involved in writing funding proposals) 3-cost_software_development___3 No (but we DID plan to write software as part of the project) 4-cost_software_development___4 No (we did NOT plan to write software as part of the project) 23 work_on_creating_software Whether respondent creates software for research Do you currently work on a project that involves the creation or maintenance of research software? Select one choice. 1-Yes 0-No Stop actions on 0 24 software_develop_knowledge Respondent’s knowledge of research software development How knowledgeable are you about research software development? Select one choice. 1 - Not at all 2 - Slightly 3 - Somewhat 4 - Moderately 5 - Extremely 25 software_develop_training Whether respondent received software development training Have you received any training in software development? Select one choice. 1-Yes - just through taught courses 2-Yes - just through self-taught training 3-Yes - through both taught courses and self-taught training 4-No 26 amt_software_develop_train Respondent’s amount of software development training How much training have you received to develop reliable software? Select one choice. 1-None, 2-Almost none, 3-A small amount, 4-A moderate amount, 5-A great deal 27 version_control_familar Respondent’s familiarity with version control Section Header: How familiar are you with the following software development technologies? Version Control 1 - Not at all 2 - Slightly 3 - Somewhat 4 - Moderately 5 - Extremely 6 - Not Applicable 28 cont_integration_familiar Respondent’s familiarity with continuous integration Continuous Integration 1 - Not at all 2 - Slightly 3 - Somewhat 4 - Moderately 5 - Extremely 6 - Not Applicable 29 version_control_use Respondent’s frequency of version control use in research Section Header: How often do you use the following software development technologies? Select one choice. Version Control 1-Never, 2-Rarely, 3-Sometimes, 4-Often, 5-Always Show the field ONLY if: [version_control_familar] = '2' or [version_control_familar] = '3' or [version_control_familar] = '4' or [version_control_familar] = '5' 30 cont_integration_use Respondent’s frequency of continuous integration use in research Continuous Integration 1 - Never 2 - Rarely 3 - Sometimes 4 - Often 5 - Always Show the field ONLY if: [cont_integration_familiar] = '2' or [cont_integration_familiar] = '3' or [cont_integration_familiar] = '4' or [cont_integration_familiar] = '5' 31 r_software_engineer_title Whether respondent’s professional title is software engineer Do you use the title research software engineer to define yourself? Select one choice. 1-Yes 0-No 32 use_univ_support_services Whether respondent has used university support services for research software development Have you used university support services for your software development needs? Select one choice. 1-Yes 0-No 2-Unsure 33 rate_univ_support_services Respondent’s rating of university support services for research software development How would you rate the university's current level of support for your software development needs? Select one choice. 1-Poor, 2-Fair, 3-Good, 4-Very Good, 5-Excellent Show the field ONLY if: [use_univ_support_services] = '1' or[use_univ_support_services] = '2' 34 software_develop_challenges Whether respondent has had software development challenges Have you had any specific challenges with your software development needs? 1-Yes 0-No 35 soft_develop_challenges_text Respondent's specific software development challenges What specific challenges have you had with university support for your software development needs? Open-ended response 36 use_gen_ai_in_soft_develop Whether respondent has used GenAI in research software development Have you used generative AI in your software development? Select one choice. 1-Yes 0-No 37 how_used_genai_in_soft_develop Respondent’s use of GenAI in research software development How has generative AI software assisted you in creating software? Open-ended response 38 cont_use_gen_ai_in_soft_develop Whether respondent still uses GenAI in research software development Do you plan to continue using generative AI in your software development? Select one choice. 1-Yes 0-No 39 hired_software_developer Whether respondent has hired a software developer for research project Have you (or someone in your group) ever hired someone specifically to develop software? Select one choice. 1-Yes 0-No 40 preserve_research_software_text How Respondent preserves research software How do you preserve your research software (e.g., for reproducibility)? Open-ended response 41 sustain_research_software Whether respondent sustains research software based upon definition provided The concept of 'software sustainability' means that the software continues to be available to its intended users for as long as it is likely to be useful. With this definition in mind, do you plan to sustain your research software? Select one choice. 1-Yes 0-No 42 how_will_sustain_research_soft How do you plan to sustain your research software? Select multiple choices. 1-how_will_sustain_research_soft___1 Grants that explicitly maintain the software 2-how_will_sustain_research_soft___2 Grants for research where some effort also will maintain the software 3-how_will_sustain_research_soft___3 Internal funding/effort 4-how_will_sustain_research_soft___4 Make the software open source 5-how_will_sustain_research_soft___5 Commercialize the software 6-how_will_sustain_research_soft___6 Other 43 research_soft_cont_in_hiring Whether respondent believes research software contribution is valued in university hiring Do you agree that research software contributions are valued in hiring at the University of Illinois? Select one choice. 1 - Strongly disagree 2 - Disagree 3 - Neither agree or disagree 4 - Agree 5 - Strongly agree 44 research_soft_cont_in_promotion Whether respondent believes research software contribution is valued in university promotions Do you agree that research software contributions are valued in promotion at the University of Illinois? Select one choice. 1- Strongly disagree 2 - Disagree 3 - Neither agree or disagree 4 - Agree 5 - Strongly agree 45 explicit_how_to_cite_software Respondent explicitly states how to cite research software Section Header: Section 3: Sharing Your Code and Software Do you make it explicit how to cite your software? Select one choice. 1-Yes 0-No 46 cite_software_in_research_results Respondent explicitly cites research software produced in research Do you cite the software you use to produce your research results? Select one choice. 1-Yes 0-No