Data analysis Protocol for a Joint Study into the Impacts of AI on professional Competencies of IT Professionals and Implications for Computing Students. ITiCSE 2024 Working Group 02.

Overview

The purpose of this protocol is to help us define a common protocol for sharing and analysing data for the ITiCSE 2024 working group: “WG02: A Multi-Institutional-Multi-National Study into the Impacts of AI on Work Practices of IT Professionals and Implications for Computing Students”.  Excerpts from the working group plan to place the protocol in context (Clear et al., 2024) are given below.

Background and Related Work

As Artificial Intelligence (AI) continues to make its presence felt in transforming workplaces around the world [1,10], and the Information Technology industry in particular, it is essential to understand its impact on the work practices of IT professionals, and the implications for computing students and curricula. This research project builds on work initiated jointly, in Sweden, New Zealand and Scotland, investigating concerns about the increasing impacts of Artificial Intelligence in IT Sector workplaces for employee work engagement [11,13,1] and the implications for tertiary study, assessment and curricula in computing [4, 8, 10, 9]. 

“Work engagement”, has been defined as the positive inner state where employees are fully present and engaged in their work, and is closely linked to motivation, learning, productivity, and accountability [11, 13]. Within the context of (Generative) AI at work, IT professionals have been noted as early adopters of AI [10, 1]. Their involvement in implementing and utilising AI technologies can provide valuable insights into the interplay between AI and work engagement.  The implications for students are significant as future IT professionals, who must acquire and enhance competencies to adapt and thrive in digital workplaces.

2           Goals of the Working Group

By exploring the relationship between work engagement and learning, this study aims to shed light on the dynamics that drive employee engagement and its connection to the professional development of competencies. The previous study has interviewed IT professionals with the following research questions (RQ):

RQ1: How does AI influence work engagement for IT professionals?

RQ2: How does AI affect the socio-technical work dynamics for IT professionals?

RQ3: What are the implications of integrating AI on the acquisition and enhancement of professional competencies and the learning processes of IT professionals?

3           Methodology

This working group aims to analyse the corpus of interview data collected from multiple countries to better understand the implications for computing students, tertiary computing education curricula and assessment of the new professional competencies emerging from this work. This study informed by the literature on work engagement, automation and motivation for IT professionals [11, 13], will use a combination of multi-vocal literature review [7] and qualitative research methods [2, 5], including thematic analysis of the interviews, to investigate the state of the practice in and challenges IT Professionals face within their local/global work contexts. The literature on professional competencies in computing [4, 3, 6] will be drawn upon to characterise the new needs identified in this analysis.  Further implications for computing curricula design and assessment will be developed from this analysis.

REFERENCES

[1]    ACM Technology Policy Council. 2023. Principles for the development, deployment, and use of generative AI technologies, ACM New York.

[2]    Braun, V. and Clarke, V. 2021. One size fits all? What counts as quality practice in (reflexive) thematic analysis? Qualitative research in psychology, 18 (3). 328-352.

[3]    Clear, A., Clear, T., Vichare, A., Charles, T., Frezza, S., Gutica, M., Lunt, B., Maiorana, F., Pears, A. and Pitt, F. 2020. Designing Computer Science  Competency Statements: A Process and Curriculum Model for the 21st Century in Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education, ACM, New York.

[4]    Clear, A., Parrish, A. and CC2020 Task Force. 2020. Computing Curricula 2020 - CC2020 - Paradigms for Future Computing Curricula ACM and IEEE-CS eds. A Computing Curricula Series Report ACM, New York.

[5]    Cruzes, D.S. and Dyba, T. 2011. Recommended steps for thematic synthesis in software engineering. in 2011 international symposium on empirical software engineering and measurement, IEEE, 2011, 275-284.

[6]    Frezza, S., Clear, T. and Clear, A. 2020. Unpacking Dispositions in the CC2020 Computing Curriculum Overview Report in 2020 IEEE Frontiers in Education Conference (FIE), IEEE, Uppsala, Sweden.

[7]    Garousi, V., Felderer, M., & Mäntylä, M. V. 2019. Guidelines for including grey literature and conducting multivocal literature reviews in software engineering. Information and Software Technology, 106. 101-121

[8]    Jacques, L. 2023. Teaching CS-101 at the Dawn of ChatGPT. ACM Inroads, 14 (2). 40-46.

[9]    Liffiton, M., Sheese, B., Savelka, J. and Denny, P. 2023. CodeHelp: Using Large Language Models with Guardrails for Scalable Support in Programming Classes. arXiv preprint arXiv:2308.06921.

[10]  Prather, J., Denny, P., Leinonen, J., Becker, B.A., Albluwi, I., Craig, M., Keuning, H., Kiesler, N., Kohn, T. and Luxton-Reilly, A. 2023. The robots are here: Navigating the generative ai revolution in computing education. arXiv preprint arXiv:2310.00658.

[11]  Roto, V., Palanque, P. and Karvonen, H., 2019. Engaging automation at work–a literature review. in Human Work Interaction Design. Designing Engaging Automation: 5th IFIP WG 13.6 Working Conference, HWID 2018, Espoo, Finland, August 20-21, 2018, Revised Selected Papers 5, Springer, 158-172.

[12]  SFIA Foundation. 2023. SFIA skills aligned to EU ICT Profiles, SFIA Institute, London.

[13]  Sharp, H., Baddoo, N., Beecham, S., Hall, T. and Robinson, H. 2009. Models of motivation in software engineering. Information and software technology, 51 (1). 219-233.