Replication Data for "How Do Different Types of Testing Goals Affect Test Case Design?"

# Replication Data for "How Do Different Types of Testing Goals Affect Test Case Design?"

## Overview

Background: Test cases are designed in service of one or more goals, e.g., assessing functional correctness or performance. We lack a clear understanding of how specific goal types influence test design.

Aims: We explore the relationship between types of testing goals and test design, including identification and importance of goal types, quantitative relations between goal types and test cases, and personal, organizational, methodological, and technological factors that may influence this relationship.

Method: We have conducted both qualitative and quantitative analysis of interviews and a survey with software developers in various domains and of varying experience.

Results: We identify nine goal types, and focus on correctness, reliability, and quality. We observe that test design for correctness forms a "default" design process that is modified when pursuing other goals. For the examined goal types, test cases tend to be simple, with many tests targeting a single goal and each test focusing on 1-2 goals at a time. Testers often start by using past tests as templates. Testing practices, tools, and system types of interest vary between goal types. Test design can be influenced by organization, process, and team makeup.

Conclusions: This study provides a foundation for future research on test case design and testing goals.

## Data Contained in This Package

- thematic_coding.pdf

This is the theme map created from the interview data. We extracted important statements from the interviews (codes) and clustered them into themes and sub-themes.

- survey_responses.pdf

This file contains all survey responses.

Both interview and survey data has been anonymized to protect the privacy of the participants.