Software Quality Grades for Seismology Software

The data provides a summary of the state of development practice for seismology software (as of August 2017). The summary is based on grading a set of 30 seismology products using a template of 56 questions based on 13 software qualities. The software qualities were further divided into 4 aspects: product, implementation, design and process.

The template used to grade the software is found in GradingTemplatedDocument.pdf file. Each quality is measured with a series of questions. For unambiguity the responses are quantified wherever possible (e.g. yes/no answers). The goal is for measures that are visible, measurable and feasible in a short time with limited domain knowledge. Unlike a comprehensive software review, this template does not grade on functionality and features. Therefore, it is possible that a relatively featureless product can outscore a feature-rich product.

A virtual machine is used to provide an optimal testing environments for each software product. During the process of grading the 30 software products, it is much easier to create a new virtual machine to test the software, rather than using the host operating system and file system.

The raw data obtained by measuring each software product is in SoftwareGrading-Seismology.xlsx. Each line in this file corresponds to between 2 and 4 hours of measurement time by a software engineer. The overall impression scores for each product are summarized in one of the tabs in AHP_seismology.xlsx spreadsheet. These overall impression numbers are then used for a relative comparison between products. The relative comparison is used to populate the AHP tables. Using the mathematics for AHP the numbers are then converted to a ranking of the 30 software products on each of the 13 qualities, and each of the 4 aspects.