A study on modelling performance in readiness review process and deep learning for automatic project effort estimation

This research comprises of two separate papers about the enhancement of software process management. In the first work, the author proposes a so-called Reengineering Readiness Review Process Performance (R3P2) model, in which a set of performance metrics and an integrated toolkit, namely the PPM Suite, are equipped to solve the problem of the complex and laborious nature of an evaluation of an organization's CMMI practice. The model contains (1) a document management tool for reusable shared papers, (2) a project management mechanism to assist mappingbetween documents and CMMI principles, and (3) a set of metrics to standardize process improvement by offering a common understanding of how to define quality. The R3P2 model is evaluated using actual assessments of 308 case studies collectedfrom 28 nations. The model performance interpretation enabled a holistic analysis of the score collected during the evaluation process for four primary metrics: maturity level delivery, customer satisfaction, cycle time delay, and assessment gap; and four secondary metrics: customer expectation, cycle time delay variability, readiness review process capacity, and gap variability. The outcome revealed that the performance of R3P2 was promising in the majority of criteria.Using the data set produced from the first work, the author presents a deep learning approach to estimate the project effort, which reflects the cost, in the secondwork. This dataset includes the 1,573 pairs of a software project description written in English and its actual man-month, gathered from the 308 companies for CMMI appraisal. Unfortunately, the dataset is not publicly available due to the privacy issueof undisclosable clients’ confidential information. For benchmarking of the model performance, the author purchased the additional datasets of 9,100 software projects from ISBSG1. The data was collected from several leading IT companies around the world. Therefore, the author's approach was evaluated using these two datasets. The total number of projects is 10,673 software projects. The data of each project is a pair of a raw project description and a structured data table explaining the character of a software project. The long-short-term memory (LSTM) sequence model is applied as the engine to estimate the project effort. In comparing four architectures; (1) averageword-vector with Multi-layer Perceptron (MLP), (2) average word-vector with Support Vector Regression (SVR), (3) Gated Recurrent Unit (GRU) sequence model, and (4) Long short-term memory (LSTM) sequence model in terms of man-months difference,the LSTM sequence model achieves the lowest and the second-lowest mean absolute errors, which are 0.705 and 14.077 man-months for 9-LAs’ datasets and ISBSGdatasets, respectively. The MLP model achieves the lowest mean absolute errors, which are 14.069 for ISBSG datasets.