82nd Training Wing technical training evaluation: an innovation study

Organizations rarely conduct impact-level training evaluation; thought to be costly, overly complicated in working environments or unwanted by organizational leaders. The 82d Training Wing (82 TRW) at Sheppard AFB, TX has never conducted comprehensive training evaluation and as a result, lacked the means to prove that training outcomes add value to major commands that employ graduates of aircraft maintenance training. The questions that guided this study asked if training directors within the 82 TRW had the knowledge, motivation, organizational support, and resources to design and implement impact-level evaluation and if the results would have value. The stakeholder group of focus for this study included eleven senior civilian instructional specialists with the capacity to conduct impact-level evaluation for the 82 TRW. This study applied Clark and Estes’ (2008) gap analysis to data derived from interviews with the stakeholders. The findings identified that directors possess a high degree of metacognition and motivation to design and implement impact-level evaluation, although resource distribution to support additional evaluation was a concern. There was a lack of conceptual and procedural knowledge about both general and specific training evaluation models and practices by the training directors, as well as a need for explicit process guidance. Another gap identified that directors believe negative perceptions exist for training conducted by the wing and of training in general by operational commands. Recommendations to address these gaps include training to provide knowledge of the Kirkpatrick New World Model (2016) and provide directed practice identifying leading indicators that correlate to learning objectives. The end goal is to prove that training generates measurable strategic impact.