Title 14 of the Code of Federal Regulations (CFR) Part 141 flight training organizations are actively pursuing ways to increase operational safety by introducing advanced risk assessment and decision-making techniques. The purpose of the dissertation was to create and validate a safety performance decision-making tool to transform a reactive safety model into a predictive, safety performance decision-making tool, specific to large, collegiate Title 14 CFR Part 141 flight training organizations, to increase safety and aid in operational decision-making. The validated safety decision-making tool uses what-if scenarios to assess how changes to the controllable input variables impact the overall level of operational risk within an organization’s flight department.
Project Details
Utilizing SPIs determined to be most indicative of flight risk within large, collegiate flight training organizations, a predictive, safety performance decision-making tool was developed utilizing Monte Carlo simulation. In a high-risk system beset with uncertainty, applying Monte Carlo simulation addresses the need to accommodate uncontrollable inputs into the model in a manner that enables the model to produce meaningful output data. This research utilizes the validated equations drawn from the non-statistical model developed by Anderson, Aguiar, Truong, Friend, Williams, & Dickson (2020) for the mathematical inputs driving the computational nodes, including the SPIs, as the foundation to develop the safety performance decision-making tool.
The probability distributions of the uncontrollable inputs were drawn from a sample of operational data from September 2017 to September 2019 from a large, collegiate 14 CFR Part 141 flight training organization in the southeastern United States. The study conducted simulation runs based on true operational ranges to simulate the operating conditions possible within large, collegiate CFR Part 141 flight training organizations with varying levels of controllable resources including personnel (Aviation Maintenance Technicians and Instructor Pilots) and expenditures (active flight students and available aircraft).
The study compared the output from three different Verification Scenarios—each using a unique seed value to ensure a different sample of random numbers for the uncontrollable inputs. ANOVA testing indicated no significant differences appeared among the three different groups, indicating the results are statistically reliable.
Four What-if Scenarios were conducted by manipulating the controllable inputs. Mean probability was the key output and represents the forecasted level of operational risk on a standardized 0-5 risk scale for the Flight Score, Maintenance Score, Damage and Related Impact, and an Overall Risk Score. Results indicate the lowest Overall Risk Score occurred when the level of personnel was high yet expenditures were moderate.
Changes to the controllable inputs are reflected by variations to the outputs demonstrating the utility and potential for the safety performance decision-making tool. The outputs could be utilized by safety personnel and administrators to make more informed safety-related decisions without expending unnecessary resources. The model could be adapted for use in any CFR Part 141 flight training organization with data collection capabilities and an SMS by modifying the input value probability distributions to reflect the operating conditions of the selected 14 CFR Part 141 flight training organization.
Research Team
CO-Investigators
