A new Government Accountability Office (GAO) audit of the information the military collects about aviation mishaps found the services do not track all relevant data and each have different standards, making it hard for the Department of Defense to analyze trends.
The report said each service’s safety center does not collect standardized aviation mishap data. The centers are the Naval Safety Center, which also includes Marine Corps information, the Army Readiness Center, and the Air Force Safety Center.
Of the 35 data elements that the Office of the Secretary of Defense (OSD) identified as related to mishaps, the Naval Safety Center does the worst, collecting only 11 standardized data elements while 17 elements were collected but not standardized. The report said three were not collected, and four were not reported at all.
The Army does a little better with 14 data points standardized, 14 not standardized, three not collected, and four not reported. The Air Force did the best by standardizing 17 of the 35 data elements, but that still left 10 not standardized, one not collected, and seven not reported at all.
With so much data not standardized, the GAO said OSD has to perform time-consuming manipulation and interpretation of some data elements to create comparative analyses. Moreover, this “introduces the risk of errors in the analysis and affects the timeliness of providing critical information to decision makers,” the GAO said.
GAO performed the audit from Sept. 2017 through Aug. 2018 after the House Armed Services Committee in its version of the FY ’18 defense authorization bill directed the auditors to assess what is known about the relationship between training completed on rotary-wing aircraft and the number of mishaps.
The office expects to release a final report in early 2019, but first released this report to highlight the problems in even collecting and analyzing the data related to military aviation mishaps.
GAO said that according to department officials, the safety centers use separate, service-specific data systems that evolved independently to collect data for unique requirements in each service. This contributes to the lack of standardized data.
GAO said part of the problem is that the safety centers and OSD do not agree on what OSD’s role should be for conducting causal analysis of aviation mishaps, limiting the analysis that OSD performs. The watchdog said the lack of consensus directly led to the safety centers not reporting all of the agreed-upon data elements, including causal factors in aviation mishaps.
Importantly, none of the safety centers reported any information on human factors that contributed to mishaps, like performance-based errors. Without the underlying human factor categories, OSD officials told GAO “they have been limited in the type of analyses that they can develop.”
This means OSD has been unable to conduct analyses on patterns in human factors that may cut across individual services since they are not getting the data.
“Until there is consensus on OSD’s role, it may be unable to obtain and analyze DOD-wide mishap data,” the GAO said.
Relatedly, GAO said the department does not consistently collect relevant training data to analyze trends.
The office specifically found that some training data related to pilots’ training records are not being collected in mishap investigations. This includes things like a pilots’ recent flying experience or training proficiency in the task or mission underway during the mishap.
“Recent studies have suggested that training shortfalls are a potential indicator of trends in aviation mishaps, but additional training data would be required for further analysis,” the GAO said.
The GAO recommended the department first take interim steps to ensure standardized mishap data elements are actually collected by the safety centers. Then it said the DoD should update department-wide and service instructions and policies to clarify the responsibility of OSD in conducting analyses and giving it access to the services’ causal information on human factors.
GAO also said DoD should identify relevant training-related data to collect as part of any update of the aviation mishap data elements and incorporate them in future analyses.
The department said it concurs with the recommendations. In DoD’s comments contained in the GAO report, the department said the Safety Information Management Working Group is developing DoD data standards for mishap information and the finalized standards will be submitted to the Chief Management Officer who will enforce implementation of the standards in future safety information systems development.
It also said the working group will include training-related data in the standards and that the Deputy Secretary of Defense directed the Office of the Under Secretary for Personnel and Readiness to review data sets collected and analytics performed to ensure DoD has the most effective means to identify risks and trends, including identifying data elements that must be standardized.
The report comes after a series of aviation mishaps that have raised concerns about underlying causes, like finishing training and flight safety.
The Air Force saw a rise in manned aviation mishaps since the start of fiscal year 2018, including a WC-130H crash on May 2 (Defense Daily, May 8) and the Army’s director of safety and the combat readiness center said in June that data does not correlate between flight hours and mishaps in that service (Defense Daily, June 14).
Last week, the DoD Inspector General found Marine Corps aviation squadron commanders were not correctly reporting the number of aircraft ready for combat, causing an inaccurate accounting of readiness rates in aviation units (Defense Daily, Aug. 10).