The Pentagon’s push to develop and scale artificial intelligence capabilities is lacking a coherent strategy and its new Joint Artificial Intelligence Center (JAIC) will require greater authorities and resources to meet objectives, according to a new RAND report commissioned by the Pentagon.
The report calls for JAIC and each of the service’s AI offices to develop five-year strategic roadmaps for AI initiatives and improve data collection, while also establishing more collaborative opportunities for industry and academia to participate in future projects.
“Although we see some positive signs, our assessment is that DoD’s posture in AI is significantly challenged across all dimensions of our assessment,” write the report’s authors. “DoD faces multiple challenges in data, including the lack of data. When data do exist, impediments to their use include lack of traceability, understandability, access, and interoperability of data collected by different systems.”
The RAND report is the result of a directive in the FY ’19 defense authorization bill for the DoD’s senior AI official to complete a study on the department’s AI posture.
While the Pentagon has released an AI strategy, the report notes the department “lacks baselines and metrics to meaningfully assess progress toward its vision.” The authors write this extends to the services, with many of their individual AI organization lacking defined mandates and authorities.
“Scaling enterprise AI will not be an easy task. Indeed, it will require careful strategy and execution coupled with significant investments in infrastructure and enablers,” the report states. “The current state of AI verification, validation, testing, and evaluation (VVT&E) is nowhere close to ensuring the performance and safety of AI applications, particularly where safety-critical systems are concerned.”
The report recommends JAIC leadership, in partnership with senior DoD acquisition and technology development officials, conduct annual portfolio reviews of department-wide AI investments, and work to ensure improved VVT&E requirements are included in all projects.
The Pentagon’s efforts to bring in industry to work on AI would also be improved with increased transparency on the range of projects, while also opening up data to create more effective algorithms, according to the report.
“DoD should recognize data as critical resources, continue instituting practices for their collection and curation, and increase sharing while resolving issues in protecting the data after sharing and during analysis and use,” RAND authors wrote in the report. “The chief data officer should make a selection of DoD data sets available to the AI community to spur innovation and enhance external engagement with DoD. DoD should embrace permeability, and an appropriate level of openness, as a means of enhancing DoD’s access to AI talent.”