The Defense Department won’t be able to make smart decisions about how much money it needs to produce a certain level of combat readiness until it begins collecting the right data in a scientific way, Center for Strategic and Budgetary Assessments senior fellow Todd Harrison said Tuesday.
In discussing his new report, “Rethinking Readiness,” Harrison told congressional staffers and others in his Capitol Hill presentation that the Pentagon has struggled to secure an adequate funding level for training and other line items that create “readiness” in part because the department has no measurable outputs it can report to lawmakers. Military officials have warned that a certain level decrease in funding would lead to a corresponding decrease in flying hours, for example, but they have not been able to go the next step and say that a certain decrease in flying hours would lead to a certain reduction in successful missile shots, or an increase in mishaps.
“If the military wants to make the case that cutting readiness inputs is actually affecting the readiness output of the force, that our men and women cannot do their jobs as effectively, show the data. Show the connection,” Harrison said. “I don’t think DoD has done that, and that’s why it’s such a weak argument on the Hill.”
What Harrison proposed instead is to carry out controlled experiments to gain data on what exactly happens when readiness inputs are altered–which would allow the Pentagon to make smarter decisions on how to balance limited funding among readiness, acquisition and personnel accounts.
First, the Pentagon would need to dive into its defense strategy and identify what readiness means within that strategy–and based on that definition of readiness, find “quantifiable performance measures for every unit according to the mission-essential tasks assigned to them.”
In many cases, Harrison said, the services already collect data like this during training exercises, but the data is not properly aggregated and analyzed to help narrow in on the relationship between resources put in and readiness achieved.
After having a baseline picture of readiness inputs and outputs, Pentagon officials could begin making hypotheses about the relationships between the two and identifying controlled experiments to run to help pinpoint how readiness changes based on fewer personnel in a unit, or fewer training hours, or substituting a certain amount of live training for simulated training.
And this process needs to continue indefinitely, he said. The balance of factors that lead to readiness are always changing, as new technologies become available, global conditions change and people join and leave the military. Current readiness funding practices are based on long-held assumption about the impact of resources on readiness, Harrison argued, whereas there should be a continuous feedback loop constantly informing decisions.
“Our current method for resourcing readiness starts with the wrong metrics, lacks experimental data to isolate causal effects, and does not have a continuous feedback loop,” he said. Harrison recommended that the Pentagon develop better metrics, that Congress require DoD to include actual outputs in their quarterly readiness reports instead of the customary flying hours and other inputs, and that the services begin conducting controlled experiments soon.
“This is not easy–it’s not just a cultural shift for the military, but this is a hard thing to do,” he said. “It requires resources, and resources are being diminished, but there are some opportunities in there as well. For example, if we know we’re not going to have enough flying hours for all of our fighter squadrons, instead of just allocating the cuts by saying, well this unit is already at a lower tier of readiness so let’s go ahead and drop them even more…instead of doing it that way, let’s actually use this as an opportunity to say, all right, let’s run a controlled experiment. I’m going to randomly select some units that will have a decrease in readiness inputs, and we’ll try decreasing different types of readiness inputs, and decreasing to different levels, and we’ll set aside a control group as well that we don’t touch and we maintain our current levels. We’ve got an opportunity to do this right now and actually use the downturn and the cuts in readiness funding as an opportunity to generate better data and a better understanding of readiness and how we can resource it in the future.”