By Geoff Fein
As the Navy moves toward open architecture (OA) and looks to upgrade software and hardware at lightning speed, putting those systems through the traditional operational test and evaluation process could create tension as programs are modified to get into the hands of warfighters.
But making sure those new and improved technologies give warfighters advanced capability is a role the Navy’s Operational Test & Evaluation Force (OPTEVFOR) is meeting head-on, Rear Adm. Dave Dunaway, commander OPTEVFOR (COMOPTEVFOR), told Defense Daily recently.
“In any organization as complex as the business we are working in, if you don’t change, you are going to be behind and you’ll be antiquated,” he said.
OA is like a framework that is going to take these complicated, very difficult mission threads that the Navy is building and make them agile and adaptable and responsive to threats, Dunaway said.
“It’s a tremendous concept. With that comes operational test that has to be adaptive and agile and responsive to the kinds of things that are going on,” he said. “We spend a lot of time thinking about changing our processes, looking at how we can take our true north, our operational realism, [and] apply it to the system. I think we are doing a great job in it.”
But it is not just information technology systems that are moving at such a rapid pace, Dunaway noted. “If you look at the speed at which we are trying to respond to this rapid deployment of LCS, or the rapid deployment of Fire Scout, we change our processes but never lose that essence…that operational realism and the operators’ need in the system.”
The process is “good constructive conflict. It’s meant to be that way,” he said.
Still, the system works and is effective in the long run, Dunaway added.
And knowing, for example, that a software change is going to come every 18 months helps establish testing plans, he added. “Absolutely, it’s a continuous process.”
The game plan Dunaway uses for a system, for example the Ship Self Defense System (SSDS), is to take each program that is going forward and do a mission threat analysis. In the case of SSDS, Rear Adm. Terry Benedict, program executive officer integrated warfare systems (PEO IWS), does a vary technically detailed decomposition of requirements down to very detailed specifications with real clear compliance issues, Dunaway said.
“I do the same decomposition, but I do it from an operational sense that goes to operators’ functions. I do this at the very beginning of the program,” he said. “When I lay out this mission thread of LPD-17 SSDS defending against a raid, I have that framework and structure. I know what I am going to look at, I know what’s important. I use design of experiments to focus my energies in the right areas and it’s a well established well known chin-up bar.”
Dunaway then articulates that bar and provides it to Benedict. “We talk frequently. He knows that’s the chin-up bar he has to meet.”
When changes to a system occur, such as might occur under OA, Dunaway must weigh the significance of each change on the system or platform’s functionality.
“If [Benedict] is making a minor software change that’s a maintenance load, I look at my framework and say ‘the little change he is making doesn’t make much of a difference.’ I am not going to test it, I don’t need to test it,” Dunaway said.
But if Benedict adds functionality to a system through an upgrade, Dunaway has to look at his matrix and determine if it impacts the functionality.
“This is a real-time, living, breathing collaborative relationship,” Dunaway said. “[Benedict’s] quest is to field and minimize unnecessary tests. He wants necessary tests. Mine is to make sure I have the operator’s view. With this clear understanding, we get there very easily.”
Dunaway added that he and Benedict would sit down and work through the architectures, looking at operational views, system views, and technical views and see what they are working on and what kind of changes might be needed.
“I have resources that go into these programs and really try to understand whether it is just a Windows 7 upgrade or if it’s a completely new operating system. I have that capability,” he said. “We are a trust but verify organization. We trust everybody, but we want to verify with data. That’s our job, we look in and we grind through it all in great detail.”
Then there comes the resource limitations, Dunaway added.
“Ideally the operational tester would sortie a battle group once a month, take [a system] into theater where there are real threats. Well that’s impractical,” he said. “That’s why I use what’s called design of experiment. It’s a sophisticated methodology of going through a major difficult matrix and bringing it down to the essence of what’s important. And when you touch through this scientific method, the important factors, you get much more insight into the total matrix.”