A draft version of a cost-benefit analysis by the Domestic Nuclear Detection Office (DNDO) that supports the use of next-generation radiation portal monitors in secondary inspection lanes to scan trucks for illicit radiological materials needs improvements before it can be used to support a decision, says the final report on the Advanced Spectroscopic Portal (ASP) program prepared by the National Research Council (NRC).
The report, which follows an interim report by the NRC in 2009, says that DNDO implemented recommendations from the interim report in several areas, including using the fact that improved performance by the ASP reduces the threat of a nuclear attack against the U.S. It also says that DNDO has improved the way it models, tests and remodels the test program for ASP and how to better analyze security benefits.
But the NRC report says “three major problems remain” in the cost-benefit analysis. First is that no strategic justification has been provided for why ASPs should be deployed in secondary inspection lanes.
The NRC, and arm of the National Academy of Science, says that “DNDO needs to articulate what is achieved by improvements in detector performance.” For example, the report says, it might be that the introduction of the ASPs forces potential adversaries to use enough shielding material around a threat object, making it easy to identify a suspicious object when it is scanned by a X-Ray system.
The report says that when the use of an ASP is “combined with, or example, random radiography of a fraction of conveyances (which would hold any conveyance at some risk of being scanned), one has an example of a coherent strategy that leads to a quantifiable probability of successful interdiction.”
The NRC report, entitled Evaluating Testing, Costs and Benefits of Advanced Spectroscopic Portals: Final Report, also says that DNDO did not consider enough alternatives to ASP nor did it consider other technologies and deployments that may be preferred.
“For example, improved handheld detectors were dismissed without analysis, base on an assertion that handheld detectors simply are not suitable for external screening of cargo containers,” the report says. “This assertion may prove true depending on the criteria established, but recent analyses within DNDO suggest that handheld detectors, even the hardware currently in use, could be far more effective than DNDO though possible in identifying threats in cargo.”
Cost Comparisons
This is important because a DNDO study of the life-cycle costs of the handheld Radio-Isotope Identification Devices (RIID) frequently used today in secondary inspections by Customs and Border Protection officers are $27,000, with the next-generation RIIDs expected to cost about $40,000 over their life. Currently the life-cycle cost of an existing PVT radiation portal monitor used with a RIID is estimated at $640,000 versus $1.2 million for an ASP and RIID, the NRC says.
“If improved software halves the difference between the current RIID and the ASP and DHS (Department of Homeland Security) is simply looking for the greatest improvement detector performance at the least cost, then the improved software is a more cost-effective improvement to the current system than replacing it with the ASP,” the NRC says.
Finally, the report says that DNDO quantified some measurements in its cost-benefit analysis that are really unquantifiable, giving “precision to the analysis that cannot be supported.”
The NRC suggests that any ASP deployments be incremental, with upgrades, particularly to the software algorithms used for data analysis, which can be introduced without significant changes to the hardware. However, the report says that at least one of the two vendors developing ASP systems for DNDO has not produced analysis modules that are compatible with the other’s detector systems.
“Within the [NRC] committee, this raised concerns about procurement: DNDO has not gotten the modularity from the vendors that was mandated in the specification,” the report says. “This deficiency should be corrected.”
Given that the ASP systems are supposed to support a modular approach to the data analysis algorithms, the NRC says that “DNDO should not limit itself to the vendors’ algorithms.” The report encourages DNDO to take advantage of experts outside of nuclear detection to improve the analysis algorithms for application to all spectroscopic detectors, including ASPs.
Thermo Fisher Scientific [TMO] and Raytheon [RTN] are each developing ASP systems for DNDO. The development effort has been troubled with questions raised about the efficacy of the systems as well as their cost on more than one occasion (TR2, July 8, 2009).
These development issues, high costs and operational effectiveness led Homeland Security Secretary Janet Napolitano last February to decide against deploying ASP systems as a primary screening tool and instead continue testing them for potential used in secondary screening.
That testing is ongoing and DHS expects to complete the program this year.
“DNDO appreciates the Academy’s work evaluating the ASP program; the report and its recommendations will be taken into careful consideration as we move forward,” DHS spokesman Chris Ortman tells TR2. “While the ASP program is important, it is just one part of our layered approach to security, which includes technology, well-trained personnel, and processes that can adapt to evolving threats.”
As to the NRC’s recommendations, DNDO has already said that it is working to improve its modeling and testing of ASP (TR2, Jan. 5).