Grantee Research Project Results
Final Report: Radiation Contamination Visualization and Mapping (RaCOM) System
EPA Contract Number: 68HERC20C0034Title: Radiation Contamination Visualization and Mapping (RaCOM) System
Investigators: Sharma, Prachee
Small Business: Physical Optics Corporation
EPA Contact: Richards, April
Phase: I
Project Period: March 1, 2020 through October 31, 2020
Project Amount: $99,999
RFA: Small Business Innovation Research (SBIR) - Phase I (2020) RFA Text | Recipients Lists
Research Category: Small Business Innovation Research (SBIR) , SBIR - Homeland Security
Description:
The Environmental Protection Agency (EPA) seeks the development of a three-dimensional (3D) gamma and neutron sensor assembly for mapping and visualization of radiological environments to aid in survey and cleanup operations while minimizing radiation exposure to work crews and equipment. Mapping the presence of radiological and nuclear materials, both indoors and outdoors, and providing 3D radiation maps fused with scene data are of special interest to the EPA. The EPA needs the development of innovative hardware and/or software solutions capable of integrating radioactivity detection, indication, and computation (RADIAC) sensors with scene data. Technical innovation is needed in current state-of-the-art gamma sensing, mapping, and information display for indoor and outdoor environments to address the EPA’s needs.
To address this need for 3D mapping and radiation data fusion technologies using multiple unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs), Physical Optics Corporation (POC) is developing a new Radiation Contamination Visualization and Mapping (RaCOM) system. RaCOM is a multi-platform system comprising a primary RaCOM platform hosted on a UGV and multiple secondary RaCOM platforms (UGVs or UAVs). The primary platform will house all sensors in the suite and computational hardware to compute point clouds as well as integrate data from secondary platforms to create a global map. The primary node will be capable of computing global 3D maps based on the integration of local maps provided by each platform and reflecting fused structural and radiation maps. The secondary nodes will include a set of sensors based on the weight and power budget of the platform. The secondary nodes assume limited computational hardware and are designed to perform point cloud computation from the sensor data acquired from the local survey path. As required by the solicitation, at least one of the platforms in the multi-platform RaCOM system can be an unmanned aerial system. All secondary nodes are designed to provide local maps to the primary node. The local data from secondary nodes is aggregated over wireless link and fused on the primary node to create global 3D maps characterizing the structures and radiological environment of the area of interest. The point cloud data computed by both primary and secondary platforms is made available for viewing at a remote viewer. The data collected by primary and secondary platforms will provide structural, gamma-ray, neutron, and thermal-specific global 3D maps using sensor data. The remote viewer will be integrated with the Tactical Assault Kit (TAK). The radioisotope maps will be displayed by mapping the energy measurements of the gamma rays to a color space.
RaCOM innovations include:
· A novel combination of sensors in a modular package on multiple platforms. Sensors can be swapped in and out, providing multi-application usability.
· A unique integration of hardware and software effectively combine a variety of commercial and customized sensors, providing a low-size, weight, power, and cost (SWaP-C) package.
· A unique modular software allows creation of 3D mapping of structures and radiological hazards for real-time viewing on a remote viewer.
In Phase I, a single platform RaCOM system was implemented. Successful demonstration of RaCOM in Phase I was completed on November 5, 2020. The demonstration included data fusion, creation of 3D point clouds using data from multiple sensors, and display of surrogate radiological data on a remote viewer. It was shown that the remote viewer is capable of visualizing the 3D environment by manipulating the scene and viewing it through various zoom, pan, and rotate angles. The rate at which the data is displayed on the remote viewer could be adjusted by selectively displaying the feeds from different sensors. The prototype implementation of a single- platform RaCOM system and design of a multi-platform system was completed by assessing
(a) the number of sensors that can be accommodated within size, weight, and power (SWaP) budgets; (b) the computational power needed to process sensor data; (c) the computational power needed to create 3D point cloud and visualization; (d) the wireless network data rate needed to support multiple platforms; and (e) the rate of computations needed to meet the EPA’s requirements to survey 100,000 m2 within 30 min. The prototype design, implementation, and feasibility demonstration completed during the Phase I work lay a solid foundation for further development of RaCOM system in Phase II. Phase II work will enhance the Phase I RaCOM system and develop a multi-platform RaCOM system yielding a complete, practical, efficient RaCOM system ready for Phase III commercialization.
Commercial RaCOM applications include use in law enforcement, industrial applications, and border security for detecting and mapping radiological environments, as well as planning cleanup operations. The ability to distribute the map computation of the 3D structural and radiological environments and view the maps in real time on remote devices sets RaCOM apart from its competitors. RaCOM could also provide thermal imaging of hot spots and supporting post-fire overhaul of structures compromised by major fires.
RaCOM’s impact includes improved health and safety of operators and a dramatic reduction in cleanup time, cost savings, and improved retrospective analysis. RaCOM addresses the radiation detection, monitoring, and safety market, which is expected to exceed $2B by 2023, registering a compound annual growth rate (CAGR) of 6.42% from 2018–2023 as estimated by market analysis reports.
Summary/Accomplishments (Outputs/Outcomes):
During Phase I, the RaCOM software architecture was developed and the RaCOM system for a single platform was implemented. The RaCOM Phase I system included a sensor suite, sensor data processing algorithms, algorithms to integrate sensor data with geographical information, and a representative remote viewer to visualize the environment of interest in 3D. RaCOM’s feasibility was established through the prototype demonstration and performance analysis, specifically focusing on following areas that are of interest to the EPA.
RaCOM’s architecture was developed. The RaCOM architecture and software design specifications were documented in the form of block diagrams and interfaces. The RaCOM software and hardware was designed to follow the client-server architecture where sensors are collocated with a server subsystem and clients are remotely accessible and include the remote viewer. The messages between the server and client were exchanged using a publish-subscribe paradigm.
The RaCOM hardware assembly for a single platform system included a sensor suite comprising low-cost 3D depth sensors with USB interfaces to collect data and multiple fields of view to provide 3D structural and thermal radiation maps. A low-cost GPS receiver with a serial interface was used to collect GPS coordinates in National Marine Electronics Association (NMEA) 0183 format as the sensor assembly moved across the area of interest. An inertial measurement unit (IMU) was integrated with the stereo infrared (IR) 3D depth/red, green, blue (RGB) sensor. A thermal sensor array was used as a surrogate for collimated radiological sensors.
RaCOM software components on the server side included the modules to collect data from sensors, correlate sensor data with geographical information, and create 3D point clouds for viewing. One of the key RaCOM features was to map the sensor observations into color space so that the location of the thermal source (representing radiological sensors) was visible. The RaCOM system was tested to detect a heat source located behind obstacles as well as those located in plain sight. The wirelessly networked client computer included a user interface to visualize the 3D structural and thermal point clouds.
RaCOM performance measurements with a single platform were completed to demonstrate the feasibility of data fusion technologies in mapping and visualizing radiation maps in 3D and in real time. A simplified representative RaCOM system prototype was used for the Phase I performance period for testing. In the test system, the local storage, data processing, and display functionalities were combined into one computer and the wireless link was replaced by a wired link for testing to reveal any potential data-link and/or bandwidth issues. Thermal sensors were used in Phase I as a surrogate for radiological sensors. The thermal sensor was chosen as the data represents the heat generated from radiological sensors. The heat source was used to test the algorithms for data collection, sensor data integration, mapping, and visualization. The RaCOM system used the Rviz remote display to show the 3D map visualization capability.
RaCOM performance analysis was completed on a single platform system and specifications for the Phase II system were generated. Key results are listed below:
· Rate of data provided by sensors: Three thermal array sensor modules were used, resulting in processing of thermal sensor data at a rate of 27 Kbps. Data from thermal, RGB, and IMU was collected and processed at an aggregated rate of 3,527 Kbps. RGB sensors contributed to maximum data at a rate of 3.5 Mbps compared to 0.009 Kbps from the thermal sensor. The USB 3.0 interface can be used to aggregate sensor data, as USB 3.0 can support a data exchange at a maximum rate of 10 GB/s. The data collection and processing is expected to change at a linear rate as additional sensors are added to the array. The addition of sensors in the array will also impact the specifications of the computer hardware used to process the sensor data and create a 3D point cloud. The complete design of the sensor array and computer hardware specifications will be designed during Phase II based on requirements such as data resolution and the size, weight, and power budget of the platform housing the assembly.
· Rate at which the sensor data is processed and 3D visualization is created: The data corresponding to 3 min of survey duration was found to generate an RGB/thermal 3D map database of 70 MB, yielding a recording rate of 3.1 Mbps.
· Storage capacity needed for the 30 min survey: For a 30 min survey, a storage capacity of approximately 0.7 GB will be needed to record raw data for offline analysis after the survey is complete. The storage capacity will need to be adjusted for additional sensors.
· CPU usage: CPU resources were assessed on an Intel i7 CPU Quad Core at 2.6 GHz with16 GB RAM for the following cases:
- Processing a live sensor feed to fuse data and creating a 3D point cloud. This case closely represents resource requirements for creating local map. For this case, 87% of the CPU and 2 GB RAM was used.
- Using recorded sensor data to create a 3D point cloud. This case represents resource requirements for fusing multiple local maps to create a global map. For this case, 31% of the CPU and 662 KB RAM were utilized.
- The CPU requirements for processing live sensor data amounts to the differences in CPU utilization for the above two cases with about 56% of the CPU usage.
The sensors for the RaCOM system were analyzed to ensure detection and discrimination of gamma-rays and neutrons. The RaCOM collimated gamma ray sensor array presented several challenges, especially in the UAV case, where weight, power consumption, and payload size needed careful consideration. Common dosimeters were compared and the decision to use a diode- based detector was made due to rapid detection compared to film sensors that may take several minutes to collect data and provide conclusive readings. Further tradeoff analysis and performance comparison of sensors will be done in Phase II as part of the requirements analysis task and a sensor selection will be refined.
Feasibility of using the remote display: The RaCOM system used a Rviz remote display with network bandwidth utilization to provide 3D visualization. The rate of data exchange between the micro-server and remote display was measured to be at 3.5 Mbps. This measurement was done by exchanging database files in XMLRPC format. Most commercial wireless technologies can support data exchange at the rate of 3.5 Mbps, demonstrating the feasibility of a remote data display capability in RaCOM. The XMLRPC may need to be replaced by another format in Phase II based on TAK requirements. Consequently, reassessment of the wireless link bandwidth needed for the remote display support will be needed.
The feasibility of integration with existing situational awareness (SA) tools, such as mobile field kit (MFK)/TAK: The ability of TAK mobile client software to support multiple overlays was investigated. It was found that structural, gamma ray, neutron, and thermal 3D maps can be displayed in the TAK mobile client using the TAK chemical, biological, radiological, and nuclear (CBRN) plugin. This plugin would require sensor inputs to be formatted to comply with the integrated sensor architecture (ISA) as developed by the U.S. Army. The 3D radiation maps can also be imported as a 3D object using the .obj file format as performed by many computer-aided design (CAD) software packages. The location of gamma ray, neutron, and thermal hotspots can be communicated over the SA multicast channel so that survey plans and blue force movement can be adjusted to avoid survivability risks.
The feasibility of using RaCOM in multiple platforms was proven. The data was recorded and played back for the software to represent multiple data sources that used input from multiple sensors and multiple platforms. RaCOM was shown to fuse multiple point clouds generated by various sensors. Phase I work also analyzed the communication link requirements to implement a multi-platform system. RaCOM system used a Rviz remote display to provide 3D visualization. The local map data will be sent at a rate of 3.1 Mbps from each secondary platform to the primary platform. Assuming the IEEE 802.11ax network, which can support a data rate of 840 Mbps using one access point, many secondary platforms can be supported (approximately 840/3.11 = 270 platforms). IEEE 802.11ax or other similar networks will be sufficient to create a multi-platform RaCOM system connection between the sensors and the remote display.
Ensuring a survey of a 100,000 m2 area can be completed in 30 min: The solicitation requires the RaCOM system to complete the survey of 100,000 m2 area within 30 min. Assuming that ten secondary platforms and one primary platform are used, the area to be surveyed is divided among secondary platforms equally and 5 mins are needed to aggregate the data from local platforms to create a global map. The estimates for performance requirements are calculated as follows:
· Area surveyed by each platform = 10,000 m2
· Each platform should complete the survey in 25 min (leaving 5 min for global map creation)
· Rate of survey should be 400 m2 per minute or 6.67 m2 per second
· To get local map for area of 6.67 m2 the following tasks must be completed in 1 s:
o Environmental sensing
o Live sensor data collection
o SLAM – Correlate sensor data with geographical information
o 3D point cloud generation
o Transmission of data from secondary to primary platforms for aggregation and global map generation.
Conclusions:
Phase I work was completed successfully and all the project objectives were met. Innovative 3D mapping software and sensor assembly was implemented in the Phase I work. The successful demonstration of RaCOM completed on November 5, 2020 showed successful creation of a 3D point cloud using multiple sensors and a display of surrogate radiological data on a remote viewer. The 3D visualization capabilities (zoom, pan, and tilt) and organization of data as “layers” was shown so that a user can selectively view the data depending on the area of focus and sensor modality of interest. The single-platform prototype implementation in Phase I lays a robust foundation for the development of multi-platform RaCOM system in Phase II that is ready for Phase III commercialization. The Phase II project is critical in that it will (a) demonstrate the viability of the proposed methodology and mature the RaCOM system and (b) serve as the foundation for Phase III improvement and implementation for use by the EPA. The matured RaCOM system will be usable by various DoD and commercial organizations. In Phase I, RaCOM innovations include:
· A novel combination of sensors in a modular multi-platform design. Sensors may be swapped in and out depending on application, providing multi-application usability.
· A unique integration of hardware and software to effectively combine a variety of commercial and customized sensors providing a low-SWaP-C package suitable for various platforms and applications.
· A unique modular software allows creation of 3D maps of the structural and radiological environment for real-time viewing on a remote viewer.
Key findings pertaining to Phase I are listed below:
· A market survey was completed and appropriate sensors were identified for use in RaCOM. The market survey completed in Phase I will serve as a foundation for further refinement of the RaCOM design in Phase II.
· Simultaneous localization and mapping (SLAM) algorithms were implemented to integrate sensor data and issues with integration of multiple data streams (such as a difference in positioning errors resulting from a loss of lock in the IMU) were identified. The need for more robust geographical context information was identified. This will serve as a foundation for selection of more accurate and robust IMU and GPS sensors in Phase II.
· Hardware sizing was completed based on the number of sensors, rate of raw data generated by the sensors, computational resources needed to compute point clouds, and the storage capability needed to archive raw sensor data for after action analysis. The analysis will serve as a foundation for further development in Phase II.
· The rate of exchange of data from sensor processing nodes and the remote viewer was determined. This information will be used to choose an appropriate wireless network technology to connect primary and secondary platforms with a remote viewer. The information will thus help in creating the RaCOM Phase II demonstration.
Target commercial applications for RaCOM include safety enhancements in civilian and law enforcement organizations that must respond efficiently to radiation situations (traffic monitoring, border patrol, and firefighting), collision avoidance (general aviation, vehicular transport, railways, and others), and search and rescue (searching for the optimal path for evacuation, especially under hazardous radiation conditions).
The estimated revenue potential for RaCOM is based on a radiation detection, monitoring, and safety market size of $2.091B by 2023. This market segment is expected to register a CAGR of 6.42% over the forecast period of 2018–2023. Assuming 1% of this market—ground contamination visualization and mapping software—yields a market size of $20M and assuming a market capture of 60% by POC, RaCOM can be expected to yield a revenue of $12M on an annual basis by 2023.
Commercialization objectives achieved in the Phase I project include identification of the market size, applications, and customers that can use RaCOM technology. Potential stakeholders include civilian, law enforcement, and search and rescue organizations. POC plans to achieve these objectives by the end of the Phase I project.
As part of Phase I efforts, a commercialization plan for Phase II was developed. Key elements of the commercialization plan developed in Phase I are listed below:
· POC will establish partnerships with UAV and UGV providers to integrate RaCOM as a package on unmanned vehicles.
· POC will identify whether RaCOM should be commercialized through equipment sales or equipment rental. For radiological cleanup operations, the rental model will alleviate difficulties for the customer in maintaining the equipment while not in use. The pricing model will be developed and anticipated profit margins will be identified based on the cost of manufacturing and maintaining production lines.
· Depending on the size of the market, production planning will be done and preparations will be made for low-rate initial production (LRIP) of RaCOM in Phase III. This will require identification of cost and production models for RaCOM.
· POC will identify and establish a supply chain for production of RaCOM. Quality control planning, manufacturing planning, and any testing and certifications needed for RaCOM production will be identified.
· The cost of the production line will be assessed. POC usually adopts a cost and feasibility-based production model, where some parts are outsourced and some parts are manufactured within POC facilities. The production model for RaCOM will be developed and resources will be lined up to start production during Phase II work.
The perspectives, information and conclusions conveyed in research project abstracts, progress reports, final reports, journal abstracts and journal publications convey the viewpoints of the principal investigator and may not represent the views and policies of ORD and EPA. Conclusions drawn by the principal investigators have not been reviewed by the Agency.