Grantee Research Project Results
Final Report: Remote Sensing of fugitive Methane Using Retroreflector-based Differential Laser Absorption Spectroscopy (DLAS) System on a UAV Pair
EPA Contract Number: 68HERC23C0019Title: Remote Sensing of fugitive Methane Using Retroreflector-based Differential Laser Absorption Spectroscopy (DLAS) System on a UAV Pair
Investigators: Heiks, Noel
Small Business: Censys Technologies Corporation
EPA Contact: Richards, April
Phase: I
Project Period: December 1, 2022 through May 31, 2023
Project Amount: $99,411
RFA: Small Business Innovation Research (SBIR) Phase I (2023) RFA Text | Recipients Lists
Research Category: SBIR - Sustainability , SBIR - Air and Climate , SBIR - Water , SBIR - Homeland Security
Description:
The detection and measurement of methane leaks are crucial for mitigating climate change and protecting the environment. While there are satellite and tower data that provide global or point measurements of greenhouse gas emissions, there is a lack of localized information about the emission and transport of methane. This hinders efforts to identify and mitigate leaks effectively.
To address these challenges, as part of its EPA SBIR Phase I efforts, Censys Technologies Corporation (Censys) built on its development efforts of the RetraSpectraTM system. Validation tests with a mounted laser spectrometer (without a retroreflector) on a UAV showed that the measured methane amounts were accurate within 5% at 25 meters, with increasing accuracy as the detector gets closer to the source. However, for a robust and commercial detection and mapping system, higher altitudes and greater precision are necessary. The goal was to achieve significant noise reduction to ensure accurate detection of fugitive gas leaks.
The RetraSpectraTM combines a differential laser absorption spectroscopy (DLAS) system mounted on one UAV with a retroreflector mounted on another UAV. In Phase I of the development, the company demonstrated a proof-of-concept measurement using the RetraSpectra system and optimized custom-developed retroreflectors while using commercial-off-the-shelf spectrometers. Subsequent phases will involve the use of two coordinated UAVs, fine-tuning for commercialization, and the development of sensor systems for measuring other gases like CO2. Censys has obtained an exclusive license for a patent related to atmospheric differential absorption tracking, which provides intellectual property protection for the detection apparatus, detection of multiple gas species, technologies to distinguish between incident and reflective beams, and different measurement configurations.
As part of its Phase I efforts , Censys Technologies Corporation met the following technical objectives:
(1) Systems Engineering/Requirements Planning
Deliverables: (a) Solutions Requirements Document (b) System Requirements Document(c) Risk Management Plan (Identify Risks and Mitigation)
(2) Optimization of Retroreflector
Deliverable: Test plan & procedure document that shows path to retroreflector testing for shape and material optimization. Report documenting retroreflector optimization test results.
(3) Development of Computer vision-based algorithm for tracking Retroreflector
Deliverable: Report on computer vision techniques that have been down-selected along with training sets of images description. Report documenting test results from computer-vision training models that demonstrate confidence for retroreflector tracking ability.
(4) Flight Integration and in-lab Demonstration
Deliverable: Report documenting test results with & without retroreflector to demonstrate reduced noise for methane detection. Report documenting results from in-lab testing of Retra-Spectra system while retroreflector is on stationary tripod.
Summary/Accomplishments (Outputs/Outcomes):
1) Retroreflector Tracking Algorithm Development: Censys developed three types of retroreflector targets to test our computer vision model and tracking accuracy. These targets needed to be highly visible, identifiable, and unique to be detected by a camera from a large distance using convolutional neural network (CNN) models. The goal was to train the system to recognize the best-performing design through tests that evaluated performance at different distances and viewing angles. The computer vision model was expected to identify the movement of the reflector and track it across the camera's field of view, even if the reflector was not parallel to the focal plane. However, there was a limit to how much of the target's side view could be accepted before its unique features became unrecognizable. To create a modeling dataset, each design was printed on an 8.5 by 11-inch sheet and photographed in various locations, tilts, and backgrounds. Over 1200 images of each design were collected, considering different lighting conditions and partial obstructions. The images were labeled using an AI platform to create inputs for training the CNN model. The results showed that the RM-3 target design performed the best, with a 98% identification rate and 95% accuracy. The number of false positives was extremely low, but false negatives increased when the target angle deviated beyond approximately 20 degrees. However, this edge case was considered less significant.
2) Retroreflector Optimization Tests: Censys conducted a series of experiments to test the impact of reflectivity on methane measurements using various COTS reflectors. We used a Laser Falcon methane detector and placed a Tedlar bag filled with propane between the reflector and detector. The experiments varied the distance between the transceiver and reflector, as well as the type of reflector used. We measured the propane concentration in the bag and compared the readings collected at different distances and with various types of reflectors. The experiments showed that the use of reflectors enhanced the detection range and reading sensitivity. The glass corner cube reflector returned nearly double the readings compared to other reflectors, The amount of light returned by the corner cube reflector was substantial, as evidenced by measurements using a photodiode.
This indicated a higher intensity of light compared to other reflectors. Retroreflectors, including the corner cube reflector, provided a higher quality return path for the light, resulting in consistent readings over subsequent 3-minute periods. In summary, the experiments demonstrated that retroreflectors improve gas concentration measurements over large distances by providing consistent readings. The use of reflectors helps maintain measurement accuracy by ensuring a higher intensity return path for the light. However, the specific behavior of the glass corner cube reflector requires further investigation to understand its unique characteristics. We also used a simple photodiode to measure the amount of light returned before it enters a Fresnel lens; lower the reading, the higher the light intensity. It is quite apparent that again the corner cube reflector dominates. Table XX summarizes the propane PPM-distance results & the photodiode test results from the same.
4) Gimbal Tracking: For this deliverable, Censys demonstrated their intelligent video-based target tracking system, which incorporates a computer vision (CV) subsystem and a movement subsystem to detect and track a retro reflector pattern. The system utilizes an AI-based model trained to detect a "cross and square" retroreflector in a video frame. The CV subsystem calculates the reflector's apparent movement between frames, and the movement subsystem adjusts the camera's pan and tilt (PT) using stepper motors to keep the reflector in the middle of the frame.
The system architecture involved a gimbaled 6-megapixel camera with PT capability achieved through two stepper motors controlled by the CV subsystem. The motors convert PWM signals from the algorithms into movement, and a Raspberry Pi PWM hat generates the necessary electrical signals.
Although the stepper motors lacked high precision, they provided smooth movement suitable for live tracking. The CPU host computer runs the AI model responsible for detecting and bounding the retroreflector's position in the video frames. The model is trained to handle oblique views, allowing detection at various angles. The Intersection over Union (IoU) metric is used to fine-tune the model and obtain the best bounding box for the reflector. The tracking algorithm relies on the calculated center of the reflector in each frame, comparing it to the frame's center. If the difference exceeds a threshold, the movement subsystem adjusts the camera's PT to align the reflector with the frame center. During the testing, adjustments were recorded to measure the system's performance. Statistical analysis showed a tight grouping of movements, with a standard deviation of 52 pixels. Rapid reflector movements resulted in rougher adjustments due to the low frame rate and the limitations of the stepper motors. Overall, the results from the experiment demonstrated the promising capabilities of the intelligent video-based target tracking system.
5) In-lab methane measurement using RetraSpectra system: a system was developed that combined various components to enable drone-based target tracking and methane sensing. The system incorporated a retroreflective target mounted on a drone, a pan-tilt (PT) mount with a sensing laser and computer vision system, and control software with tracking capabilities. Initially, a test was conducted using a tripod-mounted RGB camera on a computer-controlled PT mount. The PT mount was controlled by a Raspberry Pi or Nvidia Jetson, and the system was trained to detect the retroreflective target at different angles to ensure continuous tracking. The experiment involved testing several variables. First, the PT mount's ability to track the drone's movement was examined. It was found that the PT mount could effectively track the drone as long as the ground speed was below 15 mph and the distance exceeded 20 meters. However, at closer distances, the PT mount faced difficulties due to motor limitations. This limitation was not a significant concern as the system aimed to measure methane content over longer distances. The tracking camera's capability to detect the target was also assessed. It was observed that as the target moved farther away from the camera, the number of pixels on the target decreased, leading to decreased accuracy in detection. Beyond 95-100 meters, the computer vision system struggled to track the target. The addition of a retroreflector improved the sensitivity and accuracy of methane measurement. It allowed measurements in any direction as long as the laser hit the reflector, enabling innovative drone-based measurements such as methane flux monitoring. The proposed improved tracking system had the potential to revolutionize fugitive gas measurement, especially with the introduction of zooming capability. However, maintaining accuracy became challenging as the target size decreased with increasing distance, requiring clear visibility with at least 65 pixels on target for usable accuracy.
Conclusions:
Censys was able to demonstrate feasibility of methane measurements using a differential laser absorption spectroscopy (DLAS) system mounted on one UAV with a retroreflector mounted on another UAV. The use of a retroreflector was also demonstrated to be more beneficial than using a stand-alone retroreflector.
Middle Tennessee State University (MTSU) Partnership: MTSU contacted the project team regarding an EPA Landfill emissions detection proposal. Several conversations took place to discuss the scope of work (SOW) and system requirements. MTSU showed interest in being an early adopter of the technology and a potential collaborator for testing in Phase II.
ODNR Opportunity: A joint proposal was submitted with Davey Resource Group to detect fugitive emissions from orphan wells in Ohio. Censys would participate as a service provider to assist Davey in this project.
Dominion Energy Opportunity: Another joint proposal was submitted with Davey Resource Group for comprehensive inspection of pipeline aerial patrols to comply with DOT regulations. Censys would provide services to assist Davey in this initiative.
Additionally:
Jet Systems Helicopter Services: Conversations took place with Jet Systems Helicopter Services to explore collaboration opportunities, potential sales, or providing methane detection as a service to their O&G pipeline customers. This was in response to the EPA SBIR Phase I award announcement on LinkedIn.
The Aerospace Corporation: The project team had a conversation with The Aerospace Corporation, the patent holders, discussing the project approach, challenges, and test plan. The possibility of a consulting role for them in Phase II was mentioned.
Foresight: Multiple conversations were held with Foresight to discuss utilizing their services as part of TABA commercialization assistance. The team decided to make use of the Commercialization Readiness Assessment Report and Expert End-user identification.
EPA TPOC Call: The team had an initial conversation with the EPA TPOCs, Jason Dewees and Dave Nash. They discussed technical and regulatory requirements for the system and the desired outcomes as an end-user.
Deloitte: An introductory call was held with Deloitte to discuss the actionable data and analytics needed for creating accurate methane budgets.
Additionally, the project team had several conversations, demos, and tours with private investors to discuss the technology roadmap and the productization of the technology.
Overall, the project made progress in engaging with potential partners, submitting joint proposals, exploring collaborations, and receiving interest from investors. These activities indicate a growing interest in technology and its potential applications with several partners for Phase II.
SBIR Phase II:
Enhanced-range remote-sensing of fugitive Methane using retroreflector-based DLAS System with RTK Integration for improved location accuracy on a UAV PairThe perspectives, information and conclusions conveyed in research project abstracts, progress reports, final reports, journal abstracts and journal publications convey the viewpoints of the principal investigator and may not represent the views and policies of ORD and EPA. Conclusions drawn by the principal investigators have not been reviewed by the Agency.