Grantee Research Project Results
2021 Progress Report: A Sustainable Center for Crowd-Sourced Water Infrastructure Modeling
EPA Grant Number: R835950Center: Gulf Coast HSRC (Lamar)
Center Director: Ho, Tho C.
Title: A Sustainable Center for Crowd-Sourced Water Infrastructure Modeling
Investigators: Hodges, Ben R. , Cleveland, Theodore G. , Barrett, Michael E. , Ames, Daniel P. , Leite, Fernanda , Berglund, Emily , Urbonas, Ben , Pechacek, Linda D , Brashear, Bob
Current Investigators: Hodges, Ben R. , Cleveland, Theodore G. , Barrett, Michael E. , Ames, Daniel P. , Leite, Fernanda , Berglund, Emily , Urbonas, Ben , Brashear, Bob , Rowney, A. Charles
Institution: Brigham Young University , The University of Texas at Austin , North Carolina State University , Texas Tech University , Urban Watersheds Research Institute
EPA Project Officer: Packard, Benjamin H
Project Period: September 1, 2016 through August 31, 2021 (Extended to August 31, 2023)
Project Period Covered by this Report: September 1, 2020 through August 31,2021
Project Amount: $3,999,803
RFA: National Center for Sustainable Water Infrastructure Modeling Research (2014) RFA Text | Recipients Lists
Research Category: Water , Water Quality
Objective:
Project Title: Pipe failure modeling and analysis
Objective of Research: Develop computational framework for water distribution infrastructure asset management to predict pipe failure.
Project Title: Automated calibration of SWMM
Objective of Research: Develop and automated calibration tool for SWMM
Project Title: New linear approximation for solving the network water flow problem
Objective of Research: Develop new linear approximation for solving the network water flow and state estimation problems
Project Title: Hedging for privacy in smart water meters
Objective of Research: Develop a privacy preserving system to hedge for privacy in smart water meters
Project Title: Mining pressure transients from high frequency pressure sensors.
Objective of Research: Discover new methods for evaluating pressure transients in water distribution systems.
Project Title: Impact of water saving scenarios on performance of urban water systems
Objective of Research: Evaluate how changing water demand affects water distribution system operational metrics.
Project Title: Water quality modeling in distribution networks
Objective of Research: Develop water quality model for dead-end branches in water distribution systems.
Project Title: Flint water crisis
Objective of Research: Examine trends in recovery from lead contamination after remediation interventions.
Project Title: Review of WDSA water security
Objective of Research: Review state-of-the-art in cyber-physical threats, modeling approaches, and algorithms for water distribution systems security.
Project Title: Extending the capabilities of EPANET user interface with plugin tools.
Objective of Research: Extending the capabilities of EPANET user interface with plugin tools.
Project Title: Exploring the Operational Effects of Gentrification on Water Networks
Objective of Research: Exploring the Operational Effects of Gentrification on Water Networks
Project Title: Design and Development of a Web-Based EPANET Model Catalog and Execution Environment
Objective of Research: Develop an online registry of EPANET models and an online web-based platform for executing such models.
Progress Summary:
Research performed during this period included:
(1) Development of the hydraulic network computational engine for SWMM.
(2) Development of parallel processing code for SWMM in coarray Fortran.
(3) Development of network partitioning methods to optimize parallel code.
(4) Release of the SWMM5+ Alpha (preliminary) code.
(5) Development of parallel-processing method-of-characteristics solver for handling water transients.
(6) Deploying on-line versions of EPANET and SWMM to simplify training.
(7) Developing online introductory SWMM training, including a JupyterBook to house the training examples with internal links to on-line SWMM.
(8) Attending the 54rd International Conference on Water Management Modeling in Toronto in February 2021 and presented the Center’s research and development progress and gathered extensive community feedback.
Major output/outcomes to date include 16 peer-reviewed journal articles already in print, 39 conference/seminar presentations, and four archived repositories of technical reports and model data. The Alpha release of the new SWMM5+ code in August 2021 is a major milestone toward completion of this project.
For stormwater systems, we are implementing and testing the new approach to hydraulic network solution (previously outlined in the FY18 report), which has been reported in four peer-reviewed articles. The new hydraulic engine, known as SWMM5+, fits within the structure of the existing SWMM code and did not require an entire rewrite of SWMM. Instead, the new engine is an option that users can select that will operate transparently with the EPA SWMM code. The new hydraulics engine was provided to EPA and members of the Science Advisory Committee (SAC) and Technical Advisory Committee (TAC) as the SWMM5+ Alpha code release.
For water distribution systems, we have (1) developed methodologies for pipe failure modeling and analyses; (2) developed an approach for handling water quality dead-end branches in EPANET, (3) examined water saving scenarios for urban water networks, (4) developed methods for analyzing pressure transients from observational data in water distribution systems, and (5) developed a new vectorized and parallel implementation of the method of characteristics for transient modeling networked pipeline systems.
We have been updating and upgraded SWMM and EPANET course materials to deliver through online courses. COVID-19 has reinforced our strategy of pursuing web-based delivery that we believe will be more viable and cost-effective for a broader array of participants and can be provided within the sustainable vision for the Center. Development of EPANET-by-Example (an online tutorial) is in progress. This program leverages a previously-developed lecture tutorial and is designed for an audience who have some hydraulics background but little modeling experience. A similar SWMM-by-Example tutorial is also in progress. These will both reside in JupyterBooks for ease of future maintenance. Both tutorials will internal link to on-line instances of EPANET and SWMM.
Project Title: Pipe failure modeling and analysis
Pipe failures in water distribution infrastructure (WDI) have significant economic, environmental, and public health impacts. To alleviate these impacts, repair and replacement decisions need to be prioritized in order to effectively reduce pipe failure rates. Computational frameworks for WDI asset management have been developed that couple spatial clustering analysis with predictive modeling of pipe failures. The first part of this project focused on adopting clustering algorithms to explore spatial and spatiotemporal patterns exhibited by pipe failure. This was done by: (1) implementing spatiotemporal scan statistics for the identification of density-based clusters of failure events; and (2) applying local indicators of spatial association to identify statistically significant hotspot and coldspot clusters of exceptionally high and low pipe failure rates. The second part involved developing predictive modeling tools for forecasting pipe failures. Tests have been conducted to evaluate the predictive abilities of eight different statistical learning techniques, and the best performing method was implemented to forecast pipe failure rates within the different sectors of the WDI while explicitly accounting for the spatial patterns exhibited by the failure rate and its predictors. Survival-based models have been developed to predict the time to failure as a function of the different failure predictors, including pipe characteristics, soil conditions, traffic, and pressure. The developed frameworks where demonstrated on a real-life, large-scale WDI, and where shown to provide useful insights that can inform asset management decisions by comparing the impacts of different reactive and proactive pipe replacement scenarios. This project is ongoing for FY21, with a focus on studying spatial autocorrelation analyses of pipe failures.
Project Title: Automated calibration of SWMM
A genetic algorithm was developed to provide automated calibration of SWMM. The routine first represents the catchment network as a directed graph object using the NetworkX python package for flexibility in handling real-world observed data availability. Once the calibratable subset of the system is identified, a multi-objective, genetic algorithm (modified Non-dominated Sorting Genetic Algorithm II: NSGA-II) estimates the Pareto front for the objective functions within the feasible performance space.
Project Title: New linear approximation for solving the network water flow problem
supply uncertainty, extreme events, and security threats, depends highly on water distribution networks modeling emphasizing the importance of realistic assumptions, modeling complexities, and scalable solutions. In this study, we propose a derivative‐free, linear approximation for solving the network water flow problem. The proposed approach takes advantage of the special form of the nonlinear head loss equations, and, after the transformation of variables and constraints, the water flow problem reduces to a linear optimization problem that can be efficiently solved by modern linear solvers. Ultimately, the proposed approach amounts to solving a series of linear optimization problems. We demonstrate the proposed approach through several case studies and show that the approach can model arbitrary network topologies and various types of valves and pumps, thus providing modeling flexibility. Under mild conditions, we show that the proposed linear approximation converges. We provide sensitivity analysis and discuss in detail the current limitations of our approach and suggest solutions to overcome these. All the codes, tested networks, and results are freely available on Github for research reproducibility. This project was completed in FY20 except for revisions of manuscripts that are under review and should be completed in FY21.
Project Title: Hedging for privacy in smart water meters
Smart water meters at household connections are being installed in large numbers throughout the world due to the benefits they are expected to bring to the water utilities and water consumers. Smart metering provides high‐resolution readings and promises benefits to the water utilities, such as demand forecasting, regulating time‐of‐use watering, and making intelligent operation and planning decisions. For the consumers, smart metering promises improved billing and demand reduction by providing detailed and timely information about their water use and early notification of possible water leaks in their premises. However, the fine‐grained information collected by smart meters raises growing concerns of privacy invasion due to personal behavior exposure (private activity, daily routine, etc.). Nevertheless, there is no readily available technology for protecting water consumers from revealing their in‐home private activities.
Thus, a viable argument in favor of smart metering technologies will not be possible without proactively accounting for the associated privacy challenges. Here, we present a practical technology coupling a dedicated apparatus with a control model for increasing personal privacy. We quantify the level of privacy achieved using information‐theoretic criterion and an empirically based occupancy detection method between the smart meter readings and actual water use. Furthermore, we evaluate and compare privacy protection using the best effort approach previously developed for masking activities revealed from smart electricity meters. The main results reveal that simple control actions can disguise personal behavior patterns and, thus, hedge against privacy breach in smart water meters. Furthermore, we quantify the trade‐off between the size of the apparatus and the level of privacy protection it provides. Our results demonstrate how “privacy friendly” smart water metering technology could be implemented in real‐life systems and reduce the privacy concerns of water consumers. This project was completed in FY20, and we are using it in FY21 as a foundation for proposals to other agencies that will build upon and extend the research.
Project Title: Mining pressure transients from high frequency pressure sensors.
Pressure transients have been identified as one of the major contributing factors in many pipe failures in water distribution systems (WDSs). The behavior of these pressure transients is largely unknown and cannot be fully assessed by numerical simulation or modeling. This study investigates the behavior of pressure transients in WDSs as measured by high-frequency pressure sensors. A Time Series Data Mining (TSDM) approach is proposed to detect and cluster pressure transients to reveal recurrent and consistent patterns. The proposed technique, based on a modified two-sided cumulative sum (CUSUM) algorithm, is used to detect pressure transients. Dynamic Time Warping (DTW) is adopted to measure the similarity between the detected pressure transients, and k-means clustering algorithm is used to discover the characteristic patterns. Several performance scores are suggested to evaluate the quality of the clustering results, including sum of squared error, Silhouette index, and Calinski-Harabaz index. Results demonstrate that the proposed approach is able to reveal consistent and unique patterns across multiple sensing locations. The proposed approach provides a fast and efficient way to discover the hidden in-formation in WDSs by analyzing high-frequency pressure signals from distributed sensors. This project was completed in FY19.
Project Title: Impact of water saving scenarios on performance of urban water systems
Concerns over the impacts of urban growth have prompted the development and adoption of water-demand management strategies. Water and energy savings from increasingly efficient technologies, diversified water sources, and water savings policies are typically quantified from an individual demand-side basis, but network-wide potential is not well studied. This work studies the effects of residential demand profiles on the performance of urban water networks, in response to emerging demand management strategies. Hydraulic simulations were conducted to assess the performance of base and conservation demand scenarios. Five metrics of network performance are suggested to evaluate responses to scenarios: water loss, water age, peak flow, energy input, and loss. The results revealed that network performance for energy and flow metrics improved under the conservation scenario; however, the conservation scenario had a negative impact on water age and losses throughout the network. The results indicate that the potential benefits from these demand profiles cannot be fully realized without adjustments in network operation and may come at a cost in terms of water quality. This work provides an approach for evaluating network-wide effects of demand management strategies. This project was completed in FY20.
Project Title: Water quality modeling in distribution networks
This work introduces WUDESIM, an open-source C/C++ toolkit for modeling water quality in the dead-end branches of drinking water distribution networks. WUDESIM is linked to the programmers’ toolkit of EPANET, a widely-used public-domain water network analysis model. In place of the advection-based water quality module in EPANET, WUDESIM allows the users to simulate water quality in the pipes of dead-end branches with dispersion as a constituent transport mechanism. In addition, WUDESIM corrects for the simulation errors that arise from the spatial aggregation of water demands due to network skeletonization, and enables the users to use stochastically-generated water demands at fine time-scales. The software comprises two components: a Windows® executable file and a C/C++ dynamic link library (DLL) toolkit. Examples of how the users can employ the different toolkit functions to run water quality analysis and obtain simulation results are provided. WUDESIM is available as a public-domain software. This project was completed with both journal articles and software published in FY20.
Project Title: Flint water crisis
In the aftermath of the lead contamination crisis that plagued the water system in Flint, MI, more than 35,000 water samples were collected from the city’s premises. The majority of these samples (>85%) were collected through a voluntary crowdsourced sampling campaign. The samples were analyzed for lead and copper concentrations by the Michigan Department of Environmental Quality (MDEQ). In this study, the crowdsourced sampling data is analyzed by means of spatial autocorrelation analysis to reveal the locations of statistically significant hotspot regions of high water-lead -evels (WLLs), and to track the spatiotemporal evolution of the WLLs as the city recovered from the lead contamination crisis. The results showed that hotspot regions that experienced high WLLs were consistent with the areas where lead service line (LSL) density is the highest. Yet, galvanized service lines and other lead-containing plumbing components may have also contributed to lead release in hotspot premises. The temporal trend exhibited by the crowdsourced sampling data did not reflect a consistent decrease in the sampled WLLs despite the interventions implemented by MDEQ and EPA. Instead, sampled WLLs remained high for several months after boosting the orthophosphate dose and launching a city-wide residential flushing campaign. The findings of this study suggest that this could be partially attributed to the disproportionate sampling from hotspot premises of high WLLs and LSL density. This project was completed in FY20.
Project Title: Review of WDSA water security
This study reviewed the literature to report on the state-of-the-art in modeling methodologies that have been developed to support the security of water distribution systems. First, we reviewed the major activities that are outlined in the emergency management framework; activities include risk assessment, mitigation, emergency preparedness, response, and recovery. We reviewed simulation approaches and prototype software tools that have been developed by government agencies and research for assessing and mitigating four threat modes, including contamination events, physical destruction, interconnected infrastructure cascading failures, and cybernetic attacks. Modeling tools were mapped to emergency management activities, and analysis of research is provided to group studies based on methodologies that are used and developed to support emergency management activities. Recommendations were made for research needs that will contribute to the enhancement of the security of water distribution systems. This project was completed in FY20 with the publication of the review paper.
Project Title: Extending the capabilities of EPANET user interface with plugin tools.
Simulation of water distribution systems is crucial for the planning, management, operations, and control of municipal water systems. EPANET has been extensively used in a range of design, operation, and management problems that require hydraulic and water quality simulations. Multiple demand conditions, planning scenarios, and various computational methods for integrating with external data sources and programming tools are common challenges to water community that are not directly addressed in the current EPANET software. The US EPA has recently focused on developing an open source modular and extensible user interface, which allows plug-in and scripting support, such that new applications can be integrated and shared within the EPANET environment. In this study, we develop and test the new plugin environment compatible with the new EPANET user interface (UI). To test the new plugin environment of EPANET UI, two plugins were developed in Python scripting language: (1) optimal network design using genetic algorithms and (2) integration of the Water Network Tool for Resilience (WNTR) developed by Sandia National Labs for pressure driven analysis. The EPANET user can add the new plugins to the main EPANET UI screen and select to solve the optimal design problem using genetic algorithms optimization approach. additionally, the user can choose to perform the default demand-driven hydraulic simulation currently implemented in EPANET or choose to perform the pressure-driven hydraulic simulation, which is offered by the WNTR simulation toolkit. This project was completed in FY19.
Project Title: Exploring the Operational Effects of Gentrification on Water Networks
Gentrification can have a negative impact on the operation of the water network -- unless operational adjustments are made corresponding to changing human-infrastructure interactions. A pump upgrade, if existing pumps working at full capacity, would be required. Utilities, which often anticipate changes in the network due to total population increases, would also benefit from an assessment of human-infrastructure interactions such as that occur with gentrification. This project was completed in FY18.
Project Title: Design and Development of a Web-Based EPANET Model Catalog and Execution Environment
This project involves design and implementation of a model-sharing repository and a model-viewing application, specifically for the EPANET modeling community – the pattern and structure of which could be easily adopted by any modeling community – using existing open source cyberinfrastructure. We used HydroShare as the backend data store for the EPANET model program, model instances, and metadata, and we used the rapid app development capabilities of Tethys Platform framework to create a web-based front-end for the repository and viewer. Results of this experimental work include a functional model repository based on less than 700 lines of code and a light-weight model viewer application that encompasses nearly 100% of the legacy EPANET desktop GUI’s functionality. The tools developed in this project are live for testing at https://tethys.byu.edu/ and the source code is all provided in the open source repository at https://github.com/BYU-Hydroinformatics/tethysapp-epanet_model_repository and https://github.com/BYU-Hydroinformatics/tethysapp-epanet_model_viewer. This project was substantially completed in FY20, but there is ongoing work in FY21 to revise publications that are in review.
Future Activities:
Ongoing projects and activities for FY22 include:
- Development and release of Beta version of the SWMM5+ hydraulics engine is scheduled for Q2 of FY22
- Continue developing the code for the new parallelized pipe transient solver.
- Work closely with the SAC and TAC to move through the Beta release and toward the SWMM5+ version 1.0 code release that contains all SWMM functionality. This will include review of the code and testing it against the current SWMM model and gathering recommendations for additional model features and priorities.
- Develop a workshop to explain how the new SWMM5+ code works. The workshop will be held at the Urban Drainage Modeling conference in January 2022.
- Develop “vlog” video-log web content to provide outreach to the practicing community on the use of SWMM5+.
- If public health concerns allow, provide in-person pre-conference SWMM and EPANET courses will be taught at EWRI Congress in Milwaukee in May 2022. Alternatively, these may be replaced by online webinars if COVID is resurgent.
- Continue development of online SWMM and EPANET course curriculum and tools.
- Continue developing back-end and user manuals for web portal.
Journal Articles: 24 Displayed | Download in RIS Format
Other center views: | All 90 publications | 30 publications in selected types | All 24 journal articles |
---|
Type | Citation | ||
---|---|---|---|
|
Abokifa AA, Maheshwari A, Gudi RD, Biswas P. Influence of dead-end sections of drinking water distribution networks on optimization of booster chlorination systems. Journal of Water Resources Planning and Management 2019;145(12):04019053. |
R835950 (2019) R835950 (2020) |
Exit Exit |
|
Abokifa A A, Xing L, Sela L. Investigating the impacts of water conservation on water quality in distribution networks using an advection-dispersion transport model. Water 2020;12(4):1033 |
R835950 (2020) R835950 (Final) |
Exit Exit |
|
Abokifa A, Sela L. Integrating spatial clustering with predictive modeling of pipe failures in water distribution systems. URBAN WATER JOURNAL 2023;20(4):465-476 |
R835950 (2021) |
Exit |
|
Berglund EZ, Pesantez JE, Rasekh A, Shafiee, ME, Sela L, Haxton T. Review of modeling methodologies for managing water distribution security. Journal of Water Resources Planning and Management 2020;146(8):03120001 |
R835950 (2020) R835950 (Final) |
Exit Exit |
|
Hodges B. Conservative finite-volume forms of the Saint-Venant equations for hydrology and urban drainage. Hydrology and Earth System Sciences 2019;23(3):1281-1304. |
R835950 (2021) R835950 (Final) |
Exit Exit |
|
Hodges BR, An artificial compressibility method for 1D simulation of open-channel and pressurized-pipe flow. Wate2020;12:6:1727. |
R835950 (2020) R835950 (Final) |
Exit Exit |
|
Morales-Hernandez M, Sharif MB, Gangrade S, Dullo TT, Kao S, Kalyanapu A, Ghafoor SK, Evans KJ, Madadi-Kanjani E, Hodges BR. High performance computing in water resources hydrodynamics. Journal of Hydroinformatic 2020;22(5):1217–1235 |
R835950 (2020) R835950 (Final) |
Exit Exit |
|
Riaño-Briceño G, Sela L, Hodges B. Distributed and vectorized method of characteristics for fast transient simulations in water distribution systems. Computer-Aided Civil and Infrastructure Engineering 2021;37(2):163-184 |
R835950 (2022) R835950 (Final) |
Exit |
|
Salomons E, Sela L, Housh M. Hedging for privacy in smart water meters. Water Resources Research 2020;56(9):e2020WR027917 |
R835950 (2020) R835950 (Final) |
Exit Exit |
|
Tiernan ED, Hodges BR. A topological approach to partitioning flow networks for parallel simulation. Journal of Computing in Civil Engineering 2022;36(4):04022010. |
R835950 (2021) R835950 (Final) |
Exit |
|
Wang S, Taha A, Sela L, Giacomoni M, Gatsis N. A new derivative-free linear approximation for solving the network water flow problem with convergence guarantees. WATER RESOURCES RESEARCH 2020;56(3). |
R835950 (2021) R835950 (Final) |
Exit Exit |
|
Xing L, Sela L. Unsteady pressure patterns discovery from high-frequency sensing in water distribution systems. Water Research 2019;158:291-300. |
R835950 (2019) R835950 (2020) R835950 (Final) |
Exit |
|
Yu C, Hodges BR, Liu F. A new form of the Saint-Venant equations for variable topography. Hydrology and Earth System Sciences 24:4001-4024, August 2020. |
R835950 (2020) R835950 (Final) |
Exit Exit |
|
Zhuang J, Sela L. Impact of emerging water savings scenarios on performance of urban water networks. Journal of Water Resources Planning and Management 2019;146(1):04019063. |
R835950 (2019) R835950 (2020) R835950 (Final) |
Exit Exit Exit |
|
Abokifa AA, Sela L. Identification of spatial patterns in water distribution pipe failure data using spatial autocorrelation analysis. Journal of Water Resources Planning and Management 2019;145(12):04019057. |
R835950 (2019) R835950 (2020) R835950 (Final) |
Exit Exit |
|
Abokifa AA, Katz L, Sela L. Spatiotemporal trends of recovery from lead contamination in Flint, MI as revealed by crowdsourced water sampling. Water Research 2019:115442. |
R835950 (2019) R835950 (2020) R835950 (Final) |
Exit Exit Exit |
|
Abokifa A, Biswas P, Hodges BR, Sela L. WUDESIM:a toolkit for simulating water quality in the dead-end branches of drinking water distribution networks. Urban Water Journal 2020;17(1):54-64. |
R835950 (2020) R835950 (Final) |
Exit Exit |
|
Hodges BR, Liu F. Timescale interpolation and no-neighbour discretization for a 1D finite-volume Saint-Venant solver. Journal of Hydraulic Research 2019:1-7. |
R835950 (2019) R835950 (2020) R835950 (Final) |
Exit Exit |
|
Bayer T, Ames DP, Cleveland TG. Design and development of a web-based EPANET model catalogue and execution environment. Annals of GIS 2021;27(3):247-260. |
R835950 (Final) |
not available |
|
Riaño-Briceño G, Hodges BR, Sela L. PTSNet:A Parallel Transient Simulator for Water Transport Networks based on vectorization and distributed computing. Environmental Modelling & Software 2022;158:105554. |
R835950 (2022) R835950 (Final) |
Exit Exit |
|
Sharior S, Hodges BR, Vasconcelos JG. Generalized, dynamic, and transient-storage form of the Preissmann Slot. Journal of Hydraulic Engineering 2023;149(11):04023046. |
R835950 (Final) |
Exit |
|
Faure JC, Faust KM. Socioeconomic characteristics versus density changes:the operational effects of population dynamics on water systems. Sustainable and Resilient Infrastructure 2023;8(1):3-16. |
R835950 (Final) |
Exit |
|
Yu CW, Hodges BR, Liu F. Automated detection of instability-inducing channel geometry transitions in Saint-Venant simulation of large-scale river networks. Water 2021:2236. |
R835950 (Final) |
Exit |
|
Wang S, Taha AF, Sela L, Gatsis N, Giacomoni MH. State Estimation in Water Distribution Networks through a New Successive Linear Approximation. In2019 IEEE 58th Conference on Decision and Control (CDC) 2019 Dec 11 (pp. 5474-5479). IEEE. |
R835950 (2020) |
Exit |
Progress and Final Reports:
Original AbstractThe perspectives, information and conclusions conveyed in research project abstracts, progress reports, final reports, journal abstracts and journal publications convey the viewpoints of the principal investigator and may not represent the views and policies of ORD and EPA. Conclusions drawn by the principal investigators have not been reviewed by the Agency.
Project Research Results
- Final Report
- 2022 Progress Report
- 2020 Progress Report
- 2019 Progress Report
- 2018 Progress Report
- 2017 Progress Report
- Original Abstract
24 journal articles for this center