Grantee Research Project Results
2019 Progress Report: A Sustainable Center for Crowd-Sourced Water Infrastructure Modeling
EPA Grant Number: R835950Center: Gulf Coast HSRC (Lamar)
Center Director: Ho, Tho C.
Title: A Sustainable Center for Crowd-Sourced Water Infrastructure Modeling
Investigators: Hodges, Ben R. , Cleveland, Theodore G. , Ames, Daniel P. , Leite, Fernanda , Berglund, Emily , Brashear, Bob , Xing, Lu , Sela, Lina , Zhuang, Janice , Abokifa, Ahmed A , Kandjani, Ehsan Madadi , Tiernan, Edward , Riano-Briceno, Gerardo , Rowney, A. Charles , Faust, Kasey , Faure, Julie
Current Investigators: Hodges, Ben R. , Cleveland, Theodore G. , Barrett, Michael E. , Ames, Daniel P. , Leite, Fernanda , Berglund, Emily , Urbonas, Ben , Brashear, Bob , Rowney, A. Charles
Institution: The University of Texas at Austin , Texas State University , North Carolina State University , Brigham Young University
Current Institution: Brigham Young University , The University of Texas at Austin , North Carolina State University , Texas Tech University , Urban Watersheds Research Institute
EPA Project Officer: Packard, Benjamin H
Project Period: September 1, 2016 through August 31, 2021 (Extended to August 31, 2023)
Project Period Covered by this Report: September 1, 2018 through August 31,2019
Project Amount: $3,999,803
RFA: National Center for Sustainable Water Infrastructure Modeling Research (2014) RFA Text | Recipients Lists
Research Category: Water Quality , Water
Objective:
Project: Mining pressure transients from high frequency pressure sensors
Discover new methods for evaluating pressure transients in water distribution systems.
Project: Impact of water saving scenarios on performance of urban water systems
Evaluate how changing water demand affects water distribution system operational metrics.
Project: Water quality modeling in distribution networks
Develop water quality model for dead-end branches in water distribution systems.
Project: Flint water crisis
Examine trends in recovery from lead contamination after remediation interventions.
Project: Pipe failure modeling and analysis
Develop computational framework for water distribution infrastructure asset management to predict pipe failure.
Project: Improved hydraulic network solver for SWMM
Develop an approach to solving the Saint-Venant equations for open-channel sewer network flow that can be used for massively parallel computers.
Project: Automated calibration of SWMM
Develop and automated calibration tool for SWMM
Project:
Develop an efficient parallel solver for transient analyses in complex networks.
Project: Outreach to the SWMM and EPANET communities
Build the foundations for a sustainable community of model developers and users.
Project: Review of WDSA water security
Review state-of-the-art in cyber-physical threats, modeling approaches, and algorithms for water distribution systems security.
Project: Extending the capabilities of EPANET user interface with plugin tools.
Extending the capabilities of EPANET user interface with plugin tools. Simulation of water distribution systems is crucial for the planning, management, operations, and control of municipal water systems. EPANET has been extensively used in a range of design, operation, and management problems that require hydraulic and water quality simulations. Multiple demand conditions, planning scenarios, and various computational methods for integrating with external data sources and programming tools are common challenges to water community that are not directly addressed in the current EPANET software. The US EPA has recently focused on developing an open source modular and extensible user interface, which allows plug-in and scripting support, such that new applications can be integrated and shared within the EPANET environment. In this study, we develop and test the new plugin environment compatible with the new EPANET user interface (UI).
Project: On-line implementation of models for training
Develop and deploy a software-as-a-service implementation of EPANET/SWMM for on-line access and running of the codes for training purposes.
Project: Community web portal and model repository
Develop a web-centered community for SWMM and EPANET
Project: Exploring the Operational Effects of Gentrification on Water Networks
Exploring the Operational Effects of Gentrification on Water Networks
Progress Summary:
Project: Mining pressure transients from high frequency pressure sensors.
Pressure transients have been identified as one of the major contributing factors in many pipe failures in water distribution systems (WDSs). The behavior of these pressure transients is largely unknown and cannot be fully assessed by numerical simulation or modeling. This study investigates the behavior of pressure transients in WDSs as measured by high-frequency pressure sensors. A Time Series Data Mining (TSDM) approach is proposed to detect and cluster pressure transients to reveal recurrent and consistent patterns. The proposed technique, based on a modified two-sided cumulative sum (CUSUM) algorithm, is used to detect pressure transients. Dynamic Time Warping (DTW) is adopted to measure the similarity between the detected pressure transients, and k-means clustering algorithm is used to discover the characteristic patterns. Several performance scores are suggested to evaluate the quality of the clustering results, including sum of squared error, Silhouette index, and Calinski-Harabaz index. Results demonstrate that the proposed approach is able to reveal consistent and unique patterns across multiple sensing locations. The proposed approach provides a fast and efficient way to discover the hidden in-formation in WDSs by analyzing high-frequency pressure signals from distributed sensors.
Project: Impact of water saving scenarios on performance of urban water systems
Concerns over the impacts of urban growth have prompted the development and adoption of water-demand management strategies. Water and energy savings from increasingly efficient technologies, diversified water sources, and water savings policies are typically quantified from an individual demand-side basis, but network-wide potential is not well studied. This work studies the effects of residential demand profiles on the performance of urban water networks, in response to emerging demand management strategies. Hydraulic simulations were conducted to assess the performance of base and conservation demand scenarios. Five metrics of network performance are suggested to evaluate responses to scenarios: water loss, water age, peak flow, energy input, and loss. The results revealed that network performance for energy and flow metrics improved under the conservation scenario; however, the conservation scenario had a negative impact on water age and losses throughout the network. The results indicate that the potential benefits from these demand profiles cannot be fully realized without adjustments in network operation and may come at a cost in terms of water quality. This work suggests an initial tool for evaluating network-wide effects of demand management strategies.
Project: Water quality modeling in distribution networks
This work introduces WUDESIM, an open-source C/C++ toolkit for modeling water quality in the dead-end branches of drinking water distribution networks. WUDESIM is linked to the programmers’ toolkit of EPANET, a widely-used public-domain water network analysis model. In place of the advection-based water quality module in EPANET, WUDESIM allows the users to simulate water quality in the pipes of dead-end branches with dispersion as a constituent transport mechanism. In addition, WUDESIM corrects for the simulation errors that arise from the spatial aggregation of water demands due to network skeletonization, and enables the users to use stochastically-generated water demands at fine time-scales. The software comprises two components: a Windows® executable file and a C/C++ dynamic link library (DLL) toolkit. Examples of how the users can employ the different toolkit functions to run water quality analysis and obtain simulation results are provided. WUDESIM is available as a public-domain software.
Project: Flint water crisis
In the aftermath of the lead contamination crisis that plagued the water system in Flint, MI, more than 35,000 water samples were collected from the city’s premises. The majority of these samples (>85%) were collected through a voluntary crowdsourced sampling campaign. The samples were analyzed for lead and copper concentrations by the Michigan Department of Environmental Quality (MDEQ). In this study, the crowdsourced sampling data is analyzed by means of spatial autocorrelation analysis to reveal the locations of statistically significant hotspot regions of high water-lead -evels (WLLs), and to track the spatiotemporal evolution of the WLLs as the city recovered from the lead contamination crisis. The results showed that hotspot regions that experienced high WLLs were consistent with the areas where lead service line (LSL) density is the highest. Yet, galvanized service lines and other lead-containing plumbing components may have also contributed to lead release in hotspot premises. The temporal trend exhibited by the crowdsourced sampling data did not reflect a consistent decrease in the sampled WLLs despite the interventions implemented by MDEQ and EPA. Instead, sampled WLLs remained high for several months after boosting the orthophosphate dose and launching a city-wide residential flushing campaign. The findings of this study suggest that this could be partially attributed to the disproportionate sampling from hotspot premises of high WLLs and LSL density.
Project: Pipe failure modeling and analysis
Pipe failures in water distribution infrastructure (WDI) have significant economic, environmental, and public health impacts. To alleviate these impacts, repair and replacement decisions need to be prioritized in order to effectively reduce pipe failure rates. Computational frameworks for WDI asset management have been developed that couple spatial clustering analysis with predictive modeling of pipe failures. The first part of this project focused on adopting clustering algorithms to explore spatial and spatiotemporal patterns exhibited by pipe failure. This was done by: (1) implementing spatiotemporal scan statistics for the identification of density-based clusters of failure events; and (2) applying local indicators of spatial association to identify statistically significant hotspot and coldspot clusters of exceptionally high and low pipe failure rates. The second part involved developing predictive modeling tools for forecasting pipe failures. Tests have been conducted to evaluate the predictive abilities of eight different statistical learning techniques, and the best performing method was implemented to forecast pipe failure rates within the different sectors of the WDI while explicitly accounting for the spatial patterns exhibited by the failure rate and its predictors. Survival-based models have been developed to predict the time to failure as a function of the different failure predictors, including pipe characteristics, soil conditions, traffic, and pressure. The developed frameworks where demonstrated on a real-life, large-scale WDI, and where shown to provide useful insights that can inform asset management decisions by comparing the impacts of different reactive and proactive pipe replacement scenarios.
Project: Improved hydraulic network solver for SWMM
The new methods for open-channel and closed channel flow have been developed and tested using Python code. Results show that we can obtain mass conservation and stable solutions with much less computational effort than the existing SWMM computational engine. A new code structure has been developed using the Fortran2008 programming language. Implementation and testing of the new algorithms are in progress.
Project: Automated calibration of SWMM
A genetic algorithm was developed to provide automated calibration of SWMM. The routine first represents the catchment network as a directed graph object using the NetworkX python package for flexibility in handling real-world observed data availability. Once the calibratable subset of the system is identified, a multi-objective, genetic algorithm (modified Non-dominated Sorting Genetic Algorithm II: NSGA-II) estimates the Pareto front for the objective functions within the feasible performance space.
Project:
Several software programs have been developed to model transient flow in water systems, and more specifically, using the method of characteristics (MOC) to find a solution for the flow equations derived in accordance to the so-called elastic theory. Although the numerical scheme of the MOC approach is considered to generate numerically accurate solutions, it is often characterized by longer running times and, thus, is limited by its computational efficiency. The ability to provide high-fidelity solutions in short simulation times, and to preserve the topological and geometrical characteristics of the real system is especially important for practical applications involving modeling large-scale water systems with hundreds to thousands of pipes and junctions. This work presents a novel complete hybrid algorithm that parallelizes MOC by integrating shared-and distributed-memory schemes, as well as vectorizing the equations for inner points and basic boundary conditions, such as, pipe junctions, reservoirs, dead-ends, valves, and pumps. Specifically, the new approach exploits vectorization to compute the MOC equations in parallel using either CPUs or GPUs.
Project: Outreach to the SWMM and EPANET communities
The Center has been conducting extensive outreach at professional and technical conferences to provide opportunities for the community to evaluate the approaches being taken in the new SWMM computational engine.
Project: Review of WDSA water security
This study reviewed the literature to report on the state-of-the-art in modeling methodologies that have been developed to support the security of water distribution systems. First, we reviewed the major activities that are outlined in the emergency management framework; activities include risk assessment, mitigation, emergency preparedness, response, and recovery. We reviewed simulation approaches and prototype software tools that have been developed by government agencies and research for assessing and mitigating four threat modes, including contamination events, physical destruction, interconnected infrastructure cascading failures, and cybernetic attacks. Modeling tools were mapped to emergency management activities, and analysis of research is provided to group studies based on methodologies that are used and developed to support emergency management activities. Recommendations were made for research needs that will contribute to the enhancement of the security of water distribution systems.
Project: Extending the capabilities of EPANET user interface with plugin tools.
To test the new plugin environment of EPANET UI, two plugins were developed in Python scripting language: (1) optimal network design using genetic algorithms and (2) integration of the Water Network Tool for Resilience (WNTR) developed by Sandia National Labs for pressure driven analysis. The EPANET user can add the new plugins to the main EPANET UI screen and select to solve the optimal design problem using genetic algorithms optimization approach. additionally, the user can choose to perform the default demand-driven hydraulic simulation currently implemented in EPANET or choose to perform the pressure-driven hydraulic simulation, which is offered by the WNTR simulation toolkit.
Project: On-line implementation of models for training
A protoype on-line implementation was built using apache2 with cgi-bin to run command-line implementations of EPANET and SWMM. An up-loader script was built to allow client access to create and populate directories containing input and output files. These implementations require use of .inp files -- users can edit the files, but need to know the file structure.
A subsequent prototype web service was built using TightVNC and noVNC to allow access to the SaaS through a web browser on an x86-64 architecture. Switching from the ARM7 to an x86-64 solved the graphics problem; and fully functional instances of EPANET/SWMM with the GUI intact can be served over a browser connection. This exploratory work is to allow for a NCIMM built/managed data center to serve working instances of EPANET/SWMM with the GUI intact using a web-browser. The client only needs a relatively modern browser and a data center supplied password. The prototype is demonstrated at https://youtu.be/D6VPSfFMQvw
Project: Community web portal and model repository
Prototype development is underway for of NCIMM website and community user portal to support software and training for SWMM and EPANET models. We are developing a cyber-infrastructure back-end to allow model sharing and analyses.
A working prototype of a web-based implementation is available, with an interface that allows file editing. Users must know the .inp file structure to edit models effectively.
Project: Exploring the Operational Effects of Gentrification on Water Networks
Gentrification can have a negative impact on the operation of the water network -- unless operational adjustments are made corresponding to changing human-infrastructure interactions. A pump upgrade, if existing pumps working at full capacity, would be required. Utilities, which often anticipate changes in the network due to total population increases, would also benefit from an assessment of human-infrastructure interactions such as that occur with gentrification
Future Activities:
Project: Water quality modeling in distribution networks
Revise the paper for publication and provide software distribution from online repository.
Project: Flint water crisis
Revise submitted paper for publication.
Project: Pipe failure modeling and analysis
Revise paper for publication. An additional manuscript is in preparation on coupling of spatial autocorrelation analyses with pipe failures and will be submitted.
Project: Improved hydraulic network solver for SWMM
Development and testing of the new algorithms will continue in the next project year.
Project: Automated calibration of SWMM
Model is undergoing further testing. Manuscript describing the model is in preparation. Ph.D. dissertation that includes this work is being developed.
Project:
Code development and testing is continuing. Journal papers and a Ph.D. dissertation are in development
Project: Outreach to the SWMM and EPANET communities
Continue to develop community relationships to build sustainable open-source collaborations for the future of SWMM and EPANET.
Project: Review of WDSA water security
Revise manuscript for publication.
Project: On-line implementation of models for training
Test the demonstration site for limited users (~25 concurrent connections) at a future training camp -- the prototype may not have the capacity to support 25 concurrent connections (its an atom-based SBC running on a residential connection). Develop AWS EC2 instances to replace the physical hardware, when successful write a user manual for remote users to use the service. Arrange to spend a portion of a training class to demonstrate and test the system with trainees. A SWMM implementation is nearly completed and it will be pre-populated with training vignettes derived from the NCIMM sponsored training classes. About 72 training examples are anticipated.
Project: Community web portal and model repository
Completed cyber-infrastructure back-end to allow model sharing and analyses. A client manual with examples for the on-line implementation is in-progress.
Journal Articles: 21 Displayed | Download in RIS Format
| Other center views: | All 74 publications | 24 publications in selected types | All 21 journal articles |
|---|
| Type | Citation | ||
|---|---|---|---|
|
|
Abokifa AA, Maheshwari A, Gudi RD, Biswas P. Influence of dead-end sections of drinking water distribution networks on optimization of booster chlorination systems. Journal of Water Resources Planning and Management 2019;145(12):04019053. |
R835950 (2019) R835950 (2020) |
Exit Exit |
|
|
Abokifa A A, Xing L, Sela L. Investigating the impacts of water conservation on water quality in distribution networks using an advection-dispersion transport model. Water 2020;12(4):1033 |
R835950 (2020) |
Exit Exit |
|
|
Abokifa A, Sela L. Integrating spatial clustering with predictive modeling of pipe failures in water distribution systems. URBAN WATER JOURNAL 2023;20(4):465-476 |
R835950 (2021) |
Exit |
|
|
Berglund EZ, Pesantez JE, Rasekh A, Shafiee, ME, Sela L, Haxton T. Review of modeling methodologies for managing water distribution security. Journal of Water Resources Planning and Management 2020;146(8):03120001 |
R835950 (2020) |
Exit Exit |
|
|
Hodges B. Conservative finite-volume forms of the Saint-Venant equations for hydrology and urban drainage. Hydrology and Earth System Sciences 2019;23(3):1281-1304. |
R835950 (2021) |
Exit Exit |
|
|
Hodges B, Sharior S, Tiernan E, Jenkins E, Rano-Briceno G, Davila-Hernandez C, Madadi-Kandjani E, Yu C. Introducing SWMM5+. JOURNAL OF ENVIRONMENTAL ENGINEERING 2024;150(10) |
R835950 (Final) |
Exit |
|
|
Hodges BR, An artificial compressibility method for 1D simulation of open-channel and pressurized-pipe flow. Wate2020;12:6:1727. |
R835950 (2020) |
Exit Exit |
|
|
Morales-Hernandez M, Sharif MB, Gangrade S, Dullo TT, Kao S, Kalyanapu A, Ghafoor SK, Evans KJ, Madadi-Kanjani E, Hodges BR. High performance computing in water resources hydrodynamics. Journal of Hydroinformatic 2020;22(5):1217–1235 |
R835950 (2020) |
Exit Exit |
|
|
Riaño-Briceño G, Sela L, Hodges B. Distributed and vectorized method of characteristics for fast transient simulations in water distribution systems. Computer-Aided Civil and Infrastructure Engineering 2021;37(2):163-184 |
R835950 (2022) |
Exit |
|
|
Salomons E, Sela L, Housh M. Hedging for privacy in smart water meters. Water Resources Research 2020;56(9):e2020WR027917 |
R835950 (2020) |
Exit Exit |
|
|
Tiernan ED, Hodges BR. A topological approach to partitioning flow networks for parallel simulation. Journal of Computing in Civil Engineering 2022;36(4):04022010. |
R835950 (2021) |
Exit |
|
|
Wang S, Taha A, Sela L, Giacomoni M, Gatsis N. A new derivative-free linear approximation for solving the network water flow problem with convergence guarantees. WATER RESOURCES RESEARCH 2020;56(3). |
R835950 (2021) |
Exit Exit |
|
|
Xing L, Sela L. Unsteady pressure patterns discovery from high-frequency sensing in water distribution systems. Water Research 2019;158:291-300. |
R835950 (2019) R835950 (2020) |
Exit |
|
|
Yu C, Hodges BR, Liu F. A new form of the Saint-Venant equations for variable topography. Hydrology and Earth System Sciences 24:4001-4024, August 2020. |
R835950 (2020) |
Exit Exit |
|
|
Zhuang J, Sela L. Impact of emerging water savings scenarios on performance of urban water networks. Journal of Water Resources Planning and Management 2019;146(1):04019063. |
R835950 (2019) R835950 (2020) |
Exit Exit Exit |
|
|
Abokifa AA, Sela L. Identification of spatial patterns in water distribution pipe failure data using spatial autocorrelation analysis. Journal of Water Resources Planning and Management 2019;145(12):04019057. |
R835950 (2019) R835950 (2020) |
Exit Exit |
|
|
Abokifa AA, Katz L, Sela L. Spatiotemporal trends of recovery from lead contamination in Flint, MI as revealed by crowdsourced water sampling. Water Research 2019:115442. |
R835950 (2019) R835950 (2020) |
Exit Exit Exit |
|
|
Abokifa A, Biswas P, Hodges BR, Sela L. WUDESIM:a toolkit for simulating water quality in the dead-end branches of drinking water distribution networks. Urban Water Journal 2020;17(1):54-64. |
R835950 (2020) |
Exit Exit |
|
|
Hodges BR, Liu F. Timescale interpolation and no-neighbour discretization for a 1D finite-volume Saint-Venant solver. Journal of Hydraulic Research 2019:1-7. |
R835950 (2019) R835950 (2020) |
Exit Exit |
|
|
Riaño-Briceño G, Hodges BR, Sela L. PTSNet:A Parallel Transient Simulator for Water Transport Networks based on vectorization and distributed computing. Environmental Modelling & Software 2022;158:105554. |
R835950 (2022) |
Exit Exit |
|
|
Wang S, Taha AF, Sela L, Gatsis N, Giacomoni MH. State Estimation in Water Distribution Networks through a New Successive Linear Approximation. In2019 IEEE 58th Conference on Decision and Control (CDC) 2019 Dec 11 (pp. 5474-5479). IEEE. |
R835950 (2020) |
Exit |
Supplemental Keywords:
water distribution systems, failure analysis,water demand, water quality,EPANET,lead contamination,numerical solver, open-channel flow,model calibration, community outreach, model development,water distribution threat management,review,emergency management,security,modeling and simulation,plugin tools, genetic algorithms, WNTR, Software as a Service, on-line SWMM, on-line EPANET,web portal, model repository, gentrification, operationsRelevant Websites:
Progress and Final Reports:
Original AbstractThe perspectives, information and conclusions conveyed in research project abstracts, progress reports, final reports, journal abstracts and journal publications convey the viewpoints of the principal investigator and may not represent the views and policies of ORD and EPA. Conclusions drawn by the principal investigators have not been reviewed by the Agency.
Project Research Results
- Final Report
- 2022 Progress Report
- 2021 Progress Report
- 2020 Progress Report
- 2018 Progress Report
- 2017 Progress Report
- Original Abstract
21 journal articles for this center