Citation:

Smith, D., R. Parmar, AND K. Wolfe. Scientific Computing in the Cloud. iEMSs 2022, Brussels, BELGIUM, July 04 - 08, 2022.

Impact/Purpose:

Cloud computing provides an adaptive environment from which to execute a computational workflow. A scientific computation workflow could be executed through a web application, web application programming interface (API), or a command line interface (CLI) depending on the requirements of the problem. Compute resource scaling provides the means to tackle workflows and simulations without the need to update any code. When cloud computing and Kubernetes are paired with a technology such as dask, additional nodes can be added to the Kubernetes cluster for rapidly scaling up the available dask workers for large workflow automatically. The adaptability and scalability of Kubernetes in the cloud enables new approaches for executing environmental simulations of increased complexity and scale towards what has traditionally been the domain of high-performance computing.

Description:

Scientific computing applications can be complex to design, difficult to deploy, and expensive computationally. Using the latest advances in software engineering and cloud technologies, we can greatly mitigate these challenges while also providing new opportunities to pursue research with increased flexibility. Our Hydrologic Micro Services (HMS) technology stack has been developed as a containerized set of components, orchestrated through Kubernetes, that can be managed, updated, and deployed with relative ease to any cloud platform. The general framework can be applied to a variety of different scientific software applications, that can be containerized, providing the ability to quickly create complete technology stacks for solving new or unique problems, or integrate new features into existing applications. Combining flexible cloud compute resources, automatic horizontal scaling (distributed computing) and vertical scaling (parallel processing) using cloud platforms, Kubernetes and Dask, our scientific software applications can be used to tackle environmental problems in new and efficient ways. In addition, the deployment of applications using Kubernetes can drastically reduce the amount of time required for application and server management, freeing up more time for research and solving emerging environmental problems. Our HMS technology stack contains the following components: ·        Nginx, http traffic routing ·        Django, serves up web content ·        Flask, REST API ·        Dask, asynchronous, distributed, and parallel processing ·        .NET 6, data provision and simulation application ·        MongoDB, data storage

URLs/Downloads:

Record Details: