Lawrence Berkeley National Lab

On-Demand Overlays for Scientific Applications

PI: Deb Agarwal(ACS) and Chin Guok (ESnet)
Collaborators: University of Delaware and I2
website: N/A

Scientific computations and collaborations increasingly rely on the network to provide high-speed data transfer, dissemination of results, access to instruments, support for computational steering, etc. Networks with redundant high-speed paths need algorithms to effectively allocate bandwidth across multiple paths for a single application. This capability is required for two important communication classes within the DOE scientific community: large point-to-point transfers and periodic data dissemination from a single sender to multiple receivers. The overall goal of this project was to perform the research and development necessary to create proof-of-concept on-demand overlays for scientific applications that make efficient and effective use of the available network resources.

In this project we focused on developing mechanisms for instantiating on-demand virtual overlays on the physical ESnet network and integrating those mechanisms with the OSCARS dynamic circuit reservation system. This past year, we developed a prototype system to transparently route user traffic onto an overlay network on ESnet using the Phoebus system from the University of Delaware. This new routing mechanism allows users to take full advantage of the on-demand overlay network without any changes to their existing applications. We also conducted a range of tests to understand the effectiveness of the Phoebus overlay network for bulk data transfer applications. GridFTP was used as the bulk transfer application. Our tests show the value of the overlay network, especially when integrated with the OSCARS dynamic circuit network (DCN). One important result from these tests was an understanding that using an overlay network allows for protocol translation between nodes in the overlay.


Please see the publications page.