On-Demand Overlays for Scientific Applications
| PI: | Deb Agarwal(ACS) and Chin Guok (ESnet) |
| Collaborators: | University of Delaware and I2 |
| website: | N/A |
Scientific computations and collaborations increasingly rely on the network to provide high-speed data transfer, dissemination of results, access to instruments, support for computational steering, etc. Networks with redundant high-speed paths need algorithms to effectively allocate bandwidth across multiple paths for a single application. This capability is required for two important communication classes within the DOE scientific community: large point-to-point transfers and periodic data dissemination from a single sender to multiple receivers. The overall goal of this project was to perform the research and development necessary to create proof-of-concept on-demand overlays for scientific applications that make efficient and effective use of the available network resources.
In this project we focused on developing mechanisms for instantiating on-demand virtual overlays on the
physical ESnet network and integrating those mechanisms with the OSCARS dynamic circuit reservation
system. This past year, we developed a prototype system to transparently route user traffic onto an overlay
network on ESnet using the Phoebus system from the University of Delaware. This new routing mechanism
allows users to take full advantage of the on-demand overlay network without any changes to their
existing applications. We also conducted a range of tests to understand the effectiveness of the Phoebus
overlay network for bulk data transfer applications. GridFTP was used as the bulk transfer application. Our
tests show the value of the overlay network, especially when integrated with the OSCARS dynamic circuit
network (DCN). One important result from these tests was an understanding that using an overlay network
allows for protocol translation between nodes in the overlay.
Publications
Please see the publications page.