Welcome to the Firefish Project

Firefish is a Peer-to-Peer Grid service infrastructure for access to dynamic data featuring ease-of-use, interoperability, scalability and performance.

It is used to find, access and aggregate information and resources in a large distributed system composed of many autonomous data sources, dynamic and heterogeneous information, resources, participants and networks.

Firefish is the open source reference implementation of the P2PIO protocol specification, based on the p2pio.wsdl and the following XML schemas: p2pio.xsd and firefish-queries.xsd.

Goals

In a distributed system, many higher-level services depend on the maintenance and discovery of information about dynamic resources such as services and user communities. Existing systems typically use a central database, a hierarchical directory, multicast announcement, or an application-specific ad-hoc Peer-to-Peer network. Such technologies typically do not work well for large-scale decentralized systems that attempt to support a wide range of application queries, over autonomous and heterogeneous data sources [producing static to highly dynamic data].

Our proposal is a new model for resource discovery in decentralized Grid systems. This model is derived from our work in developing Grid Database Frameworks for use by scientists to support large-scale Grid projects.

The multi-purpose P2P resource discovery infrastructure model supports a wide range of use cases with a small set of extensible building blocks. The proposal is query language independent and XML data model independent. It supports pull and push based data delivery, and can accommodate static as well as dynamic data sources, while being agnostic wrt. transport, network, and topology.

In particular, the model supports powerful XQueries for complex XML data selection, integration, aggregation and transformation for resource discovery in dynamic large-scale decentralized systems.

In the working prototype, applications get to focus on domain specific plugins. For example, the data source can be anything that produces arbitrary XML data. An example data source is an XML file. Another example data source is a program that takes some input and produces SensorML XML output on-the-fly. Yet another example data source is a relational database (e.g. MySQL) that takes a SQL query and wraps the query output into XML. Another transforming query such as a complex XQuery can then be executed against the wrapped XML output. XQuery is the most powerful query language, supported as one of several example plugins. Examples for simple query languages are"return <hello> world </hello>" or "return currentTime()".

Features

• Data independent: Any structured or semi-structured XML data can be queried
• Integrate static data from files, relational or XML databases.
• Integrate dynamic data generated on-the-fly.
• Query language independent: Example plugins support XQuery, XPath, SQL, Regular Expressions, custom query languages.
• Network and Transport independent: E.g. TCP, WSDL/SOAP, InterGroup/SGL (currently supports WSDL WebService over SOAP/HTTP)
• Allows for arbitrary topologies (leading to hybrid system of systems).
• Fault-tolerant, scalable, interoperable.
• Easy-to-use and deploy, easy to extend and customize.
• Application developers get to focus on app-specific problems (plug-in).

How does the P2PIO protocol work?

• P2PIO specifies XML messages to find, query and retrieve information across a P2P network.
• The model is loosely analogous to streaming file system I/O:
  • Streaming file I/O: Open file; multiple seq. reads; close file
  • P2PIO: Open transaction; multiple seq. receives; close transaction
• The employed Message Exchange Patterns are as follows:

  Open    --> Ok | Error
  Receive --> Send[0..N], (FinalSend | Error)
  Close   --> Ok | Error
  Invite  --> Ok | Error
  

• A Send or FinalSend message can contain any number of XML elements.
• A Receive message can ask for single or multiple Send messages.
• Query Scope and Routing Policy is given via a Neighbor Selection Query.