About
Data integration and service orchestration deals with the combination of data and sevices that reside at different sources (e.g. in databases, in unstructured form such as Twitter data, in RDF graphs, etc.) in order to provide consumers of the data with a unified view over enriched data or a composed servcie. This integration process is very time consuming and, more critically, it often needs to be adapted to the specifics of each scenario. In addition there is often the need to transform existing data into a new format that is more appropriated or even needed for a particular application. For example, raw data that was published in naive RDF schema or quick publishing of relational data, will have to be post-processed in order for it to conform to a more sophisticated existing schema. It it this type of data transformation that RDFGears was also designed for.
The RDF Gears framework for RDF service orchestration and data transformation was developed in the context of the ImReal project to simplify the data integration task. It provides a language with formal syntax and semantics, it has an intuitive user interface and it was designed to be implemented efficiently on top of different back ends, both conventional and cloud-based.
Application within the ImREAL and DataBridges project
The RDFGears engine is used as the central orchestration in U-Sem, a user modeling infrastructure for the Social Web that is developed by the WIS group. It is used there as a flexible interface to make and adapt different com positions of services for the analysis of learner profiles in order to generate richer learner models that can be used within the ImREAL simulators.
Within the Data Bridges project, which aims at providing smart data services to the public supporting digital cities in the future, it is used as the main datatransformation en enrichment engine to gather, link and integrate data about and for citizens of the smart city.
New features within RDFGears
Since its last version in 2011 the user interface has been improved with a cleaner look and better organisation, and now allows explicit editing of nested workflows, which makes it easier to organize complex workflows in a more accessible manner. Also the collection of available services and functions has been extended with a directory structure which makes it easier to locate appropriate services.
Furthermore the type system which checks if the services in a workflow are correctly connected has been extended with generic parameterised types such that the construction of reusable components becomes easier. Moreover the well-typedness is now checked realtime while the workflow designer is editing the workflow, which leads to earlier detection of mistakes.
Finally a communication protocol between the RDFGears frontend and backend has been introduced to allow the same frontend to work with different backends. The protocol lets the frontend query the backend about the components that it knows and understands, and therefore can be included in workflows. This makes it possible to have different specialized backends that are optimized for specific types of workflows with certain components, or different backends that are optimized for memory-based, disk-based or cloud-based execution.
Slides introducing the RDFGears research project
wiser-day-2012.pdf [632Kb]
A movie introducing RDFGears
usem-rdfgears-intro.swf [5.6 Mb]
Objective
The objective of RDFGears is to provide a user-friendly, versatile and scalable engine for the integration of different datasources and services. It provides a graphical interface for the specification of simple and complex integration workflows for the enrichment and adaptation of data to make at suitable for a specific target application.
Demo
The user interface of RDF Gears can be accessed at:
http://wisserver.st.ewi.tudelft.nl:8080/rdfgears-ui/
This particular instance of RDF Gears contains all workflows that have been created for the ImREAL project.
It allows developers to build their own strategies or workflows in a graphical manner. Services that consume, transform or output RDF data can be easily combined by creating a pipeline where the output of one service functions as the input of the next service. There is no need to directly access the different components and manually assemble their input/output. RDF Gears also takes care of the eventual execution of the workflow.
A screenshot of the interface is shown below. On the left hand side the existing workflows (which can be saved/edited/deleted) are shown, while in the Designer tab, the workflow can be edited. A click on the workflow button on the left starts the execution of the workflow.
