Web Information Systems

Introduction

The WUDE (Web User Demand Elicitation) project is part of the SEALINCMedia project (work package five), an initiative of the FES program COMMIT.

SEALINCMedia has the objective to develop solutions to enrich cultural heritage collections using Internet-enabled reliable, scalable and cost effective collaborative content curation and to improve accessibility through advanced personalized content recommendation and search functionalities.

In the context of SEALINCMedia, WUDE aims at providing a framework for user modeling and demand elicitation with an application in crowdsourcing and niche-sourcing, tailored to the needs of cultural data management organizations. By capitalizing on the experience of the Web Information System group in adaptive Web-based systems and social-computation, WUDE will tap the Social Web and human computation platforms to provide the knowledge required for the design and execution of efficient and effective crowdsourcing campaigns.

To achieve such an ambitious goal, WUDE will develop methods and tools aimed at

• Eliciting the demands of Web users accessing linked cultural data
• Understanding and put in practice the "enrichment" requirements of cultural data management organizations
• Assessing the expertise required by the individuals and social communities involved in the crowdsourcing tasks
• Designing effective crowdsourcing workflows (CrowdFlows), possibly by means of games with a purpose or social campaigns
• Identifying the best set of individuals (from Web social sources) and (virtual or real) communities having the right combination of skills and motivations for the task at hand
• Launching and controlling crowdsourcing task on human-computation (e.g. Amazon Mechanical Turk, Crowdflower) and social platforms (e.g. Social Networks, Wikis)

The WUDE framework (components and tools) is built on top of the U-Sem infrastructure, and it enriches the offered set of user modeling service with components specifically targeted to the elicitation of Web user properties related to the assessment of their content enrichment capabilities. In addition, the WUDE framework integrates with Accurator (a framework for the execution and evaluation of strategies and applications for personalized nichesourcing), powering it with methods for task design and control, and external platform for human computation and games with a purpose.

The WUDE framework is currently under development and tested on a rich set of use cases.

Use Cases

Socially Enhanced Multimedia Content Annotation

Cultural heritage institutions are faced with the challenging problem of making their collections available to an increasing number and variety of users. The access to such valuable items is constrained by the availability of good quality annotations, a scarce yet expensive resource to obtain. The goal of WUDE is to make the acquisition of high quality annotations for cultural heritage collections easier and cheaper. Together with Heritage Delft and surroundings and Cit, WUDE will develop methods and techniques for the involvement of crowds and niches in the content analysis and annotation process, studying suitable identification, engagement, and incentivisation techniques.

User- and Community-centric Personalized Content Recommendation

Large collections of audiovisual materials demand advanced search and recommendation techniques to provide better access and browsing capabilities. To this end, the availability of advanced user modeling and user-needs elicitation techniques can help improving the diversification and exploration capabilities of content access techniques. Together with the project partners The Netherlands Institute for Sound and Vision, and Gridline, WUDE will study advance techniques to embed the preferences and opinions of users and communities in content recommendation and search systems.

Architecture

The architecture of the WUDE framework (depicted in the figure above) blends together state-of-the-art components for user and performer analysis and identification, social network and human computation platforms integration, task and workflow design, execution, and control.

The bottom layer contains the orchestration and analysis logic required to elicit relevant features from potential performers.

At the middle layer, the actual crowdsourcing takes place: the crowd flows are modeled and designed , the experiment launched and monitored, and the results collected and analyzed.

At the top layer, the WUDE framework is exploited for the specific application, such as the ones composing our set of use cases.

Architecture of the WUDE Framework

Team

The WUDE project is conducted in collaboration with five partners: Delft University of Technology, Heritage Delft and surroundings, Cit, The Netherlands Institute for Sound and Vision, and Gridline.

From Delft University of Technology one PhD student, Jasper Oosterman, and two staff members, Prof. Geert-Jan Houben and Alessandro Bozzon are involved.

Publications

Papers

On the impact of knowledge extraction and aggregation on crowdsourced annotation of visual artworks.
Jasper Oosterman, Jie Yang, Alessandro Bozzon, Lora Aroyo, Geert-Jan Houben. Computer Networks

Locating Domain-Specific Contents and Experts on Social Bookmarking Communities
Simon Kassing, Jasper Oosterman, Alessandro Bozzon, Geert-Jan Houben. SAC 2015: 747-752

Predicting Quality of Crowdsourced Annotations Using Graph Kernels
Archana Nottamkandath, Jasper Oosterman, Davide Ceolin, Gerben Klaas Dirk de Vries, Wan Fokkink. IFIPTM 2015: 134-148

Automated Evaluation of Crowdsourced Annotations in the Cultural Heritage Domain
Archana Nottamkandath, Jasper Oosterman, Davide Ceolin, Wan Fokkink. URSW 2014: 25-36 (PDF)

Crowdsourcing knowledge-intensive tasks in cultural heritage
Jasper Oosterman, Archana Nottamkandath, Chris Dijkshoorn, Alessandro Bozzon, Geert-Jan Houben, Lora Aroyo. ACM WebSci 2014: 267-268

Crowd vs. experts: nichesourcing for knowledge intensive tasks in cultural heritage
Jasper Oosterman, Alessandro Bozzon, Geert-Jan Houben, Archana Nottamkandath, Chris Dijkshoorn, Lora Aroyo, Mieke H. R. Leyssen, Myriam C. Traub . ACM WWW (Companion Volume) 2014: 567-568

Personalized Nichesourcing: Acquisition of Qualitative Annotations from Niche Communities
Chris Dijkshoorn, Mieke H. R. Leyssen, Archana Nottamkandath, Jasper Oosterman, Myriam Traub, Lora Aroyo, Alessandro Bozzon, Wan Fokkink, Geert-Jan Houben, Henrike Hovelmann, Lizzy Jongma, Jacco van Ossenbruggen, Guus Schreiber, Jan Wielemaker. UMAP Workshops 2013 (PDF)

Personalization in Crowd-driven Annotation for Cultural Heritage Collections
Chris Dijkshoorn, Jasper Oosterman, Lora Aroyo, Geert-Jan Houben. UMAP Workshops 2012 (PDF)

Online Articles

Are you sure? (Delta, October 8, 2015)
Digital Birdwatching in the Rijksmuseum (WIS, October 8, 2015)
Nichesourcing helpt het Rijksmuseum collecties in kaart brengen (Frankwatching, March 20, 2014)
Expertise from the crowd (Delta, February 04, 2014)

Posters

Crowd Knowledge Generation and Acceleration
Jasper Oosterman, Jie Yang at: Dies TUDelft, 9 January 2014, Delft, Netherlands (PDF)

Crowdsourcing Knowledge-Intensive Tasks In Cultural Heritage
Jasper Oosterman, Alessandro Bozzon, Geert-Jan Houben, Archana Nottankandath, Chris Dijkshoorn, Lora Aroyo at: Web Science, 23-26 June 2014, Bloomington (IN), USA (PDF)

Enriching Cultural Heritage Collections with Crowd Generated Knowledge
Jasper Oosterman, Alessandro Bozzon, Geert-Jan Houben, Archana Nottamkandath, Chris Dijkshoorn, Lora Aroyo, Mieke H.R. Leyssen, Myriam C. Traub at: WWW Web Science track, 10-11 April, Seoul, Korea (PDF)

Crowd Generated Knowledge
Jasper Oosterman at: ICT OPEN 2013, 27-28 November, Eindhoven, Netherlands (PDF)

Nichesourcing Specific Knowledge for Cultural Heritage Institutions
Jasper Oosterman, Chris Dijkshoorn, Mieke H.R. Leyssen, Myriam .C. Traub, Archana Nottamkandath at: ICT OPEN 2012, 23-23 October, Rotterdam, The Netherlands. (PDF)

---

© 2014 Delft University of Technology | Faculty of EEMCS | Software Technology | Web Information Systems