The need to tame big data is behind WeKnowIt, a large EU-funded project that has filed for several patents.

The need to tame big data is behind a large EU-funded project that has filed for several patents. The project, WeKnowIt, is designed to convert large amounts of user-generated content into 'a new, collective intelligence', according to the write-up of the project on the EU-funded Cordis Technology Marketplace. WeKnowIt is also behind some products that are destined for commercial release.

Dr Yiannis Kompatsiaris of the Greek Centre for Technology and Research is the coordinator of the project, and he says that current applications do not fully support intelligent processing and management of data so that users are failing to process information efficiently and cannot exploit the underlying knowledge.

The WeKnowIt project focused mainly on mining the type of data generated by social bookmarking, tagging and networking, where collective opinions and input creates a highly detailed dataset.

Dr Kompatsiaris categorises such data into three types of 'intelligence':

• media intelligence -digital content and contextual information
• mass intelligence comes from user feedback on a large scale
• social intelligence is drawn from personal interactions

WeKnowIt attempts to work out what category an individual item of data falls into, then combines it all into a 'collective intelligence' that is categorised so easier to search. Dr Kompatsiaris says that:

"Using a wide variety of tools, the WeKnowIt platform transforms large-scale and poorly structured information into meaningful topics, entities, points of interest, social connections and events."

Groups taking part in the project included the Brno University of Technology in the Czech Republic, the Koblenz-Landau University in Germany, Yahoo! in Spain, and Vodafone and the Centre for Research and Technology Hellas (CERTH) in Greece. Polish software company Software Mind provided software development and integration while the University of Sheffield and Sheffield City Council in the UK developed tools to use the data. The project took three years and cost EUR 7.5 million of which EUR 5.37 million was provided by the EU.

In practical terms, there's a middleware application that processes incoming data and routes it. Custom applications can then use the data for specific purposes. Over 20 applications have been created by the teams working on the project to use the data in situations including a city tourism app helps users find interesting places using the way photos of interesting places cluster. By identifying photos that belong together, the app places an 'interesting place' on a map. In essence, the app is a more structured version of the effect seen on Google Earth where users have uploaded their photos taken while on holiday.

In this video we learn how Sheffield City Council's Emergency Planning Team is testing WeKnowIt  against its existing emergency response system:

Work is continuing on other apps, with organisations including CERTH-ITI, Yahoo! and Koblenz University looking into the real-time aspects of information extraction from social media. Software Mind is also developing new semantic web tools for the telecommunications and financial sectors and Vodafone plans to use the knowledge gained in the project to make use of network infrastructure for new services.

More Information

Cordis

WeKnowIt website

To be informed about new articles on I Programmer, subscribe to the RSS feed, follow us on Google+, Twitter or Facebook or sign up for our weekly newsletter.