CWI develops application for continuous analysis of big streams of data

Researchers of the Database Architectures group at Centrum Wiskunde & Informatica (CWI) in Amsterdam have developed a database system that can quickly analyze continuously streaming big data. The researchers combined a streaming engine and a database system, thus exploiting and merging both technologies.

Publication date: 21-01-2013

Researchers of the Database Architectures group at Centrum Wiskunde & Informatica (CWI) in Amsterdam have developed a database system that can quickly analyze continuously streaming big data. The researchers combined a streaming engine and a database system, thus exploiting and merging both technologies. Researcher Erietta Liarou will defend her thesis on this new technology on Tuesday 22 January at the University of Amsterdam.

DataCell

Terabytes of data are generated on a daily basis in fields as finance, science and information technology. Numerous applications, whether stock market trading, scientific discovery or online advertising are dependent on analysis of this big data streams. Performing real-time analysis on this data, rather than storing it and analyzing it later, could be very profitable. Traders could for instance have different stocks monitored and be informed if a certain correlation occurs. Current database systems however do not support continuous processing, while data stream systems are not built to scale up to big data. Liarou and her colleagues innovatively built streaming functionality into MonetDB, the database system targeted at big data developed at CWI, integrating storage and analysis in the same engine. The resulting MonetDB/DataCell system is optimized for analyzing big streaming data, and outperforms commercial systems when stream data increases.
 
Big Data research is part of CWI's research theme Information. This line of research is aimed at developing methods and technologies to extract meaningful information from large amounts of data.
 
Thesis: 'MonetDB/DataCell: Leveraging the Column-store Database Technology for Efficient and Scalable Stream Processing'
By: Erietta Liarou (CWI Database Architectures group)
Promotor: prof. dr. M.L. Kersten (CWI/UvA)
Co-promotor : dr. S. Manegold (CWI)
Date: Tuesday 22 January 2013, 14.00h
Place: Agnietenkapel, Oudezijds Voorburgwal 231, Amsterdam

More information:

Download thesis
Homepage Database Architectures group
Homepage Erietta Liarou
MonetDB
 
The DataCell extension for MonetDB is open source and available for download (beta version).