Researchers from Centrum Wiskunde & Informatica (CWI) challenged the common belief that network is your bottleneck in Big Data. They have developed a novel database architecture where the most relevant data flows continuously through a ring network composed out of many machines. This experimental Data Cyclotron architecture is faster and gives better throughput than systems that read and write data from a traditional disk subsystem. CWI researcher Rómulo Gonçalves defends his thesis on Data Cyclotron on Friday 22 March 2013 at the University of Amsterdam.
In modern hardware, sending data through a network is much quicker than reading it from a disk. The researchers exploit this trend to build a fast and efficient database architecture. The memory limitations of a single machine are compensated by combining the memories of all computers connected in a ring and use this as a fast storage device. The most relevant data is not stored anymore in one central location, but continuously flows through the network ring. Given the rule of thumb for database systems that 80% of all queries can be handled with only 20% of all available data, the relevant data is now available from a much faster store. Furthermore, all computers in the network of the Data Cyclotron architecture benefit from this high data stream.
To determine the most used data, the system keeps score of the number of request per data fragment. The technique is based on a data flow dynamically composed of the data fragments with highest scores. The size of the set depends on the demand for the data in the network, so that the network’s capacity is used in the most optimal way.
The Data Cyclotron is especially suited for Business Analytics, data mining and web-log analysis. The Data Cyclotron architecture is developed on the SciLens(http://www.scilens.org/)-machine, a large-scale experimental database platform at CWI since 2011.
Big Data research is part of CWI's research theme Information. This line of research is aimed at developing methods and technologies to extract meaningful information from large amounts of data.
More information:
Thesis: 'The Data Cyclotron: Juggling Data and Queries for a Data Warehouse Audience’
By: Rómulo Gonçalves (CWI Database Architectures group)
Supervisor: Prof. M.L. Kersten (CWI/UvA)
Date: Friday 22 March 2013, 11.00h
Location: Aula, Spui 111, Amsterdam
The research has been partially funded by the Netherlands Organisation for Scientific Research (NWO) and COMMIT/.
homepage CWI Database Architectures groep
homepage Rómulo Gonçalves
Picture: SciLens-cluster at Centrum Wiskunde & Informatica (CWI) in Amsterdam, the Netherlands. Photographer: Guido Benschop.
