Mark Raasveldt, PhD student at CWI’s Database Architecture (DA) group, won the Student Research Competition at the SIGMOD 2018 conference for his work titled "MonetDBLite: An Embedded Analytical Database".
MonetDBLite is an extension to the database that has been developed at CWI, called MonetDB, but with one crucial change: it can be embedded directly into applications. Standard database systems run as a separate application (the "database server"). Applications can then connect to the database system over a network connection. However, this approach introduces some problems. Managing a separate database server is a lot of work. Moreover, transferring large amounts of data over a network connection is slow.
Because of these problems, many people that work with data analytics or machine learning have opted not to use database systems at all, which is a shame because database systems offer many advantages for managing data. They prevent corruption of data in the case of power failures, allow concurrent access to data by many users and introduce a rigid schema which allows you to reason over the data. In addition, they allow for fast operations on the data.
For this reason, MonetDBLite has been developed. Because it can be embedded directly inside popular analytical tools (including Python and R), no separate database server has to be managed. This makes it much easier to use. As the connection of the analytical tool and the database system does not take place over a network connection, transferring data between the database system and the analytical tool is much faster than when dealing with a standard database system.
MonetDBLite is Open Source software, and has been available for almost two years. It was built in a collaboration between Hannes Mühleisen and Mark Raasveldt in the Database Architectures group. It is currently being used by thousands of people worldwide.
More information
