Cogprints

Data Mining Using Relational Database Management Systems

Beibei, Zou and Xuesong, Ma and Bettina, Kemme and Glen, Newton and Doina, Precup (2006) Data Mining Using Relational Database Management Systems. [Preprint]

Full text available as:

[img]
Preview
 PDF
77Kb

Abstract

Software packages providing a whole set of data mining and machine learning algorithms are attractive because they allow experimentation with many kinds of algorithms in an easy setup. However, these packages are often based on main-memory data structures, limiting the amount of data they can handle. In this paper we use a relational database as secondary storage in order to eliminate this limitation. Unlike existing approaches, which often focus on optimizing a single algorithm to work with a database backend, we propose a general approach, which provides a database interface for several algorithms at once. We have taken a popular machine learning software package, Weka, and added a relational storage manager as back-tier to the system. The extension is transparent to the algorithms implemented in Weka, since it is hidden behind Weka’s standard main-memory data structure interface. Furthermore, some general mining tasks are transfered into the database system to speed up execution. We tested the extended system, refered to as WekaDB, and our results show that it achieves a much higher scalability than Weka, while providing the same output and maintaining good computation time.

Item Type:Preprint
Additional Information:Supported by the National Science and Engineering Council (NSERC), the Cnada Foundation for Innovation (CFI) and the National Research Council (NRC) Canada Institute for Scientific and Technical Information (CISTI).
Keywords:data mining, machine learning, data structures, WEKA
Subjects:Computer Science > Machine Learning
ID Code:4851
Deposited By: Newton, Glen
Deposited On:04 May 2006
Last Modified:11 Mar 2011 08:56

References in Article

Select the SEEK icon to attempt to find the referenced article. If it does not appear to be in cogprints you will be forwarded to the paracite service. Poorly formated references will probably not work.

R. Agrawal, T. Imielinski, and A. Swami. Database mining: A performance perspective.

IEEE Transactions on Knowledge and Data Engieering, 5(6), 1993.

J. Gehrke, R. Ramakrishnan, and V. Ganti. Rainforest: A framework for fast decision tree

construction of large datasets. Int. Conf. on Very Large Data Bases, 1998.

A. W. Moore and M. Lee. Cached sufficient statistics for efficient machine learning with

large data sets. Journal of Artificial Intelligence Research, 8, 1998.

W. Du Mouchel, C. Volinsky, T. Johson, C. Cortes, and D. Pregibon. Squashing flat files

flatter. ACM Int. Conf. on Knowledge Discovery and Data Mining, 1999.

D. Pyle. Data Preparation for Data Mining. Morgan Kaufmann Publishers, 1999.

B. J. Ross, A. G. Gualtieri, F. Fueten, and P. Budkewitsch. Hyperspectral image analysis

using genetic programmming. The Genetic and Evolutionary Computation Conf., 2002.

S. Sarawagi, S. Thomas, and R. Agrawal. Integrating association rule mining with relational

database systems: alternatives and implications. ACM SIGMOD Int. Conf. on Management

of Data, 1998.

Shafer, R. Agrawal, and M. Mehta. SPRINT: A scalable parallel classifier for data mining.

Int. Conf. on Very Large Data Bases, 1996.

I. H. Witten and E. Frank. Data mining software in Java. http://www.cs.waikato.ac.nz/ml/

weka/.

Carlos Ordonez. Programming the K-means Clustering Alogrithm in SQL. ACM Int. Conf.

on Knowledge Discovery and Data Mining, 2004.

Metadata

Repository Staff Only: item control page

Cogprints is powered by EPrints 3 which is developed by the School of Electronics and Computer Science at the University of Southampton. More information and software credits.