Static Load Balancing of Parallel Mining of Frequent Itemsets Using Reservoir Sampling

Authors: Robert Kessl
Publication year: 2011
Publication type: Konferenční příspěvek (mezinárodní konf.)
Periodical: Machine Learning and Data Mining in Pattern Recognition
Projects: NoSCoM: Non-Standard Computational Models and Their Applications in Complexity, Linguistics, and Learning
Bibliographic citation: KESSL, Robert. Static Load Balancing of Parallel Mining of Frequent Itemsets Using Reservoir Sampling. In: PERNER, P., ed. Machine Learning and Data Mining in Pattern Recognition. Berlin: Springer, 2011, S. 553-567. Lecture Notes in Artificial Intelligence, 6871. ISBN 978-3-642-23198-8. ISSN 0302-9743
Dostupné z: doi:10.1007/978-3-642-23199-5_41
DOI: 10.1007/978-3-642-23199-5_41
ID: 368102

Abstract

In this paper, we present a novel method for parallelization of an arbitrary depth-first search (DFS in short) algorithm for mining of all FIs. The method is based on the so called reservoir sampling algorithm. The reservoir sampling algorithm in combination with an arbitrary DFS mining algorithm executed on a database sample takes an uniformly but not independently distributed sample of all FIs using the reservoir sampling. The sample is then used for static load-balancing of the computational load of a DFS algorithm for mining of all FIs.