Finding the True Frequent Itemsets

Matteo Riondato, Fabio Vandin

Publikation: Bidrag til bog/antologi/rapport/konference-proceedingKonferencebidrag i proceedingsForskningpeer review

Abstrakt

Frequent Itemsets (FIs) mining is a fundamental primitive in data mining. It requires to identify all itemsets appearing in at least a fraction θ of a transactional dataset D. Often though, the ultimate goal of mining D is not an analysis of the dataset per se, but the understanding of the underlying process that generated it. Specifically, in many applications D is a collection of samples obtained from an unknown probability distribution π on transactions, and by extracting the FIs in D one attempts to infer itemsets that are frequently (i.e., with probability at least θ) generated by π, which we call the True Frequent Itemsets (TFIs). Due to the inherently stochastic nature of the generative process, the set of FIs is only a rough approximation of the set of TFIs, as it often contains a huge number of false positives, i.e., spurious itemsets that are not among the TFIs. In this work we design and analyze an algorithm to identify a threshold such that the collection of itemsets with frequency at least in contains only TFIs with probability at least 1 – δ, for some user-specified δ. Our method uses results from statistical learning theory involving the (empirical) VC-dimension of the problem at hand. This allows us to identify almost all the TFIs without including any false positive. We also experimentally compare our method with the direct mining of at frequency θ and with techniques based on widely-used standard bounds (i.e., the Chernoff bounds) of the binomial distribution, and show that our algorithm outperforms these methods and achieves even better results than what is guaranteed by the theoretical analysis.
OriginalsprogEngelsk
TitelProceedings of the 2014 SIAM International Conference on Data Mining
Redaktører Mohammed Zaki et al.
Antal sider9
ForlagSociety for Industrial and Applied Mathematics
Publikationsdato2014
Sider497-505
ISBN (Elektronisk)978-1-61197-344-0
DOI
StatusUdgivet - 2014
BegivenhedSIAM International Conference on Data Mining - Sheraton Society Hill Hotel, Philadelphia, USA
Varighed: 24. apr. 201426. apr. 2014

Konference

KonferenceSIAM International Conference on Data Mining
LokationSheraton Society Hill Hotel
LandUSA
ByPhiladelphia
Periode24/04/201426/04/2014

    Fingerprint

Citationsformater

Riondato, M., & Vandin, F. (2014). Finding the True Frequent Itemsets. I M. Zaki et al. (red.), Proceedings of the 2014 SIAM International Conference on Data Mining (s. 497-505). Society for Industrial and Applied Mathematics. https://doi.org/10.1137/1.9781611973440.57