Constrained Independence for Detecting Interesting Patterns

Among other criteria, a pattern may be interesting if it is not redundant with other discovered patterns. A general approach to determining redundancy is to consider a probabilistic model for frequencies of patterns, based on those of patterns already mined, and compare observed frequencies to the model. Such probabilistic models include the independence model, partition models or more complex models which are approached via randomization for a lack of an adequate tool in probability theory allowing a direct approach. We define constrained independence, a generalization to the notion of independence. This tool allows us to describe probabilistic models for evaluating redundancy in frequent itemset mining. We provide algorithms, integrated within the mining process, for determining non-redundant itemsets. Through experimentations, we show that the models used reveal high rates of redundancy among frequent itemsets and we extract the most interesting ones.