关键词: 决策树, 信息增益, 信息增益率, 平均互信息, 熵

Abstract:

Attribute selection criterion in constructing decision trees is always the focus in the area of data mining. Based on the analysis of ID3 and C4.5, this paper proposes two algorithms based on average self-information and average mutual-information in communication system. In this research, we prove that the two proposed algorithms are equivalent to ID3 and C4.5. That is, information gain is equivalent to average mutual information in communication system, and information gain ratio is the same as the ratio of average mutual information to the entropy. Experiments on AllElectrionics illustrate that compared with information gain and information gain ratio, attribute selection criteria proposed in this paper are easy to compute and understand.

Key words:  decision tree, information gain, information gain ratio, average mutualinformation, entropy