JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE) ›› 2010, Vol. 40 ›› Issue (4): 8-11.

• Articles • Previous Articles     Next Articles

A new feature selection method for text categorization

WANG Fa-bo, XU Xin-shun   

  1. School of Computer Science and Technology, Shandong University, Jinan 250101, China
  • Received:2009-11-11 Online:2010-08-16 Published:2009-11-11

PDF (PC)

2881

Abstract:

How to reduce feature dimension while maintaining categorization accuracy is a key issue of text categorization.  A new method based on information theory was proposed to solve this problem. This approach aims to eliminate sparsely distributed features and find features  useful for categorization. Working with these feature reduction methods, it could  further reduce the feature dimension. The performance of  this  proposed method was tested on benchmark text classification problems. The results showed that it could not only reduce the feature dimension to hundreds but also improve the performance.

Key words: text categorization, feature selection, entropy, mutual information, information gain, CHI square statistics

[1] MOU Lianming. Weighted k sub-convex-hull classifier based on adaptive feature selection [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(5): 32-37.
[2] QIU Lu, YE Yinzhong, JIANG Chundi. Fault diagnostic method for micro-grid based on wavelet singularity entropy and SOM neural network [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(5): 118-122.
[3] WU Jianping, JIANG Bin, LIU Jianwei. Fault diagnosis of asynchronous motor based on wavelet packet entropy and wavelet neural network [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(5): 223-228.
[4] LI Sushu, WANG Shitong, LI Tao. A feature selection method based on LS-SVM and fuzzy supplementary criterion [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(3): 34-42.
[5] FANG Hao, LI Yun. Random undersampling and POSS method for software defect prediction [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(1): 15-21.
[6] LIN Yaojin, ZHANG Jia, LIN Menglei, WANG Juan. A method of collaborative filtering recommendation based on fuzzy information entropy [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2016, 46(5): 13-20.
[7] MO Xiaoyong, PAN Zhisong, QIU Junyang, YU Yajun, JIANG Mingchu. Anomaly detection in network traffic based on online feature selection [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2016, 46(4): 21-27.
[8] ZHANG Jia, LIN Yaojin, LIN Menglei, LIU Jinghua, LI Huizong. Entropy-based collaborative filtering algorithm [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2016, 46(2): 43-50.
[9] GUO Yu, ZHANG Erhua, LIU Chi. An endpoint detection algorithm based on frequency-domain characteristics and transition fragment judgment [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2016, 46(2): 57-63.
[10] SONG Dongmei, LIU Chunxiao, SHEN Chen, SHI Xuefa, ZANG Lin, FENG Wenqiang. Multiple objective and attribute decision making based on the subjective and objective weighting [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2015, 45(4): 1-9.
[11] XIN Liling, HE Wei, YU Jian, JIA Caiyan. An outlier detection algorithm based on density difference [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2015, 45(3): 7-14.
[12] ZHOU Yongmei1, YANG Aimin1, LIN Jianghao2. A method of building Chinese microblog sentiment lexicon [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2014, 44(3): 36-40.
[13] WEI Xiaomin, XU Bin, GUAN Jihong. Prediction of protein energy hot spots based on recursion feature elimination [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2014, 44(2): 12-20.
[14] PAN Pan1, WANG Xi-zhao2, ZHAI Jun-hai2. An improved induction algorithm based on ordinal decision tree [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2014, 44(1): 41-44.
[15] HE Si-yan1, LI Peng2, LIU Cheng-yu2, WU Xue-qian2, CHEN Qi-jun3. Refining of the membership function in cross fuzzy entropy and its influence [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2014, 44(1): 63-68.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright 2018 © Editorial office of Journal of Shandong University (Engineering Science)
Supported by Beijing Magtech Co.Ltd