Journal of Shandong University(Engineering Science) ›› 2026, Vol. 56 ›› Issue (3): 62-72.doi: 10.6040/j.issn.1672-3961.0.2024.240

• Machine Learning & Data Mining • Previous Articles     Next Articles

Advances and prospects in computerized adaptive testing

CUI Chaoran, DONG Xiaolin, ZHANG Chunyun, XI Muzhi   

  1. CUI Chaoran, DONG Xiaolin, ZHANG Chunyun, XI Muzhi(School of Computing and Artificial Intelligence, Shandong University of Finance and Economics, Jinan 250014, Shandong, China
  • Published:2026-06-09

Abstract: With the development of the internet and online education, limitations of time and space were overcome, allowing people to learn anytime and anywhere. A vast amount of learning records was generated through various online learning and competition platforms. By effectively utilizing these records, computerized adaptive testing(CAT)enabled the customization of personalized assessments for examinees, achieving the goal of "tailored instruction". This paper aimed to comprehensively review the current development and state-of-the-art work in CAT, provide an outlook on future directions, and help future researchers and practitioners gain a systematic understanding of CAT. First, the background and theoretical foundations of CAT were introduced, followed by a formal description of CAT. Then, from a technical perspective, existing CAT methods were categorized into two types: CAT methods for selection process and CAT methods for ability estimation. A detailed overview of these two types of CAT methods was provided. Next, commonly used public datasets and evaluation metrics in CAT were compiled, with each dataset's source and relevant information described. Finally, the future research directions of CAT were discussed, and conclusions were drawn.

Key words: computerized adaptive testing, item response theory, question selection algorithm, ability estimation method, educational data minining

CLC Number: 

  • TP391
[1] VAN DER LINDEN W J, GLAS C A W, et al. Computerized adaptive testing: theory and practice[M]. Dordrecht, Netherlands: Springer Netherlands, 2000: 1-26.
[2] VIE J J, POPINEAU F, BRUILLARD É, et al. A review of recent advances in adaptive assessment[M] // PE(~overN)A-AYALA A. Learning analytics: fundaments, applications, and trends: a view of the current state of the art to enhance e-learning,Cham,Switzerland: Springer International Publishing, 2017: 113-142.
[3] LAN A S, WATERS A E, STUDER C, et al. Sparse factor analysis for learning and content analytics[EB/OL].(2013-03-22)[2024-09-05]. 2013: arXiv:1303.5685. https://doi.org/10.48550/arXiv.1303.5685
[4] 郑蝉金, 汪腾. 计算机化自适应测验[M]. 北京: 教育科学出版社, 2021: 23-27.
[5] WEISS D J. Improving measurement quality and efficiency with adaptive testing[J]. Applied Psychological Measurement, 1982, 6(4): 473-492.
[6] LORD F M. Some test theory for tailored testing[J]. ETS Research Bulletin Series, 1968, 1968(2):7-10.
[7] EMBRETSON S E, REISE S P. Item response theory[M]. London, UK: Psychology Press, 2013.
[8] LORD F M. Applications of item response theory to practical testing problems[M]. Hillsdale, USA: Lawrence Erlbaum Associates, 2012: 65-73.
[9] CHANG H H, YING Z. A global information approach to computerized adaptive testing[J]. Applied Psychological Measurement, 1996, 20(3): 213-229.
[10] SEGALL D O. Multidimensional adaptive testing[J]. Psychometrika, 1996, 61(2): 331-354.
[11] MAGIS D, YAN D, VON DAVIER A A, et al. An overview of computerized multistage testing[J]. Computerized Adaptive and Multistage Testing with R: Using Packages Catr and Mstr, 2017: 113-122.
[12] GILAVERT P, FREIRE V. Computerized adaptive testing: a unified approach under Markov decision process[C] //Computational Science and Its Applications-ICCSA. Cham, Switzerland: Springer, 2022: 591-602.
[13] LIN Z, CHEN P, XIN T. The block item pocket method for reviewable multidimensional computerized adaptive testing[J]. Applied Psychological Measurement, 2021, 45(1): 22-36.
[14] CHENG Y. When cognitive diagnosis meets computerized adaptive testing: CD-CAT[J]. Psychometrika, 2009, 74: 619-632.
[15] REN P, XIAO Y, CHANG X,et al. A survey of deep active learning[J]. ACM Computing Surveys(CSUR), 2021, 54(9): 1-40.
[16] MIENYE I D, SUN Y. A survey of ensemble learning: concepts, algorithms, applications, and prospects[J]. IEEE Access, 2022, 10: 99129-99149.
[17] ALMAHAMID F, GROLINGER K.Reinforcement learning algorithms: an overview and classification[C] // 2021 IEEE Canadian Conference on Electrical and Computer Engineering(CCECE). Ontario, Canada: IEEE, 2021: 1-7.
[18] MILLS C N, STEFFEN M. Computerized adaptive testing: Theory and practice[M]. Dordrecht,Netherlands: Springer Netherlands, 2000: 75-99.
[19] RUDNER L M. Implementing the graduate management admission test computerized adaptive test[M] // VAN DER LINDEN W J, GLAS C A W. Elements of Adaptive Testing. New York, USA: Springer New York, 2009: 151-165.
[20] 陈平, 丁树良, 林海菁, 等. 等级反应模型下计算机化自适应测验选题策略[J]. 心理学报, 2006(3): 461-467. CHEN Ping, DING Shuliang, LIN Haijing, et al. Item selection strategies of computerized adaptive testing based on graded response model[J]. Acta Psychologica Sinica, 2006(3): 461-467.
[21] ZHU Z, ARTHUR D, CHANG H H. A new person-fit method based on machine learning in CDM in education[J]. British Journal of Mathematical and Statistical Psychology, 2022, 75(3): 616-637.
[22] NGUYEN D, ZHANG A Y. A spectral approach to item response theory[EB/OL]. Advances in Neural Information Processing Systems.(2022)[2024-09-05]. https://dl.acm.org/doi/10.5555/3600270.3603083
[23] CHENG Y, XU W S, YU Y L.Improving the 3PLM-based computerized adaptive testing system with multi-agent item bank[C] // 2008 IEEE International Conference on Networking, Sensing and Control.Sanya, China: IEEE, 2008: 298-303.
[24] RECKASE M D. 18 Multidimensional item response theory[M] // RAO C R, SINHARAY S. Handbook of Statistics. Amsterdam, Netherlands: Elsevier, 2006: 607-642.
[25] SAMEJIMA F. Estimation of latent ability using a response pattern of graded scores[J]. Psychometrika Monograph Supplement, 1969: 23-35.
[26] MASTERS G N. A Rasch model for partial credit scoring[J]. Psychometrika, 1982, 47(2): 149-174.
[27] WANG F, LIU Q, CHEN E, et al. Neural cognitive diagnosis for intelligent education systems[C] // Proceedings of the AAAI Conference on Artificial Intelligence. New York, USA: AAAI, 2020, 34(4): 6153-6161.
[28] WANG F, LIU Q, CHEN E, et al. NeuralCD: a general framework for cognitive diagnosis[J]. IEEE Transactions on Knowledge and Data Engineering, 2022, 35(8): 8312-8327.
[29] CHEN Y. Convolutional neural network for sentence classification[D].Waterloo,Canada: University of Waterloo, 2015: 18-20.
[30] ZHUANG Y, ZHUANG Y, LIU Q, etal. A bounded ability estimation for computerized adaptive testing[C] //Proceedings of the 37th International Conference on Neural Information Processing Systems. New Orleans, USA: ACM, 2023: 2381-2402.
[31] CHANG H H. Psychometrics behind computerized adaptive testing[J]. Psychometrika, 2015, 80: 1-20.
[32] THOMPSON N A, WEISS D A. A framework for the development of computerized adaptive tests[J]. Practical Assessment, Research, and Evaluation, 2019, 16(1): 1.
[33] WANG C. On latent trait estimation in multidimensional compensatory item response models[J]. Psychometrika, 2015, 80: 428-449.
[34] WAINER H, DORANS N J, FLAUGHER R, et al. Computerized adaptive testing: a primer[M]. Oxford,UK:Routledge, 2000.
[35] CHENG Y, CHANG H H. The maximum priority index method for severely constrained item selection in computerized adaptive testing[J]. British Journal of Mathematical and Statistical Psychology, 2009, 62(2): 369-383.
[36] TATSUOKA C. Data analytic methods for latent partially ordered classification models[J]. Journal of the Royal Statistical Society Series C: Applied Statistics, 2002, 51(3): 337-350.
[37] MULDER J, VAN DER LINDEN W J. Multidimensional adaptive testing with Kullback-Leibler information item selection[M] // VAN DER LINDEN W J, GLAS C A W. Elements of Adaptive Testing. New York, USA: Springer New York, 2009: 77-101.
[38] VAN DER LINDEN W J, PASHLEY P J. Item selection and ability estimation in adaptive testing[M] // VAN DER LINDEN W J, GLAS C A W. Elements of Adaptive Testing. New York, USA: Springer New York, 2009: 3-30.
[39] MULDER J, VAN DER LINDEN W J. Multidimensional adaptive testing with optimal design criteria for item selection[J]. Psychometrika, 2009, 74: 273-296.
[40] VELDKAMP B P, VAN DER LINDEN W J. Multidimensional adaptive testing with constraints on test content[J]. Psychometrika, 2002, 67: 575-588.
[41] BI H, MA H, HUANG Z, et al. Quality meets diversity: a model-agnostic framework for computerized adaptive testing[C] // 2020 IEEE International Conference on Data Mining(ICDM). Sorrento, Italy: IEEE, 2020: 42-51.
[42] NING Y, LIU Y, HUANG Z, et al. Stable and diverse: a unified approach for computerized adaptive testing[C] // 2021 IEEE 7th International Conference on Cloud Computing and Intelligent Systems(CCIS). Xi'an, China:IEEE, 2021: 61-65.
[43] DAGAN I, ENGELSON S P. Committee-based sampling for training probabilistic classifiers[M] // PRIEDITIS A, RUSSELL S. Machine Learning Proceedings 1995. San Fransisco, USA:Morgan Kaufmann, 1995: 150-157.
[44] KWON S, KIM S, LEE S, et al. Addressing selection bias in computerized adaptive testing: a user-wise aggregate influence function approach[C] //Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. Birmingham, UK: ACM, 2023: 4674-4680.
[45] HONG Y T, HONG Y T, TONG S W, et al. Search-efficient computerized adaptive testing[C] //Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. Birmingham, UK: ACM, 2023: 773-782.
[46] GHOSH A, LAN A. Bobcat: bilevel optimization-based computerized adaptive testing[EB/OL].(2021-08-17)[2024-09-05]. https://arxiv.org/abs/2108.07386
[47] FENG W Y, GHOSH A, SIRECI S, et al. Balancing test accuracy and security in computerized adaptive testing[C] //Artificial Intelligence in Education. Cham,Switzerland: Springer, 2023: 708-713.
[48] ZHUANG Y, LIU Q, HUANG Z, et al. Fully adaptive framework: neural computerized adaptive testing for online education[C] // Proceedings of the AAAI Conference on Artificial Intelligence. Vancouver, Canada: AAAI, 2022, 36(4): 4734-4742.
[49] WANG H, LONG T, YIN L, et al. GMOCAT: a graph-enhanced multi-objective method for computerized adaptive testing[C] // Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2023: 2279-2289.
[50] VELICKOVIC P, CUCURULL G, CASANOVA A, et al. Graph attention networks[EB/OL].(2017-10-30)[2024-09-15]. https://arxiv.org/abs/1710.10903
[51] PIAN Y, CHEN P H, LU Y, et al. Improving the item selection process with reinforcement learning in computerized adaptive testing[C] // International Conference on Artificial Intelligence in Education. Cham,Switzerland: Springer Nature Switzerland, 2023: 230-235.
[52] YU J, ZHENYU M, LEI J, et al. SACAT: student-adaptive computerized adaptive testing[C] // Proceedings of the Fifth International Conference on Distributed Artificial Intelligence. New York, USA: ACM, 2023: 1-7.
[53] ZHUANG Y, ZHUANG Y, LIU Q, et al. A robust computerized adaptive testing approach in educational question retrieval[C] //Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. Madrid, Spain, ACM: 2022: 416-426.
[54] FENG M Y, HEFFERNAN N, KOEDINGER K. Addressing the assessment challenge with an online system that tutors as it assesses[J]. User Modeling and User-Adapted Interaction, 2009, 19: 243-266.
[55] WANG Y T, HEFFERNAN N T, HEFFERNAN C. Towards better affect detectors: effect of missing skills, class features and common wrong answers[C] // Proceedings of the Fifth International Conference on Learning Analytics and Knowledge. New York, USA: ACM, 2015: 31-35.
[56] ASSISTments. 2015 ASSISTments skill builder data[DB/OL].(2015)[2024-09-05]. https://sites.google.com/site/assistmentsdata/datasets/2015-assistments-skill- builder-data
[57] PATIKORN T, BAKER R S, HEFFERNAN N T. ASSISTments longitudinal data mining competition special issue: a preface[J]. Journal of Educational Data Mining, 2020, 12(2): ⅰ-xi.
[58] CHOI Y, LEE Y, SHIN D, et al. Ednet: a large-scale hierarchical dataset in education[C] // Artificial Intelligence in Education: 21st International Conference, AIED 2020, Ifrane, Morocco: Springer International Publishing, 2020: 69-73.
[59] ABDELRAHMAN G, WANG Q, NUNES B. Knowledge tracing: a survey[J]. ACM Computing Surveys, 2023, 55(11): 1-37.
[60] WANG Z, LAMB A, SAVELIEV E, et al. Instructions and guide for diagnostic questions: the neurips 2020 education challenge[EB/OL].(2020-07-23)[2024-09-05]. https://arxiv.org/abs/2007.12061. https://eedi.com/projects/neurips-education-challenge
[61] CHANG H S, HSU H J, CHEN K T. Modeling exercise relationships in e-learning: a unified approach[C] // Proceedings of the 8th International Conference on Educational Data Mining, EDM 2015, Madrid, Spain: EDM, 2015: 532-535.
[62] SHEN S, LIU Q, HUANG Z, et al. A survey of knowledge tracing: models, variants, and applications[J]. IEEE Transactions on Learning Technologies, 2024, 17: 1858-1879.
[63] 郑旭东, 高守林, 任友群. 计算机化自适应测验及应用于规模化考试的主要问题分析[J]. 开放教育研究, 2016, 22(4): 40-49. ZHENG Xudong, GAO Shoulin, REN Youqun. Computerized adaptive testing: a review and main problems in large-scale testing[J]. Open Education Research, 2016, 22(4): 40-49.
[64] HAU K T, CHANG H H. Item selection in computerized adaptive testing: should more discriminating items be used first?[J]. Journal of Educational Measurement, 2001, 38(3): 249-266.
[65] MILLS C N, STOCKING M L. Practical issues in large-scale computerized adaptive testing[J]. Applied Measurement in Education, 1996, 9(4): 287-304.
[1] DENG Bin, ZHANG Zongbao, ZHAO Wenmeng, LUO Xinhang, WU Qiuwei. Cloud-edge collaborative and graph neural network based load forecasting method for electric vehicle charging stations [J]. Journal of Shandong University(Engineering Science), 2025, 55(5): 62-69.
[2] LI Erchao, ZHANG Zhizhao. Online dynamic demand vehicle routing planning [J]. Journal of Shandong University(Engineering Science), 2024, 54(5): 62-73.
[3] YANG Jucheng, WEI Feng, LIN Liang, JIA Qingxiang, LIU Jianzheng. A research survey of driver drowsiness driving detection [J]. Journal of Shandong University(Engineering Science), 2024, 54(2): 1-12.
[4] XIAO Wei, ZHENG Gengsheng, CHEN Yujia. Named entity recognition method combined with self-training model [J]. Journal of Shandong University(Engineering Science), 2024, 54(2): 96-102.
[5] Gang HU, Lemeng WANG, Zhiyu LU, Qin WANG, Xiang XU. Importance identification method based on multi-order neighborhood hierarchical association contribution of nodes [J]. Journal of Shandong University(Engineering Science), 2024, 54(1): 1-10.
[6] Jiachun LI,Bowen LI,Jianbo CHANG. An efficient and lightweight RGB frame-level face anti-spoofing model [J]. Journal of Shandong University(Engineering Science), 2023, 53(6): 1-7.
[7] Yujiang FAN,Huanhuan HUANG,Jiaxiong DING,Kai LIAO,Binshan YU. Resilience evaluation system of the old community based on cloud model [J]. Journal of Shandong University(Engineering Science), 2023, 53(5): 1-9, 19.
[8] Ying LI,Jiankun WANG. The classification of mild cognitive impairment based on supervised graph regularization and information fusion [J]. Journal of Shandong University(Engineering Science), 2023, 53(4): 65-73.
[9] YU Yixuan, YANG Geng, GENG Hua. Multimodal hierarchical keyframe extraction method for continuous combined motion [J]. Journal of Shandong University(Engineering Science), 2023, 53(2): 42-50.
[10] ZHANG Hao, LI Ziling, LIU Tong, ZHANG Dawei, TAO Jianhua. A technology prediction model based on fuzzy Bayesian networks with sociological factors [J]. Journal of Shandong University(Engineering Science), 2023, 53(2): 23-33.
[11] WU Yanli, LIU Shuwei, HE Dongxiao, WANG Xiaobao, JIN Di. Poisson-gamma topic model of describing multiple underlying relationships [J]. Journal of Shandong University(Engineering Science), 2023, 53(2): 51-60.
[12] YU Mingjun, DIAO Hongjun, LING Xinghong. Online multi-object tracking method based on trajectory mask [J]. Journal of Shandong University(Engineering Science), 2023, 53(2): 61-69.
[13] HUANG Huajuan, CHENG Qian, WEI Xiuxi, YU Chuchu. Adaptive crow search algorithm with Jaya algorithm and Gaussian mutation [J]. Journal of Shandong University(Engineering Science), 2023, 53(2): 11-22.
[14] LIU Fangxu, WANG Jian, WEI Benzheng. Auxiliary diagnosis algorithm for pediatric pneumonia based on multi-spatial attention [J]. Journal of Shandong University(Engineering Science), 2023, 53(2): 135-142.
[15] LIU Xing, YANG Lu, HAO Fanchang. Finger vein image retrieval based on multi-feature fusion [J]. Journal of Shandong University(Engineering Science), 2023, 53(2): 118-126.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!