Journal of Shandong University(Engineering Science) ›› 2026, Vol. 56 ›› Issue (4): 75-83.doi: 10.6040/j.issn.1672-3961.0.2025.125

• Machine Learning & Data Mining • Previous Articles    

QD-RAG: Question decomposition driven iterative retrieval augmented generation

Chen Yingxin1,2, Chen Zhenhan1,2, Lin Xiaoyu1,2*, Cai Zhiling1,2, Chen Linfeng1,2, Wang Lijin1,2   

  1. Chen Yingxin1, 2, Chen Zhenhan1, 2, Lin Xiaoyu1, 2*, Cai Zhiling1, 2, Chen Linfeng1, 2, Wang Lijin1, 2(1. College of Computer and Information Sciences, Fujian Agriculture and Forestry University, Fuzhou 350002, Fujian, China;
    2. Smart Agriculture and Forestry Key Laboratory of Fujian University, Fujian Agriculture and Forestry University, Fuzhou 350002, Fujian, China
  • Published:2026-08-12

Abstract: To address the semantic mismatch between retrieval results and user needs in existing retrieval-augmented generation methods when handling complex questions requiring multi-hop reasoning, a query decomposition driven iterative retrieval augmented generation method was proposed, termed QD-RAG. This mismatch arised from the lack of hierarchical analysis of query structures in conventional RAG methods. In the proposed framework, a query decomposition module first processed user queries in a structured manner: single-hop questions were analyzed from multiple dimensions using a chain-of-thought strategy, whereas multi-hop questions were decomposed progressively in a layer-by-layer manner. Subsequently, an iterative retrieval module driven by large language models performed multiple rounds of contextual retrieval, which improved the precision of retrieved knowledge and enabled the generation of high-quality answers. Experimental results showed that QD-RAG effectively reduced hallucinations during generation and significantly improved accuracy in single-hop, multi-hop, and out-of-domain question-answering tasks.

Key words: rag, question decomposition, large language models, iterative retrieval, multi-hop

CLC Number: 

  • TP181
[1] Brown T, Mann B, Ryder N, et al. Language models are few-shot learners[J]. Advances in Neural Information Processing Systems, 2020, 33: 1877-1901.
[2] Kasai J, Sakaguchi K, Takahashi Y, et al. RealTime QA: what's the answer right now?[PP/OL]. V2.(2024-02-28)[2025-07-22]. https://arxiv.org/abs/2207.13332
[3] Mallen A, Asai A, Zhong V, et al. When not to trust language models: investigating effectiveness of parametric and non-parametric memories[C] //Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics. Toronto, Canada: ACL, 2023: 9802-9822.
[4] Hu E J, Shen Y L, Wallis P, et al. LoRA: low-rank adaptation of large language models[PP/OL]. V2.(2021-10-16)[2025-07-22]. https://doi.org/10.48550/arXiv.2106.09685
[5] Gao Y F, Xiong Y, Gao X Y, et al. Retrieval-augmented generation for large language models: a survey[PP/OL]. V5.(2024-03-27)[2025-07-22].https://doi.org/10.48550/arXiv.2312.10997
[6] Zellers R, Bisk Y, Schwartz R, et al. SWAG: a large-scale adversarial dataset for grounded commonsense inference [C] //Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium: ACL, 2018: 93-104.
[7] Izacard G, Grave E. Leveraging passage retrieval with generative models for open domain question answering [C] //Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. Online: ACL, 2021: 874-880.
[8] Borgeaud S, Mensch A, Hoffmann J, et al. Improving language models by retrieving from trillions of tokens[C] //Proceedings of the 39th International Conference on Machine Learning. New York, USA: PMLR, 2022: 2206-2240.
[9] Lewis P, Perez E, Piktus A, et al. Retrieval-augmented generation for knowledge-intensive NLP tasks[C] //Proceedings of the 34th International Conference on Neural Information Processing Systems. Vancouver, Canada: ACM, 2020: 9459-9474.
[10] Guu K, Lee K, Tung Z, et al. Retrieval augmented language model pre-training[C] //Proceedings of the 37th International Conference on Machine Learning. New York, USA: PMLR, 2020: 3929-3938.
[11] Lazaridou A, Gribovskaya E, Stokowiec W, et al. Internet-augmented language models through few-shot prompting for open-domain question answering[PP/OL]. V2.(2022-05-23)[2025-07-22]. https://doi.org/10.48550/arXiv.2203.05115
[12] Yang Z L, Qi P, Zhang S Z, et al. HotpotQA: a dataset for diverse, explainable multi-hop question answering[C] //In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium: ACL, 2018: 2369-2380.
[13] Ram O, Levine Y, Dalmedigos I, et al. In-context retrieval-augmented language models[J]. Transactions of the Association for Computational Linguistics, 2023, 11: 1316-1331.
[14] Shi F, Chen X Y, Misra K, et al. Large language models can Be easily distracted by irrelevant context[C] //Proceedings of the 40th International Conference on Machine Learning. New York, USA: PMLR, 2023: 31210-31227.
[15] Wang Y X, Chiew V. On the cognitive process of human problem solving[J]. Cognitive Systems Research, 2010, 11(1): 81-92.
[16] Khattab O, Santhanam K, Li X L, et al. Demonstrate-search-predict: composing retrieval and language models for knowledge-intensive NLP[PP/OL]. V2.(2023-01-23)[2025-07-22]. https://doi.org/10.48550/arXiv.2212.14024
[17] Sun Z Q, Wang X Z, Tay Y, et al. Recitation-augmented language models[PP/OL]. V2.(2023-02-16)[2025-07-22]. https://doi.org/10.48550/arXiv.2210.01296
[18] Yu W H, Iter D, Wang S H, et al. Generate rather than retrieve: large language models are strong context generators[PP/OL]. V3.(2023-01-25)[2025-07-22]. https://doi.org/10.48550/arXiv.2209.10063
[19] Ma X B, Gong Y Y, He P C, et al. Query Rewriting in Retrieval-Augmented Large Language Models[C] // In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Singapore: ACL, 2023: 5303-5315.
[20] Chan C M, Xu C P, Yuan R B, et al. RQ-RAG: learning to refine queries for retrieval augmented generation[PP/OL].(2024-03-31)[2025-07-22]. https://doi.org/10.48550/arXiv.2404.00610
[21] Zhou D, Schärli N, Hou L, et al. Least-to-most prompting enables complex reasoning in large language models[PP/OL].(2022-05-21)[2025-07-22]. https://doi.org/10.48550/arXiv.2205.1062
[22] Wei J, Wang X Z, Schuurmans D, et al. Chain-of-thought prompting elicits reasoning in large language models[J]. Advances in Neural Information Processing Systems, 2022, 35: 24824-24837.
[23] Shi W J, Min S, Yasunaga M, et al. REPLUG: retrieval-augmented black-box language models[C] //Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Mexico City, Mexico: ACL, 2024: 8371-8384.
[24] Izacard G, Lewis P, Lomeli M, et al. Atlas: few-shot learning with retrieval augmented language models[J]. Journal of Machine Learning Research, 2023, 24(251): 1-43.
[25] Shao Z H, Gong Y Y, Shen Y L, et al. Enhancing retrieval-augmented large language models with iterative retrieval-generation synergy[C] //Findings of the Association for Computational Linguistics: EMNLP. Singapore: ACL, 2023: 9248-9274.
[26] Yao S Y, Zhao J, Yu D, et al. ReAct: synergizing reasoning and acting in language models[PP/OL]. V3.(2023-03-10)[2025-07-22]. https://arxiv.org/abs/2210.03629
[27] Feng Z Y, Feng X C, Zhao D Z, et al. Retrieval-generation synergy augmented large language models[C] //ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing. Seoul: IEEE, 2023: 11661-11665.
[28] Wei S J, Zhang W W, Li Q S, et al. Semantic parsing for question answering over knowledge graphs[PP/OL].(2023-12-01)[2025-07-22]. https://doi.org/10.48550/arXiv.2401.06772
[29] Liu L H, Hill B, Du B X, et al. Conversational question answering with language models generated reformulations over knowledge graph[C] // Findings of the Association for Computational Linguistics: Bangkok, Thailand: ACL, 2024: 839-850.
[30] Nguyen H H, Nguyen M T. Emotion-cause pair extraction as question answering[PP/OL]. V2.(2023-01-05)[2025-07-22]. https://doi.org/10.48550/arXiv.2301.01982
[31] Kwiatkowski T, Palomaki J, Redfield O, et al. Natural questions: a benchmark for question answering research[J]. Transactions of the Association for Computational Linguistics, 2019, 7: 453-466.
[32] Joshi M, Choi E, Weld D, et al. TriviaQA: a large scale distantly supervised challenge dataset for reading comprehension[C]. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Vancouver, Canada: ACL, 2017: 1601-1611.
[33] Ho X, Duong N A K, Sugawara S, et al. Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps[J]. Proceedings of the 28th International Conference on Computational Linguistics. Barcelona, Spain: ACL, 2020: 6609-6625.
[34] Chen J W, Lin H Y, Han X P, et al. Benchmarking large language models in retrieval-augmented generation[C]. Proceedings of the 38th AAAI Conference on Artificial Intelligence,Vancouver, Canada: AAAI, 2024: 17754-17762.
[1] LI Wei, REN Qiwen, CAO Yongji, YU Sen, LI Changgang, KAN Rui, LIU Ziqi. Model-data hybrid driven estimation method of the state of charge for the lithium battery [J]. Journal of Shandong University(Engineering Science), 2026, 56(3): 156-165.
[2] GUO Junshan, ZHU Lingkai, GONG Zhiqiang, LIANG Kai, ZHONG Ziwei, SHANG Panfeng, WANG Xinyu. Analysis of flow and heat transfer characteristics in the novel energy storage battery module with immersion cooling [J]. Journal of Shandong University(Engineering Science), 2026, 56(2): 147-157.
[3] SUN Zhongqing, LAI Yening, ZHANG Jian, CAO Xuening, ZHANG Hengxu. Coordinated control strategies for energy storage and other controllable resources in power system emergencies [J]. Journal of Shandong University(Engineering Science), 2025, 55(5): 78-87.
[4] WANG Ruiqi, LIU Jiyan, JU Wenjie, WANG Weishuai, XU Wenze, ZHANG Zhenbin. Coordinated optimal scheduling of electric-hydrogen system considering hybrid energy storage in the day-ahead and intra-day stages [J]. Journal of Shandong University(Engineering Science), 2025, 55(2): 28-36.
[5] WANG Jingkui, QIAO Liping, WANG Fei, WANG Zhechao, LI Wei. The prevention and control of seawater intrusion into underground water-sealed oil storage cavern on an island [J]. Journal of Shandong University(Engineering Science), 2025, 55(2): 134-142.
[6] WANG Shibo, SUN Shumin, CHENG Yan, ZHOU Guangqi, GUAN Yifei, LIU Yiyuan, ZHANG Zhiqian, ZHANG Zhenbin. A cooperative control strategy of integrated photovoltaic-energy storage system considering SOC security boundary [J]. Journal of Shandong University(Engineering Science), 2025, 55(2): 37-44.
[7] Yingxin LIU,Jian QIN,Yanjun LIU. The analysis of key parameters of hydraulic energy storage system of wave energy converter [J]. Journal of Shandong University(Engineering Science), 2021, 51(6): 1-8.
[8] Xueshan HAN,Xinyi WANG,Ming YANG,Yixiao YU. Review and prospect of renewable energy ramp events [J]. Journal of Shandong University(Engineering Science), 2021, 51(5): 53-62.
[9] LI Ying, L(¨overU)Xuebin, LI Yan, SUN Shoujing. Method for safety management and control of tools and equipment used in substation based on ultra-wideband technology [J]. Journal of Shandong University(Engineering Science), 2021, 51(3): 84-90.
[10] Bei LI,Song ZHAO,Zhijia XIE,Meng NIU. Electric vehicle virtual energy storage available capacity modeling [J]. Journal of Shandong University(Engineering Science), 2020, 50(6): 101-111.
[11] SUN Donglei, ZHAO Long, QIN Jingtao, HAN Xueshan, YANG Ming, WANG Mingqiang. Bi-level planning of transmission network with solar-storage combination system based on learning theory [J]. Journal of Shandong University(Engineering Science), 2020, 50(4): 90-97.
[12] SHI Xuesong, GUAN Qingzheng, WANG Wenyang, XU Zhenhao, LIN Peng, WANG Xiaote, LIU Jie. Numerical simulation of rock fragmentation process by TBM cutter in double-joint rock mass [J]. Journal of Shandong University(Engineering Science), 2020, 50(4): 70-79.
[13] Caihong LI,Chun FANG,Zhiqiang WANG,Bin XIA,Fengying WANG. Complete coverage path planning for mobile robots based on hyperchaotic synchronization control [J]. Journal of Shandong University(Engineering Science), 2019, 49(6): 63-72.
[14] Liyan WANG, Fei WANG, Yongji CAO, Tao ZHANG, Yaxin ZHANG, Yi LU, Zihan LIU. Bi-level optimal configuration of energy storage system in an active distribution network [J]. Journal of Shandong University(Engineering Science), 2019, 49(5): 37-43.
[15] Huilin ZHOU,Yan QIU. Phase change characteristics of paraffin in rectangular storage unit [J]. Journal of Shandong University(Engineering Science), 2019, 49(4): 99-107.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright 2018 © Editorial office of Journal of Shandong University (Engineering Science)
Supported by Beijing Magtech Co.Ltd