Article Open Access

Retrieval-Augmented Large Language Model for Institutional Knowledge Management and Decision Assistance in Public Organizations

Yohanes Bowo Widodo, Sondang Sibuea

Abstract


The increasing volume and complexity of institutional documents in public organizations create challenges in accessing reliable knowledge for administrative processes and evidence-based decision-making. Conventional knowledge management systems often rely on keyword-based retrieval, while standalone Large Language Models (LLMs) may generate inaccurate responses when processing domain-specific institutional information. This study proposes a domain-specific Retrieval-Augmented Generation (RAG) framework to enhance institutional knowledge management and AI-assisted decision support in public-sector organizations. The framework was developed using a Design Science Research approach with Universitas Malikussaleh as a case study. The proposed architecture integrates institutional knowledge base construction, semantic retrieval, grounded language generation, and source attribution mechanisms. A knowledge base comprising 416 official institutional documents was developed through document preprocessing, semantic chunking, embedding generation, and vector database indexing. The framework was evaluated using 200 institutional queries based on retrieval performance, response quality, explainability, and system efficiency metrics. The results demonstrate effective retrieval capability, achieving Precision@5 of 0.884, Recall@5 of 0.921, and Mean Reciprocal Rank of 0.895. Generated responses achieved 94.6% factual accuracy, 91.8% contextual relevance, and 96.5% source attribution accuracy, while the hallucination rate was reduced to 3.2%. Furthermore, the framework achieved an average response latency of 1.18 seconds, indicating practical feasibility for institutional applications. These findings demonstrate that integrating semantic retrieval with grounded LLM generation can improve knowledge accessibility, transparency, and reliability for AI-assisted decision support in public organizations. The proposed framework provides a practical foundation for trustworthy institutional knowledge services and supports more efficient, explainable, and evidence-based administrative decision-making across diverse institutional contexts

Keywords


Decision Support System, Explainable Artificial Intelligence, Institutional Knowledge Management, Large Language Model, Retrieval-Augmented Generation

References


Dzhusupova, Z., Ojo, A., & Janowski, T. (2009). Organizing and managing knowledge for e-government: Issues, practices and challenges. Proceedings of the International Conference on Knowledge Management and Information Sharing (KMIS 2009), 206–211. SciTePress. https://doi.org/10.5220/0002313402060211

Laihonen, H., Kork, A.-A., & Sinervo, L.-M. (2024). Advancing public sector knowledge management: Towards an understanding of knowledge formation in public administration. Knowledge Management Research & Practice, 22(3), 223–233. https://doi.org/10.1080/14778238.2023.2187719

Prokopiadou, G., Papatheodorou, C., & Moschopoulos, D. (2004). Integrating knowledge management tools for government information. Government Information Quarterly, 21(2), 170–198. https://doi.org/10.1016/j.giq.2004.02.001

Wimmer, M. A. (Ed.). (2003). Knowledge Management in Electronic Government: 4th IFIP International Working Conference, KMGov 2003, Rhodes, Greece, May 26–28, 2003, Proceedings. Springer. https://doi.org/10.1007/978-3-540-39964-4

Heeks, R. (2006). Implementing and Managing eGovernment: An International Text. SAGE Publications. https://doi.org/10.4135/9781446220195

Wimmer, M. A. (Ed.). (2004). Knowledge Management in Electronic Government: 5th IFIP International Working Conference, KMGov 2004, Krems, Austria, May 17–19, 2004, Proceedings. Springer. https://doi.org/10.1007/b97726

Janssen, M., & Cresswell, A. M. (2005). An enterprise application integration methodology for e-government. Journal of Enterprise Information Management, 18(5), 531–547. https://doi.org/10.1108/17410390510624090

Thurston, A. (2005). Fostering trust and transparency through information systems. Records Management Journal, 15(3), 128–132. https://doi.org/10.1108/09565690510632312

Sari, R., & Muslim, M. (2023). Accountability and transparency in public sector accounting: A systematic review. Amkop Management Accounting Review, 3(2), 90–106. https://doi.org/10.37531/amar.v3i2.1440

Bolisani, E., & Scarso, E. (2014). The place of knowledge management in organizations: A new perspective. Journal of Knowledge Management, 18(5), 945–957. https://doi.org/10.1108/JKM-04-2014-0169

Massaro, M., Dumay, J., & Garlatti, A. (2015). Public sector knowledge management: A structured literature review. Journal of Knowledge Management, 19(3), 530–558. https://doi.org/10.1108/JKM-11-2014-0466

Ali, N., Mustaffa, M. M., Alkawsi, G., & Capretz, L. F. (2024). If you build it, will they come? Exploring the success factors of knowledge management systems in the Malaysian public sector. Heliyon, 10(6), e27093. https://doi.org/10.1016/j.heliyon.2024.e27093

Martín, A., & León, C. (2018). Semantic framework for an efficient information retrieval in the E-government repositories. In Information Retrieval and Management: Concepts, Methodologies, Tools, and Applications (pp. 631-652). IGI Global Scientific Publishing. https://doi.org/10.4018/978-1-5225-5191-1.ch027

Hartono, L. S., Setiawan, E. I., & Singh, V. (2025). Retrieval Augmented Generation-Based Chatbot for Prospective and Current University Students. International Journal of Engineering, Science and Information Technology, 5(3). https://doi.org/10.52088/ijesty.v5i3.951

Laihonen, H., & Mäntylä, S. (2018). Strategic knowledge management and evolving local government. Journal of Knowledge Management, 22(1), 219–234. https://doi.org/10.1108/JKM-06-2017-0232

Gómez-Pérez, A., Ortiz-Rodríguez, F., & Villazón-Terrazas, B. (2006, May). Ontology-based legal information retrieval to improve the information access in e-government. In Proceedings of the 15th international conference on World Wide Web (pp. 1007-1008). https://doi.org/10.1145/1135777.1135988

Ortiz-Rodríguez, F., Palma, R., & Villazón-Terrazas, B. (2007). EgoIR: ontology-based information retrieval intended for eGovernment. In Informatik 2007–Informatik trifft Logistik–Band 1 (pp. 237-241). Gesellschaft für Informatik e. V. https://dl.gi.de/handle/20.500.12116/22588

Brown, T. B., Mann, B., Ryder, N., et al. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33, 1877–1901. https://doi.org/10.48550/arXiv.2005.14165

Zhao, W. X., Zhou, K., Li, J., et al. (2023). A survey of large language models. arXiv. https://doi.org/10.48550/arXiv.2303.18223

Bommasani, R., Hudson, D. A., Adeli, E., et al. (2021). On the opportunities and risks of foundation models. arXiv. https://doi.org/10.48550/arXiv.2108.07258

Singhal, K., Azizi, S., Tu, T., et al. (2023). Large language models encode clinical knowledge. Nature, 620, 172–180. https://doi.org/10.1038/s41586-023-06291-2

Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., ... & Fung, P. (2023). Survey of hallucination in natural language generation. ACM computing surveys, 55(12), 1-38. https://doi.org/10.1145/3571730

Béchard, P., & Marquez Ayala, O. (2024). Reducing hallucination in structured outputs via Retrieval-Augmented Generation. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Industry Track) (pp. 228–238). https://doi.org/10.18653/v1/2024.naacl-industry.19

Cai, T., Tan, Z., Song, X., Sun, T., Jiang, J., Xu, Y., Zhang, Y., & Gu, J. (2024). FoRAG: Factuality-optimized Retrieval-Augmented Generation for web-enhanced long-form question answering. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (pp. 199–210). https://doi.org/10.1145/3637528.3672065

Singal, R., Patwa, P., Chadha, A., & Das, A. (2024). Evidence-backed fact checking using Retrieval-Augmented Generation and few-shot in-context learning with large language models. In Proceedings of the Seventh Fact Extraction and VERification Workshop (FEVER). https://doi.org/10.18653/v1/2024.fever-1.10

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W. T., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval-Augmented Generation for knowledge-intensive NLP tasks. Advances in Neural Information Processing Systems, 33, 9459–9474. https://doi.org/10.48550/arXiv.2005.11401

Kim, S., Lee, J., Park, J., Kim, M., Kang, J., & Lee, H. (2024). RE-RAG: Improving open-domain QA performance and interpretability with relevance estimator in Retrieval-Augmented Generation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 22149–22161). https://doi.org/10.18653/v1/2024.emnlp-main.1236

Wang, X., Wang, Z., Gao, X., Zhang, F., Wu, Y., Xu, Z., Shi, T., Wang, Z., Li, S., Qian, Q., Yin, R., Lv, C., Zheng, X., & Huang, X. (2024). Searching for best practices in Retrieval-Augmented Generation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 17716–17736). https://doi.org/10.18653/v1/2024.emnlp-main.981

Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., & Liu, T. (2025). A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems. https://doi.org/10.1145/3703155

Widiantoro, D., Judijanto, L., Suhartono, Pramono, S. A., Dewa, D. M. R. T., & Februati, B. M. N. (2025). Analysis of the Influence of Knowledge Management, Digital Adoption, and Technology-Based Training on Organizational Performance. International Journal of Engineering, Science and Information Technology, 5(4), 316–320. https://doi.org/10.52088/ijesty.v5i4.1222

Dalkir, K., & Liebowitz, J. (2005). Knowledge management in theory and practice (Vol. 13, No. 2). Amsterdam: Elsevier/Butterworth Heinemann.

Organisation for Economic Co-operation and Development (OECD), The OECD Digital Government Policy Framework: Six Dimensions of a Digital Government. OECD Publishing, 2020. https://doi.org/10.1787/f64fed2a-en

M. Janssen, Y. Charalabidis, and A. Zuiderwijk, “Benefits, adoption barriers and myths of open data and open government,” Information Systems Management, vol. 29, no. 4, pp. 258–268, 2012. https://doi.org/10.1080/10580530.2012.716740

P. Heisig, “Harmonisation of knowledge management—Comparing 160 KM frameworks around the globe,” Journal of Knowledge Management, vol. 13, no. 4, pp. 4–31, 2009. https://doi.org/10.1108/13673270910971798

A. Khattak, N. Ahmad, J. Mustafa, Z. Pervez, and K. Latif, “Context-aware search in dynamic repositories of digital documents,” in 2013 IEEE 16th International Conference on Computational Science and Engineering, pp. 1617–1624, IEEE, 2013. https://doi.org/10.1109/CSE.2013.207

E. Tello-Leal, A. B. Ríos-Alvarado, and A. Díaz-Manríquez, “A semantic knowledge management system for government repositories,” in Database and Expert Systems Applications, Lecture Notes in Computer Science, vol. 9261, pp. 233–247, Springer, 2015. https://doi.org/10.1007/978-3-319-22849-5_17

L. Da, P. Shah, K.-R. Liou, J. Zhang, and H. Wei, “GE-Chat: A graph enhanced retrieval-augmented generation framework for evidential response generation of large language models,” in Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI 2024), 2024.

Y. Gao et al., “Retrieval-augmented generation for large language models: A survey,” arXiv, 2024. https://doi.org/10.48550/arXiv.2312.10997

Kim, S., Lee, J., Park, J., Kim, M., Kang, J., & Lee, H. (2024). RE-RAG: Improving open-domain QA performance and interpretability with relevance estimator in Retrieval-Augmented Generation. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), 22149–22161. https://doi.org/10.18653/v1/2024.emnlp-main.1236

Y. Ates, A. Sayar, I. U. Bozlar, S. Ertugrul, and S. S. Arslan, “Semantic chunking and chain-of-thought reasoning for RAG-based document processing,” in 2025 IEEE 35th International Workshop on Machine Learning for Signal Processing (MLSP), pp. 1–6, IEEE, 2025. https://doi.org/10.1109/MLSP62443.2025.11204203

P. Elchafei, M. Swain, S. Masoudian, and M. Schedl, “Facet-level tracing of evidence uncertainty and hallucination in Retrieval-Augmented Generation,” arXiv, 2026.

Gupta, S. (2025). Retrieval-augmented generation and hallucination in large language models: A scholarly overview. Scholars Journal of Engineering and Technology, 13(05), 328-330. https://doi.org/10.36347/sjet.2025.v13i05.003

Berman, A., de Fine Licht, K., & Carlsson, V. (2024). Trustworthy AI in the public sector: An empirical analysis of a Swedish labor market decision-support system. Technology in Society, 76, 102471. https://doi.org/10.1016/j.techsoc.2024.102471

Roehl, U. B. U., & Hansen, M. B. (2024). Automated, administrative decision-making and good governance: Synergies, trade-offs, and limits. Public Administration Review, 84(6), 1184–1199. https://doi.org/10.1111/puar.13799

Bullock, J. B., & Chen, Y.-C. (2024). The brave new world of AI: Implications for public sector agents, organisations, and governance. Public Management Review. https://doi.org/10.1080/23276665.2024.2356540

S. Malanond and P. Boonyopakorn, “Building trust in automated decision-making: Explainable AI as a compliance-first adoption strategy,” in 2026 IEEE International Conference on Cybernetics and Innovations (ICCI), 2026. https://doi.org/10.1109/ICCI68752.2026.11506463

Hevner, A. R., March, S. T., Park, J., & Ram, S. (2004). Design Science in Information Systems Research. MIS Quarterly, 28(1), 75–105. https://doi.org/10.2307/25148625




DOI: https://doi.org/10.52088/ijesty.v6i3.1904

Refbacks

  • There are currently no refbacks.


Copyright (c) 2026 Yohanes Bowo Widodo

International Journal of Engineering, Science, and Information Technology (IJESTY) eISSN 2775-2674