Article Open Access

Transformer-Based Automatic Speech Recognition Development for Transcription and Clinical Information Extraction in Indonesian on Electronic Health Records

Bachtiyar Wahab, Fitriana Harahap, Khairul Anwar, Nabila Mahyuda, Nur Asiah Hasibuan

Abstract


Integrating Automatic Speech Recognition (ASR) into Electronic Health Records (EHR) has the potential to boost the efficiency of clinical documentation by automating the transcription of doctor-patient conversations. However, high-quality transcription doesn't necessarily mean the clinical information will be classified accurately. This study aims to analyze the quality of ASR transcriptions and compare the performance of Machine Learning algorithms in classifying clinical units based on Indonesian clinical texts. The dataset used consists of 1,000 medical records distributed across five clinical units: Cardiology, ENT, General, Internal Medicine, and Pediatrics. Transcription quality was evaluated using Word Error Rate (WER), while classification was done through Term Frequency-Inverse Document Frequency (TF-IDF) feature extraction and four algorithms: Random Forest, Logistic Regression, Decision Tree, and Naive Bayes. Model evaluation was carried out using accuracy, precision, recall, and F1-score on 200 test data. The research results show that the average WER is 0.40%, indicating very good ASR transcription quality. Random Forest achieved the highest accuracy at 44.50%, Logistic Regression 42.00%, Decision Tree 38.00%, and Naive Bayes 32.00%. Random Forest performed best in the Internal Medicine Clinic with a precision of 0.58 and an F1-score of 0.53. These findings suggest that the main challenge lies in the representation of textual features, mainly due to the similarity of medical vocabulary across clinical units. This study highlights the need to develop approaches based on context, clinical entities, ICD-10 codes, and Transformers to improve classification accuracy and support the automation of clinical information management in EHRs. This approach is expected to strengthen semantic understanding, reduce classification errors, and improve system reliability as a supporter of digital clinical documentation in healthcare settings

Keywords


Automatic Speech Recognition, Electronic Health Records, TF-IDF, Machine Learning, Clinical Classification

References


A. Adeniyi, J. Arowoogun, R. Chidi, C. Okolo, and O. Babawarun, “The impact of electronic health records on patient care and outcomes: A comprehensive review,” World J. Adv. Res. Rev., vol. 21, pp. 1446–1455, Feb. 2024, doi: 10.30574/wjarr.2024.21.2.0592.

K. Pereira, “Toward Enhanced Healthcare Efficiency: The Impact of Digitizing Medical Records,” Rev. Sist., vol. 14, pp. 709–714, Jul. 2024, doi: 10.56238/rcsv14n3-020.

M. Dehghan, A. Behzadi, M. H. Mehrolhassani, and M. M. Ghaemi, “Challenges and facilitators of electronic health record implementation: a scoping review,” Int. J. Med. Inform., vol. 205, p. 106094, 2026, doi: https://doi.org/10.1016/j.ijmedinf.2025.106094.

P. Mehta, P. Ajmera, A. Iyengar, and R. Botchu, “Advances in AI-Powered Medical Transcription: Implications for Clinical Practice, Radiology, and Future Interface Technologies,” Indian J. Radiol. Imaging, May 2026, doi: 10.1055/s-0046-1820451.

M. Alabbad and W. Alhoshan, “Automatic Speech Recognition in Healthcare in the Post-LLM Era: A Scoping Review,” Healthcare, vol. 14, no. 10. p. 1333, 2026. doi: 10.3390/healthcare14101333.

M. Sztabinski and P. Weichbroth, “Deep Learning in Medical Speech to Text: Methods and Challenges,” Symmetry, vol. 18, no. 6. p. 885, 2026. doi: 10.3390/sym18060885.

A. Elhadad, S. Hamad, N. Elfiky, F. Alanazi, A. I. Taloba, and R. M. A. El-Aziz, “Advancing Healthcare: Intelligent Speech Technology for Transcription, Disease Diagnosis, and Interactive Control of Medical Equipment in Smart Hospitals,” AI, vol. 5, no. 4. pp. 2497–2517, 2024. doi: 10.3390/ai5040121.

J. Ng et al., “Evaluating the performance of artificial intelligence-based speech recognition for clinical documentation: a systematic review,” BMC Med. Inform. Decis. Mak., vol. 25, Jul. 2025, doi: 10.1186/s12911-025-03061-0.

M. Zuchowski and A. Göller, “Speech recognition for medical documentation: an analysis of time, cost efficiency and acceptance in a clinical setting,” Br. J. Healthc. Manag., vol. 28, pp. 30–36, Jan. 2022, doi: 10.12968/bjhc.2021.0074.

X. Luo, L. Zhou, K. Adelgais, and Z. Zhang, Assessing the Effectiveness of Automatic Speech Recognition Technology in Emergency Medicine Settings: A Comparative Study of Four AI-powered Engines. 2024. doi: 10.21203/rs.3.rs-4727659/v1.

A. Elhadad et al., “Improved healthcare diagnosis accuracy through the application of deep learning techniques in medical transcription for disease identification,” Alexandria Eng. J., vol. 123, pp. 112–123, 2025, doi: https://doi.org/10.1016/j.aej.2025.03.047.

G. Ben Abdennour, K. Gasmi, and R. Ejbali, “An Optimal Model for Medical Text Classification Based on Adaptive Genetic Algorithm,” Data Sci. Eng., vol. 9, no. 4, pp. 378–392, 2024, doi: 10.1007/s41019-024-00257-8.

Y. Zhang, H. T. Le, N. Lopez, and K. Phan, “Comparative Study of Machine Learning Models for Textual Medical Note Classification,” Computers, vol. 15, no. 1. p. 7, 2026. doi: 10.3390/computers15010007.

T. Ling, L. Jake, J. Adams, K. Osinski, X. Liu, and D. Friedland, “Interpretable machine learning text classification for clinical computed tomography reports – a case study of temporal bone fracture,” Comput. Methods Programs Biomed. Updat., vol. 3, p. 100104, 2023, doi: https://doi.org/10.1016/j.cmpbup.2023.100104.

A. Czy?ewski et al., “A Comprehensive Polish Medical Speech Dataset for Enhancing Automatic Medical Dictation,” Sci. Data, vol. 12, no. 1, p. 1436, 2025, doi: 10.1038/s41597-025-05776-1.

A. Jarin, A. Santosa, M. T. Uliniansyah, L. R. Aini, and E. Nurfadhilah, “Automatic speech recognition for indonesian medical dictation in cloud environment,” vol. 13, no. 2, pp. 1762–1772, 2024, doi: 10.11591/ijai.v13.i2.pp1762-1772.

T. Hodgson and E. Coiera, “Risks and benefits of speech recognition for clinical documentation: a systematic review,” J. Am. Med. Informatics Assoc., vol. 23, pp. e169–e179, Nov. 2015, doi: 10.1093/jamia/ocv152.

A. Wibawa, P. Ramadhani, G. Buntoro, R. Hariadi, P. Siswanto, and S. Sabilla, “Leveraging Text-Mining Techniques On Electronic Medical Records to Analyze National Drug-insured Medication Use,” Kinet. Game Technol. Inf. Syst. Comput. Network, Comput. Electron. Control, May 2023, doi: 10.22219/kinetik.v8i2.1695.

S. McCrocklin, “Automatic Speech Recognition BT - The Palgrave Encyclopedia of Computer-Assisted Language Learning,” L. McCallum and D. Tafazoli, Eds., Cham: Springer Nature Switzerland, 2025, pp. 1–6. doi: 10.1007/978-3-031-51447-0_30-1.

S. O. Russell, I. Gessinger, A. Krason, G. Vigliocco, and N. Harte, “What automatic speech recognition can and cannot do for conversational speech transcription,” Res. Methods Appl. Linguist., vol. 3, no. 3, p. 100163, 2024, doi: https://doi.org/10.1016/j.rmal.2024.100163.

A. NithyaKalyani and S. Jothilakshmi, “Chapter 7 - Speech Summarization for Tamil Language,” N. B. T.-I. S. S. P. Dey, Ed., Academic Press, 2019, pp. 113–138. doi: https://doi.org/10.1016/B978-0-12-818130-0.00007-6.

H. Bergier et al., “Digital health, big data and smart technologies for the care of patients with systemic autoimmune diseases: Where do we stand?,” Autoimmun. Rev., vol. 20, no. 8, p. 102864, 2021, doi: https://doi.org/10.1016/j.autrev.2021.102864.

D. Sitaram and G. B. T.-M. T. T. C. Manjunath, “Chapter 5 - Paradigms for Developing Cloud Applications,” Boston: Syngress, 2012, pp. 205–253. doi: https://doi.org/10.1016/B978-1-59749-725-1.00005-6.

M. Schonlau and R. Zou, “The random forest algorithm for statistical learning,” Stata J. Promot. Commun. Stat. Stata, vol. 20, pp. 3–29, Mar. 2020, doi: 10.1177/1536867X20909688.

H. Belyadi and A. B. T.-M. L. G. for O. and G. U. P. Haghighat, “Chapter 5 - Supervised learning,” Gulf Professional Publishing, 2021, pp. 169–295. doi: https://doi.org/10.1016/B978-0-12-821929-4.00004-4.

S. U. Hassan, J. Ahamed, and K. Ahmad, “Analytics of machine learning-based algorithms for text classification,” Sustain. Oper. Comput., vol. 3, pp. 238–248, 2022, doi: https://doi.org/10.1016/j.susoc.2022.03.001.

L. Meng, B. Bai, W. Zhang, L. Liu, and C. Zhang, “Research on a Decision Tree Classification Algorithm Based on Granular Matrices,” Electronics, vol. 12, no. 21. p. 4470, 2023. doi: 10.3390/electronics12214470.

M. Schonlau, “The Naive Bayes Classifier,” 2023, pp. 143–160. doi: 10.1007/978-3-031-33390-3_8.




DOI: https://doi.org/10.52088/ijesty.v6i4.1938

Refbacks

  • There are currently no refbacks.


Copyright (c) 2026 Bachtiyar Wahab, Fitriana Harahap, Khairul Anwar, Nabila Mahyuda, Nur Asiah Hasibuan

International Journal of Engineering, Science, and Information Technology (IJESTY) eISSN 2775-2674