Implementation of digital human avatar virtual assistant with augmented generation retrieval technology in interactive systems for nutrition education
Article Sidebar
Main Article Content
Abstract
In today's digital era, artificial intelligence (AI) based chatbots utilizing Large Language Models (LLM) have become a promising innovation for nutrition education. The integration of Natural Language Processing (NLP) technology with digital animation systems creates new opportunities in developing interactive applications in the context of Indonesian public health, with nutritional challenges in Indonesia showing 21.5% of toddlers experience stunting and 12.2% of adults face obesity, indicating an urgent need for accessible and comprehensive nutrition education. This research aims to develop the GiziAI website that integrates Retrieval Augmented Generation (RAG) technology with digital human avatars to provide nutrition education to Indonesian society. The research method implementing the Nusantara 2.7B Indo Chat large language model, ChromaDB as vector database, Three.js for 3D rendering, ElevenLabs for text-to-speech, and Rhubarb for lip synchronization, with React JS, Flask, MySQL, and LangChain frameworks. Evaluation was conducted using LangSmith to measure model response time, BERTScore to measure answer accuracy, and black box testing for website functionality. Research results show that the RAG system significantly improves model performance with precision increase of 71.5%, recall 60.6%, and F1-score 65.8%, while GPU usage accelerates response by 13.5% compared to CPU. Black box testing shows all website features function as expected.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
References
C. Rozali, A. Zein, dan E. S. Eriana, “Artificial Intelligence ( AI ) Dimasa Depan : Tantangan Dan Peluang,” Jitu J. Inform. Utama, vol. 2, hal. 66–71, 2024, doi: https://doi.org/10.55903/jitu.v2i1.177.
A. Ahmad, “Mengenal Artificial Intelligence , Machine Learning , Neural Network , dan Deep Learning,” J. Teknol. Indones., 2017.
R. C. Tarumingkeng, “Natural Language Processing ( NLP ),” in RUDYCT e-PRESS, no. November, 2024.
V. Gumma, A. Raghunath, M. Jain, dan S. Sitaram, “HEALTH-PARIKSHA: Assessing RAG Models for Health Chatbots in Real-World Multilingual Settings,” arXiv, 2024, doi: https://doi.org/10.48550/arXiv.2410.13671.
R. Yang et al., “Retrieval-augmented generation for generative arti fi cial intelligence in health care,” npj Heal. Syst., 2025, doi: 10.1038/s44401-024-00004-1.
K. N. L. P. Tasks et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” arXiv, 2020, doi: https://doi.org/10.48550/arXiv.2005.11401.
M. A. Hasbi, R. Imanda, dan M. F. Fauzan, “Implementasi Chatbot Berbasis Large Language Model Untuk Pencarian Skripsi Mahasiswa Terintegrasi dengan Whatsapp,” J. Comput. Sci. Artif. Intell., vol. 5, no. 1, hal. 148–167, 2025, doi: https://dx.doi.org/10.29240/arcitech.v5i1.13974 Implementasi.
P. K. Indonesia, PROFIL KESEHATAN INDONESIA 2023. 2023.
D. Firdaus, I. Sumardi, dan Y. Kulsum, “Integrating Retrieval-Augmented Generation with Large Language Model Mistral 7b for Indonesian Medical Herb,” JISKA (Jurnal Inform. Sunan Kalijaga), vol. 9, no. 3, hal. 230–243, 2024, doi: https://doi.org/10.14421/jiska.2024.9.3.230-243.
M. Yusuf, I. P. Sari, dan V. Kristy, “Sistem Pakar Mencegah Stunting dengan Menentukan Gizi Anak Menggunakan Natural Language Processing ( NLP ),” J. JTIK ( J. Teknol. Inf. dan Komun. ), vol. 9, no. September, hal. 924–934, 2025, doi: https://doi.org/10.35870/jtik.v9i3.3614.
M. Korban dan X. Li, “A Survey on Applications of Digital Human Avatars toward Virtual Co-presence,” arXiv, no. May, 2021, doi: https://doi.org/10.48550/arXiv.2201.04168.
J. Sanchez-riera, A. Civit, dan M. Altarriba, “AVATAR : Blender add-on for fast creation of 3D human models,” arXiv, hal. 1–7, 2020, doi: https://doi.org/10.48550/arXiv.2103.14507.
M. H. Mubarak dan A. B. Santoso, “Persepsi Mahasiswa dalam Penggunaan Aplikasi Berbasis Text to Speech Pada Mata Kuliah Teknologi Pembelajaran Bahasa Arab,” J. Ilm. Iqra, vol. 17, hal. 73–84, 2023, doi: https://doi.org/10.30984/jii.v17i1.2376.
A. Fauzan dan S. Hartati, “Text to Speech untuk Bahasa Arab Menggunakan Perangkaian Diphone ( Text to Speech for Arabic Using Diphone Concatenation ),” J. UMP, vol. VI, hal. 9–14, 2018.
M. B. Nendya et al., “AUTO LIP-SYNC PADA KARAKTER VIRTUAL 3 DIMENSI,” REKAM J. Fotogr. Telev. dan Animasi, vol. 11, no. 2, hal. 137–144, 2015, doi: https://doi.org/10.24821/rekam.v11i2.1299.
K. el Haddad, F. Zajega, dan T. Dutoit, “An Open-Source Avatar for Real-Time Human-Agent Interaction Applications,” IEEE, 2019, doi: https://doi.org/10.1109/ACIIW.2019.8925115.
A. Vaswani, “Attention Is All You Need,” arXiv, no. Nips, 2017, doi: https://doi.org/10.48550/arXiv.1706.03762.
J. Camacho-collados, “Embeddings in natural language processing: Theory and advances in vector representations of meaning,” 2020.
L. Wang, N. Yang, X. Huang, dan B. Jiao, “Text Embeddings by Weakly-Supervised,” arXiv, hal. 1–17, 2022, doi: https://doi.org/10.48550/arXiv.2212.03533.
M. N. Ichsanudin dan M. Yusuf, “PERPUSTAKAAN DENGAN METODE BLACK BOX TESTING BAGI PEMULA,” STORAGE J. Ilm. Tek. dan Ilmu Komput., vol. 1, no. 2, hal. 1–8, 2022, doi: https://doi.org/10.55123/storage.v1i2.270.