Learning System for Customer Requirements to Order Related Products in Support of Customer Relationship Management
Keywords:
Chatbot, Large Language Model, Fine-Tuning, Customer Relationship, Retrieval Augmented GenerationAbstract
Glass and construction material businesses currently face significant challenges in understanding customer needs, resulting in inefficient sales growth. This research developed an intelligent chatbot system through a web application connected to the LINE platform to serve as a medium for communication, recording customer requirements, and recommending products that match customer needs. The primary target group is contractor customers who need to purchase construction materials for specialized work. This chatbot system utilizes Large Language Model (LLM) technology that has been fine-tuned, combined with Retrieval Augmented Generation (RAG) techniques to optimize for product knowledge and inter-product relationships. The research used 2,732 test datasets and evaluated the model performance by comparing the accuracy between fine-tuned Llama-3.1-8B with closed models like GPT-4o and Claude 3.5 Sonnet. The results showed that fine-tuned Llama-3.1-8B provided high accuracy in product verification (79%) and connecting products with their applications (87%), while GPT-4o performed better in product recommendations (86%). Additionally, user satisfaction evaluations found that Llama-3.1-8B received the highest overall satisfaction score (4.6 out of 5.0), thus being selected as the primary model for developing the chatbot system for real-world implementation.
References
Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., & Wang, H. (2023). Retrieval-augmented
generation for large language models: A survey. arXiv preprint arXiv:2312.10997.
https://doi.org/10.48550/arXiv.2312.10997
Gupta, A., Shirgaonkar, A., Balaguer, A. D. L., Silva, B., Holstein, D., Li, D., & Benara, V. (2024).
RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture.
arXiv preprint arXiv:2401.08406. https://doi.org/10.48550/arXiv.2401.08406
Li, Y., Chen, H., Wei, J., Huang, R., Gu, S., Zha, D., & Williams, A. (2023). Enhancing Enterprise
Customer Service with Large Language Models: A Case Study. In Proceedings of the
Conference on Empirical Methods in Natural Language Processing (pp. 952-964). Association for Computational Linguistics.
Minaee, S., Mikolov, T., Nikzad, N., Chenaghlu, M., Socher, R., Amatriain, X., & Gao, J. (2024).
Large language models: A survey. arXiv preprint. arXiv:2402.06196. https://doi.org/10.48550/arXiv.2402.06196
Neupane, S., Hossain, E., Keith, J., Tripathi, H., Ghiasi, F., Golilarz, N. A., Kaiser, J., Jiang, Y.,
Zhu, M., & Rahimi, S. (2024). From Questions to Insightful Answers: Building an
Informed Chatbot for University Resources. arXiv preprint arXiv:2405.08120. https://arxiv.org/abs/2405.08120
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., &
Polosukhin, I. (2017). Attention is all you need. arXiv preprint arXiv:1706.03762. https://doi.org/10.48550/arXiv.1706.03762
VM, K., Warrier, H., & Gupta, Y. (2024). Fine Tuning LLM for Enterprise: Practical Guidelines
and Recommendations. arXiv preprint arXiv:2404.10779. https://doi.org/10.48550/arXiv.2404.10779
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 คณะวิทยาการจัดการ มหาวิทยาลัยราชภัฏเทพสตรี

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.