Learning System for Customer Requirements to Order Related Products in Support of Customer Relationship Management

Authors

  • Kitthanya Teachanontkullawat Computer Science, School of Science, King Mongkut's Institute of Technology Ladkrabang
  • KITIPUM NORNUA Computer Science, School of Science, King Mongkut's Institute of Technology Ladkrabang
  • PRAPOJ SRINUWATTIWONG Computer Science, School of Science, King Mongkut's Institute of Technology Ladkrabang

Keywords:

Chatbot, Large Language Model, Fine-Tuning, Customer Relationship, Retrieval Augmented Generation

Abstract

Glass and construction material businesses currently face significant challenges in understanding customer needs, resulting in inefficient sales growth. This research developed an intelligent chatbot system through a web application connected to the LINE platform to serve as a medium for communication, recording customer requirements, and recommending products that match customer needs. The primary target group is contractor customers who need to purchase construction materials for specialized work. This chatbot system utilizes Large Language Model (LLM) technology that has been fine-tuned, combined with Retrieval Augmented Generation (RAG) techniques to optimize for product knowledge and inter-product relationships. The research used 2,732 test datasets and evaluated the model performance by comparing the accuracy between fine-tuned Llama-3.1-8B with closed models like GPT-4o and Claude 3.5 Sonnet. The results showed that fine-tuned Llama-3.1-8B provided high accuracy in product verification (79%) and connecting products with their applications (87%), while GPT-4o performed better in product recommendations (86%). Additionally, user satisfaction evaluations found that Llama-3.1-8B received the highest overall satisfaction score (4.6 out of 5.0), thus being selected as the primary model for developing the chatbot system for real-world implementation.

References

Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., & Wang, H. (2023). Retrieval-augmented

generation for large language models: A survey. arXiv preprint arXiv:2312.10997.

https://doi.org/10.48550/arXiv.2312.10997

Gupta, A., Shirgaonkar, A., Balaguer, A. D. L., Silva, B., Holstein, D., Li, D., & Benara, V. (2024).

RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture.

arXiv preprint arXiv:2401.08406. https://doi.org/10.48550/arXiv.2401.08406

Li, Y., Chen, H., Wei, J., Huang, R., Gu, S., Zha, D., & Williams, A. (2023). Enhancing Enterprise

Customer Service with Large Language Models: A Case Study. In Proceedings of the

Conference on Empirical Methods in Natural Language Processing (pp. 952-964). Association for Computational Linguistics.

Minaee, S., Mikolov, T., Nikzad, N., Chenaghlu, M., Socher, R., Amatriain, X., & Gao, J. (2024).

Large language models: A survey. arXiv preprint. arXiv:2402.06196. https://doi.org/10.48550/arXiv.2402.06196

Neupane, S., Hossain, E., Keith, J., Tripathi, H., Ghiasi, F., Golilarz, N. A., Kaiser, J., Jiang, Y.,

Zhu, M., & Rahimi, S. (2024). From Questions to Insightful Answers: Building an

Informed Chatbot for University Resources. arXiv preprint arXiv:2405.08120. https://arxiv.org/abs/2405.08120

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., &

Polosukhin, I. (2017). Attention is all you need. arXiv preprint arXiv:1706.03762. https://doi.org/10.48550/arXiv.1706.03762

VM, K., Warrier, H., & Gupta, Y. (2024). Fine Tuning LLM for Enterprise: Practical Guidelines

and Recommendations. arXiv preprint arXiv:2404.10779. https://doi.org/10.48550/arXiv.2404.10779

Downloads

Published

2025-04-30

How to Cite

Teachanontkullawat, K., NORNUA, K., & SRINUWATTIWONG, P. (2025). Learning System for Customer Requirements to Order Related Products in Support of Customer Relationship Management. ROMYOONGTHONG JOURNAL, 3(1), 66–80. retrieved from https://so08.tci-thaijo.org/index.php/romyoongthong/article/view/4963

Issue

Section

Reseach Articles