Smart Chatbot for Emotion Detection and Adaptive Music Recommendation

Abstract

In recent years, the development of emotionally intelligent systems has gained significant attention due to their ability to enhance human-computer interaction. This project presents a multimodal chatbot emotion recognition and music recommendation system that integrates advanced Deep Learning techniques to understand and respond to user emotions effectively. The system utilizes models such as Convolutional Neural Networks , Long Short-Term Memory , Bidirectional LSTM and a hybrid LSTM-GRU architecture, combined with BERT embeddings for accurate textual feature extraction. The proposed system supports three modes of user interaction: text input, voice input, and webcam-based facial expression analysis, enabling comprehensive emotion detection. The models are trained and evaluated using a Kaggle dataset in a Jupyter Notebook environment, with performance assessed through metrics such as accuracy, precision, recall, and F1-score. Among the implemented models, BiLSTM demonstrates superior performance in emotion classification. Based on the detected emotion, the system recommends appropriate music that aligns with the user’s mood through an interactive Flaskbased web application. The chatbot enables realtime communication and provides personalized responses, allowing users to express their emotions naturally. This approach not only improves user engagement but also highlights the potential of integrating emotion recognition with recommendation systems to create more adaptive and human-centric AI applications.