Design and Evaluation of a System for Hand Mudra Recognition and Benefits Analysis

Abstract

Hand gestures, known as mudras, play a vital role in conveying expressions and narratives in Indian classical dance forms. However, automated recognition of such gestures, particularly in less-explored forms like Sattriya, remains a challenging task due to variations in hand orientation, lighting conditions, and limited availability of diverse datasets. Existing approaches primarily rely on traditional computer vision techniques or basic machine learning models, which often fail to generalize across complex gestures and real-world scenarios. To address these limitations, this study proposes a deep learning–based system for accurate hand mudra recognition using a comprehensive multi-class dataset of classical dance gestures captured from multiple angles. The proposed approach leverages transfer learning with MobileNetV2, combined with data augmentation, normalization, and finetuning strategies to enhance feature extraction and classification performance. The system is integrated into a Django-based web application that supports user interaction, real-time prediction, and administrative control. Experimental results demonstrate a high classification accuracy of 93.4% across 50 mudra classes, validating the effectiveness of the proposed model. The study concludes that integrating deep learning with cultural heritage applications can significantly improve gesture recognition systems while contributing to the preservation and digital documentation of traditional art forms.