Convolutional Neural Network Based Hand Gesture Recognition in Sophisticated Background for Humanoid Robot Control

Basit Öğe Kaydı
dc.authorid0000-0003-4236-3646
dc.authorid0000-0001-6888-5755
dc.contributor.authorYildiz, Ali
dc.contributor.authorAdar, Nurettin Gokhan
dc.contributor.authorMert, Ahmet
dc.date.accessioned2026-02-12T21:05:29Z
dc.date.available2026-02-12T21:05:29Z
dc.date.issued2023
dc.departmentBursa Teknik Üniversitesi
dc.description.abstractHand gesture recognition is a preferred way for human-robot interactions. Conventional approaches are generally based on image processing and recognition of hand poses with simple backgrounds. In this paper, we propose deep learning models, and humanoid robot integration for offline and online (real-time) recognition and control using hand gestures. One thousand and two hundred of hand images belonging to four participants are collected to construct the hand gesture database. Five class (forward, backward, right, left and stop) images in six sophisticated backgrounds with different illumination levels are obtained for four participants, and then one participant's images are kept as testing data. A lightweight Convolutional Neural Network (CNN), and transfer learning techniques using VGG16, and Mobilenetv2 are performed on this database to evaluate user independent performance of the hand gesture system. After offline training, real-time implementation is designed using a mobile phone (Wi-Fi and camera), Wi-Fi router, computer with embedded deep learning algorithms, and NAO humanoid robot. Streamed video by the mobile phone is processed and recognized using the proposed deep algorithm in the computer, and then command is transferred to robot via TCP/IP protocol. Thus, the NAO humanoid robot control using hand gesture in RGB and HSV color spaces is evaluated in sophisticated background, and the implementation of the system is presented. In our simulations, 95% and 100% accuracy rates are yielded for the lightweight CNN, and transfer learning, respectively.
dc.identifier.doi10.34028/iajit/20/3/9
dc.identifier.endpage375
dc.identifier.issn1683-3198
dc.identifier.issue3
dc.identifier.scopus2-s2.0-85160252055
dc.identifier.scopusqualityQ2
dc.identifier.startpage368
dc.identifier.urihttps://doi.org/10.34028/iajit/20/3/9
dc.identifier.urihttps://hdl.handle.net/20.500.12885/6978
dc.identifier.volume20
dc.identifier.wosWOS:001046095400009
dc.identifier.wosqualityQ3
dc.indekslendigikaynakWeb of Science
dc.indekslendigikaynakScopus
dc.language.isoen
dc.publisherZarka Private Univ
dc.relation.ispartofInternational Arab Journal of Information Technology
dc.relation.publicationcategoryMakale - Uluslararası Hakemli Dergi - Kurum Öğretim Elemanı
dc.rightsinfo:eu-repo/semantics/openAccess
dc.snmzKA_WoS_20260212
dc.subjectHand gesture recognition
dc.subjecthuman -robot interaction
dc.subjectrobot control
dc.subjectconvolutional neural network
dc.subjectdeep learning
dc.titleConvolutional Neural Network Based Hand Gesture Recognition in Sophisticated Background for Humanoid Robot Control
dc.typeArticle

Dosyalar

Koleksiyon

WoS İndeksli Yayınlar Koleksiyonu
Scopus İndeksli Yayınlar Koleksiyonu