YOLOv8 Algorithm to Improve the Sign Language Letter Detection System Model

Abstract

Sign language is the language used by someone with a deaf disability, their limitations in socializing certainly need a tool to help them socialize. Therefore, by utilizing the YOLOv8 model, this study seeks to improve the accuracy and efficiency of visual-based sign language detection, especially the Indonesian Sign System (SIBI) alphabet. The purpose of this study is to evaluate the performance  of the YOLOv8  model in detecting sign language to identify the SIBI alphabet. The method used includes training and testing  the YOLOv8 model  on a dataset consisting of videos and photos using a laptop camera. Each video is converted into an image and taken each frame of the video, then preprocessing, data transformation and data mining are carried out  using YOLOv8 to generate bounding boxes, labels, and confidence scores of detected objects. With a complexity of 168 layers and more than 11 million parameters, the model is able to consistently detect sign language with an average inference rate of 4.6 ms per image. The detection results for each letter class showed a very high success rate, especially on letters such as D, F, N, O, and Q which achieved an accuracy of up to 96%. Overall, 22 out of 26 letters showed "very good" detection results (above 90%), while 4 letters (H, M, T, Z) had "quite good" detection results (86%-89%).