Rancang Bangun AirMouse Menggunakan Sarung Tangan Bersensor Berbasis ESP32

Sholahuddin Muhammad Irsyad, Achmad Basuki, Bima Sena Bayu Dewantara


Digital interactions are still commonly using indirect media such as mouse and keyboard to provide user input in the form of two-dimensional data. Therefore, to provide intuition in virtual interactions, it is possible to add media that can draw directly in the air or a flat surface that will track hand movements and overall finger position. In this research, we try to track hand movements in real time by capturing the position of the hand and finger curvature using a wearable sensor equipped with an Inertial Measurement Unit (IMU) sensor and a flex sensor installed by the user. Then the system will identify the position of the user's finger bending. and the location indicated by the sensors installed to move the cursor on the screen and simulate left-click and right-click hand movements as with a traditional mouse. By using this system, users can interact with the computer more naturally and get the accuracy of cursor movement with the accuracy of finger movement translation reaching more than 85% and the translation of hand movements to mouse cursor movements is on average 73% for shapes that use straight lines. and 23.4% on curved lines such as circles and other shapes.


hand movement recognition; line drawing; inertial measurement unit; human-computer interaction

Full Text:



Chan S. Usage of Mobile Video Conferencing Apps Including Zoom Grew 150% in the First Half of 2021 n.d. https://sensortower.com/blog/video-conferencing-apps-mau-growth.

Chang H-T, Chang J-Y. Sensor Glove Based on Novel Inertial Sensor Fusion Control Algorithm for 3-D Real-Time Hand Gestures Measurements. IEEE Trans Ind Electron 2020;67:658–66. https://doi.org/10.1109/TIE.2019.2912765.

Flores MBH, Siloy CMB, Oppus C, Agustin L. User-oriented finger-gesture glove controller with hand movement virtualization using flex sensors and a digital accelerometer. 2014 Int. Conf. Humanoid Nanotechnol. Inf. Technol. Commun. Control Environ. Manag. HNICEM, Palawan, Philippines: IEEE; 2014, p. 1–4. https://doi.org/10.1109/HNICEM.2014.7016195.

Wang J-S, Hsu Y-L, Liu J-N. An Inertial-Measurement-Unit-Based Pen With a Trajectory Reconstruction Algorithm and Its Applications. IEEE Trans Ind Electron 2010;57:3508–21. https://doi.org/10.1109/TIE.2009.2038339.

Sadek MI, Mikhael MN, Mansour HA. A new approach for designing a smart glove for Arabic Sign Language Recognition system based on the statistical analysis of the Sign Language. 2017 34th Natl. Radio Sci. Conf. NRSC, Alexandria: IEEE; 2017, p. 380–8. https://doi.org/10.1109/NRSC.2017.7893499.

Rizwan SB, Khan MSZ, Imran M. American Sign Language Translation via Smart Wearable Glove Technology. 2019 Int. Symp. Recent Adv. Electr. Eng. RAEE, Islamabad, Pakistan: IEEE; 2019, p. 1–6. https://doi.org/10.1109/RAEE.2019.8886931.

Lee S, Park K, Lee J, Kim K. User Study of VR Basic Controller and Data Glove as Hand Gesture Inputs in VR Games. 2017 Int. Symp. Ubiquitous Virtual Real. ISUVR, Nara, Japan: IEEE; 2017, p. 1–3. https://doi.org/10.1109/ISUVR.2017.16.

Kao C-H, Chen C-C, Jhu W-Y, Tsai Y-T, Chen S-H, Hsu C-M, et al. Novel digital glove design for virtual reality applications. Microsyst Technol 2018;24:4247–66. https://doi.org/10.1007/s00542-018-3747-z.

Yan H, Shan Q, Furukawa Y. RIDI: Robust IMU Double Integration. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y, editors. Comput. Vis. – ECCV 2018, vol. 11217, Cham: Springer International Publishing; 2018, p. 641–56. https://doi.org/10.1007/978-3-030-01261-8_38.

Sarker S, Hoque MM. An Intelligent System for Conversion of Bangla Sign Language into Speech. 2018 Int. Conf. Innov. Sci. Eng. Technol. ICISET, Chittagong, Bangladesh: IEEE; 2018, p. 513–8. https://doi.org/10.1109/ICISET.2018.8745608.

Yudhana A, Rahmawan J, Negara CUP. Flex Sensors and MPU6050 Sensors Responses on Smart Glove for Sign Language Translation. IOP Conf Ser Mater Sci Eng 2018;403:012032. https://doi.org/10.1088/1757-899X/403/1/012032.


What is a Graphical User Interface? Definition and FAQs | OmniSci n.d. https://www.omnisci.com/technical-glossary/graphical-user-interface (accessed January 2, 2022).

Processing Overview. Processing n.d. https://processing.org//tutorials/overview (accessed January 2, 2022).

Valli C, Lucas C. Linguistics of American Sign Language: an introduction. 3rd ed. Washington, D.C: Gallaudet University Press; 2000.

DOI: https://doi.org/10.17529/jre.v18i3.25816


View My Stats


Creative Commons License

Jurnal Rekayasa Elektrika (JRE) is published under license of Creative Commons Attribution-ShareAlike 4.0 International License.