Mechatronic system design with LLM integration: a voice-controlled servo-driven 7-segment clock

Abstract

Digital clocks have transformed from basic timekeeping tools into versatile gadgets. Modern clocks now offer features like automatic time synchronization, multiple time zones, and customizable alarms. The development of a voice-controlled servo-driven 7-segment clock seeks to bridge the gap in comprehensive, voice-controlled time management. This project aims to integrate large language models (LLMs) and automatic speech recognition (ASR) with mechatronic components to create a precise and user-friendly system. This digital clock's display is driven by an ESP32 microcontroller that uses servo motors and servo drivers. Users can interact with the clock via a smartphone web application that communicates with the ESP32 through an adaptive.io server. The ESP32 is also programmed to show the time, control other functions, and provide feedback through a buzzer and an LED. The system was tested under various conditions to evaluate the accuracy and responsiveness of the LLM-based voice recognition. To test for the accuracy of the voice commands, a dataset of ten (10) commands was used. The system testing and evaluation were based on the accuracy and responsiveness of the LLM-based voice recognition and timekeeping accuracy. Performance evaluation results for ten trials each, obtained, were, 90%, 90% and 80% for the average timer setting of 665ms, average updating current time setting after timer operation of 815ms and average alarm setting time of 475ms, respectively. The time was also correctly updated 54 times giving errors 6 times, for a 60 minutes test. Errors were as a result of poor internet connection. This confirmed that LLMs significantly improve reliability and user satisfaction of the developed mechatronic systems. This research demonstrates its potential to enhance system performance, paving the way for further innovations in voice-controlled automation.