On-Device Automatic Speech Recognition for IIoT and Extended Reality Industrial Metaverse Applications

This paper presents a comprehensive study on enhancing Industrial Internet of Things (IIoT) and Industrial Metaverse Applications through the integration of On-Device Automatic Speech Recognition (ASR) using Microsoft HoloLens 2 smart glasses. Specifically, this paper focuses on the utilization of the HoloLens 2's microphone array and sound capture APIs to benchmark the performance and accuracy of on-device ASR models. The evaluation of these models includes metrics such as Character Error Rate (CER), Word Error Rate (WER), and latency. Furthermore, the paper explores various optimization techniques, including quantization tools and model refinement strategies, aimed at minimizing latency while maintaining high accuracy. The study also emphasizes the importance of supporting low-resource languages, using Galician—a language spoken by less than 3 million people worldwide—as a case study. By benchmarking different variations of a Wav2Vec2.0-based ASR model fine-tuned for Galician, the research identifies the most effective models and their optimal runtime configurations. This work underscores the critical role of low-latency, on-device ASR systems in real-time IIoT and Industrial Metaverse applications, highlighting how these technologies can enhance operational efficiency, privacy, and user experience in industrial environments. The findings contribute to the broader applicability of ASR's potential in supporting emerging Metaverse applications across various industrial contexts.