Open Source for Real-Time Voice Understanding: An Edge AI Solution

This talk presents the architecture and implementation of a real-time voice understanding system for retail environments, powered by open source AI technologies and deployed on NVIDIA Jetson devices. The system integrates automatic speech recognition (ASR) and large language models (LLMs) to process customer interactions in real-time, enabling conversation understanding, summarization, and sentiment analysis. We will explore system architecture design, selection and evaluation of ASR and LLM models, design trade-offs between latency and accuracy, and share performance test results.