Visualization of Latency Metrics

Below is a diagram representing latency metrics across the pipeline components and their optimizations:
+--------------+    +--------------+    +--------------+    +--------------+
| Audio Input  | -> |    STT       | -> |    LLM       | -> |    TTS       |
|              |    |  (700ms)    |    |  (200ms)    |    |  (600ms)    |
+--------------+    +--------------+    +--------------+    +--------------+
       |                   |                  |                  |
       v                   v                  v                  v
    Logging          Interim Results      Token Streaming   Audio Chunks
                     Fuzzy Matching       Caching           Force Flush


Performance Improvements Over Time

The following bar chart illustrates the latency reduction achieved across the components:
Component      | Initial Latency (ms) | Optimized Latency (ms)
------------------------------------------------------------
Speech-to-Text | 700-900              | 500-700
Large Language | 200                  | 150
Text-to-Speech | 600                  | 300-400


Through these techniques, VoiceX achieved significant latency reductions, paving the way for seamless, real-time user interactions.