Fixing the UI Flickering Issue in an AI Voice Chat App

Introduction

While developing an AI voice chat application, I encountered a bug where the AI's response text would flicker when the user sent multiple messages in rapid succession.

In this article, I will share the thought process behind identifying the cause of this "flickering" issue and how I resolved it.

The Issue

When a user sends messages continuously without waiting for the AI's response, the AI text displayed on the screen repeatedly appears and disappears at high speed.

"Yes, I'm doing well." → "" → "Today's goal is..." → "" → "Yes, I'm doing well." → ...

Background

This app has two distinct features:

1. Streaming responses
   AI responses arrive bit by bit via streaming. When multiple requests (A/B) are processed in parallel, chunks from responses A and B arrive alternately.

2. TTS synchronized display
   As the AI's response is read aloud by the voice, the text is displayed character by character in sync with the audio.

Streaming: "Yes, I'm doing well." ← Full text arrives in several hundred milliseconds.
Display: "Y" → "Ye" → "Yes" ... ← Character by character over the duration of the audio (2 seconds).

When these two are combined, state management becomes complex.

Cause Analysis

Let's trace the sequence of the problem.

When Message A is sent → Message B is sent immediately

T0: Response A is being displayed
Currently syncing message = A
Syncing text = "Yes, I'm"
→ Display: "Yes, I'm"

T1: Streaming of Response B starts
Currently syncing message switches to B
Syncing text resets to "" ← This is the problem!
→ Display: ""

T2: Chunk of Response A arrives
Latest message = A, but syncing = B, so it returns the full text of A
→ Display: "Yes, I'm doing well."

T3: Chunk of Response B arrives
Latest message = B, syncing = B, so it returns the syncing text ("")
→ Display: ""

... "" → Full text of A → "" → Full text of B → "" flickers repeatedly.

Root Cause

• The syncing text is reset to empty when a new response begins.
• Chunks of A/B arrive alternately due to parallel streaming.
• The judgment of "which message to display" changes every time.

Solution: Message ID Commitment Strategy

Concept

"Lock onto a message once display begins, and do not switch until the next TTS playback starts."

The key is to restrict the switching timing to the start of TTS playback. This allows for stable display that is not affected by the arrival order of chunks.

Implementation Outline

const committedIdRef = useRef(null)

// Switch commit target when TTS sync begins
if (syncedText.length > 0) {
  committedIdRef.current = syncingMessageId
  return syncedText
}

// If already committed, continue displaying that message
if (committedIdRef.current) {
  return messages.find(m => m.id === committedIdRef.current)?.content
}

Sequence After Fix

T0: Response A is being displayed
Commit target = A
→ Display: "Yes, I'm"

T1: Streaming of Response B starts
Syncing text → "", but commit target = A remains
→ Display: "Yes, I'm doing well." ← Locked to A (stable)

T2: Chunks of A/B arrive alternately
Commit target = A remains
→ Display: No flickering

T3: TTS playback of Response B starts
Syncing text = "T" (1st character)
→ Switch commit target = B
→ Display: "T"

T4 onwards: B increases character by character

Summary

• Issue
  • Display flickers due to timing mismatch in state switching during parallel streaming
• Cause
  • Syncing text becomes empty when a new response starts, causing oscillation when A/B chunks arrive alternately
• Solution
  • Lock the message ID using a ref and delay switching until TTS playback begins

Learnings

• In real-time processing like streaming × audio synchronization, controlling the timing of state transitions is crucial
• One must consider not only avoiding "empty states" but also "unnecessary alternating switches"
• useRef is effective for maintaining values within useMemo

Reference

Why useRef?

Reasons to use useRef instead of useState:

• Does not trigger re-rendering: Updating the ID itself does not require a re-render
• Can be updated within useMemo: Updating state inside useMemo violates React rules
• Synchronous reference: Can be referenced immediately without waiting for the next render