VoiceRAG

• blog
  https://techcommunity.microsoft.com/blog/azure-ai-services-blog/voicerag-an-app-pattern-for-rag--voice-using-azure-ai-search-and-the-gpt-4o-real/4259116

• learn
  https://zenn.dev/tomodo_ysys/articles/azure-audio-rag-sample

• repo
  https://github.com/Azure-Samples/aisearch-openai-rag-audio

real time
https://github.com/Azure-Samples/aoai-realtime-audio-sdk/

https://github.com/Azure-Samples/aoai-realtime-audio-sdk/tree/main

ローカル環境
js
/Users/s.y/dev/01.react/aoai-realtime-audio-sdk/javascript/samples/web

js + python
/Users/s.y/dev/05.next/aisearch-openai-rag-audio

生成AIとリアルタイムな会話をする

https://openai.com/index/introducing-the-realtime-api/

sequenceDiagram
actor User as End User
participant MiddleTier as /realtime host
participant AOAI as Azure OpenAI
User->>MiddleTier: Begin interaction
MiddleTier->>MiddleTier: Authenticate/Validate User
MiddleTier--)User: audio information
User--)MiddleTier:
MiddleTier--)User: text information
User--)MiddleTier:
MiddleTier--)User: control information
User--)MiddleTier:
MiddleTier->>AOAI: connect to /realtime
MiddleTier->>AOAI: configure session
AOAI->>MiddleTier: session start
MiddleTier--)AOAI: send/receive WS commands
AOAI--)MiddleTier:
AOAI--)MiddleTier: create/start conversation responses
AOAI--)MiddleTier: (within responses) create/start/add/finish items
AOAI--)MiddleTier: (within items) create/stream/finish content parts