TTS Streaming Delayed Until Conversation Agent Completes

[mikenorgate]

Describe the feature

When using streaming conversation agents with TTS, the audio playback is delayed until the entire conversation response completes, even though text is available and streaming to the TTS engine immediately.

When creating complex assist pipelines there can be a fairly large delay between it starting and finishing, being able to give status updates as the flow is progressing would be really useful

Example commands

N/A

Use cases

I have used a webhook conversation integration to connect assist to n8n, this will run some flows to either take direct actions or redirect to an LLM for more conversational queries.

Due to some of the flows being slow, like waiting for external services, it want to be able to send updates while it's processing. e.g. One moment while I process your request...

Anything else?

I have got this working up to a point. I have the conversation agent sending back multiple responses correctly however TTS doesn't kick in until the last message is received and the webhook connection is closed.

From what I can see from the code around this the processing currently synchronous, the conversation agent has to complete before TTS is run, even though with streaming enabled it sends the text to the TTS buffer as it's received .

My current series of events looks like this, based on my rough debugging of the pipeline

T+0.0s  User stops speaking
T+0.5s  Conversation agent yields first message
T+0.5s  Text reaches Wyoming TTS immediately via `SynthesizeChunk` events
T+0.5s  TTS background task starts, cache created, audio generated
T+0.5s  But: Clients don't fetch yet (waiting for TTS_START event)
        [8 second delay while external processes happens]
T+8.5s  Conversation agent yields second message
T+8.5s  Generator completes, `async_converse()` returns
T+8.5s  Intent stage completes
T+8.5s  TTS stage begins, TTS_START event emitted
T+8.5s  Now clients start fetching audio
T+9.0s  First message finally plays