Description:
The ext_proc already extracts the model field from the upstream response, but discards it before building dynamic metadata. Adding it will allow the proxy to log the actual serving LLM even if it has been replaced/upgraded. Should be a small change.
Use case: operators can swap the backing model by patching a single Backend resource without needing to update every users AIGatewayRoute just for metadata accuracy.
Happy to open a PR or provide more information.
Description:
The ext_proc already extracts the model field from the upstream response, but discards it before building dynamic metadata. Adding it will allow the proxy to log the actual serving LLM even if it has been replaced/upgraded. Should be a small change.
Use case: operators can swap the backing model by patching a single Backend resource without needing to update every users AIGatewayRoute just for metadata accuracy.
Happy to open a PR or provide more information.