Skip to content

fix(langchain): normalise provider-specific content blocks in ModelFallbackMiddleware#36601

Open
VishwasPatel (Vishwaspatel2401) wants to merge 1 commit intolangchain-ai:masterfrom
Vishwaspatel2401:fix/modelfallback-normalise-function-call
Open

fix(langchain): normalise provider-specific content blocks in ModelFallbackMiddleware#36601
VishwasPatel (Vishwaspatel2401) wants to merge 1 commit intolangchain-ai:masterfrom
Vishwaspatel2401:fix/modelfallback-normalise-function-call

Conversation

@Vishwaspatel2401
Copy link
Copy Markdown

@Vishwaspatel2401 VishwasPatel (Vishwaspatel2401) commented Apr 7, 2026

Fixes #36531

When ModelFallbackMiddleware falls back to an OpenAI-based model, the response
can contain function_call content blocks in OpenAI's internal format. If that
response is added to message history and forwarded to a primary model that only
understands LangChain-standard tool_use blocks (e.g. ChatBedrockConverse),
the primary model raises an error on the unrecognised block type.

This fix adds two normalisation helpers to ModelFallbackMiddleware:

  • _normalise_ai_message: converts function_call blocks to standard tool_use,
    handling string/dict arguments and id/callId/call_id aliases
  • _normalise_fallback_response: applies normalisation across all result messages
    after a fallback model succeeds

The primary model path is deliberately left unchanged.

Co-authored with Snir Balgaly (@balgaly) who drafted the initial implementation in #36600.

Verification

Unit tests added in tests/unit_tests/agents/middleware/test_model_fallback_normalise.py
covering string/dict arguments, id alias resolution, mixed content blocks,
invalid JSON handling, and verification that the primary model path is NOT normalised.

…llbackMiddleware

When ModelFallbackMiddleware falls back to an OpenAI-based model, the
response can contain function_call content blocks in OpenAI's internal
format. If that response is added to message history and forwarded to a
primary model that only understands LangChain-standard tool_use blocks
(e.g. ChatBedrockConverse), the primary model raises an error on the
unrecognised block type.

This fix adds two normalisation helpers:
- _normalise_ai_message: converts function_call blocks to standard
  tool_use, handling string/dict arguments and id/callId/call_id aliases
- _normalise_fallback_response: applies normalisation across all result
  messages after a fallback model succeeds

The primary model path is deliberately left unchanged.

Co-authored-by: Snir Balgaly <balgaly@users.noreply.github.com>
closes: langchain-ai#36531
Copy link
Copy Markdown

@balgaly Snir Balgaly (balgaly) left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good coverage overall. One test case worth adding: a check that the primary model path returns unchanged (no normalisation). The current tests verify the fallback path converts correctly, but there is no test asserting that a primary handler result with function_call blocks passes through as-is.

Something like:

def test_primary_path_not_normalised() -> None:
    """Primary handler result is returned unchanged -- normalisation only applies in fallback path."""
    from langchain.agents.middleware.model_fallback import ModelFallbackMiddleware

    primary_msg = AIMessage(
        content=[{"type": "function_call", "id": "c1", "name": "fn", "arguments": "{}"}]
    )
    primary_response = ModelResponse(result=[primary_msg])

    middleware = ModelFallbackMiddleware.__new__(ModelFallbackMiddleware)
    middleware.models = []

    result = middleware.wrap_model_call(object(), lambda _req: primary_response)  # type: ignore[arg-type]
    assert result.result[0].content[0]["type"] == "function_call"

This locks in the invariant that normalisation is scoped only to fallback, which is the key design decision here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external fix For PRs that implement a fix langchain `langchain` package issues & PRs new-contributor size: M 200-499 LOC

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ChatBedrockConverse crashes on function_call content blocks from OpenAI Responses API when used with ModelFallbackMiddleware

2 participants