Skip to content

feat: add Responses API support for Azure OpenAI provider#4977

Closed
devin-ai-integration[bot] wants to merge 1 commit intomainfrom
devin/1773990865-azure-responses-api
Closed

feat: add Responses API support for Azure OpenAI provider#4977
devin-ai-integration[bot] wants to merge 1 commit intomainfrom
devin/1773990865-azure-responses-api

Conversation

@devin-ai-integration
Copy link
Copy Markdown
Contributor

Summary

Closes #4974. Adds native Responses API support to AzureCompletion using a composition/delegation pattern: when api="responses" is specified, an internal OpenAICompletion instance is created and its clients are replaced with AzureOpenAI/AsyncAzureOpenAI from the openai SDK (which natively supports the Responses API on Azure). All Responses API calls are then delegated to this internal instance.

This avoids reimplementing all Responses API logic (tool format conversion, structured output, auto-chaining, reasoning items, streaming, etc.) and reuses the existing OpenAICompletion implementation.

Key changes to AzureCompletion:

  • New __init__ parameters: api, instructions, store, previous_response_id, include, builtin_tools, parse_tool_outputs, auto_chain, auto_chain_reasoning, seed, reasoning_effort, max_completion_tokens
  • _init_responses_delegate() creates OpenAICompletion and swaps in AzureOpenAI/AsyncAzureOpenAI clients
  • call()/acall() delegate to the internal instance when api="responses"
  • Pass-through properties: last_response_id, last_reasoning_items
  • Pass-through methods: reset_chain(), reset_reasoning_chain()
  • AZURE_RESPONSES_API_VERSION = "2025-03-01-preview" default
  • Interceptors now permitted in responses mode (handled by OpenAI SDK)

Review & Testing Checklist for Human

  • self.client / self.async_client not set in responses mode: When api="responses", the Azure AI Inference ChatCompletionsClient is never created. Verify that no code path (e.g., supports_function_calling(), aclose(), supports_stop_words(), event emission, or BaseLLM base class) accesses self.client on a responses-mode instance — this would raise AttributeError.
  • Client monkey-patching fragility: The delegate is constructed as a normal OpenAICompletion (creating default OpenAI/AsyncOpenAI clients), then its .client and .async_client are immediately replaced with AzureOpenAI instances. Confirm OpenAICompletion.__init__ doesn't perform any network calls or store client references elsewhere that would bypass this swap.
  • is_azure_openai_endpoint is False in responses mode: Since self.endpoint is the raw base URL (e.g., https://test.openai.azure.com without /openai/deployments/), the is_azure_openai_endpoint flag will be False. Verify this doesn't break supports_stop_words(), get_context_window_size(), or _prepare_completion_params() behavior for responses-mode instances.
  • Error handling gap: In call()/acall(), the delegation to the responses delegate happens before the try/except block and llm_call_context(). Errors from the delegate won't go through _handle_api_error or emit call_failed events. Consider whether this is acceptable.
  • E2E validation: All tests are unit tests with mocks. Test against a real Azure OpenAI endpoint with api="responses" to confirm the delegation actually produces correct Responses API calls over the wire.

Notes

  • The uv.lock file was not modified (it was temporarily regenerated locally for test runs but reverted before commit).
  • 30+ new tests added covering initialization, delegation, parameter forwarding, auto-chaining, reasoning items, interceptors, endpoint handling, and backward compatibility.
  • The self.endpointself.base_endpoint rename preserves backward compat by still setting self.endpoint — but the initialization order changed, which is worth a quick check.

Link to Devin session: https://app.devin.ai/sessions/01c25abb1a1849918399731940f82443

When api='responses' is specified for Azure, creates an internal
OpenAICompletion instance configured with AzureOpenAI/AsyncAzureOpenAI
clients from the openai Python SDK, which natively supports the
Responses API on Azure.

Key changes:
- Extended AzureCompletion.__init__() with Responses API parameters
  (api, instructions, store, previous_response_id, include,
  builtin_tools, parse_tool_outputs, auto_chain, auto_chain_reasoning,
  seed, reasoning_effort, max_completion_tokens)
- Added _init_responses_delegate() method using composition/delegation
- Added delegation logic in call() and acall() methods
- Added pass-through properties: last_response_id, last_reasoning_items
- Added pass-through methods: reset_chain(), reset_reasoning_chain()
- Preserved base endpoint before validation for Azure client config
- Interceptors now allowed in responses mode (via OpenAI SDK)
- Added AZURE_RESPONSES_API_VERSION constant (2025-03-01-preview)
- Added 30+ comprehensive tests covering all new functionality

Closes #4974

Co-Authored-By: João <joao@crewai.com>
@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

Prompt hidden (unlisted session)

@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

with patch.object(
llm._responses_delegate, "call", return_value='{"temperature": 72.0, "condition": "sunny"}'
) as mock_call:
result = llm.call(
@greysonlalonde
Copy link
Copy Markdown
Contributor

Duplicate of #5201. Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Add native responses API support for other providers than OpenAI

1 participant