Skip to content

MAINT: Consolidate OpenAI response handling - #2381

Open
Roman Lutz (romanlutz) wants to merge 2 commits into
microsoft:mainfrom
romanlutz:romanlutz-consolidate-openai-responses
Open

MAINT: Consolidate OpenAI response handling#2381
Roman Lutz (romanlutz) wants to merge 2 commits into
microsoft:mainfrom
romanlutz:romanlutz-consolidate-openai-responses

Conversation

@romanlutz

Copy link
Copy Markdown
Contributor

Description

OpenAI Chat Completions, Completions, and Responses targets previously implemented response validation, filtering, truncation detection, partial-content extraction, and metadata capture across multiple target classes. This made behavior harder to compare and maintain as the API formats evolved.

This change introduces a small typed response-adapter seam with dedicated adapters for each wire format and a no-op default for other OpenAI targets. Existing protected target hooks remain compatibility delegates, including subclass truncation overrides. The consolidation preserves each format's existing behavior, including legacy Completions semantics, content-filter handling, partial output, stop reasons, token usage, and graceful truncated responses.

Tests and Documentation

  • Added adapter contract and parity tests for malformed, filtered, truncated, and metadata-bearing responses across all three formats.
  • Added regression coverage ensuring protected truncation overrides remain honored.
  • Ran 336 relevant OpenAI target and parser tests.
  • Ran Ruff check and format, ty, all changed-file pre-commit hooks, and git diff --check.
  • Documentation changes were not applicable because public APIs and behavior are unchanged.
  • JupyText was not applicable.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@hannahwestra25 hannahwestra25 self-assigned this Aug 13, 2026
Comment thread pyrit/prompt_target/openai/_response_adapter.py Outdated
bool: True if content filter detected, False otherwise.
"""
return False
return self._response_adapter.is_content_filter(response=response)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we just eliminate these functions in favor of creating the adapter objects ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kept these protected hooks as compatibility delegates. They predate this adapter and subclasses can override them; calling the adapter directly from _handle_openai_request_async would bypass those extension points. In particular, _validate_response deliberately computes truncation through the overridable _is_truncated_response hook before delegating to the adapter. The format-specific logic still lives in the adapters.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but should we update the docstring to point people to the adapter ?

Comment thread pyrit/prompt_target/openai/_response_adapter.py Outdated
@hannahwestra25

Copy link
Copy Markdown
Contributor

should we also include OpenAI video ?

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 7b3a8d42-bfe2-4b5e-90ae-fc211028c2ad
@romanlutz

Copy link
Copy Markdown
Contributor Author

On including OpenAI video: I kept it out of this consolidation because video uses a distinct asynchronous job/status response model, while this PR is specifically aligning Chat Completions, Completions, and Responses text formats. Video currently only shares the base request wrapper and has its own content-filter/status construction semantics; forcing it into this adapter set would broaden the PR without meaningful shared validation, truncation, partial-content, or metadata behavior. The concrete base adapter still lets video retain its existing protected hook behavior, and the video target tests pass unchanged.

@hannahwestra25 hannahwestra25 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small nit related to docstring :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants