feat(claude-agents)!: emit invoke_agent, chat and execute_tool spans by apucacao · Pull Request #31 · launchdarkly/python-ai-sdk

apucacao · 2026-08-11T20:13:13Z

Replaces one flat span per call with the tree the TypeScript SDK emits, for claude-agents.

invoke_agent                     one per call, carries the LD identity and the run total
├── chat {model}                 one per inference the Agent SDK reports
└── execute_tool {name}          one per tool call, a sibling of chat

Subagent identity was invisible

The Agent SDK reports a request id, a session id and a subagent type for every inference, and none of it reached a span. A trace could not say which subagent ran, or group turns by conversation.

gen_ai.response.id and gen_ai.agent.name now go on the chat span.
gen_ai.conversation.id goes on all three span types: the root learns it from the CLI's init message, the chat span from the inference, and the tool span from the hook input, which is where this side sees it without waiting for a message.

The streaming path had a real bug

The blocking path already held the vendor's query generator in a variable and awaited aclose() on it, because a bare return inside async for abandons it and asyncio's finalizer then raises RuntimeError when it is suspended inside a real await in the SDK. The streaming path iterated the generator inline with no held reference and no such cleanup, so it carried the same bug the blocking path was patched for. Both now close it in the same finally that ends the spans, and the streaming path gets the test the blocking path already had.

A run abandoned mid-stream now also closes any tool span whose PostToolUse hook never fired, which is otherwise the one span with no path to being ended.

Other changes

Cache tokens reach the span, folded into the input total.
The finish reason is written when the SDK reports one, which in practice is almost never: measured against Agent SDK 0.3.220, stop_reason is null on every assistant message and only the run-level result carries one. Deriving a reason from the presence of a tool-use block would put a value on the span the provider never returned, so the write stays guarded and usually absent.
gen_ai.response.model on the chat span is the model the turn actually used, not the requested name. This handler and openai-messages are the only two where those differ.

Breaking change

The span is renamed from claude.query to invoke_agent. Queries selecting on the old name will not match. Prompt and completion content is no longer on spans unless the caller passes capture_content=True.

Where this sits

Needs the usage layer (#28) and the content layer (#29). Independent of the other five handler PRs; the stack orders them only because gh stack is linear.

Tests: 729 to 763. This is the largest of the six handlers.

Note

Overview
Breaking: Replaces the flat claude.query span with an invoke_agent root plus chat {model} and sibling execute_tool {name} children, aligned with the TypeScript handler and TELEMETRY-CONTRACT. Prompt/completion on spans only when capture_content=True (default off).

Telemetry is split into new spans.py (InferenceSpans, ToolCatalog, root/chat/tool lifecycle). The handler derives one chat span per Anthropic response id from the Agent SDK message stream, opens execute_tool spans via Pre/Post tool hooks, and puts LD identity and run-level usage on the root only. Adds gen_ai.conversation.id, gen_ai.response.id, gen_ai.agent.name for subagents, cache tokens folded into input usage, and gen_ai.response.model on chat spans from the turn’s actual model.

Lifecycle fixes: Blocking and streaming paths explicitly aclose() the vendor query() generator; streaming abandonment/cancellation end open tool spans (UNSET + launchdarkly.stream.abandoned vs launchdarkly.run.cancelled); finally paths handle CancelledError and partial usage without writing all-zero totals when nothing was reported. tools=[] is no longer sent when there are no native tools (preserves Claude Code built-ins).

Tests move to TracerProvider + InMemorySpanExporter and scripted SDK message streams (~729→763 cases).

^{Reviewed by Cursor Bugbot for commit d4f1b6f. Bugbot is set up for automated code reviews on this repo. Configure here.}

apucacao · 2026-08-11T20:20:37Z

bugbot run

apucacao · 2026-08-11T20:44:51Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 41575b5. Configure here.}

apucacao · 2026-08-11T21:02:18Z

bugbot run

apucacao · 2026-08-11T21:19:30Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 078c7a5. Configure here.}

apucacao · 2026-08-11T21:46:41Z

bugbot run

apucacao · 2026-08-12T17:23:30Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

2 issues from previous reviews remain unresolved.

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 8e1dd90. Configure here.}

apucacao · 2026-08-12T17:46:34Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

2 issues from previous reviews remain unresolved.

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 9c7d549. Configure here.}

apucacao · 2026-08-12T18:03:34Z

bugbot run

apucacao · 2026-08-12T18:08:39Z

bugbot run

apucacao · 2026-08-12T18:20:58Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

4 issues from previous reviews remain unresolved.

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 05567c2. Configure here.}

apucacao · 2026-08-12T18:40:19Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

4 issues from previous reviews remain unresolved.

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit fa26c6f. Configure here.}

apucacao · 2026-08-12T18:51:56Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit cd679f1. Configure here.}

apucacao · 2026-08-14T19:02:03Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit fddb8b7. Configure here.}

apucacao · 2026-08-14T19:16:55Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 324733b. Configure here.}

apucacao · 2026-08-14T19:30:57Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 17ea219. Configure here.}

One flat span named claude.query becomes the tree the TypeScript SDK emits: an invoke_agent root, one `chat {model}` child per inference the Agent SDK reports, one `execute_tool {name}` child per tool call. BREAKING CHANGE: the span this handler emits is renamed from `claude.query` and `claude.query.stream` to `invoke_agent`. Queries selecting on the old names will not match. Prompt and completion content is no longer on spans unless the caller passes capture_content=True. Subagent identity was invisible. The Agent SDK reports a request id, a session id and a subagent type for every inference, and none of it reached a span, so a trace could not say which subagent ran or group turns by conversation. The chat span now carries gen_ai.response.id and gen_ai.agent.name, and gen_ai.conversation.id goes on all three span types: the root learns it from the CLI's init message, the chat span from the inference, and the tool span from the hook input, which is where this side sees it without waiting for a message. Cache tokens now reach the span, folded into the input total, because Anthropic reports cache reads and writes beside the input count rather than inside it. The finish reason is written when the SDK reports one, which in practice is almost never: measured against Agent SDK 0.3.220, stop_reason is null on every assistant message and only the run-level result carries one. Deriving a reason from the presence of a tool-use block would put a value on the span the provider never returned, so the write stays guarded and usually absent. The streaming path needed more than a finally. The blocking path already held the vendor's query generator in a variable and awaited aclose() on it, because a bare return inside `async for` abandons it and asyncio's finalizer then raises RuntimeError when it is suspended inside a real await in the SDK. The streaming path iterated the generator inline with no held reference and had no such cleanup, so it carried the same bug the blocking path was patched for. Both now close the generator in the same finally that ends the spans, and the streaming path gets the test the blocking path already had. A run abandoned mid-stream now closes any tool span whose PostToolUse hook never fired, which is otherwise the one span with no path to being ended. Tests: 56 to 90. The telemetry tests are rewritten rather than extended, because they pinned the old flat span, and every test that was not about telemetry is preserved under its original name.

_build_query_options passed tools=[] whenever a config had no native tools. An explicit empty list is not the same as omitting the key: it tells the Agent SDK there are no tools, which switches off the Claude Code built-ins. A run with only MCP tools, or none at all, silently lost Read, Bash and the rest. main omitted the key in that case, leaving the SDK default. Restored, so the condition is back where it was and only a non-empty list is ever passed. Two tests, one per branch. The empty case fails when the regression is put back. Found by Bugbot on #31 at High severity. This is a behaviour regression the span port introduced, not a telemetry change.

…spans The streaming teardown reached close_open_spans, which records an exception and sets ERROR on every tool span still open. That is right for a failure and wrong for abandonment. A consumer stopping early is normal, and the root and chat spans on that same path are deliberately left UNSET with launchdarkly.stream.abandoned, so a tool span whose PostToolUse hook never fired reported an error nobody had. Adds abandon_open_spans beside close_open_spans and uses it on the abandonment path only. The openai-agents and langchain-agents handlers already drew this distinction, so this also settles a three-way disagreement about what one abandoned run looks like across the SDK. Two tests: abandonment leaves UNSET with the marker, and a genuine failure still sets ERROR.

Both paths wrote the all-zero per-response sum when the stream ended without a ResultMessage and without absorbing a single assistant turn. Zeros on the root say the run cost nothing, which is a different claim from not knowing what it cost, and a config-scoped cost query cannot tell the two apart once they are written. Absent usage means unknown, which is the honest answer here. The error and abandonment paths in this same file already guarded on reported, which is what makes the success paths' omission look accidental rather than considered. It was. Three tests: neither path writes usage when nothing reported, and a run that did report one turn still reports it. Found by Bugbot on #31.

The prompt write ran before the try that fails the root, so a raise while serialising it left the root open: never ended, never exported, so the run disappeared from AI Config Monitoring along with the feature_flag event it carries. Both paths had it. Two tests, one per path. Found by Bugbot on #34, which is this shape in langchain-messages.

A timeout or a task.cancel() raises asyncio.CancelledError, which inherits from BaseException, so it walks past the except Exception this handler's blocking path relies on. That path ended its spans only from that clause, so a cancelled run exported nothing at all: not a wrong attribute, no span. The root carries the feature_flag event and every launchdarkly.* attribute, so the whole run vanished from AI Config Monitoring instead of showing as incomplete. A finally in _call_impl now owns the end the except clause cannot reach. It finishes the root with whatever usage the completed turns already billed, the same figure the failure path reports, then hands off to the shared end_unfinished_spans helper. The streaming path already worked this way; this brings the blocking path into line with it. Tool spans opened through the SDK's own hook dispatch needed the same treatment. They live inside a closure in build_tool_hooks, so it now returns a fourth function, cancel_open_spans, that ends whatever is still open there through the same shared helper, distinct from the existing abandon_open_spans because a cancelled run and an abandoned stream are marked differently. The root is tracked with a local cleared once a path ends it, not by asking the span whether it is still recording. A mock span answers that truthily, and this suite still builds some fixtures on mock spans, so asking would have made the finally end a successful run's span a second time. Two tests drive a real task.cancel() against a query() call that never returns, and assert every span opened was ended exactly once, left at UNSET, and marked launchdarkly.run.cancelled. Gutting the finally to a bare pass fails the first of them, which is the proof this fix rests on. Found by Bugbot on the langchain-messages layer, then found here by audit.

…not serialise A tool result comes from the caller's own function, so it can be any Python object, including one json.dumps refuses. Both content writes on the tool path could raise, and neither was positioned so the span survived it. The start hook wrote the arguments before filing the span in tool_spans. A raise there left a span nothing knew about: close_open_spans and the cancellation teardown both walk that dict, so a span missing from it can never be ended and never exports. Filing now happens first. The end hook popped the span, then wrote the result outside any guard. Once popped, ending it is that function's job alone, and a raise meant nobody did it. The write is now guarded, and the guard fails the span with `error or exc` so a tool that had already failed keeps the reason it failed. A serialisation problem is that span's second-worst fact, not its first. langchain-agents already had both of these right, and is what this now matches. Two tests, one per write, each driving a result and an argument that cannot be serialised. Reverting either change fails one. Found by Bugbot on the openai-agents layer, then found here by audit.

…oned A CancelledError never enters except Exception, so the streaming teardown always ran its abandonment path and marked launchdarkly.stream.abandoned on the root, the chat span and every open tool span. A consumer that calls aclose chose to stop, and that is abandonment. A CancelledError is not a choice: something cancelled the run, usually a timeout, and the consumer was still reading. The blocking path already reports launchdarkly.run.cancelled for that, so the two paths disagreed about the same event. abandon_open_spans now carries the distinction through to the tool spans, so a cancelled run's children agree with its root about why they stopped. The existing abandonment test uses aclose and still asserts stream.abandoned, which is that word's case, and needed no change. A new test cancels the draining task and asserts the other marker. No new attribute. Both keys already exist and are in the vocabulary lock. Setting the flag to False fails the new test. Found by Bugbot on the openai-agents layer, then found here by audit.

apucacao · 2026-08-14T19:40:36Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit d4f1b6f. Configure here.}

apucacao · 2026-08-14T19:57:25Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit d4f1b6f. Configure here.}

…y ran The request-model row claimed the requested name for all six handlers. claude-agents writes the model the inference actually used on its chat span, the same value it gives gen_ai.response.model, which the TypeScript source does deliberately: the CLI reports the model it really ran and there is no separate per-turn requested name to report. Stated as it was, the document made a correct handler look like a defect. A review of #31 raised it against the handler rather than against this file, which is the failure mode a specification has when it overgeneralises. Its root span still writes the requested name, so only the chat span differs.

cursor Bot reviewed Aug 11, 2026

View reviewed changes

Comment thread packages/claude-agents/src/launchdarkly_ai_claude_agents/spans.py

Comment thread packages/claude-agents/src/launchdarkly_ai_claude_agents/spans.py

apucacao force-pushed the ag/py-telemetry-claude-agents branch from c73f2a4 to 41575b5 Compare August 11, 2026 20:43

cursor Bot reviewed Aug 11, 2026

View reviewed changes

apucacao force-pushed the ag/py-telemetry-claude-agents branch from 41575b5 to 8c8604f Compare August 11, 2026 21:01

cursor Bot reviewed Aug 11, 2026

View reviewed changes

Comment thread packages/claude-agents/src/launchdarkly_ai_claude_agents/handler.py

cursor Bot reviewed Aug 11, 2026

View reviewed changes

Comment thread packages/claude-agents/src/launchdarkly_ai_claude_agents/handler.py

Comment thread packages/claude-agents/src/launchdarkly_ai_claude_agents/handler.py

apucacao force-pushed the ag/py-telemetry-claude-agents branch from 078c7a5 to 8e1dd90 Compare August 12, 2026 17:23

cursor Bot reviewed Aug 12, 2026

View reviewed changes

apucacao force-pushed the ag/py-telemetry-claude-agents branch from 8e1dd90 to 9c7d549 Compare August 12, 2026 17:45

cursor Bot reviewed Aug 12, 2026

View reviewed changes

apucacao force-pushed the ag/py-telemetry-claude-agents branch from 9c7d549 to e58ffc4 Compare August 12, 2026 18:07

cursor Bot reviewed Aug 12, 2026

View reviewed changes

Comment thread packages/claude-agents/src/launchdarkly_ai_claude_agents/spans.py

Comment thread packages/claude-agents/src/launchdarkly_ai_claude_agents/handler.py

cursor Bot reviewed Aug 12, 2026

View reviewed changes

Comment thread packages/claude-agents/src/launchdarkly_ai_claude_agents/handler.py

cursor Bot reviewed Aug 12, 2026

View reviewed changes

apucacao force-pushed the ag/py-telemetry-claude-agents branch from 05567c2 to fa26c6f Compare August 12, 2026 18:39

cursor Bot reviewed Aug 12, 2026

View reviewed changes

cursor Bot reviewed Aug 14, 2026

View reviewed changes

apucacao force-pushed the ag/py-telemetry-claude-agents branch from cd679f1 to fddb8b7 Compare August 14, 2026 19:01

cursor Bot reviewed Aug 14, 2026

View reviewed changes

apucacao force-pushed the ag/py-telemetry-claude-agents branch from fddb8b7 to 324733b Compare August 14, 2026 19:16

cursor Bot reviewed Aug 14, 2026

View reviewed changes

apucacao force-pushed the ag/py-telemetry-claude-agents branch from 324733b to 17ea219 Compare August 14, 2026 19:30

cursor Bot reviewed Aug 14, 2026

View reviewed changes

apucacao added 8 commits August 14, 2026 15:38

apucacao force-pushed the ag/py-telemetry-claude-agents branch from 17ea219 to d4f1b6f Compare August 14, 2026 19:39

cursor Bot reviewed Aug 14, 2026

View reviewed changes

andrewklatzke approved these changes Aug 14, 2026

View reviewed changes

apucacao marked this pull request as ready for review August 14, 2026 21:18

apucacao merged commit 6a82ef5 into main Aug 14, 2026
12 checks passed

apucacao deleted the ag/py-telemetry-claude-agents branch August 14, 2026 21:20

Uh oh!

Conversation

apucacao commented Aug 11, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Subagent identity was invisible

The streaming path had a real bug

Other changes

Breaking change

Where this sits

Uh oh!

apucacao commented Aug 11, 2026

Uh oh!

Uh oh!

Uh oh!

apucacao commented Aug 11, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

apucacao commented Aug 11, 2026

Uh oh!

Uh oh!

apucacao commented Aug 11, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

apucacao commented Aug 11, 2026

Uh oh!

Uh oh!

Uh oh!

apucacao commented Aug 12, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

apucacao commented Aug 12, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

apucacao commented Aug 12, 2026

Uh oh!

apucacao commented Aug 12, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

apucacao commented Aug 12, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

apucacao commented Aug 12, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

apucacao commented Aug 12, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

apucacao commented Aug 14, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

apucacao commented Aug 14, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

apucacao commented Aug 14, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

apucacao commented Aug 14, 2026

Uh oh!

apucacao commented Aug 11, 2026 •

edited by cursor Bot

Loading