GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — test(harness): propagate AGX1-377/378 fix into conformance — un-suppr… · scaleapi/scale-agentex-python@df3461c

File tree

Expand file treeCollapse file tree

Open diff view settings

Filter options

tests/lib/core/harness/conformance
- runner.py
- test_conformance.py

Expand file treeCollapse file tree

Open diff view settings

Collapse file

`‎tests/lib/core/harness/conformance/runner.py‎`

Copy file name to clipboardExpand all lines: tests/lib/core/harness/conformance/runner.py

+72-26Lines changed: 72 additions & 26 deletions

Original file line number	Diff line number	Diff line change
`@@ -13,11 +13,12 @@`
`13`	`13`	`conformance test compares those normalised sequences.`
`14`	`14`
`15`	`15`	`payload` carries the content that callers actually consume:
`16`		`- - text/reasoning: the accumulated concatenated delta string`
`17`		`- - tool_request: the arguments dict (JSON-sorted)`
	`16`	`+ - text: initial_content.content prepended, then accumulated delta string`
	`17`	`+ - reasoning: initial_content.summary joined, then accumulated delta string`
	`18`	`+ - tool_request: the arguments dict (JSON-sorted), from Start content`
`18`	`19`	`- tool_response: the content value (str)`
`19`		`- This catches a channel that delivers the right structural shape but corrupts`
`20`		`- or drops the payload.`
	`20`	`+ This catches a channel that delivers the right structural shape but corrupts,`
	`21`	`+ drops, or omits initial_content (including reasoning summary) or payload.`
`21`	`22`
`22`	`23`	`2. Span signal equivalence: each channel is driven with its own recording`
`23`	`24`	`tracer that captures every SpanSignal it actually receives in handle(); the`
`@@ -51,6 +52,13 @@`
`51`	`52`	`Start(content)+Done from auto_send are equivalent. The recorded span signals are`
`52`	`53`	`identical because both adapters drive the same SpanDeriver.observe() call`
`53`	`54`	`sequence and forward every signal to their tracer.`
	`55`	`+`
	`56`	`+AGX1-377 fix: auto_send now DELIVERS streamed tool-request messages (Start+Done)`
	`57`	`+instead of dropping them. The conformance normaliser previously suppressed the`
	`58`	`+delivery for Start(tool_request)+Done on the yield channel to match auto_send's`
	`59`	`+old drop behaviour. That suppression is now removed: both channels produce a`
	`60`	`+LogicalDelivery for a streamed tool_request, and the cross-channel assertion`
	`61`	`+verifies it is delivered on both.`
`54`	`62`	`"""`
`55`	`63`
`56`	`64`	`from __future__ import annotations`
`@@ -113,9 +121,11 @@ class LogicalDelivery(NamedTuple):`
`113`	`121`	`pairs that uniquely identify the content (e.g. tool_call_id for tool`
`114`	`122`	messages, or index for text/reasoning). `payload` is a stable string
`115`	`123`	`representation of the content callers actually consume:`
`116`		`- - text/reasoning: accumulated delta string`
`117`		`- - tool_request: JSON-sorted arguments`
`118`		`- - tool_response: str(content)`
	`124`	`+ - text: initial_content.content prepended to accumulated delta strings`
	`125`	`+ - reasoning: initial_content.summary joined, prepended to accumulated`
	`126`	`+ reasoning-content delta strings`
	`127`	`+ - tool_request: JSON-sorted arguments from Start content`
	`128`	`+ - tool_response: str(content) from Full event`
`119`	`129`	`"""`
`120`	`130`
`121`	`131`	`content_type: str`
`@@ -128,24 +138,39 @@ def _yield_logical_deliveries(events: list[StreamTaskMessage]) -> list[LogicalDe`
`128`	`138`
`129`	`139`	`The yield channel forwards events verbatim. A logical delivery is:`
`130`	`140`	`- A Full event (tool_request / tool_response): content delivered as-is.`
`131`		`- - A Start + ... + Done sequence for text/reasoning content.`
`132`		`-`
`133`		- The `payload` field captures the content callers consume: accumulated delta
`134`		`- text for text/reasoning, arguments for tool_request, and response body for`
`135`		`- tool_response. This ensures a channel that delivers the right structure but`
`136`		`- corrupts the payload is caught.`
	`141`	`+ - A Start + ... + Done sequence for text/reasoning/tool_request content.`
	`142`	`+`
	`143`	+ The `payload` field captures the content callers consume:
	`144`	`+ - text: initial_content.content (from Start) prepended to accumulated deltas`
	`145`	`+ - reasoning: initial_content.summary joined (from Start) prepended to`
	`146`	`+ accumulated reasoning-content deltas (this catches a channel that drops`
	`147`	`+ the summary)`
	`148`	`+ - tool_request: JSON-sorted arguments from the Start content (AGX1-377: now`
	`149`	`+ delivered on both channels, no longer suppressed)`
	`150`	`+ - tool_response: str(content) from Full event`
`137`	`151`	`"""`
	`152`	`+ from agentex.types.text_content import TextContent`
	`153`	`+ from agentex.types.reasoning_content import ReasoningContent`
	`154`	`+ from agentex.types.tool_request_content import ToolRequestContent`
	`155`	`+`
`138`	`156`	`deliveries: list[LogicalDelivery] = []`
`139`	`157`	`# Track which indices had a Start so we can pair with Done`
`140`	`158`	`started: dict[int, Any] = {} # index -> initial content`
`141`		`- # Accumulate delta text per index`
	`159`	`+ # Accumulate delta text per index (seed with initial_content text if present)`
`142`	`160`	`accumulated: dict[int, list[str]] = {} # index -> list of delta strings`
`143`	`161`
`144`	`162`	`for event in events:`
`145`	`163`	`if isinstance(event, StreamTaskMessageStart):`
`146`	`164`	`if event.index is not None:`
`147`	`165`	`started[event.index] = event.content`
`148`		`- accumulated[event.index] = []`
	`166`	`+ # Seed accumulator with initial_content so a channel that drops`
	`167`	`+ # initial_content but delivers deltas correctly will fail.`
	`168`	`+ seed: list[str] = []`
	`169`	`+ if isinstance(event.content, TextContent) and event.content.content:`
	`170`	`+ seed = [event.content.content]`
	`171`	`+ elif isinstance(event.content, ReasoningContent) and event.content.summary:`
	`172`	`+ seed = list(event.content.summary)`
	`173`	`+ accumulated[event.index] = seed`
`149`	`174`	`elif isinstance(event, StreamTaskMessageDelta):`
`150`	`175`	`if event.index is not None and event.delta is not None:`
`151`	`176`	`if isinstance(event.delta, TextDelta) and event.delta.text_delta:`
`@@ -165,11 +190,22 @@ def _yield_logical_deliveries(events: list[StreamTaskMessage]) -> list[LogicalDe`
`165`	`190`	`payload="".join(deltas),`
`166`	`191`	`)`
`167`	`192`	`)`
`168`		`- # tool_request Start+Done just means the span opens; the message`
`169`		`- # itself is delivered via Full (ToolRequestContent Full), so we`
`170`		`- # don't emit a delivery here for Start(tool_request)+Done.`
`171`		`- # AGX1-377: once auto_send handles the streamed tool-request shape,`
`172`		`- # this suppression will be removed and the delivery counted here.`
	`193`	`+ elif ctype == "tool_request" and isinstance(content, ToolRequestContent):`
	`194`	`+ # AGX1-377 fix: auto_send now delivers streamed tool-request`
	`195`	`+ # messages. Emit a delivery here so the cross-channel`
	`196`	`+ # assertion verifies it is present on both channels.`
	`197`	`+ deliveries.append(`
	`198`	`+ LogicalDelivery(`
	`199`	`+ content_type=ctype,`
	`200`	`+ identity=frozenset(`
	`201`	`+ {`
	`202`	`+ ("tool_call_id", content.tool_call_id),`
	`203`	`+ ("name", content.name),`
	`204`	`+ }`
	`205`	`+ ),`
	`206`	`+ payload=json.dumps(content.arguments, sort_keys=True),`
	`207`	`+ )`
	`208`	`+ )`
`173`	`209`	`elif isinstance(event, StreamTaskMessageFull):`
`174`	`210`	`content = event.content`
`175`	`211`	`ctype = getattr(content, "type", None) or ""`
`@@ -323,9 +359,17 @@ def _auto_send_logical_deliveries(sink: list[Any]) -> list[LogicalDelivery]:`
`323`	`359`	`("close", ctype) -- context closed`
`324`	`360`
`325`	`361`	`A logical delivery corresponds to each open+close pair. For text/reasoning`
`326`		`- we identify by sequential position and accumulate the delta payload; for`
`327`		`- tool messages we use tool_call_id + name and capture arguments/content.`
	`362`	`+ we identify by sequential position and build the payload by prepending the`
	`363`	`+ initial_content text (TextContent.content) or summary (ReasoningContent.summary)`
	`364`	`+ to accumulated deltas. This matches _yield_logical_deliveries so a channel`
	`365`	`+ that drops initial_content or reasoning summary fails the comparison.`
	`366`	`+ For tool messages we use tool_call_id + name and capture arguments/content.`
`328`	`367`	`"""`
	`368`	`+ from agentex.types.text_content import TextContent`
	`369`	`+ from agentex.types.reasoning_content import ReasoningContent`
	`370`	`+ from agentex.types.tool_request_content import ToolRequestContent`
	`371`	`+ from agentex.types.tool_response_content import ToolResponseContent`
	`372`	`+`
`329`	`373`	`deliveries: list[LogicalDelivery] = []`
`330`	`374`	`open_idx = 0`
`331`	`375`	`while open_idx < len(sink):`
`@@ -335,6 +379,12 @@ def _auto_send_logical_deliveries(sink: list[Any]) -> list[LogicalDelivery]:`
`335`	`379`	`content: Any = entry[2]`
`336`	`380`	`found_open = False`
`337`	`381`	`delta_parts: list[str] = []`
	`382`	`+ # Seed delta_parts with initial_content so payload comparison`
	`383`	`+ # catches a channel that drops initial_content but delivers deltas.`
	`384`	`+ if isinstance(content, TextContent) and content.content:`
	`385`	`+ delta_parts = [content.content]`
	`386`	`+ elif isinstance(content, ReasoningContent) and content.summary:`
	`387`	`+ delta_parts = list(content.summary)`
`338`	`388`	`for j in range(open_idx + 1, len(sink)):`
`339`	`389`	`if sink[j][0] == "open" and sink[j][1] == ctype and not found_open:`
`340`	`390`	`found_open = True`
`@@ -358,8 +408,6 @@ def _auto_send_logical_deliveries(sink: list[Any]) -> list[LogicalDelivery]:`
`358`	`408`	`)`
`359`	`409`	`)`
`360`	`410`	`elif ctype == "tool_response":`
`361`		`- from agentex.types.tool_response_content import ToolResponseContent`
`362`		`-`
`363`	`411`	`if isinstance(content, ToolResponseContent):`
`364`	`412`	`deliveries.append(`
`365`	`413`	`LogicalDelivery(`
`@@ -374,8 +422,6 @@ def _auto_send_logical_deliveries(sink: list[Any]) -> list[LogicalDelivery]:`
`374`	`422`	`)`
`375`	`423`	`)`
`376`	`424`	`elif ctype == "tool_request":`
`377`		`- from agentex.types.tool_request_content import ToolRequestContent`
`378`		`-`
`379`	`425`	`if isinstance(content, ToolRequestContent):`
`380`	`426`	`deliveries.append(`
`381`	`427`	`LogicalDelivery(`

Collapse file

`‎tests/lib/core/harness/conformance/test_conformance.py‎`

Copy file name to clipboardExpand all lines: tests/lib/core/harness/conformance/test_conformance.py

+49-6Lines changed: 49 additions & 6 deletions

Original file line number	Diff line number	Diff line change
`@@ -11,8 +11,9 @@`
`11`	`11`	`- yield channel delivers StreamTaskMessageFull(ToolResponseContent) as-is.`
`12`	`12`	`- auto_send delivers the same tool-response by opening a streaming context`
`13`	`13`	`with the full content and closing it immediately.`
`14`		`- Both collapse to LogicalDelivery(content_type, identity) tuples that compare`
`15`		`- equal.`
	`14`	`+ Both collapse to LogicalDelivery(content_type, identity, payload) tuples`
	`15`	`+ that compare equal. The payload includes initial_content (TextContent.content`
	`16`	`+ and ReasoningContent.summary) so a channel that drops initial content fails.`
`16`	`17`
`17`	`18`	`2. Span signal equivalence: both channels feed the same pure SpanDeriver`
`18`	`19`	`over the same event sequence, so the derived span signals must be identical.`
`@@ -22,6 +23,12 @@`
`22`	`23`	`Raw wire-level event shapes are NOT compared (that would fail by design: the`
`23`	`24`	`Full vs Start+Done envelope difference is a documented, acceptable choice in`
`24`	`25`	`auto_send — see runner.py for the rationale).`
	`26`	`+`
	`27`	`+AGX1-377 fix: auto_send now delivers streamed tool-request messages. The`
	`28`	`+suppression that previously prevented the yield normaliser from emitting a`
	`29`	`+LogicalDelivery for Start(tool_request)+Done is removed. Both channels now`
	`30`	`+produce a delivery for streamed tool_request, verified by the`
	`31`	`+"streamed-tool-request" fixture.`
`25`	`32`	`"""`
`26`	`33`
`27`	`34`	`from __future__ import annotations`
`@@ -54,7 +61,8 @@`
`54`	`61`	`# ---------------------------------------------------------------------------`
`55`	`62`
`56`	`63`	`_FIXTURES: list[Fixture] = [`
`57`		`- # fixture 1: single tool call (the canonical builtin example)`
	`64`	`+ # fixture 1: single tool call — tool_request delivered via Full (classic path)`
	`65`	`+ # plus a streamed tool_response via Full. Both channels should deliver both.`
`58`	`66`	`Fixture(`
`59`	`67`	`name="builtin-single-tool",`
`60`	`68`	`events=[`
`@@ -75,14 +83,16 @@`
`75`	`83`	`),`
`76`	`84`	`],`
`77`	`85`	`),`
`78`		`- # fixture 2: streaming text — exercises the text start/delta/done path`
	`86`	`+ # fixture 2: streaming text — exercises the text start/delta/done path.`
	`87`	`+ # Uses non-empty initial_content so the payload comparison catches a channel`
	`88`	`+ # that drops StreamTaskMessageStart.content (Greptile id 3438655533, P1).`
`79`	`89`	`Fixture(`
`80`	`90`	`name="streaming-text",`
`81`	`91`	`events=[`
`82`	`92`	`StreamTaskMessageStart(`
`83`	`93`	`type="start",`
`84`	`94`	`index=0,`
`85`		`- content=TextContent(type="text", author="agent", content=""),`
	`95`	`+ content=TextContent(type="text", author="agent", content="Init"),`
`86`	`96`	`),`
`87`	`97`	`StreamTaskMessageDelta(`
`88`	`98`	`type="delta",`
`@@ -97,7 +107,9 @@`
`97`	`107`	`StreamTaskMessageDone(type="done", index=0),`
`98`	`108`	`],`
`99`	`109`	`),`
`100`		`- # fixture 3: reasoning block — exercises reasoning span open/close + delivery`
	`110`	`+ # fixture 3: reasoning block — exercises reasoning span open/close + delivery.`
	`111`	`+ # ReasoningContent.summary is included in the payload so a channel that drops`
	`112`	`+ # the reasoning-summary fails (Greptile id 3438655533, P1).`
`101`	`113`	`Fixture(`
`102`	`114`	`name="reasoning-block",`
`103`	`115`	`events=[`
`@@ -122,6 +134,37 @@`
`122`	`134`	`StreamTaskMessageDone(type="done", index=0),`
`123`	`135`	`],`
`124`	`136`	`),`
	`137`	`+ # fixture 4: streamed tool_request (AGX1-377 fix) — tool_request delivered`
	`138`	`+ # via Start+Done (no Full). auto_send now delivers this instead of dropping`
	`139`	`+ # it. Both channels must produce a LogicalDelivery for this fixture.`
	`140`	`+ Fixture(`
	`141`	`+ name="streamed-tool-request",`
	`142`	`+ events=[`
	`143`	`+ StreamTaskMessageStart(`
	`144`	`+ type="start",`
	`145`	`+ index=0,`
	`146`	`+ content=ToolRequestContent(`
	`147`	`+ type="tool_request",`
	`148`	`+ author="agent",`
	`149`	`+ tool_call_id="tr-1",`
	`150`	`+ name="Read",`
	`151`	`+ arguments={"path": "/tmp/foo"},`
	`152`	`+ ),`
	`153`	`+ ),`
	`154`	`+ StreamTaskMessageDone(type="done", index=0),`
	`155`	`+ StreamTaskMessageFull(`
	`156`	`+ type="full",`
	`157`	`+ index=1,`
	`158`	`+ content=ToolResponseContent(`
	`159`	`+ type="tool_response",`
	`160`	`+ author="agent",`
	`161`	`+ tool_call_id="tr-1",`
	`162`	`+ name="Read",`
	`163`	`+ content="file contents",`
	`164`	`+ ),`
	`165`	`+ ),`
	`166`	`+ ],`
	`167`	`+ ),`
`125`	`168`	`]`
`126`	`169`
`127`	`170`	`# Register all fixtures for backward-compatible use via all_fixtures()`

0 commit comments

Comments

0 (0)

You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.