跳转至

Chapter 1 experiment requirement/evidence ledger

The Chinese manuscript is authoritative. Runtime completion and manuscript behavior claims are kept separate: a real, correctly controlled ablation may finish even when one claimed degradation is not observed.

Experiment Exact manuscript gate Status Canonical evidence / qualification
1-1 One complete baseline plus removal of tool definitions, tool results, assistant reasoning, and prior history on the same multi-currency ReAct task Exact five-arm real run completed; one prose behavior not reproduced context/validation/latest.json retains direct Kimi K3 requests/responses, all tool observations, expected totals, and context contracts. Baseline was correct; no tools removed action; hidden tool results and history caused repeated calls. Removing reasoning still completed correctly, so the manuscript’s “contradictory decisions” claim is explicitly false for this run rather than fabricated.
1-2 Exact Kimi K3 with provider-hosted Formula web search, model-directed multiple search rounds, reasoning, current answer, and authoritative links Passed web-search-agent/validation/latest.json: direct Moonshot endpoint, exact model, 15 succeeded distinct Formula fibers over multiple rounds, official ASEAN/Indonesian sources, real response IDs and usage.
1-3 Multi-provider policy (author-mandated 2026-07-31): model-directed multi-round hosted web search + hosted code execution closed server-side on a Responses API, clarification-before-tools, and current answers with authoritative sources; the official OpenAI GPT-5.6 Sol path stays canonical, but acceptance is not gated on it — any provider with genuinely equivalent hosted tools is eligible Passed on DashScope qwen3.7-plus; official OpenAI still quota-blocked search-codegen/validation/latest.json (+ validation/runs/real_20260731T170529Z/{evidence,receipts,manifest}.json, SHA-256 sidecars, credential-free receipts). ASEAN: one hosted web_search_call batching 10 model-issued coordinate queries, then a hosted code_interpreter_call enumerating all 45 haversine pairs — Kuala Lumpur–Singapore 316.35 km, the same pair as the independent local reference (309.3 km with standard coordinates). Bitcoin: first turn clarified data source/indicators with no tools; continuation via previous_response_id ran 3 model-directed search rounds and 4 code_interpreter_calls computing MA7/MA20/RSI14/MACD, period return and max drawdown, and plotted a chart in the sandbox. The OpenAI gpt-5.6-sol attempt is recorded honestly as credit_balance_exhausted before inference. Qualifications: the DashScope sandbox has no outbound network, so daily closes were search-extracted (the model disclosed this in its report); the chart PNG stays inside the sandbox because the API returns logs only; qwen3.7-plus needed an explicit clarify-first system prompt to ask before using tools.

Legacy demos and provider-compatible substitutes are teaching aids only. A provider rejection before inference is not converted into a model failure and does not authorize accepting a narrower proxy.