Items in this briefing come from Simon Willison and arXiv.org.
The Signal
Up-front scores for agent tools and model prices may miss what matters when tools fail or when cost is measured per task. The authors of a tool-picking study report, from simulated outages, that high offline ranking accuracy on real queries does not guarantee robust tool selection when tools fail. Simon Willison says per-token prices are not really comparable, because what matters is how many tokens a model needs to reach a goal when working as an agent.
Via: Simon Willison (blog post) · Published: 2026-10-13
Why this matters: Readers comparing model costs can note that Argon's launch price is set to double on December 1st, and that Willison calls total benchmark running cost one useful clue to real cost, since list prices are not directly comparable.
Summary: This is a release note from Simon Willison for version 0.35 of llm-gemini, which adds support for the new Gemini 4 Argon model. Argon is available through the Gemini API and Google's $100/month subscription plan. Willison says the launch price is $2 per million input and $10 per million output, and that it is set to double on December 1st. He calls Artificial Analysis benchmark running costs one useful clue for real cost, and says Argon looks very good there, close to GLM-5.3 and significantly cheaper than the latest models from OpenAI, Anthropic and Meta. After the doubling he says its benchmark running cost will still be a little cheaper than GPT-6 Astra's.
Via: Simon Willison (blog post) · Published: 2026-10-13
Why this matters: Teams whose code change volume is climbing, especially if AI tools are driving it, can compare their own repository size and checkout load against the figures Anthropic gives for why Git stopped fitting its needs.
Summary: This is a blog post by Simon Willison about an Anthropic article on why the company built its own replacement for Git. Anthropic reports that merged code changes rose from about 2,500 per day in May 2026 to over 30,000 per day by September. The article says its main branch history is almost 100 gigabytes, and that it runs on the order of 25 million code checkouts per day for continuous integration, not counting developer environments, agent sandboxes and research jobs. Seven people built the new system, called Scion, and the move from Git took five months. Willison guesses that newer models are a key factor in the faster pace of changes and says Scion sounds like a reinvention of centralized source control along the lines of Subversion. The article also notes that adversarial review can become myopic, so engineers had to steer Claude away from over-engineering.
Why this matters: Teams building agents that call many similar tools could use the released code and EcoAgent data to check how their own tool picking copes with failing tools, though the authors have not yet tested live agent runtimes.
Summary: This arXiv paper presents a system that helps artificial intelligence agents pick tools by combining text matching with a small learning component that re-ranks candidates from live success or failure results. The authors also built EcoAgent, a benchmark of 5,172 real tools and over 210,000 query-tool pairs. They report a 52.2% relative gain in ranking quality over the ToolLLM application programming interface retriever on real queries, though it still trails three other methods on that measure, Ada-002, BGE-Reranker and SPLADE-v3. In a simulated outage with 30% of tools failing, they report the rate of picking a failed tool was 2.9%, against 86.2% for the ToolLLM retriever, with under 1 millisecond added delay. Tests used injected failures only, and checking new tools and live agent runtimes is left as future work.
Why this matters: Red-team groups testing personal agents across many services could compare this graph-based approach with prompting an LLM to propose tool chains, since the authors report it reaches more tools and more harmful paths in their simulation.
Summary: This arXiv preprint from Meta researchers presents X-AgentRed, a red-teaming framework that looks for harmful outcomes when a personal agent fulfills a benign request across several connected services. It builds a graph of how one tool's effects can satisfy another tool's requirements, samples paths from it, and turns them into test scenarios. Feedback from each run is used to swap in neighboring tools or to fix blocked steps. In a simulated setup with 30 services and 914 tools, across four agentic models, the authors report 1.77 times the harmful path rate of an earlier method called STAC and 2.99 times its tool coverage. Each setup was run once, and the authors say their diversity measures show structure, not distinct causes of failure.
Sources:
Simon Willison, arXiv.org
Daily Signals currently covers one topic: Artificial Intelligence. More topics may follow.