DHH on AI reviewing code better than writing it
"You'll soon come to the realization too that AI is even better at reviewing code than it is at writing it."
— DHH
Tags
16 posts tagged
"You'll soon come to the realization too that AI is even better at reviewing code than it is at writing it."
— DHH
After each reply, Anthropic adds one hidden turn and Claude returns a JSON safety flag in ~10 tokens, under 2% of compute. The KV cache makes it cheap.
Anthropic's Safeguards team published how Claude monitors its own conversations for misuse. After each reply, the system appends a short hidden instruction aski...
via XA Harvard study of 718 firms: coding agents raised lines of code 30% and PRs 23%, but shipped features didn't rise significantly as review time grew 49%.
Artificial Intelligence in the Firm: Bottlenecks in Software Production by Fiona Chen and James Stratton, Harvard University (current version August 4, 2026). C...
via Ars TechnicaSurge AI's DAYJOB: Tax tests agents on 50 messy tax assignments. Sonnet 5.5 leads at 10.8%. Tasks reward spotting issues the client never raised.
DAYJOB: Tax is Surge AI's new benchmark for AI agents doing professional tax work, published October 27, 2026. It has 50 assignments across eight areas of tax p...
via Hacker NewsAnthropic gave 50 analysts' Claude Code agents a hidden coin-flip lean. Analysts matched it 86% of the time and 15% credited the agent.
How AI agents shape experts' judgment calls is a new study from Anthropic's Alignment team (October 27, 2026). They gave 50 professional analysts a realistic ca...
OpenAI released an AI-written disproof of the Exponential Time Hypothesis with a Lean proof, shepherded by the researchers who proposed ETH.
OpenAI's Working with academic communities to build scientific understanding (October 26, 2026) announces three results its internal models produced: a proof of...
Ubicloud took Kimi K2.6 tool-call schema accuracy in vLLM from 92% to 100% with constrained decoding. What it does and why vLLM leaves it off by default.
Fixing Tool-Call Accuracy in vLLM (Jesse Donkervliet and Junhao Li, Ubicloud, October 20, 2026) Ubicloud serves open-weight models, and this post explains why a...
via Hacker NewsA 2015 Inception v3 CNN beat GPT-6 Astra and Claude Fable 5.1 at diagnosing 200 skin lesions, 62% to 42%. What it says about small models.
A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis (Dr Magnus Lynch, October 25, 2026) Magnus Lynch, a consultant dermatologist at Guy's Hosp...
via Hacker NewsHow to evaluate AI agents: build tasks from real failures, grade outcomes, run several trials and report pass@k and pass^k. With a runnable Python demo.
The evaluator-optimizer pattern pairs a generator LLM with an evaluator that sends feedback in a loop until the output passes. When it works, with Python.
How to choose an embedding model: read MTEB the right way, weigh dimensions, context length and cost, then test on your own queries with a Python harness.
SWE-bench tests whether an AI can fix real GitHub issues, graded by the repo's own tests. How grading works, why Verified was retired, and a runnable demo.
LLM benchmarks are fixed test sets that score models on knowledge, reasoning, coding or agents. What the main ones measure, why they saturate, how to read them.
LLM-as-a-judge uses a strong model to grade another model's output against a rubric. How it works, its known biases and how to calibrate it, with a Python demo.
How to evaluate RAG: measure retrieval with recall@k, MRR and nDCG, and answers with faithfulness and relevance. The metrics explained, with runnable Python.
AI red teaming means attacking your own model or LLM app to find harms and security holes before users do. Methods, tools and a runnable Python harness.