Case Study15 of 105 recoverable AI code-generation events (14.3%) introduced a genuine error that was corrected before any commit, caught by the AI's own test suite; roughly 1 in 4-5 of the AI's interactive responses to the user contained at least one factual error (23.0% of responses scored by Claude, 21.6% by Codex, rising to 29.5% when design-fix proposals are included).
Between the Commits: Process, Error, and Claim Reliability in a Wholly AI-Authored CodebaseWhat it means Treat the agent's prose claims as unverified: the error rate in what it TELLS you is higher than the error rate in what it commits. Fund the test suite as the primary defect gate, and stop reading commit-history quality metrics as agent accuracy -- they are structurally blind to the errors the agent introduced and fixed before committing.
arXiv preprint · SEP 24
Case StudyOne experienced developer using a CLI coding agent generated hundreds of commits touching thousands of lines in 15 days and saturated both build-on-commit CI and reviewer attention; switching from naive per-file commits to directory-based batching with a cap on files per change restored throughput, but still required explicit review solicitation and iterative follow-up on build and static-analysis failures.
Orchestrating AI-Assisted Code Remediation: Socio-Technical Bottlenecks in a Large Industrial RepositoryWhat it means Check CI capacity and reviewer throughput before scaling agent-driven remediation -- once editing is cheap those become the binding constraint, not agent capability. Batch by directory with an explicit per-change file cap, and treat a semantic change set ('fix all instances of warning X') as the unit of work, sliced differently for developers, reviewers and CI.
arXiv preprint · SEP 24
Controlled StudyAppending the model's own channel-control tokens to an untrusted user message drove the reasoning channel from a mean of 52.5 tokens to zero on every trial while the tool call still fired on every trial, and converted 39.6% of the model's refusals on overtly malicious requests into completed exfiltrations.
Control-Token Injection Suppresses Chain-of-Thought and Defeats Reasoning-Based Oversight in Tool-Using AgentsWhat it means Agent safety is a joint property of the model and your harness, so sanitize special and channel-control tokens out of untrusted input before it reaches the chat template, and pin exactly one strict tool-call parser -- a truncation-tolerant regex fired calls a strict parser correctly dropped. Stop treating an empty or absent reasoning trace as benign: flagging it catches the basic attack but not an adaptive decoy, so reasoning-trace monitoring cannot be your only control.
arXiv preprint · SEP 23
Large BenchmarkAcross 1,248 GitHub Agentic Workflow files, agent instructions ran a median of 556.5 words with code blocks in 62.1% of files, and tasks, outputs, constraints and process instructions each appeared in over 93% of labeled workflows, yet only 9.4% explicitly addressed prompt-injection defense. Among files with at least 120 days of observed activity, 78.2% still received updates in month 4.
Specifying and Maintaining Agentic Workflows: An Empirical Study of GitHub Agentic WorkflowsWhat it means Treat agentic workflow files as long-lived operational specifications, not prompts: 78.2% were still being updated in month 4, so version, review and test them like code. Add the three things almost nobody has -- an explicit prompt-injection defense, a resource budget, and an evidence-credibility check -- and re-audit workflows you copied from another repository, because the copy does not inherit the original's fixes.
arXiv preprint · SEP 23
Controlled StudyMerged AI-agent pull requests attract verified follow-up fixes at 1.62 times the within-repository odds of merged human PRs over the same observation window (cumulative incidence 4.5% vs 2.6% by day 30; Mantel-Haenszel OR 1.62, 95% CI 1.10-2.39, p=0.015), and 69.6% of those fixes are opened by the same agent.
Who Finishes the Job? A Study of Follow-Up Fixes and Commit Authorship on AI Coding Agent Pull RequestsWhat it means Treat an agent merge as provisional for at least a week (half of 30-day fix incidence lands in week one): budget a post-merge watch, and read the diff rather than the PR metadata, since merge-time signals barely separate PRs that later needed fixing. Within a repository, a PR with ten times the commits carried 6.1x the odds of a verified fix.
arXiv preprint · SEP 22
Controlled StudyCompile rate, a standard progress metric for LLM vulnerability repair, is dominated by harness and dataset artifacts (about 64% of compile failures are not attributable to the model), shifts 1.8 to 2.7x on identical patches under a single compiler-standard flag, ranks models in the opposite order to reference-similarity metrics, and rewards deletion- and placeholder-style non-repairs when used as an optimization target; whole-function CodeBLEU also fails, since an unchanged copy of the vulnerable input outscores every model.
Metrics Failure in LLM-Based Code Vulnerability Repair: An Empirical Study and a Change-Aware ScreenWhat it means Discount any auto-repair claim reported as compile rate or whole-function similarity; ask vendors for change-aware, execution-grounded evidence before trusting a fix rate. The paper's own diffF1 is offered as a cheap pre-screen, not a repair-quality metric.
arXiv preprint · SEP 22
Small BenchmarkOn the 19 SWE-Serve tasks with model-serving end-to-end tests (627 patches), E2E tests rejected roughly one third of agent patches that passed every unit and regression test (45.9% pass under the full verifier vs 69.4% with E2E tests excluded, 95% bootstrap CI on the gap 13.4-34.4 points), with the increase present for all 11 top-per-model configurations; a matched removal of the same number of non-E2E tests produced half as many fail-to-pass flips.
SWE-Serve: Benchmarking Agentic Engineering For Production Inference ServingWhat it means Add end-to-end tests to whatever gate an agent must clear before merge; in this domain local test passes over-count production-correct patches by about a third. Higher reasoning effort narrowed the E2E gap for GPT-5.6 Sol, Luna and Terra but did not eliminate it (16-21 points at max effort), so budget effort per task risk rather than per default.
arXiv preprint · SEP 22
Small BenchmarkParallel coding agents produced patches that each passed alone but failed when merged in 97% of runs on constructed tasks using 12 real Django helpers, and a single message describing the completed concurrent change recovered 82% of those runs; among 834 runs on 417 mined merged Django PR pairs, only one showed interference.
Passes Alone, Fails Together: Benchmarking Semantic Coordination in Parallel LLM-Agent DevelopmentWhat it means If you fan out parallel agents on one repo, pass each agent a summary of what its siblings changed before merge; the mined-PR tier found almost no interference in already-reviewed PRs, so the 97% figure is a controlled-condition ceiling, not a field rate.
arXiv preprint · SEP 21
Large BenchmarkAutomated harness self-improvement overfits the tasks it evolves against: unregularized evolution posted the highest evolve-set score but an out-of-distribution average within a point of the unevolved harness, and two of four prior methods finished below their starting harness out of distribution; regularized evolution gained up to 14.1 points on an evolve split (Gemini 3.5 Flash on Terminal-Bench; 6.0 with Claude Opus 4.8) but only up to 4.7 points across five out-of-distribution benchmarks, while cutting policy tokens by about 30%.
RRSI: Regularized Recursive Self-Improvement of Agent HarnessesWhat it means Treat any 'self-improving agent' or auto-tuned harness result as in-distribution until it is scored on tasks the loop never saw; expect the transferable gain to be roughly a third of the headline. Hold out a task set the optimizer cannot touch before adopting an evolved harness.
arXiv preprint · SEP 21
Controlled StudyA fixed 267-word specification preamble reduced defects in one-shot generated backend code for all five frontier models tested (mean 0.16 to 0.70 fewer findings per task, every Holm-adjusted sign test significant, every bootstrap CI excluding zero), won 95 of the 100 task pairs where the arms differed, never made any model worse in any domain, and cut Bandit medium-or-high findings from 53 to 11.
Specification Before Generation: A Pre-Registered, Five-Model Paired Evaluation of a Specification Frame for LLM-Generated Code in Money, Time, Idempotency, and Access TasksWhat it means Put a short, fixed specification of what must be true (money arithmetic, time handling, idempotency, access control) in front of generation prompts for regulated backends; the lift was largest where a model's unprompted defaults were weakest, and it does not depend on which vendor you use. Evidence is one-shot generation only, so it is untested inside an agent loop.
arXiv preprint · SEP 20