B4 Research August 10, 2026

From neutral prompts, three frontier models (Fable 5, Opus 4.8, Opus 5) shipped code with unprompted SOC 2 conformance of 47-88% and real vulnerabilities (a reachable Werkzeug debugger RCE, an unauthenticated download, an endpoint returning every stored name and email); adding one sentence naming the SOC 2 standard lifted every case to 86-100% (worth 23-50 points) and removed every insecure construction.

What it means Name the compliance standard explicitly in the prompt and gate AI-generated infra/auth/PII code behind a security review; unprompted output ships RCE-class defects, controls outside the model's default conception (MFA, cookie flags, account lifecycle) survive even a named standard, and a newer model won't close the gap.

arXiv preprint · AUG 07

Across 3.52M production code changes at a billions-of-users enterprise (Apr 2025-Apr 2026), AI-generated C++ carried a distinct quality profile - higher interface/coupling burden, more copy and allocation overhead, explicit loops over optimized standard APIs - translating to increased review effort and a 5-8% rise in compute consumption; taxonomy-informed static-analysis feedback cut targeted warnings 11.1%.

What it means Budget for the hidden tax on AI-authored code - extra review effort and ~5-8% more compute at runtime - and route it through taxonomy-informed static-analysis feedback; velocity gains are real but not free.

arXiv preprint · AUG 06

Recurrent context compression in long-horizon agents weakens the influence of recent interactions and measurably increases blocked actions, repeated exploration, and run-to-run instability; a verifier-guided compaction framework (TRACE) recovers task performance and multi-run reliability.

What it means Treat context compaction as a reliability risk rather than a free cost optimization — measure multi-run stability before enabling it on long-horizon agents, since naive compression trades tokens for blocked actions and nondeterminism.

arXiv preprint · AUG 06

A survey of 119 practitioners found LLM use has become a habitual component of professional software engineering, with reported patterns of functional dependence and overreliance — prioritizing LLMs over documentation or peer consultation — alongside less-common addiction-related difficulty moderating use.

What it means Build trust-calibration and mandatory verification into team practice — the productivity that drives adoption also drives overreliance (skipping docs and peers), so make 'verify the output' and 'consult a human/source' explicit workflow steps rather than assumed judgment.

arXiv preprint · AUG 06

Coding agents spend most of their token budget finding the file to patch rather than patching it — a 30B OpenHands agent averages 23 rounds and 631K tokens per resolved SWE-Bench issue — and a dedicated RL-trained retrieval step preserved resolve rate (27.0% vs 25.8%) while cutting 15% of rounds and 19% of tokens, but only above a retrieval-precision threshold below which retrieval degraded the agent.

What it means Attack agent cost at repository retrieval, not just the model — a precise file-finding step cuts ~19% of tokens, but a weak retriever (BM25-grade) makes things worse, so measure retrieval precision before bolting one on.

arXiv preprint · AUG 06

Engineers report a productivity-experience paradox: 84% report productivity improvement at both time points, yet the share of matched participants reporting worsened developer experience in at least one dimension nearly doubled from 14% to 27% over six months, with flow state and cognitive load eroding while feedback loops improved.

What it means Track developer experience alongside throughput when rolling out AI tooling — the work is shifting to supervisory correction of AI output, and that erosion shows up months after the productivity win.

arXiv preprint · MAY 22

In legacy-code modernization, semantic-trap snippets silently changed behavior in 39.7% of attempts (versus 7.0% on benign controls), and the producing model silently endorsed 31.7% of its own drift cases — sometimes while correctly articulating the exact Python 2/3 semantic distinction that broke its output.

What it means Never use the model that performed a migration as its own behavioral reviewer — verify semantic preservation with executable oracles (characterization tests), especially around numeric semantics.

arXiv preprint · MAY 20

10.7% of passing SWE-agent trajectories are 'Lucky Passes' — regression cycles, blind retries, or missing verification that happen to end green — with per-model Lucky rates ranging 0.5% to 23.2%, enough to move models up to five leaderboard rank positions when scored by process quality.

What it means When comparing agents for production use, inspect trajectories, not just pass rates — a model that passes by chaotic trial-and-error will be more expensive and less predictable on your real work than its score suggests.

arXiv preprint · MAY 13