How I Built a 1,600-Category Software Decision Engine

I oversee marketing and technology at a major apparel manufacturer. Three brands, B2B and DTC channels, a 300+ application stack. And I’ve had a growing suspicion that we were spending money on software that AI could handle.

As I took on the challenge of understanding everything that we’ve bought, built, and put to pasture over the years, that suspicion turned into a framework that helped me make better decisions. The framework turned into a database. The database turned into the B4 Index — more than 1,600 independently scored software categories across 169 domains and 92 industries. Here’s how it happened, and how it keeps itself honest now.

The problem that started it

Two years ago, I tried to build a custom workflow tool using AI. It failed. Well, I got frustrated with how hard it was and gave up. The technology wasn’t ready. Last year, I tried the same thing. I built 90% of what I wanted over a weekend. Was it perfect. No. But it was almost magical how far development had come in such a short time.

That experience broke something in my mental model. If the cost to build dropped that fast in one category, what about the other 300 tools in our stack? Which ones were still worth paying for? Which ones had AI breathing down their neck, or were something you could just build internally? And which ones were we locked into for no good reason?

I looked for a resource that answered these questions. Something that scored software categories — not individual vendors, but the categories themselves — on whether you should build, buy, or do something in between. G2 ranks vendors. Gartner’s Magic Quadrants compare vendors within markets. Nobody was asking the actual question: in 2026, given what AI can do, should we buy this capability, extend a platform, or build a scoped alternative for our needs?

So I built one.

The framework: Build, Buy, Bridge, or Beware

I started with five dimensions. Each one is scored 1 to 5.

Specificity — how unique is this to your company? Payroll is the same everywhere. Your sales compensation logic is one-of-a-kind.

AI Feasibility — can AI build most of the core value today? Not in a demo. In production. The current methodology anchors this on buildability, not theory: could you actually build and run the thing, with parts you can order today.

Strategic Control — is this a competitive differentiator or table stakes?

Vendor Value — what percentage of the vendor’s features do you actually use?

Cost Trajectory — how does the full cost of building and operating compare with the vendor over the same scope and time horizon?

Two of those dimensions (Specificity and Strategic Control) average together to form the X-axis: Strategic Differentiation. AI Feasibility is the Y-axis. An axis is high only when it is strictly above 3.5. Methodology v4.0 enumerates 27 nearby scoring scenarios and reports the leading quadrant with its classification stability:

BUILD — high differentiation, high AI feasibility. Your competitive edge. Test the case for owning a scoped capability.
BUY — low differentiation, low AI feasibility. Commodity. Buy what works for your use case.
BRIDGE — high differentiation, low AI feasibility. Strategic, with limited feasibility for the full build. Buy the platform and extend it where the business benefits.
BEWARE — low differentiation, high AI feasibility. The ground is shifting. Test a narrower build, a cheaper purchase, or a capability you already own.

The other two dimensions — Vendor Value and Cost Trajectory — don’t change the quadrant. They determine urgency. How fast should you act?

From 162 to 299 in one session

I started with 162 categories across 12 domains. The original scoring, mostly my work. CRM, ERP, marketing tech, security, analytics… the horizontal software every company evaluates.

Then I realized the gaps. No AI & Machine Learning domain. No Customer Service & Support. No Commerce & Payments. No industry verticals beyond a handful. If someone in healthcare or construction looked at the index, they wouldn’t find their world.

So I expanded it. Through a custom multi-agent research system, I added 125 new categories. Seven parallel AI agents, each handling a domain batch, each researching vendors via Perplexity, scoring against the methodology, writing justifications for every dimension. The agents looked up current vendor pricing, checked AI capability claims, and applied the rubric. Those outputs still need review.

Six new horizontal domains. Thirteen industry groups within the Vertical domain. Healthcare, financial services, construction, education, energy, government, automotive, legal, agriculture, media, nonprofits, professional services, transportation.

That sprint brought it to 299 categories across 19 domains. It hasn’t stopped growing since. Today the index covers more than 1,600 categories across 169 domains and 92 industries, with thousands of vendor records and category mappings underneath them.

What the data tells you

The August 2026 v4 snapshot used in the book is revealing:

BUY: about 53% — The largest quadrant. Vendor scale still wins for most infrastructure.
BRIDGE: about 21% — The sweet spot for mid-market companies. Buy the platform, build the edge.
BEWARE: about 22% — The controversial one. These are categories worth testing for overpayment against credible alternatives.
BUILD: about 4% — Smaller than people expect. Most software is still better to buy. A gain in AI feasibility can move BRIDGE toward BUILD or BUY toward BEWARE; differentiation still decides which side of the map applies.

How I actually built this

The B4 Index itself is a case study in the framework. I built it instead of buying a database product. Here’s the stack:

Supabase (PostgreSQL) for the database. Triggers auto-compute quadrants when scores change.
Astro 5 + React for the website. Tailwind v4. Deployed on Vercel. I update everything directly from Claude Code.
Claude Code for implementation. The AI coding tool wrote most of the components, handled the data pipeline, and executed the expansion.
Perplexity for most of my vendor research. Current pricing, market positioning, feature comparisons.

The whole thing runs on a few dollars a month in infrastructure. The expansion from 162 to 299 categories happened in a single afternoon. Seven parallel agents, each scoring 15 to 25 categories with full vendor research and five-dimension justifications.

That’s the point. The economics of building have changed. What would have taken a team of analysts months to produce, I did in an afternoon with AI assistance. AI handles the research and scoring grunt work while I keep hold of the methodology and the judgment calls.

The part that never ends

Here’s what nobody tells you about a decision engine: building it is the easy part. Keeping it true is the job.

A score isn’t a fact you write down once. AI capability moves every week. Better production evidence can move a category from BRIDGE toward BUILD. Moving from BEWARE to BUILD also requires higher differentiation. A vendor price increase changes urgency, not the quadrant. An index that doesn’t move with the world is just a snapshot, and snapshots rot.

So it re-scores itself, every night. At 1 a.m. an engine wakes up and scores whatever slice of the index it has to in order to get through the full category list before the quarter’s deadline. Miss a night, and the next night’s batch grows to make up the difference. It’s deadline-driven, not a fixed rate, with the aim of giving every category a fresh look each quarter. The yardstick never moves. The scores do.

Those nightly scores are drafts. They’re live right away for anyone using the API or the MCP server, but the public index doesn’t change until a quarterly release promotes them all at once. That’s on purpose. A public reference that flickers every night is noise. A public reference that locks, ships, and shows its work every quarter is something you can plan around.

And a release has to earn its way out. Before a quarter’s scores go public, a red team runs at every score that moved: three independent skeptics per change, each forced to go pull its own evidence, majority vote. It’s adversarial by design, trying to kill a bad change before it reaches you. Then a release pipeline checks coverage, verifies the live and published data agree, writes the changelog, and only then publishes. If anything fails a check, it rolls back.

Then on Wednesday mornings, a second system goes to work: a sweep of the actual frontier. It reads the week’s funding rounds, model releases, and product launches across a dozen sources, checks funding claims against source announcements and reporting, and flags when something real should move a category or spin up a brand-new one. That’s how a category like AI-agent security gets born the same week the money shows up for it. It’s also where my weekly Frontier column comes from — the one I follow the money in every week.

None of this is a weekend project anymore. Eight-stage pipelines. Adversarial passes that try to kill a finding before it’s allowed to count. A recall audit that grades its own blind spots. I built the first version in an afternoon. Keeping it honest is a standing operation.

That’s really the whole pitch. You could build your own. I just spent a year learning why you probably don’t want to.

What’s next

The public index locks and republishes each quarter, but underneath it, categories are always moving. A gain in feasibility can move BRIDGE toward BUILD or BUY toward BEWARE. Changes to differentiation or corrections to the evidence can move categories in other directions. Consolidation alone does not set the verdict. Each shift is a story, and each story becomes content.

A lot of what I called “coming” a few months ago is live now. There’s an MCP server, so you can query the index from inside Claude or any agent you build. There’s a Claude Skill for running assessments against the methodology. And the framework has a book behind it that ties the whole thing together.

The index is live at benroberts.ai/framework, free to browse. Pick your industry. See your categories. Make better decisions. I’d love to hear what you find.

Ben Roberts is an operator, AI builder, and strategist, and creator of the B4 Framework. He writes about software decisions, AI-assisted development, and what happens when you let AI challenge your assumptions about what to build and what to buy.

Updated September 19, 2026: Updated the distribution to the book’s dated v4 snapshot and corrected the distinction between feasibility, differentiation, and price urgency.