Only 16.6% of Agent Sessions Ended With a Working Hosted Database: Scope's First AX Benchmark
Scope's first database agent benchmark: only 16.6% of 1,035 agent sessions ended with a working hosted database. Product-side signup walls, not models, dominated the failures.
Scope ran the scorecard most agent operators have been waiting for: can a coding agent, starting with nothing, sign up for a hosted database and actually use it? Across 1,035 scored sessions covering 18 providers and 20 agent configurations (Claude Code, Codex, Cursor, OpenClaw, Hermes and OpenCode), only 16.6% — 172 sessions — ended with a working database that Scope’s auditors verified themselves, never taking the agent’s word for it. A session counts as a success when the database the agent set up actually works, and some of those successes arrived through account-less paths rather than a completed signup.
The headline finding is not about models. Per Scope’s results section, success rates ran from 0.0% to 88.3% across products, but only 3.7% to 29.6% across agents — a spread the publisher reads as “the product matters more than the agent.” Note that each “agent” is a specific model-harness configuration, so harness and model effects are bundled together.
The leaderboard
| Product | Success | Main wall |
|---|---|---|
| Prisma Postgres | 88.3% | Sign-up refused, no reason (60% of its rare failures) |
| Upstash | 63.3% | Email never verified |
| Neon | 43.3% | Bot block (53% of failures) |
| Nile | 43.3% | OAuth only (94% of failures) |
| ClickHouse Cloud | 38.3% | Captcha |
| Supabase, Convex, MongoDB Atlas, CockroachDB Cloud, Render, Xata, Tiger Cloud, SingleStore, Aiven, Appwrite, Nhost | ~0–3% | Captcha and bot blocks |
What actually stops agents
Scope’s audit attributed 67% of failed sessions to the site, 16% to the agent and 17% to neither (unattributed). Captchas alone stopped 373 sessions; bot blocks another 105; 51 sessions were refused because the agent’s inbox domain was treated as disposable; 32 met Google-or-GitHub-only OAuth walls; 31 never got past email verification — a category Scope classifies as unattributed, not a confirmed product-side wall.
One transcript in the report captures the whole problem. Claude Opus 5.5 hit Neon’s “Verify you are human” check, correctly refused to circumvent it — and then finished the task anyway through Neon’s official account-less instant-database path, delivering a working notes table with the row read back, plus a claim link for the human. Fifty of the 172 successes came through exactly these try-before-signup paths. Where products let agents start before the account exists, agents take the exit.
What operators should do with this
- Read the table as a ranking, not a guarantee. Prisma Postgres (88.3%), Upstash (63.3%), Neon and Nile (43.3%) and ClickHouse Cloud (38.3%) scored highest, but even the leaders failed 12–62% of the time, and some successes bypassed signup entirely. Providers near zero in this run — Supabase, Convex, MongoDB Atlas — are candidates for pre-provisioning the account yourself and handing the agent a connection string, which is usually cheaper than retrying into a captcha. This matches the pattern we covered in agent environment setup being two-thirds of the failure: the blocker usually lives in the environment, not the prompt.
- Pre-provision instead of fighting walls. A captcha that stopped 373 of 1,035 sessions is a stop sign, not a puzzle. Give the agent a connection string through your secret channel rather than asking it to drive a headless browser through a human-verification loop — and when you do hand it database access, gate it the way our Postgres MCP server read-only role guide describes, so agent access stays scoped.
- Prefer account-less bootstrap where it exists. Neon’s instant-database paths (
neon.new,npx neondb) succeeded in the benchmark even when signup failed, though the resulting databases expire after 72 hours unless claimed.
Vendors are already moving
The fix looks like templates and protocols, not bigger models. Neon’s Function Registry, announced Nov. 5, installs working templates with neon functions new <id> -y — no linked project, no signed-in CLI — so agents start from pinned, runnable code instead of burning tokens reconstructing best practices. Cloudflare’s Trusted Agent Delegation post argues the other side: sites need identity, reputation and accountability mechanisms (Web Bot Auth, MOLE, per-agent access policies) so they can let agents in without losing the user behind them — and its Security Analytics now breaks out personal-assistant traffic so site owners can see which agents arrive and what they hit.
Read the numbers with the setup in mind
The 1,035 sessions ran from November 2 to November 6, 2026, each in its own Modal cloud sandbox with a fresh AgentMail inbox (and AgentPhone numbers on request), browser agents going through a residential proxy. Every agent tried every product up to three times — OpenCode ran once — and Scope’s 95% intervals come from 20,000 bootstrap resamples, so most of the agent ranking is not statistically ordered. The environment is part of the result: some products refuse agent inbox domains outright, and Scope says what it could prove was its own setup is taken out. Treat all of this as a dated snapshot — signups, captchas and free plans change. But as of this window, the benchmark’s structural lesson stands: agent experience is now a product feature, and if your database’s signup assumes a human with a browser, the scorecard will find it.
