New startup ideas · AI and software · Agent infrastructure
startup idea
Rubricon
An exchange where domain experts sell held-out eval suites that enterprises run on demand.
Rubricon is a marketplace for agent evaluation: radiologists, tax accountants, claims adjusters and network engineers publish task sets with graders, and enterprises pay per run to test their agents against them.
- Marketplace
- Enterprise
- $10-100B market
- Creates a new category
- US first
4/5
venture judge
3
similar startups, last 2 years (14 all-time)
83%
of 4 nearest real companies still alive
yes
3 matching federal grants and programs
Direction supported by government programs and grants
Test it before you build it
$1,850 · 4 weeks · 30 prospects
Proves for $1,850 in four weeks that teams shipping claims-handling agents will prepay $500 per run for an adjuster-written held-out suite, and that ten licensed adjusters will write one.
Riskiest assumption · Teams shipping claims-handling agents will pay real money per run for an independent adjuster-written held-out suite instead of relying on their own spreadsheets and vendor-supplied evals, which is exactly the budget Atla and Baserun never found.
1Focus group: who and where
The head of AI or product at a US claims-automation vendor (10-100 people) whose carrier prospects keep asking for proof the agent decides like a competent adjuster, plus claims technology leads at mid-size P&C carriers and TPAs evaluating those vendors this quarter
where to find 30 · The Guidewire Marketplace partner directory of claims-automation vendors, the InsurTech NY community and its monthly events, the public exhibitor list of InsureTech Connect (ITC Vegas, October 2026) for direct outreach without buying a ticket, and claims AI threads on r/Insurtech
2Sell first, build later
One run of the adjuster suite: 150 held-out claims decision tasks written by 10 licensed adjusters, the buyer's agent outputs graded against their rubrics, scored report delivered within 14 days of the October run slot.
the ask · $500 per run at the founding price, locked for 12 months; list price $1,500 per run after the first cohort.
a real yes · A completed $500 prepaid checkout counts as yes; waitlist signups, 'send us the tasks to review first', and requests for a free pilot run do not.
3Small experiments
The first one attacks the riskiest assumption; each ends with a number that says whether to run the next.
1. Pre-sell fifteen buyer calls
$150 · 12 days
One founder emails 25 buyer-side leads pulled from the Guidewire Marketplace directory, the ITC Vegas exhibitor list, and InsurTech NY intros, and books 15 calls. Pitch from a three-task sample deck: 150 held-out adjuster-written claims decisions, scored report in 14 days, October cohort, and ask for the $500 prepay on the call.
keep going if · 4 of 15 calls end with a verbal yes and a checkout link sent the same day
2. Priced landing with prepaid checkout
$200 · 5 days
Put up a one-page site showing the three sample tasks, the October run window, and a $500 prepaid checkout; card checkout fits here because self-serve card signup is this product's own buying motion. Send the link after every call and post it where claims AI vendors already argue about evals on r/Insurtech and in InsurTech NY channels.
keep going if · 5 prepaid runs collected, or 3 prepaid plus 10 waitlist signups from company email domains
3. Sign ten adjuster authors
$900 · 10 days
Post a paid call for contributors on r/adjusters and the Insurance Nerds community: $300 for 20 held-out claims decision tasks with grading rubrics, under a contributor agreement with a revenue share on every future run. Pay the first three adjusters to author 60 tasks now so the sample deck and first runs are real.
keep going if · 10 licensed adjusters sign contributor agreements and the first 60 tasks pass cross-review by a second adjuster
4. Deliver first runs manually
$600 · 14 days
Run the first prepaid buyers by hand: the buyer submits their agent's outputs on 60 tasks through a secure form, two paid adjusters grade against the rubrics, and the founders compile the scored report within 14 days. Manual grading stands in for the sandbox because the run count is tiny and the graders are the marketplace's future sellers.
keep going if · 3 of 5 report recipients ask to book a second run or a monthly package
4Collect a deposit up front
Tesla took $1,000 refundable reservations for the Model 3 and $100 for the Cybertruck before building either: the deposit is the measurement, not the revenue.
$500
per prospect, refundable
how · A prepaid first run through the landing page checkout: the buyer's AI lead pays $500 by card, which they can expense without procurement, and self-serve card payment is exactly the buying motion the marketplace will live on, so the checkout itself is the measurement. set up: Stripe Checkout ↗
what it reserves · A slot in the October grading cohort and the $500 founding price per run locked for 12 months.
refund · Refunded in full if the scored report is not delivered by October 31, 2026.
target · 5 prepaid runs from 25 buyer conversations within 28 days
before taking money · Use only synthetic or fully de-identified claim scenarios and bar buyers from submitting real claimant records until a privacy and security review is complete, since claims files carry personal and health information.
Go: build it if
5 prepaid runs ($2,500 collected) from 25 buyer conversations, 10 adjusters under contributor agreements, and at least 3 report recipients booking a second run.
Kill: stop if
Fewer than 2 prepaid runs after 25 buyer conversations, or fewer than 6 licensed adjusters willing to author under a held-out, revenue-share agreement.
5 Scripts to run itoutreach message, landing copy, deposit terms · click to open
outreach message
Carriers keep asking you to prove your claims agent decides like a good adjuster, and a demo on cherry-picked claims does not close that gap. Ten licensed adjusters are writing a held-out suite of 150 claims decisions with grading rubrics; you point your agent at it and get a scored, third-party report in 14 days. The first cohort runs in October at $500 per run, locked for a year. Do you have 20 minutes this week to see three sample tasks?
landing page
Test your claims agent against 150 decisions by working adjusters $500 prepays your first run: scored report in 14 days, founding price locked for 12 months Prepay your run and reserve an October grading slot
deposit terms
$500 prepays one full run of the adjuster suite: 150 held-out claims decision tasks, grading by licensed adjusters, and a scored report within 14 days of your October run slot. Refunded in full if your report is not delivered by October 31, 2026. Your $500 founding price per run is locked for 12 months.
Would you run this test?
One tap. The yes-share feeds the Demand pillar of this idea's score; nobody sees who answered.
Budgets are out-of-pocket estimates for a team of one to three, US market. Size the deposit to the deal, and check the terms before taking money in a regulated line.
Scorecard
One score that balances how trendy the idea is, the demand for it and its potential for 100x, with competition measured relative to every other idea in the catalog. Recent startup trends first, government priorities second.
74
Idea Score, 0-100 · raw 45.7 x 1.61
Warm
competition: more crowded than 41% of ideas · headwind x0.80
+1.5
government priorities, secondary (3 matching grants)
Trend
50
Is the wave forming now? 2025-26 entrants vs 2023-24, rounds since 2025, the sector's live-batch direction, the 2026 trend analyst.
- Entrants 2025-26 vs 2023-24 (similar companies)70
- Rounds announced 2025+ in the sector4
- Sector direction (live batch)100
- 2026 trend analyst25
Demand
52
Does anyone want it? YC's current RFS, companies already paid for something similar, the operator judge, founders' yes-rate in decks, readers who would run the test.
- YC asks for it (current RFS: idea / sector)30
- Someone already pays (similar companies, recent / all-time)100
- Operator judge: real pain25
100x potential
67
Can it return a fund? The venture judge (double weight), market-size and moat axes, neighbours still alive, the technologist judge.
- Venture judge75
- Market size axis67
- Moat axis100
- Neighbours still alive34
- Technologist judge50
Score = 100 x cbrt(Trend x Demand x 100x) x (1 - 0.5 x crowding) + government bonus (max 5), calibrated so the 95th-percentile idea scores 90 (order never changes). A geometric mean: a weak pillar cannot be papered over. Percentiles are among the 272 ideas in the catalog; the terms matched were exchange, domain, experts, sell, held-out, eval, suites, marketplace.
The idea in full
- What
- Rubricon is a marketplace for agent evaluation: radiologists, tax accountants, claims adjusters and network engineers publish task sets with graders, and enterprises pay per run to test their agents against them. Sellers never release the answer keys, since suites run only inside Rubricon's sandbox with private held-out splits, and sellers take a revenue share on every run. Buyers sign up with a card and point their agent endpoint at a suite in an afternoon.
- Why now
- CoArena is building the biggest crowdsourced benchmark for computer-use, and the NSF funded an I-Corps translation project for a GenAI result comparison tool for evaluating foundation model outputs at roughly $50k, so third-party comparison of agent outputs is becoming a traded good; meanwhile Atla and Baserun both died selling single-vendor eval platforms.
- Wedge: first customer and entry point
- US insurance claims teams buying one suite: adjuster decision tasks written by ten working adjusters, priced per run, sold self-serve.
- Path to 100x
- Enterprise assurance and QA spend moving onto agent systems is a $10-100B pool, and an exchange wins most of it because the suite with the most buyers attracts the best experts, whose suites then attract more buyers. Once procurement teams cite Rubricon scores in agent purchases, the scoreboard itself is the category and a second exchange has nothing to compare against.
- Ceiling
- Enterprises keep accepting vendor-supplied evals and internal spreadsheets, leaving Rubricon a niche developer tool at low tens of millions in revenue.
- Closest real companies, as the generator saw them
- CoArena crowdsources one horizontal computer-use benchmark; Rubricon is a two-sided market of priced vertical suites. Buildbox and Agnost AI analyze how users experience one company's agent rather than comparing agents across a shared standard. Atla and Baserun were single-vendor eval tooling, not an exchange.
- Main risk
- Suites leak into training data or public repositories and lose their value faster than sellers can replace them.
Five judges
Each judge scores every idea in the catalog with a named rubric; the venture judge decides whether a card is shown at all (4-5 is venture-grade).
Venture investor
4/5
Capital-light exchange with revenue within a year where the scoreboard cited in procurement compounds sellers and buyers into one defensible standard.
Bootstrapper
4/5
Ten working adjusters writing one suite, sold self-serve per run within a year, is cheap to launch and cash-positive fast.
Operator
2/5
Atla and Baserun both died proving nobody has budget for eval, and procurement citing Rubricon scores requires the market to move first.
Technologist
3/5
Held-out sandbox execution is modest engineering and suite leakage into training data erodes the asset faster than the network effect builds.
Risk
3/5
Capital light with no platform dependency, but suite value evaporates the moment held-out splits leak into training data.
trends
2/5
Paid held-out eval suites cite only a $50k NSF award, and the graveyard entries Atla and Baserun show the eval trend already tried and thinned.
Similar startups in the directory
Companies whose pitch matches most of the idea's terms (exchange, domain, experts, sell, held-out, eval, suites, marketplace): 14 all-time, 3 from the last two years. Same matching as Idea Check.
Evals for robots
Evals for frontier research agents
Misprint is building Robinhood for Pokemon cards
Marketplace for automotive enthusiasts to buy and sell speciality equipment
Cross-Border Payments and FX Simplified
The safest way to buy and sell collectibles
Operator of a peer-to-peer fashion marketplace intended to connect buyers and sellers for online shopping. The company offers a marketplace that is a mix of a social network and a forum where people like and comment on the articles posted and give each other fashion tips, enabling customers to buy, sell, exchange, and review new and used fashion items with ease.
Asia’s leading media marketplace in the US and SEA to Buy & Sell Film, TV & Sports Content
Operator of artificial intelligence (AI)-powered marketplace intended to connect finance and accounting freelance professionals with companies. The company's platform matches businesses of all sizes and industries with highly vetted finance experts, Chief Financial Officers (CFOs), Certified Public Accountants (CPAs), and bookkeepers who have the domain expertise to tackle industry and company-specific problems on demand, enabling businesses to access the tools and insights they need to manage and grow their books of business.
Provider of an online marketplace designed to sell construction supplies. The company's online marketplace offers drop shipping and contractor-level pricing services along with hardware and building supplies, enabling general contractors to procure building materials.
Shibuya Blend Green Energy is a operator of a carbon trading platform.
The generator's reference companies
Real companies the model named as closest when it wrote the card, with their fate. A check mark is a company the radar could verify in its directory.
Public money in this direction
US federal grants, SBIR/STTR awards and open opportunities from the radar's public-money feed, matched to the idea's terms; the sector totals give the context.
3
grants and programs matching the idea
36
startup-relevant grants in Agent infrastructure
$16M
awarded in the sector, tracked
- Collaborative Research: Cognitively Optimized Mathematics through Peer Analogous Reflective Exchangesawardmedium relevance
National Science Foundation · Cyberlearn & Future Learn Tech · $506K · posted 2026-08-12
- Collaborative Research: Cognitively Optimized Mathematics through Peer Analogous Reflective Exchangesawardmedium relevance
National Science Foundation · Cyberlearn & Future Learn Tech · $224K · posted 2026-08-12
National Science Foundation · Secure &Trustworthy Cyberspace · $200K · posted 2026-08-07
Market signal
What the radar sees in Agent infrastructure: new companies by cohort year, the forming YC batch, and outcomes since the February snapshot.
Agent infrastructure · 28 → 64 → 108 → 150 → 171 new companies 2022 → 2026 · 94% aliveYC F26 live: 22 in this cluster, 18% of the batch (was 15% in S26)Since February, of 239 YC companies here: 9 acquired, 4 shut down, 111 rewrote their pitch
Design attributes
The card is one cell of a designed set: every axis below was chosen before the text was written, and the text had to realize it.
- Buyer
- Enterprise
- Business model
- Marketplace
- Path to 100x
- Creates a new category
- Market size
- $10-100B market
- Capital intensity
- Capital-light (software margins)
- Speed to revenue
- Revenue within a year
- Technical depth
- Real engineering
- Go-to-market
- Self-serve
- Moat
- Network effects
- Geography
- US first
- Regulation
- Some regulation
- Vibe
- Boring business
Listed under
An idea sits in its own sector and in any sector its text clearly touches.
More ideas like this
AI and software · Agent infrastructure
Socketry
The exchange where software vendors sell maintained, agent-ready API connections.
Socketry is a two-sided marketplace where SaaS vendors publish guaranteed-current, agent-callable versions of their APIs - tested sandboxes, auth, rate contracts, change notices - and enterprises subscribe to them for their internal agents with one bill and one security review.
AI and software · Agent infrastructure
Latchwork
Self-healing connectors for the long tail of small business software agents cannot reach.
Latchwork records a session against a vertical tool that has no usable API - a dental scheduler, a salon booking system, a freight dispatch app - and turns it into a connector that agents call like an API, then repairs itself when the vendor changes a screen.
AI and software · Agent infrastructure
Civitrace
Casework agents for US agencies that turn every verified fact into reusable public evidence.
Civitrace runs eligibility and permit casework as a managed agent service for US state, county and city agencies: it reads submitted documents, checks them against source systems, drafts determinations, and hands a human caseworker a decision packet with citations.
AI and software · Agent infrastructure
Actledger
Signed action records for enterprise agents, with a policy pack ecosystem on top.
Actledger sits between an enterprise's agents and the systems they touch: every tool call is authorized against policy, signed, and written to an immutable action record mapped to audit controls.
AI and software · Agent infrastructure
Toolharbor
The governed registry every enterprise agent must pass through to touch a tool.
Toolharbor is a control plane that sits between an enterprise's AI agents and every tool, API, and MCP server they call, enforcing policy, credentials, and rate limits per agent.
AI and software · Agent infrastructure
Openstall
No-code layer that makes every small business readable and transactable for AI agents.
An SMB connects its booking, inventory, and payment tools (Shopify, Square, Calendly, QuickBooks) in a no-code dashboard, and Openstall publishes them as one hosted, self-maintaining endpoint that any AI agent can query and transact against.
Fictional company written 2026-08-22 from MarkosWeb data; the companies, grants and numbers around it are real and tracked. Treat the idea as a research prompt, not a plan.