Forward
Does the code keep its promise?
Assay extracts every implicit claim your code makes. Handles null. Validates input. Retries on failure. Then it checks the code against each claim, one at a time.
Your AI wrote the code.
Assay tells you what’s actually true.
AI writes code that compiles clean, reads right, and still does not do what it says it does. Assay extracts every claim your app makes, verifies each against the code, then triages every failure. Real defect: fix it. Spec drift: leave the code alone.
One production app: 319 claims verified. 25 bugs. 8 of them were the spec’s fault, not the code’s.
npx tryassay assess /path/to/projectOpen the Assay portalNo signup to run the CLI.
Triage
Verification that knows a real bug from a stale spec.
A blind auto-fixer ships a security regression here. Assay tells your agent to stand down.
bug-report.md — from a live assess run
## Critical — User Registration
Claim: registration returns access + refresh tokens
Verdict: FAIL
Triage: Likely spec drift (not a code bug). Tokens are deliberately withheld until email verification.
Fix prompt: do NOT change the code. Update the spec to match reality.
npx tryassay assess /path/to/projectnpx tryassay fix /path/to/projectRuns with your Anthropic API key, roughly $1–10 in tokens by repo size. Reports publish to the Assay portal by default; --no-publish keeps everything local. fix applies the real defects. Never the spec-drift ones.
The method
Two directions, one verdict.
Every claim the code makes, checked both ways.
Reverse
Does anything happen you never claimed?
Assay also works backward, from the code to the claims. The swallowed error, the path with no guard, the branch that returns the wrong shape. It surfaces behavior no claim covers.
Engine
Patent-backed verification.
Claim extraction and dual-direction checking are a patent-backed method, not a linter pass. Assay reasons about what the code intends, then proves it or breaks it.
Under the hood it is a neurosymbolic verification loop: neural systems extract claims, symbolic oracles verify them.
1Extract
An LLM reads your code and identifies every implicit claim it makes. "This function handles null input." "This API returns sorted results." No regex or AST parser can do this — it requires semantic understanding. This is the neural layer.
2Verify
A deterministic oracle tests each claim. For code, that means actual test execution in an isolated subprocess. The oracle does not hallucinate. It runs the code and reports what happened. This is the symbolic layer.
3Triage & Fix
Every failed claim is classified: real defect, spec drift, or uncertain. Real defects get a fix prompt your coding agent can apply — or `assay fix` applies them for you. Spec drift gets flagged so nobody "fixes" correct code.
Proof
We also point it at the big stuff.
OpenAI. Kubernetes. Shopify. Cloudflare. Vulnerabilities found and reported in all of them.
OpenAIcritical · CWE-94
RCE via vm.Script sandbox escape
Buffer.from('').constructor.constructor('return process')()Found and reportedKuberneteshigh · CWE-918
SSRF chain across kubeadm discovery endpoints
client.Get(httpsURL) // user-controlled, full response consumedFound and reported
Kuberneteshigh · CWE-22
Path traversal to root code execution in GCI mounter
path, _ := filepath.Split(os.Args[0]) // attacker controls argv[0]Found and reported
Cloudflarehigh · CWE-918 / 284 / 601
SSRF + CORS + Open Redirect chain
await fetch(switchRemote(url, remote), { ... }) // no assertValidURL()Found and reportedCloudflarehigh · CWE-732
OAuth tokens written world-readable
writeFileSync(configPath, TOML.stringify(config)) // mode defaults 0o644Found and reported
Shopifymedium · CWE-338
Predictable CSP nonce via Math.random fallback
new Uint8Array(16).map(() => (Math.random() * 255) | 0)Found and reported
The gap
Human-built code scores 91. Here is what AI platforms score.
4 platforms verified. 18 bugs found. 0 passed.
Bolt.new
42/100
Lovable (App 1)
42/100
Lovable (App 2)
42/100
Replit
32/100
Community result
TurfUmpire — 91/100
Cricket scoring app, Next.js + Supabase. 205 claims verified, 18 bugs found (2 critical, 1 high, 15 medium). Real developer, real repo, published report.
CVE recall
We measure our own miss rate.
Most benchmarks hide the misses. The misses are the roadmap.
32.9%
of 164 blind holdout CVEs caught at the exact vulnerable code — advisories no rule was learned from, strictest definition.
+6.1
points of blind recall in the last teaching cycle — 26.8% → 32.9% on the same 164-CVE holdout.
222
real vulnerabilities scored from the GitHub Advisory Database, each scanned as the code existed before the fix.
A finding only counts as a catch if it lands on the code the fix later changed, and stops firing once the fix is applied. Looser definitions inflate the number; we lead with the strictest and publish all three. The benchmark re-runs every Monday and appends to a public TSV.
The full benchmark, misses includedTraction & research
From research to production.
Verified results across benchmarks, pilots, and live deployments.
464
tasks verified
100%
pass@5 HumanEval (164/164)
354
claims in the LVR pilot
27
bugs auto-fixed
Published and protected
- CHI 2026 submission
- DOI 10.5281/zenodo.18522644
- U.S. Patent App. #63/980,048 (Feb 2026)
- Architecture paper: 5 theorems, arXiv-ready
- Verified Agent Runtime v0.12.2
- Source: github.com/gtsbahamas/hallucination-reversing-system
Verification can’t be baked in
RLVF showed negative scaling: more training data makes models worse at self-verification. Verification must remain external. That is the product.
Full codebase reconstruction works
LVR pilot: 126 files, 354 claims, 27 bugs fixed. Zero scaffolding.
Investing, partnership, enterprise pilot, or research collaboration — we answer our own email.
Get in touchRun it
Three ways in.
MCP / editors
Run Assay inside Claude Code, Cursor, or Windsurf via the lucid-mcp server. Free scans for community members, too.
MCP setup guide ↗Part of Seatrial
Run it on your own repo.
npx tryassay assess /path/to/project