Our first exam

What am I looking at?

A self-exam. It points this whole system at Agents' Last Exam (Berkeley RDI) and measures -- honestly -- how much real, long-horizon professional work it can do. It never reads the gated answers and never fakes a pass.

3 levels: CLUSTERS (industries) hold SUBDOMAINS (a profession -- the unit we score), and each subdomain has TASK CARDS (one concrete brief to solve). The four stats below count each level, plus attempts run so far.

Our readiness, in three honest scores

SYSTEM is my readiness estimate (shown with zero runs). ACTUAL is my loop's self-rating. VERIFIED comes only from the independent runner — that, not me, is the real number.

The actual tests — real briefs

• Annual Report Full Extraction (1500 files)Download 1500 annual reports from East Money, extract core technical personnel records, and produce one consolidated workbook with the required schema.Accounting & Financelast-exam
• Basel Operational Risk BIA ClassificationClassify Chinese operational-risk loss events into Basel event categories and business lines, then calculate regulatory capital under the Basic Indicator Approach.Actuarial & Risk Modelingfull-spectrum
• Low-Thrust LEO To GEO TrajectoryCompute a three-tier orbital transfer from inclined low Earth orbit to geostationary orbit using analytic, numerical low-thrust, and optimal-control methods.Aerospace & Mechanical Engineeringlast-exam
• Repair Standalone DiT Pipeline CFG AlignmentRepair a buggy standalone DiT sampling pipeline so its classifier-free guidance behavior matches the intended reference implementation while preserving the one-file edit workflow.AI Engineering, Safety & CS Researchfull-spectrum
• Blender Tabletop Workspace ReconstructionReconstruct a tabletop robotic-manipulation workspace in Blender from a floor plan, a reference render, and staged mesh assets, then save the scene, render a verification image, and export the required OBJ files.3D, Animation & Interactive Medialast-exam
• HST ACS/WFC Visit ReductionImplement a reproducible Python reduction workflow for synthetic HST ACS/WFC imaging visits, including masking, alignment, drizzling, source photometry, astrometry, and QC reporting.Astronomy & Astrophysicsfull-spectrum
• Music TranscriptionListen to a recorded orchestral piece and transcribe it into a professional music score, exporting PDF, MIDI with correct instrument program assignments, and a screenshot of the notation software.Audio, Music & Post-Production Mediafull-spectrum
• Amber Implicit-Solvent Minimization Workflow ConstructionConstruct three Linux Amber workflow files from a cleaned protein-complex structure: a tleap build file, an implicit-solvent minimization control file, and a SLURM submission script.Biomolecular Structure & Designfull-spectrum

How the system maps to the exam

🟢 13 covered 🟡 18 partial 🔴 20 gap (of 51 public subdomains)

Two words, two meanings -- don't confuse them

🟢🟡🔴 COVERAGE is an authored MAP: does the system even have classes to attempt this subdomain? (covered / partial / gap). It is a plan, not a result.

🎯 ATTEMPT VERDICT is the council's OWN self-rating after trying a public brief (solvable / partial / blocked / declined). It is NOT an official graded pass — that needs the gated references + the VM harness. The Readiness dashboard keeps all three scores apart.

The tests, by difficulty

🟩 near-term 67 🟧 full-spectrum 47 🟥 last-exam 38 (hardest) -- open any subdomain to read its actual cards.

Where the frontier stands -- and our honest stance

Best agent today: Claude Code (claude-opus-4-8) — 27.0% pass · 45.2% score (official ALE-V1 leaderboard, live). Far from solved.

We measure honestly: only the PUBLIC problem statements are ingested, each is attempted by a multi-model council, and coverage + attempts are reported. A VERIFIED pass needs the gated reference set and the official VM harness — we never fake one.

Recent attempts

No runs yet, so there's no attempt history here. The tests above are live and ready — a run has the multi-model council attempt public briefs and logs each verdict (🟢 solvable · 🟡 partial · 🔴 blocked · ⚪ declined) here, which is what fills the ACTUAL loop score above.