Decentralized Leaderboard
The Forge Matrix™
A crowdsourced, tamper-proof model reliability leaderboard built from real /break runs across the global Forge network. No editorializing. No sponsorships. Just data.

What It Is

The Forge Matrix aggregates Break and Assurance results from Forge installations worldwide into a 3D visualization of model reliability. Every data point is Ed25519-signed and verified server-side before acceptance. Fleet consensus with outlier detection prevents manipulation.

How It Works

Reports are uploaded to the Matrix when telemetry_enabled: true is set in the user's config. This is an explicit opt-in — Forge never sends data without consent. The --share flag on /break commands signals upload intent, but telemetry must be enabled for it to work. The verification server checks the Ed25519 signature, validates the tamper-evident hash chain, and adds the report to the index. No unsigned or tampered data enters the Matrix.

What Makes It Different

Traditional benchmarks measure capability in controlled environments. The Matrix measures reliability in the wild — how models actually behave when real developers push them with real adversarial scenarios on real hardware. It's the difference between a lab test and a field test.
  • MMLU — Massive Multitask Language Understanding. Academic knowledge benchmark across 57 subjects.
  • HumanEval — OpenAI's code generation benchmark. 164 hand-written programming problems.
  • LMSYS Chatbot Arena — Crowdsourced human preference ranking via blind A/B comparisons.

These measure what a model can do. The Matrix measures what a model does when you try to break it.

Data Integrity

  • Proof of Inference — Ed25519 challenge-response proves real inference occurred
  • Signature verification — every report is cryptographically signed
  • Hash chain validation — per-scenario tamper-evident linking
  • Fleet consensus — outlier detection flags anomalous submissions
  • Protocol versioning — scores are interpreted relative to the scoring protocol that produced them