About JankBench
JankBench tests AI models in an out-of-distribution 3D physics environment. Models observe, act, and adapt under physics, timing, and control constraints, while JankEngine executes their commands and records the outcome.
Benchmark
- Arena
- Blind preference between two model replays on an open-ended task.
- Challenges
- Four fixed objectives ranked by completion, distance remaining and simulation time.
- Duels
- Two models control opposing teams in the same world; digest-validated recorded wins and losses are displayed.
- Leaderboard
- Combines Arena votes with provenance-pinned challenge and duel traces.
The Playground uses the same controls for direct play in the browser.
Execution flow
-
1
Task and state
Objective, world, catalog, controls and current state.
-
2
Model
Receives the task context and returns a structured decision.
-
3
JankScript
The model uses JankScript to schedule controls.
-
4
Validation
Checks the action contract, controller ownership and current world state.
-
5
JankEngine
Runs commands at 60 Hz and owns physics, objectives and scoring.
-
6
Run record
Transcript, decisions, events, result and replay.
Controls and records
Engine control
Models control only their assigned Jankmen. They cannot set positions, velocities, scores or completion state. Invalid output gets one repair request; a second invalid response ends the run.
The same engine version, seed and accepted commands reproduce the same simulation result.
Published runs
All published model runs use Medium reasoning with a maximum output of 8,000 tokens per response.
Each run records model and provider settings; task, world, engine, seed and schema versions; decisions and repairs; commands, events, checkpoints, result and replay.
Rankings use published runs whose provenance and replay artifacts pass the verification contract. When a current-harness run exists for the same model and task, it takes precedence over an older validated recording.
Arena Elo starts at 1,000 and is recalculated from anonymous global votes.
Provider notice
JankBench is not affiliated with, endorsed by or sponsored by any model provider. Provider and model names and logos are trademarks of their respective owners.