JumalAIta IntelligenceIndependent evaluation

Changelog

Changelog

A dated record of what changed on this site: new benchmarks, new results, methodology updates and corrections. Version histories of individual benchmarks are on the benchmark pages.

  1. Results

    Claude Fable 5.2 ranked first

    Claude Fable 5.2, run in Claude Code, was placed first by the original authors. Every other model moves down one place.

  2. Benchmark

    Creative Synthesis Benchmark introduced

    A new task: research Jumalauta and jml-engine-webgl, then create an original audiovisual demo for expert ranking. The prompt and protocol are published; no results have been published yet.

  3. Results

    Mistral Medium 3.5 ranked 12th

    Mistral Medium 3.5, run in Mistral Vibe CLI, was evaluated and placed last by the original authors.

  4. Results

    Claude Haiku 4.5 evaluated

    The original authors ranked Claude Haiku 4.5 sixth. Updated ranking: 1. Claude Opus 5.5, 2. Claude Fable 5.1, 3. GPT-6 Astra, 4. GPT-6 Sol, 5. Claude Opus 4.7, 6. Claude Haiku 4.5.

  5. Results

    Claude Sonnet 5 evaluated

    The original authors ranked Claude Sonnet 5 sixth, above Claude Haiku 4.5. Updated ranking: 1. Claude Opus 5.5, 2. Claude Fable 5.1, 3. GPT-6 Astra, 4. GPT-6 Sol, 5. Claude Opus 4.7, 6. Claude Sonnet 5, 7. Claude Haiku 4.5.

  6. Results

    DeepSeek V4 Pro ranked, Gemini 3.8 Flash disqualified

    DeepSeek V4 Pro was ranked tenth. Gemini 3.8 Flash downloaded the original demo during its run and was disqualified; its output is published, marked as a rule violation, without a rank.

  7. Results

    Grok 4.7 ranked, DeepSeek V4.1 Flash disqualified

    Grok 4.7 was ranked eighth. DeepSeek V4.1 Flash read the original release's info file on pouet.net during its run and was disqualified; its output is published, marked as a rule violation, without a rank.

  8. Results

    Kimi K3 and Qwen3.8 Max evaluated

    The first models from Chinese labs, each run in its maker's own agent harness. Updated ranking: 1. Claude Opus 5.5, 2. Claude Fable 5.1, 3. GPT-6 Astra, 4. Kimi K3, 5. GPT-6 Sol, 6. Qwen3.8 Max, 7. Claude Opus 4.7, 8. Claude Sonnet 5, 9. Claude Haiku 4.5.

  9. Results

    All versions side by side

    One video with the original and every reimplementation in rank order, in sync, on the benchmark page.

  10. Results

    GPT-6 Sol evaluated

    The original authors ranked GPT-6 Sol fourth. Updated ranking: 1. Claude Opus 5.5, 2. Claude Fable 5.1, 3. GPT-6 Astra, 4. GPT-6 Sol, 5. Claude Opus 4.7.

  11. Results

    Claude Opus 5.5 evaluated

    The original authors ranked Claude Opus 5.5 first. Updated ranking: 1. Claude Opus 5.5, 2. Claude Fable 5.1, 3. GPT-6 Astra, 4. Claude Opus 4.7.

  12. Benchmark

    Creative Reimplementation Benchmark v1.0

    First publication of the benchmark: three models, one prompt, single shot, ranked by the original authors of the reference demo.

  13. Results

    Full ranking published

    The original authors of the reference demo ranked the three reimplementations: 1. Claude Fable 5.1, 2. GPT-6 Astra, 3. Claude Opus 4.7. Their notes are on the benchmark page.

  14. Results

    Recordings added for three models

    Side-by-side comparison recordings published for Claude Fable 5.1, Claude Opus 4.7 and GPT-6 Astra.

  15. Site

    Site published at ai.jumalauta.org