Split Decision: Nine AI Judges, Two AI Journalists, and a Verdict You Can Verify

This article introduces Split Decision, an AI experiment simulating Supreme Court deliberations to test whether multi-agent debate improves decision accuracy.

Honorable Mention & Blog Post Award, Global AI Hackathon Series with Qwen Cloud

cover

Project Introduction

Here's a premise that sounds like satire and plays like civic technology: nine AI judges, each holding a different judicial philosophy, deliberate real Supreme Court cases. Two AI journalists turn every deliberation into a podcast episode and a pixel-art courtroom replay. And every prediction is scored against the real Court's decision, so the whole thing is secretly a rigorous experiment asking one question — does multi-agent debate actually beat a single model? This is Split Decision.

Project Value

Right now, everyone is wiring multiple AI agents together and assuming the committee is smarter than the individual. Almost nobody measures it. Worse, agents in a group tend to agree with each other fast — sycophancy — which can turn a "debate" into an echo chamber that just sounds thoughtful.

Split Decision tests the assumption on a hard judgment task with public, verifiable answers: Supreme Court outcomes. It only scores cases decided after the model's training cutoff, so the model can't have memorized the answers, and it puts predictions on pending cases on the record so reality grades them later. There's also a legibility payoff. AI reasoning is usually invisible. Here you can hear machine minds disagree, watch votes flip, and read a verbatim record of who convinced whom — which makes it civic education about both courts and AI at once.

Solution

The pipeline is a courtroom in software. A Clerk agent reads case data from CourtListener, Oyez, and the Supreme Court Database and writes a bench memo. Nine juror agents cast secret ballots, argue in rounds, debate in pairs chosen by a Foreperson, and revote until a 7-2 supermajority emerges or round five is reached.

The anti-sycophancy engineering is the clever part: private votes before anyone speaks, personas re-injected every round so judges don't drift into agreement, and written rules governing when each judge is even allowed to change its vote. Every utterance, ballot, flip, and verdict is written to one schema-validated event log. That single log feeds everything downstream — a scoreboard grading the panel against the real Court (solo model vs. silent jury vs. full debate vs. neutral control), a two-anchor podcast that plays only verbatim clips, and a Phaser pixel courtroom with a live transcript.

And then, the finding that makes this project intellectually honest rather than promotional: debate made the panel more articulate, not more accurate — 66.7% either way across 24 paired cases. The committee didn't outthink the individual. It just talked a better game.

Highlights

Qwen Cloud models do all the reasoning: nine jurors on qwen3.7-plus, the Foreperson and Clerk on qwen3.7-max, all via the OpenAI-compatible DashScope endpoint. Qwen Voice Design created twelve distinct voices from text prompts — nine judges, a foreperson, two anchors — synthesized with qwen3-tts-vd, and episode art was generated with wan2.6-t2i.

Alibaba Cloud Simple Application Server runs the whole pipeline: nginx serves the courtroom app, systemd runs episode production, and a small HTTP service powers the Live Bench, where anyone can convene a real deliberation on demand. Alibaba Cloud OSS stores the podcast MP3s, the RSS feed, and episode art — copied there immediately, because model-generated image URLs expire in 24 hours. Every API call's token usage is logged to costs.jsonl as a full audit trail, in keeping with the project's measure-everything ethos.

Alibaba Cloud Products Used

About the Team

Tarik Moody is the Director of Strategy and Innovation for Radio Milwaukee. He built Split Decision entirely on his own — not for his company — as an experiment in whether machine deliberation can be made visible, measurable, and genuinely civic. The answer, it turns out, is more interesting than a simple win: the agents learned to argue beautifully without getting any better at being right.

Explore the project: https://split-decision.tarikmoody.com/

This project was developed as part of the Global AI Hackathon Series with Qwen Cloud.

Community

Split Decision: Nine AI Judges, Two AI Journalists, and a Verdict You Can Verify

Project Introduction

Project Value

Solution

Highlights

Alibaba Cloud Products Used

About the Team

Read previous post:

Read next post:

Alibaba Cloud Project Hub

You may also like

Comments

Alibaba Cloud Project Hub

Related Products

Token Plan

QwenWork

Qwen

Alibaba Cloud Model Studio