How it works

Two paper-trading systems run on a schedule and publish whether they would be trusted with real money. Start here if you want the idea in one minute.

In one minute

the crypto arena: Three language models and four fixed rules each get $10,000 of imaginary money. Every hour they see the same market data and each submits one trade. Costs and the models' own running bills come out of their returns. The question is whether any model can beat buying bitcoin and holding it.
the equity committee: A $100,000 US stock book. Ordinary code screens about 500 companies each night and proposes buys on the eve of the monthly rebalance. Language models write opinions afterwards. They cannot change a holding. The book only moves on the first trading day of the month.
why both: Different questions. The arena asks whether models can trade. The committee asks whether a locked value rulebook can run itself in public, with models as advisers only.

Checks

neither system passes yet

Until every condition below passes, the verdict is NO-GO and the book remains simulated. That is the expected state for most ideas.

crypto arena

at least 500 hourly decisions on record
a model beats buy-and-hold bitcoin after costs, by a margin that is not chance
no scheduling gaps over 2.5 hours in the last two weeks

equity committee

56 days of tracking and two monthly rebalances completed
no missed research nights
losses stay inside a −25% halt
advisory language-model reviews can be read

Fairness

what stops an agent gaming the comparison

identical inputs: Every participant in a cycle receives the same packet, hashed for reproducibility. Prices, indicators and recent closes are shared; only open positions and past decisions differ, because those belong to the account.
costs are charged: Fills pay spread, slippage and taker fees. LLM agents also pay their own inference cost out of returns, so an expensive model has to out-earn its bill.
risk validated after the fact: Caps on leverage, gross and net exposure, per-symbol notional and aggregate BTC-beta are applied to the action the model returned. An action that breaches one is rejected in code, so a model cannot talk its way past a limit.
point-in-time fundamentals: The equity side lags fundamentals 90 days behind quarter-end, so a backtest cannot use numbers that were not public on the day it claims to trade.

Bugs found

kept public

agents could not close positions: Models emit CLOSE with a size of zero, meaning all of it. The simulator sized every fill from that number, so a close became a no-op and the position survived. 114 close orders did nothing. One agent spent 48 consecutive cycles trying to exit a short and wrote so in its thesis each time, which is how the bug was found.
an hourly system running every two hours: Scheduled CI runs arrived late or were dropped. Measured median gap was 2.02h against a 1h target, worst case 6.14h. Extra staggered schedules did not help, so the scheduler moved to dedicated infrastructure. Cadence is now measured and published on the front page.
a flat equity curve hiding real movement: The book was valued at its last fill price, never at the current close, so it sat at exactly the seeding number for eight days while the holdings moved through a 1.6% range. The drawdown halt reads that number, so it would not have fired.

Cadence

why an hourly system publishes its own timekeeping

The arena is supposed to decide once an hour. For months it did not, and nothing noticed. Scheduled runs on the old infrastructure arrived late or were dropped entirely: the typical gap was 2.02 hours against a 1 hour target, and the worst was 6.14 hours. An agent with 78 decisions when it should have made 168 is being scored on the scheduler.

The fix was to stop using scheduled CI and run a dedicated scheduler. The gap between cycles is now measured and published on the arena page, so the same failure cannot repeat quietly.

Architecture

for readers who want the machinery

scheduler, files, database

scheduler (one container, timezone-aware cron)
   hourly      -> crypto arena cycle
   21:30 ET    -> nightly research pipeline
   08:30 ET    -> monthly rebalance execution

   |-> trading scripts, run as subprocesses
   |      -> JSON / JSONL artifacts          [authoritative]
   |-> sync, idempotent upserts
   |      -> Postgres                        [query layer for this site]
   |-> daily git snapshot                    [archive]

The files stay authoritative and Postgres is a read replica. The arena rebuilds every account by replaying its own event log, so making the database the write path would mean rewriting position reconstruction. If Postgres goes down, trading continues and the next sync fills in the missing rows.

Terms

plain versions of the words used elsewhere

14 definitions

paper trading: Trading with imaginary money against real prices. Every position, fill and profit on this site is simulated. Nothing is held at a broker and no order ever leaves the machine.
agent: A language model given a situation and asked to decide what to do about it. In the crypto arena the models genuinely decide. In the equity committee they only advise, and ordinary code makes the decisions.
baseline: A simple fixed rule, like buying bitcoin and never selling, used as the thing to beat. If a language model cannot beat a rule this basic, it is not adding anything worth paying for.
benchmark: The particular baseline a system is graded against. The arena's is buy-and-hold bitcoin, because any strategy that cannot beat it would have been better off doing nothing.
cycle: One decision round in the crypto arena. Every participant sees the same market data and submits one action. Cycles run hourly, so 24 a day.
net of costs: After the money it cost to trade has been subtracted. Fills pay spread, slippage and commission. Language models are also charged what they cost to run, so an expensive one has to out-earn its bill.
inference cost: What it costs to ask a language model a question. It is charged against that agent's returns. A model that trades constantly can therefore lose to one that sits still.
sharpe ratio: Return measured against how much the value swung around to get it. Above 1 is respectable, below 0 means the risk taken was not rewarded at all.
drawdown: The worst fall from a peak. A book that reached £110,000 and now sits at £99,000 is in a 10% drawdown, whatever it started at.
turnover: How much was traded relative to the size of the account. High turnover means costs eat the returns.
book value: What the whole account is worth: cash plus the current market value of everything held.
rebalance: Selling what no longer qualifies and buying what does. The committee does this once a month, not continuously, because trading costs money.
point-in-time: Using only what was actually known on the day. A company's results for a quarter are not public the moment the quarter ends, so the committee waits 90 days before acting on them. Skipping this makes a strategy look far better than it was.
GO / NO-GO: Whether a system has met every published condition for being trusted with real money. Both currently read NO-GO.