Setlist Prophet

Each version's ten highest-confidence picks, ranked by its own logic, scored on the 1994 night. Ten points for the top pick down to one for the tenth, 55 available.

Where each model actually scored — by how hard the song was to find

Every song #94 played has a rank inside the eligible pool — its position once the 1994 play counts are sorted and the 50 songs burned on earlier nights are struck out. Sorting the night that way splits it three ways, and the split is the whole story of this project.

What the chart shows, and why two numbers moved

Forbin v7 caught the top-20 band exactly — all ten — and nothing outside it. That is not a coincidence, it is what the rule is: Forbin v7 picks the top 20 of the eligible pool, so it hits every played song in that band by construction and can never reach past it. The frequency core is a solved problem, and it was worth 10 of 16.

Every point of disagreement between these models lives in the six songs outside the top 20. The best any version managed there was two, and the whole reason Forbin v8 exists is to reach them. So look at what that reaching cost.

Two numbers on the table above are lower than previously published, and the correction is worth naming. Earlier this page showed Forbin v5 at 8/20 and Forbin v8 at 12/20. Those slates were assembled by hand after the setlist was known. The chart runs each rule mechanically instead — Forbin v5 as top 8 plus twelve from the 20–40% depth band, Forbin v8 as Forbin v7's top 17 plus three reserved slots — and applied blind they score 6 and 9. Forbin v6 and Forbin v7 are unchanged at 9 and 10. A rule is only worth what it scores when it is not allowed to peek.

What the ladder table shows

On the ten-pick ladder the three real bots land within three points of each other on a 55-point scale. Forbin v5, Forbin v6 and Forbin v7 disagree about method loudly and about outcomes barely — because they share a frequency core that does essentially all the work. The arguing this week was over the last few slots.

Forbin v6 would have beaten Forbin v7. The version retracted hours before showtime — cover quota, album-year reasoning, the one I called most likely to look foolish — scores 43 to Forbin v7's 42, because it carried Tweezer high and had Wolfman's Brother. It is a fitted number, but the ranking is uncomfortable and worth keeping visible.

Forbin v8's 54 is meaningless and is listed only for completeness. It picks Back in the U.S.S.R., Purple Rain and Catapult — three songs already known to have been played. A bot built after the fact will always win a contest scored on that night.

Settled since, at scale. All five versions have now run against 100 held-out shows: Forbin v9 828 · Forbin v7 806 · Forbin v8 765 · Forbin v6 699 · Forbin v5 592, random 309. The #94 ordering was not noise after all — Forbin v8's reserved slots lose 41 songs to plain frequency and lose head-to-head on 43 shows against 11. Forbin v6, Forbin v7 and Forbin v8 post identical ladder scores there, because they share a frequency head and differ only in slots worth no ladder points. Full results →

The honest reading of this table alone: on #94 evidence there is no demonstrated skill gap between the model versions, only a large gap between all of them and random. See the scorecard for why beating random is a weaker claim than it sounds.

Setlist structure the song list does not capture, counted the same way every night so it can be tracked rather than admired.

One sandwich a night, three for three

Every night of this run has produced exactly one, never two and never zero. That alone is worth pre-registering.

On both nights without a fest, the filling was that night's deepest bustout. The Vibration of Life had gone 1,027 shows and came back inside a Bowie. Big Ball Jam had gone 1,170 and came back inside a Tube. Those are the two largest gaps of the entire run, and neither was played on its own — both were delivered wrapped. On #94 the fest took that role instead, and the sandwich fell to a pair of Down with Diseases around a Tweezer.

Pre-registered for #96: one sandwich each night. If a night produces no fest, expect the filling to be that night's largest gap rather than a rotation song — which means the bustout call and the sandwich call are the same question asked twice.

The rest of the deep end

Deep bustouts have run 3, 4, 3 — the range the called-shot uses. What the count hides is the spread: #92 and #93 reached past a thousand shows, #94 topped out at 659. The 1990s-themed nights are pulling from genuinely different depths, and the earlier the theme year, the deeper the reach — which makes sense, since a 1992 rotation contains more songs that have since fallen out of the band's hands entirely.

Why “beats random” is a weaker claim than it sounds

The comparator throughout is a random 20 songs from the same eligible pool. That is a fair pool — but it is not the baseline a competent fan represents. The natural human baseline is “the twenty most-played songs of the theme year, minus whatever they already played this run” — and that is Forbin v7. Beating random therefore demonstrates two things any phish.net-literate fan had for free: the theme-year premise, announced publicly by the band, and the no-repeat rule.

The model's edge over an informed fan is currently indistinguishable from zero, because the model is what an informed fan would do. Every cleverer idea tried this week lost to the plain frequency list.

Why tonight's score will not be comparable to the others

86 songs are burned and 87 remain once The Curtain is struck as a duplicate of The Curtain With, the thinnest pool of the run, so blind picks score better tonight than on any earlier night — chance alone expects 4.5 of 20. Two effects now pull in opposite directions: the shrinking pool lifts the baseline, and it also strips out the high-frequency material the model is best at, which measurably cuts the edge from 2.5× to about 1.3×. Read tonight as a multiple of chance, never as a raw count. Judge Forbin v9 against the random baseline for its own night.

The like-for-like improvement is 6 to 10, Forbin v5 to Forbin v7 on the same night. And Forbin v8 does not improve on Forbin v7 at all — run as a rule rather than assembled with hindsight it scores 9, because its three reserved slots cost three certain top-20 hits to buy two unreachable ones. The 12/20 previously shown here was built after the fact and has been withdrawn.

The costume album — what Saturday actually proved, and the wrong turn taken after it

Phish's Halloween tradition is to cover an entire album in costume. #94 opened with Back in the U.S.S.R., a White Album track — and the White Album was the Halloween 1994 costume. That is one observation, and it is worth reading carefully, because the first reading of it here was wrong.

The error was to ask “did the costume songs enter the following year's rotation?” That is a question about 1990s setlist mechanics, and the answer is mostly no. But it is the wrong question, because the song they actually played never entered any rotation at all:

Four lifetime plays: 31 Oct 1994, 6 Dec 1994, 23 Oct 2013, and Saturday. Frequency ranked it at zero. It was chosen because it was on that year's Halloween album and for no other reason — so rotation data cannot explain it, and the audit built on rotation data was answering a question nobody asked. The rule is not about what a costume album contributed to the repertoire. It is about the band, in 2026, nodding to that year's Halloween show.

Two consequences follow, and both reverse what this page said an hour ago:

1. The album is the themed year's own, not the previous year's. The 1994 night took the 1994 costume. So #95 looked to Quadrophenia (Halloween 1995) and tonight looks to Remain in Light (Halloween 1996). Drowned was carried on #95's ladder at rank 7 as a declared override — Forbin v10 itself ranked it near zero — and it hit, taking four ladder points and the night's longest jam. While My Guitar Gently Weeps is off both slate and ladder: 12 plays puts it 21st in the eligible pool, one place outside the cut.

2. Frequency is not the selector inside the album. They passed over the 38-play repertoire song for the 3-play album opener. #95 therefore carried Drowned (48 all-time, the only Quadrophenia song that stuck) while noting the rival reading: Quadrophenia's opening vocal is The Real Me, 2 lifetime plays, unplayed in 1,176 shows — the exact shape of Saturday's pick. Wednesday carries both readings openly: Crosseyed and Painless, and Born Under Punches, Remain in Light track one at one lifetime play.

The sharper test: costume set → heavy rotation the next year

A better way to ask the question, and one that requires the frequency evidence instead of assuming it: which songs were played in a Halloween costume set and then played a lot the following year? That is computable against the whole catalogue, and the answer is startlingly thin.

One song in the whole catalogue passes this test, and it is While My Guitar Gently Weeps. And even it falls one place short — 12 plays ranks 21st in the eligible 1995 pool, just outside the twenty. The Quadrophenia set is the opposite result and it could not be cleaner — every one of its fifteen tracks was played exactly zero times in 1996. Whatever happened with the White Album did not happen with Quadrophenia at all, so this test hands Wednesday nothing.

One discipline point that follows from it: for While My Guitar Gently Weeps the costume effect is already priced in. The 12 plays are not evidence sitting alongside the frequency number — they are the frequency number, and boosting the song above 14% would count the same fact twice. Drowned is the reverse: its two 1995 plays contain no information about the costume link at all, which is why it carries the top ladder slot at 45% and While My Guitar Gently Weeps does not carry one at all.

Stated honestly: this is n=1. One night, one song. It was enough to put a 2%-frequency song at rank 7 on #95 and not more, and #95 was a clean test of it — Drowned was carried by nothing except this rule. It landed, so the rule is 2 for 2 and n=2. Tonight's Crosseyed override is the third and final test this run offers.

Crosseyed at 75% — and what the gap objection got wrong

The first version of this page put Crosseyed and Painless on Wednesday's slate but kept it off the ladder, reasoning that a 7-show gap was the wrong shape for a costume pick next to U.S.S.R.'s 534. That objection does not survive contact with the data. U.S.S.R. was chosen for the opener — a slot where a bustout is the point. Nothing about it says the band avoids a current song for a mid-set nod, and the two picks are not competing for the same job.

Six of Crosseyed's eight MSG appearances fall on 29 December–1 January — the marquee nights of the New Year's runs. It is the band's big-statement song in this building, and Wednesday closes a five-night run on the night themed to the year the song was born. That case stands on its own without the costume rule; the costume rule then adds to it. It goes to the top of Wednesday's ladder at 80%.

Consistency then forces a second change. If the same-year nod is near-certain on Wednesday, it cannot be a coin flip on Monday — Quadrophenia is the Halloween 1995 costume and Drowned is its only survivor, 48 all-time against 4 for the next-best track. Drowned went from 30% to 45% and took the override slot on #95 — and hit. It is held under Crosseyed for one honest reason and only one: Crosseyed is a living song at 8% a night, while Drowned has not been played since October 2023 and runs about 2%. These two are now the largest positions on the page, and both are judgement, not measurement.

Still true, and unaffected: this is not an opener rule. Having just opened #94 with a cover, a cover opener two nights running is less likely, not more. The Real Me and Born Under Punches are not opener calls. Openers revert to frequency.

What each night taught

#92 → run no-repeat. The played nights share zero songs. The cold blind guess for #93 collapsed to 2/20 because six of its picks had been played the night before.

#93 → rank inside the eligible pool, never the full year. Forbin v5's “mid-depth band” was an artifact of ranking within the full theme-year pool. Worth remembering: that same Forbin v5 was sold here as “held out, not fitted” at 19/40, then scored 6/20 live. That gap is the best evidence for how much any backtest here is worth, including Forbin v8's.

#94 → the costume idea, on one observation, now corrected for the year lag.

“Era-narrative wins the marquee slots” has a live record of 0 for 4

Every called shot on #94 missed. The narrative reading of that night is an explanation fitted afterwards, not a prediction that landed. Forbin v8 spends 3 of 20 slots on it anyway, re-admitting an idea this project measured and rejected a day earlier. That was a bet, and #95 exposed it: Forbin v8 was retired before the night ran, on 100 simulations that had it 41 songs behind plain frequency.

What the numbers audit found — Sat 25 Jul, after the show

Every play count on this page was re-pulled from the Phish-only feed and checked. The 1995 slate came back clean: Free 32, Strange Design 29, Theme from the Bottom 27, Acoustic Army 27, You Enjoy Myself 25 — all confirmed. The 1996 slate did not. Its counts were inflated across the board — Character Zero shown at 30 against a true 21, the Star-Spangled Banner at 17 against 10, Swept Away and Steep at 18 against 13. That whole night has been rebuilt from verified numbers, its expected ladder score falling from 16.8 to 14.9, and its opener call moving from the Banner to Wilson on a corrected count of 1996 set-one openers.

The lesson is narrower than it looks and worth stating plainly: the failures on this page have been arithmetic and premature publication, not modelling. A wrong number presented confidently is worse than a hedge. And the costume-rule reversal above is a third kind of error again — not bad arithmetic but a well-executed answer to the wrong question, which is the harder one to catch, since every number in it was correct.

Still standing

Do not filter on dormancy — each night runs a current-rotation core plus 3–4 deep bustouts. Secret Language is collinear with play count. Skipping the obvious loses hits every way it was cut. Bustouts are worth predicting as a count, though “3–4” is a range drawn around the only three observations that exist.