ACX Contest x Manifold
The unofficial side event to the annual forecasting tournament
The Astral Codex Ten annual forecasting competition was my introduction to the world of forecasting, several years back. It’s a mainstay of the forecasting space, the largest and most venerated spot-scoring tournament — where you make your forecasts once at the beginning of the year and then check back at the end for your score — in at least the English-speaking world. It’s now hosted on and run by Metaculus, where you can register your predictions and compete at the start of the year.
But while spot-scoring is fun, and lets you reward accurate forecasts rather than constant updating, some people (like you, the reader) like to forecast and trade on markets all year long. And there’s even an internal Manifold leaderboard for the traders that earn the most on this question set.
All these questions resolve at the end of the year, and Scott Alexander with Metaculus has done a great job of creating a diverse question set of fairly significant questions.
Currently, the market drawing the most attention and traders is on whether an AI-created song will chart in the top 20 of the Billboard Hot 100 this year.
Traders are heavily divergent on this, with #1 on the all-time leaderboard, SemioticRivalry and one of the Manifold founders, Ian, staking out massive YES positions against the #24 all-time trader, Panfilo.
While AI-generated songs have reached as high as #1, on other charts, there is discussion as to whether Billboard would allow an AI song to chart, or selectively exclude it from the rankings. AI-generated music hasn’t yet had the kind of viral moments as text, image, or video-generation, but this is probably due to the nature of how users share this content. While images and videos can go viral from anyone, songs often require a bit of lift-off. This can happen organically through things like TikTok audio clips, but generally a platform is required, and artists are hesitant to turn over creative control to the AI to the extent that a song could meaningfully be described as “AI-created,” rather than, perhaps, utilizing AI to create a single track that might be imperceptible to the listener.
The market remains at 31%, well below the aggregated forecast on Metaculus. The divergence between the markets and the aggregated forecast is interesting here, as typically Metaculus tends to be more conservative in the direction of things-not-happening.
Another AI capabilities market, on whether an AI model will reach a 3-hour time horizon at 80% reliability this year on METR’s benchmark, seems all but certain at this point.
The remaining uncertainty centers around whether METR will change their methodology or stop adjudicating models by this benchmark due to saturation or other priorities.
There have also been some large swings this year on the market on whether OpenAI will manage to file for (not complete!) an IPO this year.
This market trades a few percentage points above similar real-money markets on whether OpenAI will complete their IPO by the end of December, as filings typically happen anywhere from 15 days to six months in advance of the actual offering.
A couple of questions recently resolved, such as NASA’s Artemis II completing its mission successfully, and SCOTUS somewhat surprisingly issuing a firm ruling that Trump’s emergency tariffs were unlawful.
There are plenty of remaining questions that have split the Manifold community. For example, Keir Starmer’s fate has traded between 40% and 60% for most of this year:
And in lieu of any public statements by Supreme Court justices Alito or Thomas, the best we can do is this prediction market on whether SCOTUS will have a retirement this year before the midterms, which Metaculus prices decently lower at 24% and real-money markets place higher, above 40%!
On some of these questions, with just a couple dozen traders, there’s room for newbies to stake out a claim for themselves. For example, my favorite question in the tournament, on Montenegro’s accession to the EU, is one of the least-traded markets in the set!
Do none of you care about EU expansion? Croatia was the last country to join back in 2013; a lot has happened in the world since then. Montenegro, which began the year with just 12 negotiating chapters closed, has set the ambitious goal to close the remainder of these chapters this year, and hopefully join the EU by 2028. This is the EU equivalent of trying to walk 100,000 steps in one day.
The world’s second-most deadly ongoing conflict, in Sudan, might finally reach a ceasefire this year, as well, with a 25% chance of at least a temporary end to the civil war. Perhaps more forecasters should devote their analytical attention to this:
There’s also a meta-question in the ACX tournament, on how well the top bot will place, relative to the top 5 humans.
Forecasters expect that the top bot will be about as good as last year’s 92% score, which some forecasters viewed as anomalously high. They give a 24% chance that the best bot does as well or better than the average of the top 5, though. Someone should have made a bot to trade on the ACX question set, so we could see how it matched up on the same question set in a different modality. Currently no bot ranks on the leaderboard:
These markets have large liquidity subsidies and many will come down to the wire, so there’s still time to win the Unofficial Manifold x ACX 2026 contest. Special thank you to Manifold user “walmart” for creating all the markets.
Get those limit orders up and happy forecasting!
-Above the Fold