Quotient CEO Jordan Olmstead argues that publicly cited probabilities need an audit trail, and says his AI forecasting agent Q should be judged against the markets it tracks.
Prediction market odds are turning up in headlines alongside polls and financial data. A price can suggest what traders expect to happen, but it does not tell readers whether the market is liquid, whether its participants are acting on independent information, or how reliable similar odds have been in the past.
Sponsored
Crypto Prediction Markets
18+ · Gambling involves risk. Play responsibly.
That gap is where Quotient wants to work. Its AI forecasting agent, Q, researches questions with defined outcomes and deadlines, publishes probabilities with supporting analysis, and compares its forecasts with prediction market prices after the events resolve.
Jordan Olmstead, Quotient’s CEO, argues that a market price can be useful without being the final word. In one market on whether traffic through the Strait of Hormuz would return to normal by May 31, Q assigned a 3% probability when Polymarket’s price implied 68%. The market ultimately resolved NO. But Olmstead also acknowledges that the model does not outperform every category or every measurement period.
In this interview with DailyCoin, Olmstead discusses how journalists should read prediction market odds, what Q’s results do and do not show, and why AI agents may need forecasts whose accuracy can be checked.
What did you see happening in prediction markets that made you believe Quotient needed to exist?
Prediction markets give us clear questions, deadlines, prices to compare against, and outcomes we can score. That makes them a useful way to test Q’s forecasts and improve the model.
We turn that work into research and intelligence that help people make better-informed decisions. Q forecasts world events and asset prices, with supporting analysis people can examine. Our thesis is that prediction markets can be a leading indicator for real-world asset prices, and a calibrated forecasting system can be a leading indicator for prediction markets themselves.
Prediction market odds are increasingly quoted alongside polls and traditional market data. Are they being given more credibility than they deserve?
Sometimes. The promise is that people bring different information and independent judgments, and the price brings those views together. But participation does not necessarily mean independent thinking.
Traders may be reading the same headlines, seeing the same posts, or following the same people. A price move can look like broad agreement when the underlying judgments are closely connected.
Prediction market odds are prices. Liquidity, incentives, and whoever happens to be trading can all move them. They can be informative, but we should be careful about treating them as the complete wisdom of the crowd. If those odds are going to be treated as data, they should be scored like data.
What should someone check before citing a prediction market probability?
Start with the exact question: what has to happen, and by when? Then check the venue and when the price was recorded. A probability on its own does not tell you how much to trust it.
Look at the trading behind the price, too. Could one trade move it substantially? If so, readers should know that. And where a track record is available, ask how accurate prices on similar questions have been. That gives the number some context.
How does Q arrive at its own probability, and how do you test it against the market?
Q starts with the question, the deadline, and what would count as a YES or NO outcome. It researches the evidence, considers how events could unfold, and weighs information supporting or challenging each possibility. It then produces a probability with supporting analysis.
We record Q’s forecast and the market price at the same moment. When the market resolves, we score both and publish the comparison. We use the Brier score, a standard measure of probability accuracy. Lower is better, and confident mistakes carry a larger penalty.
In our rolling 60-day comparison, we average the scores for each market and then give every market equal weight. That way, a market we forecast frequently does not dominate the results. A forecast can be useful whether Q agrees with the market or sees a different outcome; our Signals product focuses in particular on those differences.
Over the 60 days ending August 19, Q recorded an 8.44% lower Brier score than forecast-time Polymarket prices across 965 forecasts on 172 resolved markets. What does that result tell you?
It tells us Q had lower average error in that sample of covered geopolitics and global-election markets. It does not mean Q always beats the market.
The results change. In the September 24 rolling 60-day comparison, Q was ahead across all covered, resolved Polymarket markets, with a Brier score of 0.1655 versus 0.1691. But on geopolitics and global elections, Polymarket was ahead: 0.1289 versus Q’s 0.1338.
Looking at the categories separately helps us see where Q adds information and where it needs work. If we only published the good windows, the scoreboard would be worthless. We want people to evaluate the record as it develops.
On April 20, Q put the probability of Strait of Hormuz traffic returning to normal by May 31 at 3%, while Polymarket priced it at 68%. What was Q seeing?
It saw a logistics and timing problem. For that market to resolve YES, the seven-day average had to reach 60 ship transits a day by May 31. Even a diplomatic agreement would have to translate into ships moving through the strait at that rate.
Q considered insurance costs, unfinished mine clearance, and how long it would take fleets rerouted around the Cape of Good Hope to return. Restoring traffic required coordination among shipowners, insurers, and others responsible for safe passage. Those steps could take longer than the deadline allowed.
Across its 16-forecast record, Q stayed at or below 20% and never changed its NO call. The market resolved NO. The useful part of the research was connecting the political developments to the practical steps required for traffic to recover.
How should journalists and traders interpret the price of a thinly traded market?
Liquidity matters because it affects how easily a trade can move the price. But it does not, by itself, tell you whether the forecast is accurate. Kalshi’s research found that forecasts generally improved with more trading, while lightly traded markets could still be informative. A busy market can also be wrong.
Related markets can help. A market on a party’s vote share, for example, can be read alongside one on who becomes prime minister. If they seem to tell different stories, investigate why. It might be new information, a different deadline, or different conditions for the outcome.
For journalists, name the market, give the time and exact question, and say if trading is thin. Add a track record for similar questions if one is available. A market price is not a poll, and a trader count alone does not tell you how much to trust it.
As AI agents begin using financial data to make decisions and execute trades, where does Quotient fit?
Machine-readable probabilities need measurable track records. An agent should be able to retrieve a forecast, understand the question and deadline, inspect the evidence, and check how the source has performed. That gives developers a basis for deciding how to use the information.
Traders can use Q’s forecasts and Signals to evaluate opportunities. Readers can use the research to understand the events moving markets. Developers can bring Q’s forecasts and supporting evidence into their products through our API, CLI, and MCP tools.
Our ambition is for Q to become a research and intelligence source that people and agents routinely consult when evaluating a market price. The record has to remain visible so they can decide how much trust to put in it.
Dive into DailyCoin’s top crypto news right now:
Solana ETF Demand Builds Towards a $150 Test
Consumer Confidence Index About to Send a Signal to Bitcoin — Here’s What to Watch
Be the first to comment