Markets Are Not Random, They Are Hard to Predict
Pith reviewed 2026-06-27 18:51 UTC · model grok-4.3
The pith
Financial markets are causal economic systems whose unpredictability arises from hidden causes, strategic feedback, and capacity limits rather than ontic randomness.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Markets are not random in the ontic sense in which a quantum measurement is random. They are causal economic systems whose future is hard to predict because relevant causes are hidden, costly to observe, strategically used, capacity constrained, and sometimes governed by a changing law. The formal language of finance already encodes the distinction between ontic randomness and other forms of unpredictability through filtered probability spaces, risk-neutral measures Q distinct from the physical measure P, and no-arbitrage conditions that deliver martingality only under pricing measures.
What carries the argument
The ontic-versus-epistemic distinction in unpredictability, carried by filtered probability spaces that encode partial information, risk-neutral pricing measures Q ≠ P, and the Doob decomposition that isolates risk-compensated predictable drift from martingale innovation.
If this is right
- Prices live on filtered spaces because agents possess only partial information, so martingality under no-arbitrage does not imply unpredictability under every real-world information set.
- The wedge between physical measure P and pricing measure Q links directly to stochastic discount factor geometry and relative entropy without requiring markets to be fully random.
- Capacity and survival constraints imply that positive signals need not scale, even when they exist.
- Reflexivity, microstructure noise, and Knightian ambiguity can be tracked through a unified entropy ledger that records model-selection and intervention risks.
Where Pith is reading between the lines
- Regulatory design could shift from assuming pure randomness toward accounting for information costs and strategic responses.
- Empirical tests of regime stability might focus on measurable changes in the P-Q wedge during periods of high intervention.
- Prediction resources could be allocated preferentially to settings where filtration sufficiency is high and model landscape risk is low.
Load-bearing premise
The formal structures already standard in finance, such as probability filtrations and changes of measure, suffice to separate ontic randomness from other sources of unpredictability.
What would settle it
A demonstration that asset returns display the same invariance properties under all information sets as quantum measurement outcomes, or an empirical case in which no hidden causes, strategic actions, or capacity constraints affect forecast accuracy.
read the original abstract
Financial returns are often called ``random,'' but the word conflates ontic chance, epistemic ignorance, strategic feedback, and model instability. This essay argues that financial markets are not random in the ontic sense in which a quantum measurement is random. They are causal economic systems whose future is hard to predict because relevant causes are hidden, costly to observe, strategically used, capacity constrained, and sometimes governed by a changing law. The formal language of finance already encodes this distinction. Prices live on filtered probability spaces because agents have partial information; derivatives are priced under a risk-neutral measure $\Q\ne\Prob$ because pricing is an instrumental change of measure rather than a statement about the real data-generating law; and no-arbitrage gives martingality under an equivalent pricing measure, not full predictability failure under every real-world information set. The paper separates no-arbitrage, informational efficiency, and net exploitability; uses the Doob decomposition to isolate risk-compensated predictable drift from martingale innovation; adds a capacity-and-survival layer explaining why positive signals need not be scalable; relates the $\Prob$--$\Q$ wedge to stochastic-discount-factor geometry and relative entropy; formalises filtration sufficiency, model-selection landscape risk, and intervention-stable causality; and connects reflexivity, microstructure, and Knightian ambiguity to a unified entropy ledger. The disciplined thesis is therefore not that markets are unknowable, nor that they are literally random. Markets are hard to predict, and hardest exactly where prediction is costly, competitive, self-defeating, capacity limited, or invalidated by regime change.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript argues that financial returns are not 'random' in an ontic sense comparable to quantum measurements. Instead, markets are causal economic systems whose future is difficult to predict because relevant causes are hidden, costly to observe, strategically deployed, capacity-constrained, or subject to regime change. It claims that the standard apparatus of mathematical finance already encodes the relevant distinctions: filtered probability spaces capture partial information; pricing under an equivalent risk-neutral measure Q ≠ P is an instrumental device rather than a claim about the physical data-generating process; no-arbitrage implies martingality under Q but does not entail unpredictability under every real-world filtration; and the Doob decomposition isolates risk-compensated drift from innovation. The essay further separates no-arbitrage, informational efficiency, and net exploitability; introduces a capacity-and-survival layer; relates the P–Q wedge to stochastic-discount-factor geometry and relative entropy; and unifies reflexivity, microstructure, and Knightian ambiguity under an entropy ledger.
Significance. If the interpretive claims are accepted, the paper supplies a coherent conceptual map that prevents conflation of distinct sources of unpredictability and shows how existing formal objects (filtered spaces, equivalent measures, Doob decomposition) already support finer distinctions than the colloquial label 'random' allows. The synthesis of capacity constraints with intervention-stable causality and model-selection landscape risk could usefully inform discussions of why positive signals often fail to scale. Because the manuscript contains no new derivations, theorems, or empirical tests, its contribution is one of clarification and unification rather than technical advance.
minor comments (2)
- The abstract lists several formal devices (filtered spaces, Q ≠ P, Doob decomposition, entropy ledger) in a single dense sentence; breaking this list into shorter clauses or a brief enumerated summary would improve readability without altering content.
- The claim that the paper 'formalises filtration sufficiency, model-selection landscape risk, and intervention-stable causality' would be clearer if each formalisation were explicitly located by section or equation number rather than asserted in the abstract.
Simulated Author's Rebuttal
We thank the referee for the supportive report and for correctly identifying the manuscript as a work of conceptual clarification rather than a source of new theorems or empirical results. We agree with the recommendation for minor revision and will incorporate any editorial suggestions in the revised version.
read point-by-point responses
-
Referee: Because the manuscript contains no new derivations, theorems, or empirical tests, its contribution is one of clarification and unification rather than technical advance.
Authors: We fully agree. The paper is explicitly framed as an interpretive essay that organises existing objects (filtered spaces, equivalent measures, Doob decomposition, relative entropy) to distinguish distinct sources of unpredictability. No technical novelty is claimed or required for the stated purpose. revision: no
Circularity Check
No significant circularity; standard concepts used for reinterpretation
full rationale
The paper is a conceptual essay whose central claim reinterprets standard tools of mathematical finance (filtered spaces, Q ≠ P, no-arbitrage martingales, Doob decomposition) to separate ontic randomness from epistemic/strategic sources of unpredictability. These devices are drawn from external literature and do not reduce to any equation or claim internal to the manuscript; no new derivation, fitted parameter, or self-citation chain is load-bearing. The argument therefore remains self-contained against external benchmarks and receives the default non-circularity finding.
Axiom & Free-Parameter Ledger
axioms (3)
- domain assumption Prices live on filtered probability spaces because agents have partial information
- standard math Derivatives are priced under a risk-neutral measure Q different from the real-world P
- standard math No-arbitrage implies martingality under an equivalent measure but not full predictability failure under every real-world filtration
Reference graph
Works this paper leans on
-
[1]
Bachelier, Louis , title =
-
[2]
, title =
Bell, John S. , title =. Physics Physique Fizika , volume =
-
[3]
The Journal of Finance , volume =
Black, Fischer , title =. The Journal of Finance , volume =
-
[4]
Journal of Political Economy , volume =
Black, Fischer and Scholes, Myron , title =. Journal of Political Economy , volume =
-
[5]
Journal of Econometrics , volume =
Bollerslev, Tim , title =. Journal of Econometrics , volume =
-
[6]
, title =
Brudno, Alexander A. , title =. Transactions of the Moscow Mathematical Society , volume =
-
[7]
and Kelly, Michael , title =
Buchen, Peter W. and Kelly, Michael , title =. Journal of Financial and Quantitative Analysis , volume =
-
[8]
and Shiller, Robert J
Campbell, John Y. and Shiller, Robert J. , title =. The Journal of Finance , volume =
-
[9]
, title =
Cochrane, John H. , title =
-
[10]
Quantitative Finance , volume =
Cont, Rama , title =. Quantitative Finance , volume =
-
[11]
and Thomas, Joy A
Cover, Thomas M. and Thomas, Joy A. , title =
-
[12]
and Morton, Andrew and Willinger, Walter , title =
Dalang, Robert C. and Morton, Andrew and Willinger, Walter , title =. Stochastics and Stochastic Reports , volume =
-
[13]
Mathematische Annalen , volume =
Delbaen, Freddy and Schachermayer, Walter , title =. Mathematische Annalen , volume =
-
[14]
, title =
Doob, Joseph L. , title =
-
[15]
The Quarterly Journal of Economics , volume =
Ellsberg, Daniel , title =. The Quarterly Journal of Economics , volume =
-
[16]
, title =
Engle, Robert F. , title =. Econometrica , volume =
-
[17]
, title =
Fama, Eugene F. , title =. The Journal of Finance , volume =
-
[18]
and French, Kenneth R
Fama, Eugene F. and French, Kenneth R. , title =. Journal of Financial Economics , volume =
-
[19]
Stochastic Finance: An Introduction in Discrete Time , edition =
F. Stochastic Finance: An Introduction in Discrete Time , edition =
-
[20]
Mathematical Finance , volume =
Frittelli, Marco , title =. Mathematical Finance , volume =
-
[21]
Journal of Mathematical Economics , volume =
Gilboa, Itzhak and Schmeidler, David , title =. Journal of Mathematical Economics , volume =
-
[22]
, title =
Girsanov, Igor V. , title =. Theory of Probability and Its Applications , volume =
-
[23]
and Milgrom, Paul R
Glosten, Lawrence R. and Milgrom, Paul R. , title =. Journal of Financial Economics , volume =
-
[24]
Granger, C. W. J. , title =. Econometrica , volume =
-
[25]
and Stiglitz, Joseph E
Grossman, Sanford J. and Stiglitz, Joseph E. , title =. The American Economic Review , volume =
-
[26]
, title =
Hansen, Peter R. , title =. Journal of Business and Economic Statistics , volume =
-
[27]
, title =
Hansen, Lars Peter and Sargent, Thomas J. , title =
-
[28]
Michael and Kreps, David M
Harrison, J. Michael and Kreps, David M. , title =. Journal of Economic Theory , volume =
-
[29]
Michael and Pliska, Stanley R
Harrison, J. Michael and Pliska, Stanley R. , title =. Stochastic Processes and their Applications , volume =
-
[30]
Hasbrouck, Joel , title =
-
[31]
, title =
Jaynes, Edwin T. , title =. Physical Review , volume =
-
[32]
Keynes, John Maynard , title =
-
[33]
, title =
Knight, Frank H. , title =
-
[34]
, title =
Kullback, Solomon and Leibler, Richard A. , title =. The Annals of Mathematical Statistics , volume =
-
[35]
, title =
Kyle, Albert S. , title =. Econometrica , volume =
-
[36]
, title =
Lo, Andrew W. , title =. Journal of Portfolio Management , volume =
-
[37]
Advances in Financial Machine Learning , publisher =
L. Advances in Financial Machine Learning , publisher =
-
[38]
Lucas, Robert E. Jr. , title =. Econometrica , volume =
-
[39]
The Journal of Business , volume =
Mandelbrot, Benoit , title =. The Journal of Business , volume =
-
[40]
, title =
Merton, Robert C. , title =. The Bell Journal of Economics and Management Science , volume =
-
[41]
, title =
Merton, Robert C. , title =. Journal of Financial Economics , volume =
-
[42]
Pearl, Judea , title =
-
[43]
, title =
Pesin, Yakov B. , title =. Russian Mathematical Surveys , volume =
-
[44]
, title =
Protter, Philip E. , title =
-
[45]
, title =
Samuelson, Paul A. , title =. Industrial Management Review , volume =
-
[46]
, title =
Shannon, Claude E. , title =. Bell System Technical Journal , volume =
-
[47]
, title =
Sharpe, William F. , title =. The Journal of Finance , volume =
-
[48]
, title =
Shiller, Robert J. , title =. The American Economic Review , volume =
-
[49]
, title =
Shleifer, Andrei and Vishny, Robert W. , title =. The Journal of Finance , volume =
-
[50]
Journal of Economic Methodology , volume =
Soros, George , title =. Journal of Economic Methodology , volume =
-
[51]
The Journal of Finance , volume =
Stutzer, Michael , title =. The Journal of Finance , volume =
-
[52]
Econometrica , volume =
White, Halbert , title =. Econometrica , volume =
-
[53]
, title =
Zurek, Wojciech H. , title =. Reviews of Modern Physics , volume =
-
[54]
Journal of Risk , volume =
Almgren, Robert and Chriss, Neil , title =. Journal of Risk , volume =
-
[55]
Bailey, David H. and L. The Deflated. Journal of Portfolio Management , volume =
-
[56]
, title =
Chen, Zengjing and Epstein, Larry G. , title =. Econometrica , volume =
-
[57]
Journal of Political Economy , volume =
Hansen, Lars Peter and Jagannathan, Ravi , title =. Journal of Political Economy , volume =
-
[58]
and Liu, Yan and Zhu, Heqing , title =
Harvey, Campbell R. and Liu, Yan and Zhu, Heqing , title =. Review of Financial Studies , volume =
-
[59]
and MacKinlay, A
Lo, Andrew W. and MacKinlay, A. Craig , title =. Review of Financial Studies , volume =
-
[60]
, title =
LeRoy, Stephen F. , title =. Journal of Economic Literature , volume =
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.