Theoretical adversarial collaboration: a template

Kelvin J. McQueen; Markus P. Mueller

REVIEW 3 major objections 4 minor 30 references

Theoretical adversarial collaboration: a template

T0 review · 3 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that theoretical disputes experiment cannot settle can still yield publishable insight through a structured five-step adversarial collaboration.

desk verdict A clear, honest proposal for structuring theoretical disputes, but the success criterion is self-assessed and the only evidence is the authors' own two collaborations, so treat the effectiveness claim as a hypothesis to test rather than a proven result. read the letter →

arxiv 2607.16374 v1 pith:HKJ7VMXN submitted 2026-07-17 physics.hist-ph quant-ph

Kelvin J. McQueen , Markus P. Mueller This is my paper · ORCID

classification physics.hist-phquant-ph PACS 01.70.+w

keywords adversarialcollaborationtheoreticaldisputephilosophyofsciencefoundationsphysicssteelmanningmethodologypublicationstrategyresolution

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a way to make progress in theoretical disputes when experiments cannot settle them. It offers a five-step procedure in which an advocate and a critic work together to clarify the disagreement, have the critic build and defend a supervised steelman version of the theory, develop objections under the advocate's supervision, and then test whether the theory's existing resources can answer them. The collaboration counts as successful only when it produces insight that substantially exceeds the existing literature: either a novel, non-obvious defense of the theory, or an acknowledged need to revise it. The authors present this template as a general method for philosophy, foundational physics, and other fields where experimental adjudication is unavailable, and they ground it in two of their own collaborations.

What carries the argument

The five-step template itself is the central mechanism: (1) clarify central points of contention and agree on what would count as progress; (2) the critic constructs a steelman version of the theory, supervised by the advocate; (3) the critic develops objections to the theory, also supervised; (4) the advocate responds using the theory's existing conceptual resources; steps 3 and 4 repeat until (5) one of two agreed success outcomes is reached—5a, a novel and non-obvious defense, or 5b, a conceded need for revision. The pivotal element is the condition for success in step 5, which defines what counts as publishable progress.

What would settle it

Take the two collaborations described in the paper and ask a panel of independent researchers in the relevant fields to judge whether the outputs meet the paper's own criterion of 'substantially exceeding the existing literature'; if the panel consistently disagrees with the collaborators' self-assessed success decisions, the success condition is not objective.

Watch

Extended reading notes

Core claim

The central claim is that a theoretical adversarial collaboration can be defined by a clear condition for success that does not require one side to convert or surrender. Success occurs when the collaboration yields insight 'substantially exceeding the existing literature,' in one of two forms: (5a) the critic concedes that an objection can be met without modifying the theory, but only through a novel and non-obvious argument grounded in the theory's published commitments; or (5b) the advocate concedes that new conceptual resources or revisions are required. These outcomes mark genuine progress without demanding the unrealistic extremes of total abandonment or complete vindication.

Load-bearing premise

The load-bearing premise is that the participants' own judgment that their insights 'substantially exceed the existing literature' is a reliable marker of genuine progress—if both sides are content, nothing external checks that the bar was actually met.

Editorial extensions

If this is right

The template gives fields like philosophy and foundational physics a publishable format for disputes that currently default to unproductive critique–reply–rejoinder cycles.
It makes progress possible without requiring the advocate to abandon the theory or the critic to withdraw all skepticism, which are often unrealistic in deep theoretical disagreements.
Supervised steelmanning reduces strawmanning and misunderstanding, producing both stronger objections and stronger defenses of the target theory.
The condition for success ensures a tangible result that clearly marks progress, even when the dispute cannot be settled by experiment.
The procedure applies not only to scientific theories but to any contested claim, including ideas, arguments, proofs, and policy proposals.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the success condition is self-assessed by the participants, its reliability could be tested by adding an external arbiter or pre-registered criteria before the collaboration begins.
In fields like ethics or policy, the 5a outcome might need independent evaluation of whether a defense is genuinely novel, since advocates may overestimate the non-obviousness of their own arguments.
The two pilot collaborations show the method can generate results in the authors' own areas, but its generalizability to other disciplines remains untested; a systematic comparison across several fields would help.
One testable extension is measuring whether outputs produced under the template receive different downstream recognition (citations, uptake) than outputs from ordinary debate volumes.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Quick take: this is a genuinely useful proposal for structuring theoretical disagreements, and it is honestly written. The five-step template is new in its specific form—particularly the supervised steel-manning and the two-pronged success condition (advocate concedes revision, or critic concedes a novel non-obvious defense). It is clearly distinguished from empirical adversarial collaboration, and the authors cite the relevant psychology and philosophy literature, including recent applications in ecology and psychiatry. For a methodological essay, this is solid.

The weak spot is exactly what you and the stress-test note flag: the success condition is self-assessed by the collaborators, and the only reported implementations are the authors' own two projects. That means the central claim—that the template reliably yields publishable progress—is not independently validated. I do not think this is a fatal flaw. The paper is a proposal, not a proven instrument, and it is transparent about its sourcing from personal experience. The success condition being subjective is not unique to this paper; it is endemic to theoretical fields where independent verification of 'depth' is hard. Still, a more robust paper would suggest how a neutral third party might evaluate whether the output truly exceeds the literature.

One minor issue: the template assumes a clear advocate/critic split, which may not fit broad framework disputes. The authors define T broadly to include ideas and arguments, so this is not a major objection, but it does limit immediate applicability.

Overall, I would send this to peer review. A good referee should push for explicit criteria for external validation and for more case studies, but the template itself is clear and actionable. It deserves to be tried. I would bring it to a reading group and would cite it as a reference for theoretical adversarial collaboration methods.

Referee Report

3 major / 4 minor

Summary. The paper distinguishes theoretical adversarial collaboration from its empirical counterpart and proposes a five-step template for conducting it. The procedure requires the advocate and critic to clarify points of contention, have the critic construct a supervised steel-man of the theory, develop objections, and respond using existing resources. The condition for success is that the collaboration produces insights that substantially exceed the existing literature, via either a novel non-obvious defense of the theory (5a) or the advocate's concession that revisions are needed (5b). The template is presented as a general framework for constructive, publishable progress in philosophy, foundational science, and other non-experimental fields, supported by the authors' own two collaborations.

Significance. If accepted, the paper would fill a genuine gap in the methodology of philosophical and foundational dispute resolution. Its strengths include a clear procedural structure, an explicit success condition, useful distinctions between empirical and theoretical adversarial collaboration, and an honest report of two implementing case studies. The paper is readable and thoughtfully engages with neighboring literature. However, its central claim is supported only by the authors' own collaborations, one of which is an unpublished manuscript, and the success condition is largely self-assessed. The general reliability of the template is therefore plausible but not established; the paper functions better as a proposal than as a validated method.

major comments (3)

[Step 1 and condition for success] The condition for success is that the collaboration produce insights that 'substantially exceed the existing literature,' but the template places judgment of this criterion entirely with the collaborators. Step 1 asks the parties to agree on what would count as progress, and step 5 is satisfied when the critic (5a) or advocate (5b) concedes that progress has occurred. There is no independent check. This creates a double role: the same people who set the bar also determine whether it is met. Given the incentives to produce a publishable result, the condition can be satisfied by mutual agreement even if an external expert would judge that nothing substantial has been produced. The two supporting examples are self-reported by the authors, one of which (ref [5]) is an unpublished manuscript. Since the paper's central claim is that the template yields 'constructive and publishable progress' g
[Step 5a] Outcome 5a requires the critic to concede that the objection is met only by a 'novel and non-obvious argument' grounded in existing publications. No criteria are given for novelty or non-obviousness, and no one outside the collaboration checks this. The critic may be insufficiently familiar with the full literature to assess novelty, and social pressure can make a concession the path of least resistance. Without an operational definition or an external referee, 5a is indistinguishable from 5b and from straightforward clarification. The paper should specify how novelty is to be determined, or acknowledge that the distinction is a judgment call left to participants.
[Our own collaboration] The template is validated by only two projects, both conducted by the authors (refs [4,5]). One is not yet published. This is a small, non-independent sample from which to infer general reliability. The paper would be strengthened by additional case studies from other groups or a clearer statement that the template is a proposal that requires broader testing.

minor comments (4)

[Introduction / footnote 1] The paper claims 'we have not found a single journal article in philosophy that applies a rigorous adversarial procedure to a purely theoretical philosophical dispute,' but footnote 1 lists several works that seem close, e.g., refs [16] and [17]. Please clarify why these do not count as examples; otherwise the claim appears too strong.
[Step 2] Step 2 has the critic construct a steel-man 'under the advocate's supervision.' While this ensures accuracy, it risks turning the steel-man into the advocate's preferred interpretation, reducing the independence that makes steel-manning useful. The paper should discuss how to manage this tension, e.g., by having the neutral mediator verify the steel-man or by having the critic defend it to a third party.
[Step 4] The paper says that if every objection is answered straightforwardly, the collaboration has not produced a substantive result. This could encourage participants to manufacture unnecessary novelty or revisions to satisfy the success condition. A brief caution about this perverse incentive would be helpful.
[References] Reference [5] is cited as 'manuscript'; if it is a key case study, the authors should either provide more information or mark it clearly as unpublished. Reference [28] contains a duplicated author ('A. Plantinga' appears multiple times), likely a formatting error.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; minor self-referential validation pattern only.

full rationale

This paper is a methodological proposal rather than a derivation. It offers a five-step template, and its central claim is that the procedure can produce constructive, publishable progress. There is no equation or fitted parameter whose output is reused as an input. The strongest candidate for circularity is the success condition: the abstract defines success as 'insights that substantially exceed the existing literature' and Step 5 operationalizes this as concessions by the advocate or critic. Because the parties themselves judge novelty and substantiality, the criterion is self-assessed and could in principle be satisfied by mutual agreement. This is a genuine verifiability limitation, but not a circular reduction: the paper does not use the concessions to prove the template's value as a theorem, and the template's content (clarification, supervised steel-manning, supervised objections, responses, resolution) does not presuppose its conclusion. The only cited implementations are the authors' own two projects (refs [4,5]), which is a form of self-citation; however, those projects are externally available and the paper uses them as illustrative provenance, not as a load-bearing proof. Thus there is no specific step that reduces by construction to its own inputs. Score 2 reflects the minor self-referential validation pattern, not substantive circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The template rests on a few domain assumptions about the nature of theoretical disputes and the value of steel-manning. No free parameters are present. The 'insight exceeding literature' criterion is an ad hoc definition introduced by the authors.

assumptions (3)

domain assumption Theoretical disputes in philosophy and foundational science cannot be settled by experiment.
Motivates the need for a non-empirical variant of adversarial collaboration; stated in the abstract and intro.
domain assumption Steel-manning and supervised objections reduce strawmanning and misunderstanding.
Assumed positive effect of step 2; no empirical evidence provided.
ad hoc to paper The condition for success (insights substantially exceeding existing literature) is a well-defined and appropriate goal.
Defined by the authors as the criterion for a publishable outcome; not derived from any external standard.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Theoretical adversarial collaboration: a template." pith.science (2026). https://pith.science/paper/HKJ7VMXN

@misc{pith2026260716374,
  author       = {Pith},
  title        = {Pith review of: Theoretical adversarial collaboration: a template},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HKJ7VMXN}},
  note         = {Machine review of arXiv:2607.16374}
}

read the original abstract

Adversarial collaborations have become increasingly prominent in empirical science, where proponents of competing theories jointly design experiments capable of discriminating between them. Much less attention has been given to theoretical disputes that cannot readily be resolved by experiment. We distinguish theoretical adversarial collaboration from its empirical counterpart and propose a five-step template for conducting it. The procedure is designed to move disputants beyond familiar critique-reply-rejoinder exchanges by requiring sustained clarification, supervised steel-manning of the target theory, and the collaborative development and refinement of objections. Its condition for success is that the collaboration produce insights that substantially exceed the existing literature: either novel, non-obvious arguments show that the theory can meet the objections, or the theory's advocate concedes that new conceptual resources or revisions are required. The template was developed through reflection on, and implemented in, our own adversarial collaboration in philosophy and foundations of physics. We offer it here as a general framework for constructive and publishable progress in philosophy, foundational science, and other areas of inquiry in which disputes cannot readily be resolved by experiment.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references

[5]

K. J. McQueen and M. P. M ¨uller, Realism about the external world: An adversarial collaboration (2026), manuscript

2026
[1]

G. P. Latham, M. Erez, and E. A. Locke, Resolving scientific disputes by the joint design of crucial experiments by the antagonists: Application to the erez–latham dispute regarding participation in goal setting, Journal of Applied Psychology73, 753 (1988)

1988
[2]

Kahneman, Experiences of collaborative research., American Psychologist58, 723 (2003)

D. Kahneman, Experiences of collaborative research., American Psychologist58, 723 (2003)

2003
[3]

C. J. Clark, T. Costello, G. Mitchell, and P. E. Tetlock, Keep your enemies close: Adversarial collaborations will improve behavioral science., Journal of Applied Research in Memory and Cognition11, 1 (2022)

2022
[4]

K. J. McQueen, I. T. Durham, and M. P. M¨uller, Quantum superpositions of conscious states in a minimal integrated informa- tion model, Entropy28, 394 (2026)

2026
[6]

Mellers, R

B. Mellers, R. Hertwig, and D. Kahneman, Do frequency representations eliminate conjunction effects? an exercise in adver- sarial collaboration, Psychological Science12, 269 (2001)

2001
[7]

A. W. Corcoran, J. Hohwy, and K. J. Friston, Accelerating scientific progress through bayesian adversarial collaboration, Neuron111, 3505 (2023)

2023
[8]

F. W. Dyson, A. S. Eddington, and C. Davidson, Ix. a determination of the deflection of light by the sun’s gravitational field, from observations made at the total eclipse of may 29, 1919, Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character220, 291 (1920)

1919

Show all 30 references

[9]

Melloni, L

L. Melloni, L. Mudrik, M. Pitts, K. Bendtz, O. Ferrante, U. Gorska, R. Hirschhorn, A. Khalaf, C. Kozma, A. Lepauvre, et al., An adversarial collaboration protocol for testing contrasting predictions of global neuronal workspace and integrated information theory, PLoS One18, e0...

2023
[10]

Consortium, O

C. Consortium, O. Ferrante, U. Gorska-Klimowska, S. Henin, R. Hirschhorn, A. Khalaf, A. Lepauvre, L. Liu, D. Richter, Y . Vidal,et al., Adversarial testing of global neuronal workspace and integrated information theories of consciousness, Nature 642, 133 (2025)

2025
[11]

Bateman, D

I. Bateman, D. Kahneman, A. Munro, C. Starmer, and R. Sugden, Testing competing models of loss aversion: An adversarial collaboration, Journal of Public Economics89, 1561 (2005)

2005
[12]

Cowan, C

N. Cowan, C. Belletier, J. M. Doherty, A. J. Jaroslawska, S. Rhodes, A. Forsberg, M. Naveh-Benjamin, P. Barrouillet, V . Camos, and R. H. Logie, How do scientific views change? notes from an extended adversarial collaboration, Perspec- tives on Psychological Science15, 1011 (2020)

2020
[13]

M. A. Killingsworth, D. Kahneman, and B. Mellers, Income and emotional well-being: A conflict resolved, Proceedings of the National Academy of Sciences120, e2208661120 (2023)

2023
[14]

Fischer, P

E. Fischer, P. E. Engelhardt, and J. Sytsma, Inappropriate stereotypical inferences? an adversarial collaboration in experimental ordinary language philosophy, Synthese198, 10127 (2021)

2021
[15]

Crusius, M

J. Crusius, M. F. Gonzalez, J. Lange, and Y . Cohen-Charash, Envy: An adversarial review and comparison of two competing views, Emotion Review12, 3 (2020)

2020
[16]

Richter and T

D. Richter and T. Steinert, Can coercion in psychiatry be justified? a theoretical adversarial collaboration approach, Interna- tional Journal of Law and Psychiatry105, 102171 (2026)

2026
[17]

Gefaell and T

J. Gefaell and T. Uller, From rivals to partners: Adversarial collaboration in ecology and evolution, Trends in Ecology & Evolution41, 37 (2026)

2026
[18]

Rakow, V

T. Rakow, V . Thompson, L. Ball, and H. Markovits, Rationale and guidelines for empirical adversarial collaboration: A 6 Thinking & Reasoning initiative, Thinking & Reasoning21, 167 (2015)

2015
[19]

R. T. Hurlburt and E. Schwitzgebel,Describing Inner Experience? Proponent Meets Skeptic(MIT Press, Cambridge, MA, 2007)

2007
[20]

J. M. Fischer, R. Kane, D. Pereboom, and M. Vargas,F our views on free will(John Wiley & Sons, 2024)

2024
[21]

A. Kind, F. Jackson, and D. Stoljar,What is Consciousness?: A Debate(Routledge, 2023)

2023
[22]

Shoemaker and S

S. Shoemaker and S. Swinburne,Personal Identity: Great Debates in Philosophy, edited by R. Swinburne (Blackwell, Oxford, England, 1984)

1984
[23]

D. C. Dennett, A. Plantinga, A. Plantinga, E.-U. Philosophe, and A. Plantinga,Science and religion: Are they compatible? (Oxford University Press Oxford, 2011)

2011
[24]

J. J. C. Smart and B. Williams,Utilitarianism: F or and against(Cambridge University Press, 1973)

1973
[25]

Glasgow, S

J. Glasgow, S. Haslanger, C. Jeffers, and Q. Spencer,What is race?: F our philosophical views(Oxford University Press, 2019)

2019
[26]

Leal and H

F. Leal and H. Marraud,How philosophers argue: An Adversarial Collaboration on the Russell–Copleston Debate(Springer International Publishing, 2022)

2022
[27]

D. M. Armstrong,Dispositions: A Debate, edited by C. B. Martin, U. T. Place, and T. Crane (Routledge, New York, 1996)

1996
[28]

D. C. Dennett and G. D. Caruso,Just deserts: Debating free will(John Wiley & Sons, 2021)

2021
[29]

D. J. Chalmers and K. J. McQueen, Consciousness and the collapse of the wave function, inConsciousness and Quantum Mechanics, edited by S. Gao (Oxford University Press, Oxford, 2022) pp. 11–63

2022
[30]

M. P. M ¨uller, Law without law: from observer states to physics via algorithmic information theory, Quantum4, 301 (2020)

2020

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.

[1] [5]

K. J. McQueen and M. P. M ¨uller, Realism about the external world: An adversarial collaboration (2026), manuscript

2026

[2] [1]

G. P. Latham, M. Erez, and E. A. Locke, Resolving scientific disputes by the joint design of crucial experiments by the antagonists: Application to the erez–latham dispute regarding participation in goal setting, Journal of Applied Psychology73, 753 (1988)

1988

[3] [2]

Kahneman, Experiences of collaborative research., American Psychologist58, 723 (2003)

D. Kahneman, Experiences of collaborative research., American Psychologist58, 723 (2003)

2003

[4] [3]

C. J. Clark, T. Costello, G. Mitchell, and P. E. Tetlock, Keep your enemies close: Adversarial collaborations will improve behavioral science., Journal of Applied Research in Memory and Cognition11, 1 (2022)

2022

[5] [4]

K. J. McQueen, I. T. Durham, and M. P. M¨uller, Quantum superpositions of conscious states in a minimal integrated informa- tion model, Entropy28, 394 (2026)

2026

[6] [6]

Mellers, R

B. Mellers, R. Hertwig, and D. Kahneman, Do frequency representations eliminate conjunction effects? an exercise in adver- sarial collaboration, Psychological Science12, 269 (2001)

2001

[7] [7]

A. W. Corcoran, J. Hohwy, and K. J. Friston, Accelerating scientific progress through bayesian adversarial collaboration, Neuron111, 3505 (2023)

2023

[8] [8]

F. W. Dyson, A. S. Eddington, and C. Davidson, Ix. a determination of the deflection of light by the sun’s gravitational field, from observations made at the total eclipse of may 29, 1919, Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character220, 291 (1920)

1919