Paper Citation Record · LEDGER

Has the Machine Learning Review Process Become More Arbitrary as the Field Has Grown? The NeurIPS 2021 Consistency Experiment

As of 9 August 2026, this Paper Citation Record lists 0 of 0 outbound references and 16 inbound Pith citation observations for arXiv:2306.03262.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2306.03262 v1

Coverage vector

measured 0 of 0 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links

measured 16 of 16 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-08T06:32:00.761636+00:00

measured 16 of 16 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-08-07T05:22:49.441452Z

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: arxiv_reference, observed 2026-07-02T09:26:51.596027Z

Reference resolution

0 of 0 outbound references displayed

verified exact0
verified fuzzy0
unresolved0
parse uncertain0
malformed identifier0
metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

No outbound reference observations are available for this paper version.

Pith citing papers

Observation 4ba85af2-80d6-441a-891a-bf488af7ffb2 · inbound

Position: The ML Community Must Build an AI-Augmented Peer-Review Ecosystem cites this paper.

Position: The ML Community Must Build an AI-Augmented Peer-Review Ecosystem Has the Machine Learning Review Process Become More Arbitrary as the Field Has Grown? The NeurIPS 2021 Consistency Experiment

Reference 5

Resolution: unresolved; no resolver link, observed 2026-08-07T05:22:49.441452Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b936c0a1-94e5-4cd6-b5c3-40e9449c27bf · inbound

How Far Are AI Scientists from Changing the World? cites this paper.

How Far Are AI Scientists from Changing the World? Has the Machine Learning Review Process Become More Arbitrary as the Field Has Grown? The NeurIPS 2021 Consistency Experiment

Reference 13

Resolution: unresolved; no resolver link, observed 2026-08-06T10:55:14.582127Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3fb21796-4962-43d9-aba8-8417b455ed5b · inbound

Justice in Judgment: Unveiling (Hidden) Bias in LLM-assisted Peer Reviews cites this paper.

Justice in Judgment: Unveiling (Hidden) Bias in LLM-assisted Peer Reviews Has the Machine Learning Review Process Become More Arbitrary as the Field Has Grown? The NeurIPS 2021 Consistency Experiment

Reference 6

Resolution: verified exact; arxiv_id, observed 2026-05-18T15:41:33.404001Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation d125b366-d74c-4841-bfc7-14ba55afab6c · inbound

Can We Volunteer Out of the Peer Review Crisis? cites this paper.

Can We Volunteer Out of the Peer Review Crisis? Has the Machine Learning Review Process Become More Arbitrary as the Field Has Grown? The NeurIPS 2021 Consistency Experiment

Reference 7

Resolution: metadata mismatch; arxiv_id, observed 2026-05-12T10:11:27.029628Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation 74149989-1429-48af-a81d-00cac6376919 · inbound

Intern-Atlas: A Methodological Evolution Graph as Research Infrastructure for AI Scientists cites this paper.

Intern-Atlas: A Methodological Evolution Graph as Research Infrastructure for AI Scientists Has the Machine Learning Review Process Become More Arbitrary as the Field Has Grown? The NeurIPS 2021 Consistency Experiment

Reference 24

Resolution: verified exact; arxiv_id, observed 2026-05-12T10:16:28.539507Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation c5efe562-0436-41c9-b1d0-2e2fb0e0bc41 · inbound

PRISM: A Multi-Dimensional Benchmark for Evaluating LLM Peer Reviewers cites this paper.

PRISM: A Multi-Dimensional Benchmark for Evaluating LLM Peer Reviewers Has the Machine Learning Review Process Become More Arbitrary as the Field Has Grown? The NeurIPS 2021 Consistency Experiment

Reference 17

Resolution: verified exact; arxiv_id, observed 2026-06-29T18:03:48.025770Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation 6361fd28-30ca-4479-9f3b-7b87cc8e238b · inbound

The State of Peer Review in Empirical Software Engineering: A Community Survey on Review Load, Quality, and GenAI Use cites this paper.

The State of Peer Review in Empirical Software Engineering: A Community Survey on Review Load, Quality, and GenAI Use Has the Machine Learning Review Process Become More Arbitrary as the Field Has Grown? The NeurIPS 2021 Consistency Experiment

Reference 5

Resolution: metadata mismatch; arxiv_id, observed 2026-07-02T09:26:51.597503Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation 63de9d53-7739-4fa7-94e3-278a02423cd5 · inbound

FARS: A Fully Automated Research System Deployed at Scale cites this paper.

FARS: A Fully Automated Research System Deployed at Scale Has the Machine Learning Review Process Become More Arbitrary as the Field Has Grown? The NeurIPS 2021 Consistency Experiment

Reference 3

Resolution: metadata mismatch; arxiv_id, observed 2026-07-01T10:35:42.693170Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation b68471c7-0cac-48ee-b6b9-8c517ab31174 · inbound

FARS: A Fully Automated Research System Deployed at Scale cites this paper.

FARS: A Fully Automated Research System Deployed at Scale Has the Machine Learning Review Process Become More Arbitrary as the Field Has Grown? The NeurIPS 2021 Consistency Experiment

Reference 3

Resolution: unresolved; no resolver link, observed 2026-07-14T16:55:37.417202Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 008a4778-ce76-416f-850b-9f497cd27527 · inbound

Publishing Without Journals: An Open, Forkable Archive with Attributed Review cites this paper.

Publishing Without Journals: An Open, Forkable Archive with Attributed Review Has the Machine Learning Review Process Become More Arbitrary as the Field Has Grown? The NeurIPS 2021 Consistency Experiment

Reference 6

Resolution: unresolved; no resolver link, observed 2026-07-11T18:32:37.712339Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 5c3f30fb-068e-47bc-a655-dcd7ed695872 · inbound

Publishing Without Journals: An Open, Forkable Archive with Attributed Review cites this paper.

Reference 1

Resolution: unresolved; no resolver link, observed 2026-08-02T08:38:05.007738Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2b92252f-f925-4451-8273-daca0f3c7717 · inbound

Publishing Without Journals: An Open, Forkable Archive with Attributed Review cites this paper.

Reference 2023

Resolution: unresolved; no resolver link, observed 2026-08-02T08:38:05.012531Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6fc29780-2b35-4c90-a138-a28d82d5528a · inbound

Bias at the Borderline: Who Gets the Benefit of the Doubt in Peer Review? cites this paper.

Bias at the Borderline: Who Gets the Benefit of the Doubt in Peer Review? Has the Machine Learning Review Process Become More Arbitrary as the Field Has Grown? The NeurIPS 2021 Consistency Experiment

Reference 9

Resolution: unresolved; no resolver link, observed 2026-08-01T00:21:36.791762Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a77bebf8-c0b9-4e06-81bb-5431eaefbf16 · inbound

Reviewer Scores Are Not Comparable Across Research Areas in ML Peer Review cites this paper.

Reviewer Scores Are Not Comparable Across Research Areas in ML Peer Review Has the Machine Learning Review Process Become More Arbitrary as the Field Has Grown? The NeurIPS 2021 Consistency Experiment

Reference 4

Resolution: unresolved; no resolver link, observed 2026-08-02T15:15:45.837048Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0a6270f2-f440-4e61-9ecb-f0bd738cfd9e · inbound

ReVoicer: Conversational Voice Annotation for Human-Centered, LLM-Assisted Peer Review cites this paper.

ReVoicer: Conversational Voice Annotation for Human-Centered, LLM-Assisted Peer Review Has the Machine Learning Review Process Become More Arbitrary as the Field Has Grown? The NeurIPS 2021 Consistency Experiment

Reference 3

Resolution: unresolved; no resolver link, observed 2026-08-04T00:49:01.007449Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1ab5c9c9-4927-4dbd-b23b-e3924b5db83b · inbound

ReVoicer: Conversational Voice Annotation for Human-Centered, LLM-Assisted Peer Review cites this paper.

Reference 3

Resolution: unresolved; no resolver link, observed 2026-08-06T04:15:20.095443Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.