Pith. sign in

REVIEW 3 cited by

Position: The AI Conference Peer Review Crisis Demands Author Feedback and Reviewer Rewards

Not yet reviewed by Pith; the record is open.

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2505.04966 v1 pith:FNQNW6DV submitted 2025-05-08 cs.AI cs.CY

classification cs.AIcs.CY
keywords reviewsystemauthorspeerreviewerqualityaccountabilitybi-directional
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The peer review process in major artificial intelligence (AI) conferences faces unprecedented challenges with the surge of paper submissions (exceeding 10,000 submissions per venue), accompanied by growing concerns over review quality and reviewer responsibility. This position paper argues for the need to transform the traditional one-way review system into a bi-directional feedback loop where authors evaluate review quality and reviewers earn formal accreditation, creating an accountability framework that promotes a sustainable, high-quality peer review system. The current review system can be viewed as an interaction between three parties: the authors, reviewers, and system (i.e., conference), where we posit that all three parties share responsibility for the current problems. However, issues with authors can only be addressed through policy enforcement and detection tools, and ethical concerns can only be corrected through self-reflection. As such, this paper focuses on reforming reviewer accountability with systematic rewards through two key mechanisms: (1) a two-stage bi-directional review system that allows authors to evaluate reviews while minimizing retaliatory behavior, (2)a systematic reviewer reward system that incentivizes quality reviewing. We ask for the community's strong interest in these problems and the reforms that are needed to enhance the peer review process.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ReVoicer: Conversational Voice Annotation for Human-Centered, LLM-Assisted Peer Review

    cs.HC 2026-07 conditional novelty 6.0 of 10

    ReVoicer is a prototype that turns spoken, in-the-moment reactions to a paper into cleaned, tagged annotations and a draft review aligned with the reviewer's own style.

  2. Reviewer Scores Are Not Comparable Across Research Areas in ML Peer Review

    cs.DL 2026-04 conditional novelty 6.0 of 10

    An audit of 50,289 ICLR papers shows acceptance odds vary up to 8x across topics at equal reviewer scores, indicating scores are not comparable across research areas.

  3. Position: The ML Community Must Build an AI-Augmented Peer-Review Ecosystem

    cs.AI 2025-06 conditional novelty 4.0 of 10

    The paper argues that AI-assisted peer review is an urgent priority and that its success depends on collecting richer, structured peer review process data.

Pith tools