Pith. sign in

REVIEW 3 major objections 5 minor 23 references

Accept More, Reject Less: Reducing up to 19% Unnecessary Desk-Rejections over 11 Years of ICLR Data

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An optimization-based desk-rejection rule enforces per-author submission limits while keeping up to 19.23% more papers than current ID-order policies.

desk verdict Useful empirical study of a real policy problem, but the rounding algorithm's correctness proof has a load-bearing gap and some claims are overstated. read the letter →

arxiv 2506.20141 v1 pith:BLQDJJCZ submitted 2025-06-25 cs.DS cs.CYcs.DLcs.IRcs.LG

Xiaoyu Li , Zhao Song , Jiahao Zhang This is my paper
classification cs.DScs.CYcs.DLcs.IRcs.LG MSC 90C0590C10
keywords deskrejectionper-authorsubmissionlimitlinearprogrammingrelaxationMaxRoundingICLRauthorwelfareintegerOpenReview
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AI conferences that cap submissions per author currently desk-reject every excess paper in submission-ID order, which can discard more work than the limit actually requires. This paper asks whether the same limit can be enforced while keeping as many papers as possible, and answers yes. It formalizes the policy as an integer program, solves a linear-program relaxation, and rounds the fractional solution to a feasible accept/reject list. On eleven years of ICLR submission records the method keeps up to 19.23% more papers than the current ID-order policy, with every run finishing in under a minute. If the result holds, conferences can adopt a simple optimization step that preserves thousands of authors' papers without changing the reviewer workload cap.

What carries the argument

The central object is the authorship matrix $A \in \{0,1\}^{n \times m}$ and the maximum desk-acceptance problem $\max_{x \in \{0,1\}^m} \mathbf{1}^\top x$ subject to $Ax \le b \cdot \mathbf{1}_n$, which maximizes the number of papers sent to review under each author's $b$-paper cap. The two-stage solver first relaxes the binary constraint to $x \in [0,1]^m$, solves the resulting linear program, and then runs MaxRounding. MaxRounding repeatedly takes the still-fractional paper with the largest value, sets it to 1, and for each of its authors whose load would exceed $b$, zeroes out enough remaining fractional papers to restore feasibility. This rounding step is what converts the fractional optimum into a provably feasible binary desk-acceptance set.

What would settle it

Take ICLR 2024's official submission roster with author identities verified by the organizers, and rerun both the ID-order baseline and the LP-plus-MaxRounding rule at $b=22$. If the baseline desk-rejects 26 papers while the method rejects 21, the central claim holds; if the baseline rejects 26 or fewer, or the method's output leaves any author with more than 22 papers, the reported 19.23% reduction collapses.

Watch

Extended reading notes

Core claim

The paper's central claim is that enforcing a per-author submission limit does not force the rejections that current ID-order policies produce. The authors show that the choice of which papers to desk-reject can be treated as an integer program, and that solving its linear-program relaxation and then applying a deterministic rounding rule yields a binary accept/reject list that respects every author's limit while keeping more papers. On eleven years of ICLR submission records, this method never rejects more papers than either baseline in any tested year or limit, and it rejects strictly fewer in most of the cases where limits bite. The largest single case is ICLR 2024 at limit $b=22$, where the method rejects 21 papers instead of the baseline's 26, a 19.23% relative improvement. The method also scales: every experiment finishes in at most 53.64 seconds on a two-vCPU machine.

Load-bearing premise

The load-bearing premise is that the authorship matrix crawled from OpenReview assigns every paper to its true, complete set of authors; if records are missing or name variants are merged incorrectly, the per-author limit checks and the reported rejection counts would be wrong.

Editorial extensions

If this is right

  • A conference using per-author limits can replace ID-order rejection with the LP-plus-rounding rule and keep the same cap on each author's submissions while never desk-rejecting more papers, and often fewer.
  • The gain grows with conference size: ICLR 2024 and 2025 show double-digit relative improvements at nearly every tested limit from $b=4$ to $b=25$.
  • At ICLR 2024 with $b=22$, the method rejects 21 papers rather than 26, the 19.23% improvement; at ICLR 2025 with $b=4$, it rejects 2668 instead of 2984.
  • Operationally, all runs finish within 53.64 seconds on a two-vCPU, 13GB machine, so the optimization is fast enough to run during a real submission cycle.
  • In cases where the limit is high enough that nobody exceeds it, the method matches the baselines at zero desk-rejections, so adopting it carries no downside in easy settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension, not tested in the paper, is to replace the unit objective with per-paper weights, such as expected citations or review cost, since the LP relaxation handles weighted objectives identically.
  • The formulation could also absorb fairness caps, such as limiting desk-rejections per institution or research group, as extra linear constraints on the same system.
  • The 19.23% figure is measured on ICLR's authorship graph; applying the method to CVPR or KDD would require those venues' private submission data, and the realized gain could differ depending on how co-authorship overlaps there.
  • A production deployment would need an explicit name-resolution step; without one, authors could evade caps through name variants, which would change both the baseline and the optimized counts.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper considers desk-rejection policies used by top AI conferences, which enforce per-author submission limits by rejecting papers in submission-ID order. It formalizes the problem of maximizing the number of papers that can proceed to review while respecting per-author caps, proposes an LP-relaxation plus a custom rounding algorithm (MaxRounding, Algorithm 4), and evaluates the resulting pipeline on 11 years of ICLR submission data. The empirical tables report consistent improvements over the AllReject and ForwardReject baselines, with the headline figure of 19.23% fewer desk-rejections at ICLR 2024 with b=22. The paper claims to establish computational hardness of the problem but gives no reduction, and the main correctness theorem for MaxRounding is both unproven and, as stated, false.

Significance. If the empirical claim is correct, the paper identifies a simple, practical way for conferences to reduce needless desk-rejections while respecting author limits, with a real-data evaluation over 11 years. The problem is well motivated, the baseline comparison is sensible, and the reported running times (under a minute on ICLR scale) make the approach attractive. The paper also has useful empirical content: it collects and cleans a large public dataset and presents a clear per-year/per-limit comparison. However, the core algorithmic guarantee underlying the 'Ours' column is not established: the proof of Theorem 4.5 omits the key existence argument, and a concrete feasible LP solution causes Algorithm 4 to fail. The paper also claims NP-hardness without proof. These issues are load-bearing because the empirical results are only meaningful if MaxRounding always produces a feasible binary vector.

major comments (3)
  1. [Section 4.2, Algorithm 4, Theorem 4.5] The correctness proof does not establish the existence of the set S_i required in line 14, and the statement as written is false. Consider one author i with b=2 and three papers: an integral paper with x=1, a paper l with x_l=0.9, and a paper f with x=0.05. The LP solution is feasible since 1+0.9+0.05=1.95 <= 2. After setting ex_l=1, the author's load is 2.05 > b. The algorithm then asks for a subset S_i of the remaining fractional papers with total mass at least 1-x_l=0.1, but the only such paper has mass 0.05, so no qualifying subset exists and the algorithm cannot proceed. Thus Theorem 4.5 is not merely missing a proof step; its asserted guarantee is false for feasible fractional inputs. This directly affects Table 4, whose 'Ours' rows depend on MaxRounding returning a feasible {0,1} vector. The fix is to require a subset with total mass at least the actual excess D_i = max(0, 1 + sum_{j in T_i \ {l}} ex_j - b); by the LP constraint, D_i never exceeds the total remaining fractional mass of author i, so such a subset always exists. The current threshold 1-x_l is unjustified and can be too large.
  2. [Introduction, Section 4.2] The paper states in the introduction that it 'establish[es] the computational hardness of the problem' and in Section 4.2 says the maximum desk-acceptance problem 'cannot be solved efficiently in general,' but no hardness reduction or formal theorem is provided. The only supporting argument is a reference to the multidimensional knapsack problem. Since the problem is a cardinality-constrained set packing over hyperedges, an NP-hardness proof would be nontrivial and should be given, or the claim should be removed and the paper should instead state that no polynomial-time algorithm is known. As written, this contribution is unsupported.
  3. [Section 5.1, Reproducibility] The experiments are described as deterministic ('The experiments are deterministic and contain no randomness'), but Algorithm 5 begins with 'Randomly initialize x0' and no random seed is reported. If the randomness in x0 is irrelevant to the solver's output, please state so explicitly; if not, the single-run results without variances are not reproducible from the information given.
minor comments (5)
  1. [Section 3.3, Proposition 3.5 proof] The proof of Proposition 3.5 says 'the algorithm iterates through all the n papers and each iteration i in [n] ends in O(k1) time,' but Algorithm 1 iterates over authors, not papers; the text should say 'n authors.'
  2. [Section 4.2, Algorithm 4, line 14] There is a typographical error in the displayed condition: 'P j∈Si. exj' has a stray period. Also, the claim that finding S_i takes O(k1) time is not justified for arbitrary fractional weights unless a greedy selection that always succeeds is specified; this is related to the major correctness issue.
  3. [Section 5.1, Datasets] The statement that the OpenReview API may miss papers but 'this gap is small ... and does not affect the validity of our empirical results' is asserted without any quantitative support. Please provide a count of the missing records or another concrete comparison with official numbers, even approximate.
  4. [Section 5.2, first paragraph] The phrase 'we compare our Algorithm 5 with the desk-rejection policies now policies in AI conferences' contains a repeated word; remove the second 'policies.'
  5. [Section 5.1] The word 'exaustively' should be 'exhaustively.'

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the central result is an external-data benchmark with no fitted constants.

full rationale

The central claim is an empirical benchmark, not a derivation from fitted constants. The objective in Definition 4.2 is the same quantity counted in Table 4, but the comparison is against external OpenReview ICLR data and the baseline policies of Section 3.3; no parameter is fit to the reported outcomes. The submission limit b is swept as a policy parameter, the LP solver and rounding have no tunable hyperparameters, and the experiments are reported as deterministic, so the 19.23% figure is not forced by construction. Self-citations such as [CLL+25] and the LP-runtime references are background or standard technical results and are not load-bearing; no uniqueness theorem from the authors' prior work is invoked to rule out alternatives. The main caveats are non-circular: Theorem 4.5's proof asserts without derivation that line 14 can always find a subset Si with sum of ex_j at least 1 - x_l, and the paper does not quantify the OpenReview crawl gap; these are soundness and data-fidelity concerns for the empirical claim, not reductions of the claim to its own inputs. Accordingly, no circular step is identified.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The empirical claim rests on the fidelity of the OpenReview-derived authorship matrix, the identification of current policy with ForwardReject, and the feasibility of the greedy rounding. The paper introduces no invented entities and fits no free parameters; the submission limit b is a swept policy variable rather than a fitted constant.

assumptions (4)
  • domain assumption The OpenReview-derived authorship matrix A correctly represents the official authors of each ICLR submission.
    Section 5.1 states data are crawled via the OpenReview API and that some papers may be missing, but asserts the gap is small without quantifying it. All desk-rejection counts are computed on this matrix.
  • domain assumption ForwardReject (Algorithm 2) is a correct model of the ID-order desk-rejection policy used by venues such as CVPR.
    Sections 3.3 and 5.1 treat rejecting higher submission IDs as the current policy, citing the CVPR 2025 guideline. The baseline comparison depends on this identification.
  • ad hoc to paper BackwardReject (Algorithm 3) is equivalent to ForwardReject and can be omitted from comparison.
    Remark after Algorithm 3 states the equivalence. It is false in general: for b=1 with papers {A}, {A,B}, {B}, ForwardReject accepts 2 papers while BackwardReject accepts 1. Omitting it may miss a stronger conventional baseline.
  • standard math A subset Si with sum_{j in Si} ex_j >= 1 - x_l always exists in Algorithm 4 line 14.
    Theorem 4.5 relies on this existence to restore feasibility after rounding a fractional paper up. The paper does not prove it; it likely follows from LP feasibility and integral paper counts, but the argument is omitted.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Accept More, Reject Less: Reducing up to 19% Unnecessary Desk-Rejections over 11 Years of ICLR Data." pith.science (2026). https://pith.science/paper/BLQDJJCZ

@misc{pith2026250620141,
  author       = {Pith},
  title        = {Pith review of: Accept More, Reject Less: Reducing up to 19% Unnecessary Desk-Rejections over 11 Years of ICLR Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BLQDJJCZ}},
  note         = {Machine review of arXiv:2506.20141}
}
abstract

The explosive growth of AI research has driven paper submissions at flagship AI conferences to unprecedented levels, necessitating many venues in 2025 (e.g., CVPR, ICCV, KDD, AAAI, IJCAI, WSDM) to enforce strict per-author submission limits and to desk-reject any excess papers by simple ID order. While this policy helps reduce reviewer workload, it may unintentionally discard valuable papers and penalize authors' efforts. In this paper, we ask an essential research question on whether it is possible to follow submission limits while minimizing needless rejections. We first formalize the current desk-rejection policies as an optimization problem, and then develop a practical algorithm based on linear programming relaxation and a rounding scheme. Under extensive evaluation on 11 years of real-world ICLR (International Conference on Learning Representations) data, our method preserves up to $19.23\%$ more papers without violating any author limits. Moreover, our algorithm is highly efficient in practice, with all results on ICLR data computed within at most 53.64 seconds. Our work provides a simple and practical desk-rejection strategy that significantly reduces unnecessary rejections, demonstrating strong potential to improve current CS conference submission policies.

Figures

Figures reproduced from arXiv: 2506.20141 by the authors.

Figure 1
Figure 1. ICLR Submission Trend (2013-2025). Total number of paper submissions to ICLR each year from 2013 to 2025, plotted on a log scale. Data for 2015 and 2016 are omitted due to missing records in OpenReview. The era of artificial intelligence (AI) is rapidly unfolding, with numerous breakthroughs across a wide range of real-world applications. Notable examples include visual generation [SME21, HSG+22, BRL+23], language r… view at source ↗
Figure 2
Figure 2. Submission Frequency Bar Charts for ICLR 2023–2025. 5.1 Experimental Settings Datasets. Desk-rejection data from most conferences (e.g., CVPR, KDD) is not public and can be accessed only by the conference chairs. ICLR is the only venue with public submission records, so we evaluate our method on ICLR data. We obtain the data through the OpenReview API3 and run simulation experiments. Specifically, we exaustively col… view at source ↗
Figure 3
Figure 3. Submission Frequency Bar Charts for ICLR 2013–2014. 1 2 3 4 5 6 7 8 Number of Submissions 10 0 10 1 10 2 10 3 Frequency 1247 162 32 19 7 3 3 1 Submission Frequency Bar Chart for ICLR 2017 (a) ICLR 2017 1 2 3 4 5 6 7 8 9 11 12 Number of Submissions 10 0 10 1 10 2 10 3 Frequency 2366 326 79 28 8 5 1 3 1 1 2 Submission Frequency Bar Chart for ICLR 2018 (b) ICLR 2018 [PITH_FULL_IMAGE:figures/full_fig_p017_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Submission Frequency Bar Charts for ICLR 2017–2018. B Conference Submission Limit Policies In the introduction, [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]
Figure 5
Figure 5. Figure 5: Submission Frequency Bar Charts for ICLR 2019–2020. 1 2 3 4 5 6 7 8 9 10 12 13 14 15 16 18 21 30 Number of Submissions 10 0 10 1 10 2 10 3 10 4 Frequency 6394 991 314 116 49 35 21 18 8 5 2 4 2 1 1 1 1 1 Submission Frequency Bar Chart for ICLR 2021 (a) ICLR 2021 1 2 3 4…
Figure 6
Figure 6. Figure 6: Submission Frequency Bar Charts for ICLR 2021–2022. 17 [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

23 extracted references · 16 canonical work pages

  1. [1]

    Gpt-4 technical report

    [AAA+23] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Flo- rencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

    arXiv

  2. [9]

    wsdm-conference.org/2024/call-for-papers/ ,

    https://www. wsdm-conference.org/2024/call-for-papers/ ,

    work page 2024

  3. [10]

    thecvf.com/Conferences/2025/CVPRChanges,

    https://cvpr. thecvf.com/Conferences/2025/CVPRChanges,

    work page 2025

  4. [11]

    thecvf.com/Conferences/2025/AuthorGuidelines,

    https://iccv. thecvf.com/Conferences/2025/AuthorGuidelines,

    work page 2025

  5. [12]

    https:// ieee-icde.org/2025/cfp/,

    work page 2025

  6. [13]

    ijcai.org/call-for-papers-main-track/ ,

    https://2025. ijcai.org/call-for-papers-main-track/ ,

    work page 2025

  7. [14]

    wsdm-conference.org/2025/call-for-papers/ ,

    https://www. wsdm-conference.org/2025/call-for-papers/ ,

    work page 2025

  8. [15]

    A faster small treewidth sdp solver

    [GS22] Yuzhou Gu and Zhao Song. A faster small treewidth sdp solver. arXiv preprint arXiv:2211.06033,

    arXiv

Show all 23 references
  1. [16]

    A fast optimization view: Reformulating single layer attention in llm based on tensor and svm trick, and solving it in matrix multiplication time

    [GSWY23] Yeqi Gao, Zhao Song, Weixin Wang, and Junze Yin. A fast optimization view: Reformulating single layer attention in llm based on tensor and svm trick, and solving it in matrix multiplication time. arXiv preprint arXiv:2309.07418 ,

    arXiv

  2. [17]

    A nearly-linear time algorithm for structured support vector machines

    [GSZ23] Yuzhou Gu, Zhao Song, and Lichen Zhang. A nearly-linear time algorithm for structured support vector machines. arXiv preprint arXiv:2307.07735 ,

    arXiv

  3. [18]

    A faster quantum algorithm for semidefinite programming via robust ipm frame- work

    [HJS+22a] Baihe Huang, Shunhua Jiang, Zhao Song, Runzhou Tao, and Ruizhe Zhang. A faster quantum algorithm for semidefinite programming via robust ipm frame- work. arXiv preprint arXiv:2207.11154 ,

    arXiv

  4. [19]

    Solv- ing sdp faster: A robust ipm framework and efficient implementation

    [HJS+22b] Baihe Huang, Shunhua Jiang, Zhao Song, Runzhou Tao, and Ruizhe Zhang. Solv- ing sdp faster: A robust ipm framework and efficient implementation. In 2022 IEEE 63rd Annual Symposium on Foundations of Computer Science (FOCS) , pages 233–244. IEEE,

    2022

  5. [23]

    Streaming semidefinite pro- grams: o(\sqrt{n}) passes, small space and fast runtime

    [SYZ23] Zhao Song, Mingquan Ye, and Lichen Zhang. Streaming semidefinite pro- grams: o(\sqrt{n}) passes, small space and fast runtime. arXiv preprint arXiv:2309.05135,

    arXiv

  6. [1983]

    Path finding i: Solving linear programs with \˜ o (sqrt (rank)) linear system solves

    [LS13] Yin Tat Lee and Aaron Sidford. Path finding i: Solving linear programs with \˜ o (sqrt (rank)) linear system solves. arXiv preprint arXiv:1312.6677 ,

    arXiv

  7. [2003]

    Speeding up sparsification using inner product search data structures

    [SXZ22] Zhao Song, Zhaozhuo Xu, and Lichen Zhang. Speeding up sparsification using inner product search data structures. arXiv preprint arXiv:2204.03209 ,

    arXiv

  8. [2012]

    Time to rethink the publication process in machine learning

    [Ben20] Yoshua Bengio. Time to rethink the publication process in machine learning. https://yoshuabengio.org/2020/02/26/time-to-rethink-the-publication- process-in-machine-learning/,

    2020

  9. [2014]

    Solving linear programs with sqrt (rank) linear system solves

    [LS19] Yin Tat Lee and Aaron Sidford. Solving linear programs with sqrt (rank) linear system solves. arXiv preprint arXiv:1910.08033 ,

    1910 arXiv

  10. [2020]

    wsdm-conference.org/2020/call-for-papers.php ,

    https://www. wsdm-conference.org/2020/call-for-papers.php ,

    2020

  11. [2021]

    wsdm-conference.org/2021/call-for-papers.php ,

    https://www. wsdm-conference.org/2021/call-for-papers.php ,

    2021

  12. [2022]

    wsdm-conference.org/2022/calls/,

    https://www. wsdm-conference.org/2022/calls/,

    2022

  13. [2023]

    wsdm-conference.org/2023/calls/call-papers/,

    https://www. wsdm-conference.org/2023/calls/call-papers/,

    2023

  14. [2024]

    thecvf.com/Conferences/2024/AuthorGuidelines,

    https://cvpr. thecvf.com/Conferences/2024/AuthorGuidelines,

    2024

  15. [2025]

    [BBM+24] Tamay Besiroglu, Sage Andrus Bergerson, Amelia Michael, Lennart Heim, Xueyun Luo, and Neil Thompson

    https://ssrn.com/abstract=5127931. [BBM+24] Tamay Besiroglu, Sage Andrus Bergerson, Amelia Michael, Lennart Heim, Xueyun Luo, and Neil Thompson. The compute divide in machine learning: A threat to academic contribution and scrutiny? arXiv preprint arXiv:2401.02452 ,

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.