Paper Citation Record · LEDGER

Changing Answer Order Can Decrease MMLU Accuracy

As of 12 August 2026, this Paper Citation Record lists 0 of 0 outbound references and 12 inbound Pith citation observations for arXiv:2406.19470.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2406.19470 v2

Coverage vector

measured 0 of 0 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links

measured 12 of 12 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-12T06:34:41.77262+00:00

measured 12 of 12 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-08-12T16:16:01.149922Z

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: arxiv_reference, observed 2026-05-20T19:13:40.348932Z

Reference resolution

0 of 0 outbound references displayed

verified exact0
verified fuzzy0
unresolved0
parse uncertain0
malformed identifier0
metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

No outbound reference observations are available for this paper version.

Pith citing papers

Observation e02a791e-bf25-4157-9fa1-512b9e929dbd · inbound

GenBFA: An Evolutionary Optimization Approach to Bit-Flip Attacks on LLMs cites this paper.

GenBFA: An Evolutionary Optimization Approach to Bit-Flip Attacks on LLMs Changing Answer Order Can Decrease MMLU Accuracy

Reference 34

Resolution: unresolved; no resolver link, observed 2026-08-12T16:16:01.149922Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2857396a-2651-45c1-8a74-0d9036d5753b · inbound

Arabic Stable LM: Adapting Stable LM 2 1.6B to Arabic cites this paper.

Arabic Stable LM: Adapting Stable LM 2 1.6B to Arabic Changing Answer Order Can Decrease MMLU Accuracy

Reference 14

Resolution: unresolved; no resolver link, observed 2026-08-11T21:37:26.940619Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 90f8dd49-73ab-4ce4-93e8-3b5545222d3d · inbound

HARP: A challenging human-annotated math reasoning benchmark cites this paper.

HARP: A challenging human-annotated math reasoning benchmark Changing Answer Order Can Decrease MMLU Accuracy

Reference 18

Resolution: unresolved; no resolver link, observed 2026-08-11T17:34:19.662284Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b30f42db-93d8-4bae-9d56-2409d23f0d8f · inbound

Too Big to Fool: Resisting Deception in Language Models cites this paper.

Too Big to Fool: Resisting Deception in Language Models Changing Answer Order Can Decrease MMLU Accuracy

Reference 18

Resolution: unresolved; no resolver link, observed 2026-08-11T15:55:23.169547Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a19bbbb6-d29e-45bf-bdab-8992c9615c43 · inbound

MMLU-CF: A Contamination-free Multi-task Language Understanding Benchmark cites this paper.

MMLU-CF: A Contamination-free Multi-task Language Understanding Benchmark Changing Answer Order Can Decrease MMLU Accuracy

Reference 17

Resolution: unresolved; no resolver link, observed 2026-08-11T11:36:56.261554Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation cdbfaff9-978e-4087-96fd-5f1321e991a6 · inbound

RoToR: Towards More Reliable Responses for Order-Invariant Inputs cites this paper.

RoToR: Towards More Reliable Responses for Order-Invariant Inputs Changing Answer Order Can Decrease MMLU Accuracy

Reference 11

Resolution: unresolved; no resolver link, observed 2026-08-08T16:11:43.132926Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b6d1f93e-e3dd-4009-9000-70e8e9a4973f · inbound

Benchmarking the Pedagogical Knowledge of Large Language Models cites this paper.

Benchmarking the Pedagogical Knowledge of Large Language Models Changing Answer Order Can Decrease MMLU Accuracy

Reference 9

Resolution: unresolved; no resolver link, observed 2026-08-06T23:20:18.169890Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 52594fd5-0e98-41bc-a90e-0ea4c98c651b · inbound

Position: Stop Evaluating AI with Human Tests, Develop Principled, AI-specific Tests instead cites this paper.

Position: Stop Evaluating AI with Human Tests, Develop Principled, AI-specific Tests instead Changing Answer Order Can Decrease MMLU Accuracy

Reference 29

Resolution: verified exact; arxiv_id, observed 2026-05-19T02:12:55.777888Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 4f8cf4c5-2b88-49df-9087-696db27e308e · inbound

EsoLang-Bench: Evaluating Genuine Reasoning in Large Language Models via Esoteric Programming Languages cites this paper.

EsoLang-Bench: Evaluating Genuine Reasoning in Large Language Models via Esoteric Programming Languages Changing Answer Order Can Decrease MMLU Accuracy

Reference 1

Resolution: verified exact; arxiv_id, observed 2026-05-15T13:35:51.808802Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 372b4a57-b4d2-44ac-bc76-49c7410da6f5 · inbound

From 0-Order Selection to 2-Order Judgment: Combinatorial Hardening Exposes Compositional Failures in Frontier LLMs cites this paper.

From 0-Order Selection to 2-Order Judgment: Combinatorial Hardening Exposes Compositional Failures in Frontier LLMs Changing Answer Order Can Decrease MMLU Accuracy

Reference 10

Resolution: verified exact; arxiv_id, observed 2026-05-11T03:15:55.896876Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 088feab2-b4fc-47ed-975b-e960f681f599 · inbound

Syntax Without Semantics: Teaching Large Language Models to Code in an Unseen Language cites this paper.

Syntax Without Semantics: Teaching Large Language Models to Code in an Unseen Language Changing Answer Order Can Decrease MMLU Accuracy

Reference 4

Resolution: verified exact; arxiv_id, observed 2026-05-20T19:13:40.350702Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 31f1a6c9-2d62-42d4-9011-825f53ad4b26 · inbound

Syntax Without Semantics: Teaching Large Language Models to Code in an Unseen Language cites this paper.

Syntax Without Semantics: Teaching Large Language Models to Code in an Unseen Language Changing Answer Order Can Decrease MMLU Accuracy

Reference 2025

Resolution: unresolved; no resolver link, observed 2026-08-04T05:10:06.677483Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.