Codifying the Judge: Scalable Evaluation via Program Distillation | Paper Citation Record

Observation 86e1de7f-a778-4f37-b864-25f0a17054cb · outbound

This paper cites The Innovation2024,.

Codifying the Judge: Scalable Evaluation via Program Distillation The Innovation2024,

Reference 1

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:08.649159Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 01558392-8d9f-4d4f-8b29-ad399e7bec6c · outbound

This paper cites LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods.

Codifying the Judge: Scalable Evaluation via Program Distillation LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods

Reference 2

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:08.695963Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ecf22b85-92cc-40fe-b100-5e6b4c305e73 · outbound

This paper cites JudgeLM: Fine-tuned Large Language Models are Scalable Judges.

Codifying the Judge: Scalable Evaluation via Program Distillation JudgeLM: Fine-tuned Large Language Models are Scalable Judges

Reference 3

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:08.778979Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 982707aa-4572-4e05-8f3c-0c82a4e48702 · outbound

This paper cites Prometheus: Inducing Fine-Grained Evaluation Capability in Language Models.

Codifying the Judge: Scalable Evaluation via Program Distillation Prometheus: Inducing Fine-Grained Evaluation Capability in Language Models

Reference 4

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:08.848395Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d5dff37d-8461-4dd7-9a71-42399f72f72a · outbound

This paper cites PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization.

Codifying the Judge: Scalable Evaluation via Program Distillation PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization

Reference 5

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:08.936162Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f2322a79-8fa5-4ede-aa5f-3755fba4a169 · outbound

This paper cites Learning LLM-as-a-Judge for Preference Alignment.

Codifying the Judge: Scalable Evaluation via Program Distillation Learning LLM-as-a-Judge for Preference Alignment

Reference 6

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:09.014344Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation fa6b12f0-4025-49d1-bd16-51766b628baa · outbound

This paper cites E.; Sukhbaatar, S.

Codifying the Judge: Scalable Evaluation via Program Distillation E.; Sukhbaatar, S

Reference 7

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:09.154738Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 551babe2-fe27-43d0-a6b4-11cc905ba28b · outbound

This paper cites Self-Taught Evaluators.

Codifying the Judge: Scalable Evaluation via Program Distillation Self-Taught Evaluators

Reference 8

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:09.307551Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a3e219fa-4939-48a4-9944-7c4e242e9ca4 · outbound

This paper cites an unresolved cited work.

Codifying the Judge: Scalable Evaluation via Program Distillation Unresolved cited work

Reference 9

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:09.465680Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 692c4f01-7ed5-4e87-a5f3-152978951f54 · outbound

This paper cites RubiCap: Rubric-Guided Reinforcement Learning for Dense Image Captioning.arXiv preprint arXiv:2603.091602026,.

Codifying the Judge: Scalable Evaluation via Program Distillation RubiCap: Rubric-Guided Reinforcement Learning for Dense Image Captioning.arXiv preprint arXiv:2603.091602026,

Reference 10

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:09.566172Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2e4da422-f945-49b9-97d0-d9073f02948c · outbound

This paper cites DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research.

Codifying the Judge: Scalable Evaluation via Program Distillation DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research

Reference 11

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:09.660219Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1bb26942-3e47-4c4e-9ad7-9c275163ca29 · outbound

This paper cites OpenAI GPT-5 System Card.

Codifying the Judge: Scalable Evaluation via Program Distillation OpenAI GPT-5 System Card

Reference 12

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:09.743492Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 79f67867-9df5-455f-8c47-902e03dca5a8 · outbound

This paper cites an unresolved cited work.

Codifying the Judge: Scalable Evaluation via Program Distillation Unresolved cited work

Reference 13

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:10.029536Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ac4a0ddb-416b-4301-9931-24de8facd88e · outbound

This paper cites Tuning LLM Judge Design Decisions for 1/1000 of the Cost.

Codifying the Judge: Scalable Evaluation via Program Distillation Tuning LLM Judge Design Decisions for 1/1000 of the Cost

Reference 14

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:10.144752Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ee393540-d4c3-4fec-88af-3568e6d6f7fc · outbound

This paper cites One Token to Fool LLM-as-a-Judge.

Codifying the Judge: Scalable Evaluation via Program Distillation One Token to Fool LLM-as-a-Judge

Reference 15

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:10.192392Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2d5f8df9-50ee-452d-9a34-259c34739a6d · outbound

This paper cites Investigating the Vulnerability of LLM-as-a-Judge Architectures to Prompt-Injection Attacks.International Journal of Open Information Technologies2025,13, 1–6.

Codifying the Judge: Scalable Evaluation via Program Distillation Investigating the Vulnerability of LLM-as-a-Judge Architectures to Prompt-Injection Attacks.International Journal of Open Information Technologies2025,13, 1–6

Reference 16

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:10.318374Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8470c192-28f4-48fe-8de0-cb5deaf10e2e · outbound

This paper cites H.; Chen, S.; Liu, Z.; Jiang, F.; Wang, B.

Codifying the Judge: Scalable Evaluation via Program Distillation H.; Chen, S.; Liu, Z.; Jiang, F.; Wang, B

Reference 17

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:10.402306Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3492a5f2-1299-4b1c-8cbd-1596e2d581e3 · outbound

This paper cites V .; Zhang, X.

Codifying the Judge: Scalable Evaluation via Program Distillation V .; Zhang, X

Reference 18

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:10.534828Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c79fa538-6edf-4dfa-bf87-390c4602cf7e · outbound

This paper cites Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge.

Codifying the Judge: Scalable Evaluation via Program Distillation Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge

Reference 19

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:10.661038Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 98c7724a-5157-4c4d-a965-2f801e888550 · outbound

This paper cites Can You Trust LLM Judgments? Reliability of LLM-as-a-Judge.

Codifying the Judge: Scalable Evaluation via Program Distillation Can You Trust LLM Judgments? Reliability of LLM-as-a-Judge

Reference 20

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:10.833195Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 067c43fb-443f-48de-9d0d-33839da46a04 · outbound

This paper cites an unresolved cited work.

Codifying the Judge: Scalable Evaluation via Program Distillation Unresolved cited work

Reference 21

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:10.976810Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 14251072-9c7e-4146-a569-207157793ddb · outbound

This paper cites an unresolved cited work.

Codifying the Judge: Scalable Evaluation via Program Distillation Unresolved cited work

Reference 22

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:11.205985Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 5f203a7a-6cf1-46db-9670-f99a231fe8f8 · outbound

This paper cites Qwen2.5 Technical Report.

Codifying the Judge: Scalable Evaluation via Program Distillation Qwen2.5 Technical Report

Reference 23

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:11.369244Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f98e9cd4-c47e-4fdc-ac47-45109f789985 · outbound

This paper cites Y .; Chandu, K.; Dziri, N.; Kumar, S.; Zick, T.; Choi, Y .; Smith, N.

Codifying the Judge: Scalable Evaluation via Program Distillation Y .; Chandu, K.; Dziri, N.; Kumar, S.; Zick, T.; Choi, Y .; Smith, N

Reference 24

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:11.475810Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation fd0e2a7d-54ca-45d1-9fa0-67158dde2430 · outbound

This paper cites Can Large Language Models Be an Alternative to Human Evaluations? Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Papers).

Codifying the Judge: Scalable Evaluation via Program Distillation Can Large Language Models Be an Alternative to Human Evaluations? Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Papers)

Reference 25

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:11.552215Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ccf2ed11-0870-4836-a005-5c59b7508932 · outbound

This paper cites Length-Controlled AlpacaEval: A Simple Debiasing of Automatic Evaluators.

Codifying the Judge: Scalable Evaluation via Program Distillation Length-Controlled AlpacaEval: A Simple Debiasing of Automatic Evaluators

Reference 26

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:11.717579Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 4137f46c-b3bd-48d6-9f55-1b4777ede575 · outbound

This paper cites an unresolved cited work.

Codifying the Judge: Scalable Evaluation via Program Distillation Unresolved cited work

Reference 27

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:11.908400Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 931a5c6f-6440-43c1-b0f0-36d76d60e6c4 · outbound

This paper cites an unresolved cited work.

Codifying the Judge: Scalable Evaluation via Program Distillation Unresolved cited work

Reference 28

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:12.026141Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 47b2055a-3bec-4df3-822a-77ad7dc25a64 · outbound

This paper cites Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback.

Codifying the Judge: Scalable Evaluation via Program Distillation Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback

Reference 29

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:12.179514Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e5e194c0-6c27-454a-b7d0-83c4cbfdddd6 · outbound

This paper cites J.; Choi, E.

Codifying the Judge: Scalable Evaluation via Program Distillation J.; Choi, E

Reference 30

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:12.289782Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7c207e63-7eca-40eb-a82e-2dfde7aa19cc · outbound

This paper cites H.; Ehrenberg, H.; Fries, J.; Wu, S.; Ré, C.

Codifying the Judge: Scalable Evaluation via Program Distillation H.; Ehrenberg, H.; Fries, J.; Wu, S.; Ré, C

Reference 31

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:12.401722Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b1f9d5c9-47a4-4d32-9247-8748132fa2a1 · outbound

This paper cites J.; De Sa, C.

Codifying the Judge: Scalable Evaluation via Program Distillation J.; De Sa, C

Reference 32

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:12.506541Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation be779dee-688c-42c5-8bd7-ab74b7bac719 · outbound

This paper cites Training complex models with multi-task weak supervision.

Codifying the Judge: Scalable Evaluation via Program Distillation Training complex models with multi-task weak supervision

Reference 33

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:12.652292Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e4cdfcc3-750c-40c4-978f-517366be634c · outbound

This paper cites C.; Sala, F.

Codifying the Judge: Scalable Evaluation via Program Distillation C.; Sala, F

Reference 34

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:12.710884Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6b9539bb-38ba-468a-9a4a-aecae3af7576 · outbound

This paper cites Snorkel metal: Weak supervision for multi-task learning.

Codifying the Judge: Scalable Evaluation via Program Distillation Snorkel metal: Weak supervision for multi-task learning

Reference 35

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:12.797978Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 69688967-3854-4867-b05d-ef24a6b0b7e7 · outbound

This paper cites The ALCHEmist: Automated Labeling 500x CHEaper than LLM Data Annotators.

Codifying the Judge: Scalable Evaluation via Program Distillation The ALCHEmist: Automated Labeling 500x CHEaper than LLM Data Annotators

Reference 36

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:12.871922Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9074d841-9b15-4eaa-afda-0103ef88c704 · outbound

This paper cites ScriptoriumWS: A Code Generation Assistant for Weak Supervision.

Codifying the Judge: Scalable Evaluation via Program Distillation ScriptoriumWS: A Code Generation Assistant for Weak Supervision

Reference 37

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:12.955916Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0cbb58af-0106-4b1f-8f1a-46171ab88dfb · outbound

This paper cites Autows-bench-101: Benchmarking automated weak supervision with 100 labels.Advances in Neural Information Processing Systems2022,35, 8912–8925.

Codifying the Judge: Scalable Evaluation via Program Distillation Autows-bench-101: Benchmarking automated weak supervision with 100 labels.Advances in Neural Information Processing Systems2022,35, 8912–8925

Reference 38

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:13.104765Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 52b68521-9fa9-43d3-a7b5-acdabd9cc56f · outbound

This paper cites Multimodal Data Curation via Object Detection and Filter Ensembles.

Codifying the Judge: Scalable Evaluation via Program Distillation Multimodal Data Curation via Object Detection and Filter Ensembles

Reference 39

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:13.266271Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0f091e68-a15d-4e97-a867-0803e419f75a · outbound

This paper cites H.; Kellman, P.; Xue, H.; Sala, F.; Langlotz, C.; Re, C.

Codifying the Judge: Scalable Evaluation via Program Distillation H.; Kellman, P.; Xue, H.; Sala, F.; Langlotz, C.; Re, C

Reference 40

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:13.421063Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation cb36c387-e8d9-46d5-bfe6-cb19d39bc201 · outbound

This paper cites F.; Orr, L.; Guha, N.; Bhatia, K.; Chami, I.; Re, C.

Codifying the Judge: Scalable Evaluation via Program Distillation F.; Orr, L.; Guha, N.; Bhatia, K.; Chami, I.; Re, C

Reference 41

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:13.584975Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c742bd23-a3c9-4cec-8981-1e09c21872ce · outbound

This paper cites K.; Chen, M.

Codifying the Judge: Scalable Evaluation via Program Distillation K.; Chen, M

Reference 42

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:13.667192Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2ded89f0-b699-425e-82ca-d639e3dde50c · outbound

This paper cites Evaluating Sample Utility for Efficient Data Selection by Mimicking Model Weights.

Codifying the Judge: Scalable Evaluation via Program Distillation Evaluating Sample Utility for Efficient Data Selection by Mimicking Model Weights

Reference 43

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:13.711886Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ebe747bd-5e49-4240-b4b2-87c993747339 · outbound

This paper cites Lifting weak supervision to structured prediction.Advances in Neural Information Processing Systems2022,35, 37563–37574.

Codifying the Judge: Scalable Evaluation via Program Distillation Lifting weak supervision to structured prediction.Advances in Neural Information Processing Systems2022,35, 37563–37574

Reference 44

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:13.847612Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7f9cd947-6aaa-4bf4-a86e-f97b5e211a91 · outbound

This paper cites E.; Kadous, M.

Codifying the Judge: Scalable Evaluation via Program Distillation E.; Kadous, M

Reference 45

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:13.897344Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 47a729ab-78e8-48ca-8e24-396f0b71bbfd · outbound

This paper cites J.; Bieker, J.; Li, X.; Jiang, N.; Keigwin, B.; Ranganath, G.; Keutzer, K.; Upadhyay, S.

Codifying the Judge: Scalable Evaluation via Program Distillation J.; Bieker, J.; Li, X.; Jiang, N.; Keigwin, B.; Ranganath, G.; Keutzer, K.; Upadhyay, S

Reference 46

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:14.064491Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1191ef7a-8406-489d-9ec4-ed2fc6cb1a96 · outbound

This paper cites an unresolved cited work.

Codifying the Judge: Scalable Evaluation via Program Distillation Unresolved cited work

Reference 47

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:14.217591Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3108bc72-b255-446c-81ab-ef06443dacdb · outbound

This paper cites S.; Juneja, J.; Wang, C.; Wang, Z.; Go, A.; Lee, C.-Y .; Shenoy, P.; Panigrahy, R.; Menon, A.

Codifying the Judge: Scalable Evaluation via Program Distillation S.; Juneja, J.; Wang, C.; Wang, Z.; Go, A.; Lee, C.-Y .; Shenoy, P.; Panigrahy, R.; Menon, A

Reference 48

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:14.333230Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ec40c102-8d13-4c0a-9139-057f0720bb22 · outbound

This paper cites Large Language Model Routing with Benchmark Datasets.

Codifying the Judge: Scalable Evaluation via Program Distillation Large Language Model Routing with Benchmark Datasets

Reference 49

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:14.475806Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 02d9e9dc-bb67-449c-9c5d-15c508457a90 · outbound

This paper cites Tryage: Real-time, intelligent Routing of User Prompts to Large Language Models.

Codifying the Judge: Scalable Evaluation via Program Distillation Tryage: Real-time, intelligent Routing of User Prompts to Large Language Models

Reference 50

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:14.576579Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 47568622-746b-47b1-905d-6e57b949d26a · outbound

This paper cites Routing to the expert: Efficient reward-guided ensemble of large language models.

Codifying the Judge: Scalable Evaluation via Program Distillation Routing to the expert: Efficient reward-guided ensemble of large language models

Reference 51

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:14.636858Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 88c1bd85-3293-420c-80d1-31d6fc73e99c · outbound

This paper cites RLHF Workflow: From Reward Modeling to Online RLHF.

Codifying the Judge: Scalable Evaluation via Program Distillation RLHF Workflow: From Reward Modeling to Online RLHF

Reference 52

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:14.702968Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation fb930b25-575e-4929-a44f-a4ad4280e1be · outbound

This paper cites an unresolved cited work.

Codifying the Judge: Scalable Evaluation via Program Distillation Unresolved cited work

Reference 53

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:14.760278Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 98619d9d-e035-4a24-add6-8a15912df9cd · outbound

This paper cites GPT-4 Technical Report.

Codifying the Judge: Scalable Evaluation via Program Distillation GPT-4 Technical Report

Reference 54

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:14.827234Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6e13ec76-17d1-4044-9b42-bd60f501f8fd · outbound

This paper cites Gemma 3 Technical Report.

Codifying the Judge: Scalable Evaluation via Program Distillation Gemma 3 Technical Report

Reference 55

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:14.884047Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 5415b1cf-b0a0-4ac7-bbd4-dd7e73dccdfe · outbound

This paper cites good” vs “bad.

Codifying the Judge: Scalable Evaluation via Program Distillation good” vs “bad

Reference 56

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:14.961390Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 60d6cea8-3c47-4cdb-bb81-302f12f4c89f · outbound

This paper cites Evaluate how semantically relevant the response is to the question asked.

Codifying the Judge: Scalable Evaluation via Program Distillation Evaluate how semantically relevant the response is to the question asked

Reference 57

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:15.124540Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1246e7e4-982a-4b90-aab7-0220ec51c94a · outbound

This paper cites Evaluate language quality and readability.

Codifying the Judge: Scalable Evaluation via Program Distillation Evaluate language quality and readability

Reference 58

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:15.280292Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 299a4bd9-bc21-4af5-af5d-b9c4ac7377a2 · outbound

This paper cites Evaluate completeness and thoroughness of the answer.

Codifying the Judge: Scalable Evaluation via Program Distillation Evaluate completeness and thoroughness of the answer

Reference 59

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:15.367431Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e2b18d21-5906-44df-9df0-d57c288b464a · outbound

This paper cites Evaluate indicators of factual reliability.

Codifying the Judge: Scalable Evaluation via Program Distillation Evaluate indicators of factual reliability

Reference 60

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:15.439853Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a063c678-6285-43f2-aa9e-5db083abcf03 · outbound

This paper cites Evaluate logical coherence.

Codifying the Judge: Scalable Evaluation via Program Distillation Evaluate logical coherence

Reference 61

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:15.496430Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a6fd832a-e985-4d7f-8647-ed91a818dcf6 · outbound

This paper cites Evaluate clarity and conciseness.

Codifying the Judge: Scalable Evaluation via Program Distillation Evaluate clarity and conciseness

Reference 62

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:15.566520Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 658a2b09-1910-4d29-aaee-a1197dc2c780 · outbound

This paper cites Evaluate how transparently the response shows its reasoning process.

Codifying the Judge: Scalable Evaluation via Program Distillation Evaluate how transparently the response shows its reasoning process

Reference 63

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:15.670868Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 721afa0a-df22-4097-af73-4fd4ad3bc272 · outbound

This paper cites likely”, “research suggests.

Codifying the Judge: Scalable Evaluation via Program Distillation likely”, “research suggests

Reference 64

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:15.769133Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e998f238-1b0d-4448-b982-d584ee52b18d · outbound

This paper cites Evaluate the structural organization of the response.

Codifying the Judge: Scalable Evaluation via Program Distillation Evaluate the structural organization of the response

Reference 65

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:15.895168Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2cc68719-540e-4b66-8942-9309a55c9f26 · outbound

This paper cites many people think.

Codifying the Judge: Scalable Evaluation via Program Distillation many people think

Reference 66

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:16.057537Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8c6bc9f8-7b6c-4d4f-9c9d-da8eb1160fa0 · outbound

This paper cites One answer is rewritten from a male-only perspective.

Codifying the Judge: Scalable Evaluation via Program Distillation One answer is rewritten from a male-only perspective

Reference 67

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:16.134604Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c268a055-77ab-40ae-964f-5a079011a096 · outbound

This paper cites One answer is reformatted with markdown, headers, bullet points, and emojis, while preserving the same substantive content.

Codifying the Judge: Scalable Evaluation via Program Distillation One answer is reformatted with markdown, headers, bullet points, and emojis, while preserving the same substantive content

Reference 68

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:16.294752Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f9ea5a41-3f31-4e0d-87b6-7fac20b20396 · outbound

This paper cites One answer is augmented with fake or irrelevant citations.

Codifying the Judge: Scalable Evaluation via Program Distillation One answer is augmented with fake or irrelevant citations

Reference 69

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:16.507514Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8dbe153d-bc2f-4005-a641-8b8c17b3ec5d · outbound

This paper cites One answer is padded with filler text to make it longer.

Codifying the Judge: Scalable Evaluation via Program Distillation One answer is padded with filler text to make it longer

Reference 70

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:16.721390Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b9c32b16-3864-4585-bec3-4c7f878e784b · outbound

This paper cites an unresolved cited work.

Codifying the Judge: Scalable Evaluation via Program Distillation Unresolved cited work

Reference 71

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:16.904883Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ecffc8bb-77f9-4ca8-964b-072ec958fafa · outbound

This paper cites an unresolved cited work.

Codifying the Judge: Scalable Evaluation via Program Distillation Unresolved cited work

Reference 72

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:16.994935Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f6a09cdb-fb88-4015-b582-97b1c5a6728c · outbound

This paper cites 18 Table 3: Dataset statistics after filtering, along with the source of ground-truth preference labels.

Codifying the Judge: Scalable Evaluation via Program Distillation 18 Table 3: Dataset statistics after filtering, along with the source of ground-truth preference labels

Reference 73

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:17.137920Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 27ce8947-14df-4858-84ab-7e3de9e68a81 · outbound

This paper cites an unresolved cited work.

Codifying the Judge: Scalable Evaluation via Program Distillation Unresolved cited work

Reference 74

Resolution: unresolved; no resolver link, observed 2026-08-02T12:49:17.255854Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 33dc076d-9319-4d7f-bdb8-4e6975f454d6 · outbound

This paper cites Data Filtering.To construct our evaluation set, we retain only samples with reliable preference signals and discardambiguous, tied, or low-confidence cases.

Codifying the Judge: Scalable Evaluation via Program Distillation Data Filtering.To construct our evaluation set, we retain only samples with reliable preference signals and discardambiguous, tied, or low-confidence cases

Reference 75

Resolution: malformed identifier; no resolver link, observed 2026-08-02T12:49:17.313990Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Coverage vector

Reference resolution

Outbound references

Pith citing papers