Paper Citation Record · LEDGER

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering

As of 18 August 2026, this Paper Citation Record lists 65 of 65 outbound references and 6 inbound Pith citation observations for arXiv:2502.06193.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2502.06193 v3

Coverage vector

measured 65 of 65 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-08T16:31:38.707684Z

measured 71 of 71 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-17T06:30:58.91139+00:00

measured 6 of 6 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-08-15T19:32:36.028070Z

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: pith, observed 2026-08-06T22:55:22.701914Z

Reference resolution

65 of 65 outbound references displayed

verified exact1
verified fuzzy8
unresolved56
parse uncertain0
malformed identifier0
metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation 3868d53e-fb24-4bfc-83f0-d5e6420ae3bb · outbound

This paper cites Program Synthesis with Large Language Models.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Program Synthesis with Large Language Models

Reference 1

Resolution: unresolved; no resolver link, observed 2026-08-08T16:31:38.500723Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e068fdfd-4cd9-4104-a28b-8c29d8280bf3 · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 2

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:39.261024Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation ab8ebfde-a82c-4f38-8e82-d8dffec19534 · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 3

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:39.252552Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation ec301540-cd7d-4044-a9b9-4fd2a3864f9a · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 4

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:39.244734Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 3e2f3999-7198-438a-8654-770fbb9171b8 · outbound

This paper cites DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence

Reference 5

Resolution: unresolved; no resolver link, observed 2026-08-08T16:31:38.518553Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation af6ca29c-4399-45d6-af62-61bd049563cd · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 6

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:39.237275Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation a131398c-471e-415a-9e50-79f619d19e36 · outbound

This paper cites ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generation.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generation

Reference 7

Resolution: unresolved; no resolver link, observed 2026-08-08T16:31:38.525989Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 5bfc84ef-99ff-47b5-87f8-2f2a7df77bb0 · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 8

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:39.228688Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation e08bd5b7-87f0-468a-add7-328667f053fd · outbound

This paper cites ComplexCodeEval: A Benchmark for Evaluating Large Code Models on More Complex Code.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering ComplexCodeEval: A Benchmark for Evaluating Large Code Models on More Complex Code

Reference 9

Resolution: verified exact; local_arxiv, observed 2026-08-08T16:31:38.875140Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 956a22f1-63ee-4de7-b238-c4cfbb466594 · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 10

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:39.220654Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 6c49c30b-b42b-46ba-bfbd-c64740c3c994 · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 11

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:39.210455Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 10ff1f27-ce36-479c-8ab8-6103ae517494 · outbound

This paper cites LLM-based NLG Evaluation: Current Status and Challenges.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering LLM-based NLG Evaluation: Current Status and Challenges

Reference 12

Resolution: unresolved; no resolver link, observed 2026-08-08T16:31:38.543764Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 4e49bdf6-813e-4a77-b009-27f48a3c89e9 · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 13

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:39.201088Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 5ecdf1d1-5b53-41c6-9554-f8d186b0d679 · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 14

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:39.191394Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation c5afba38-8efb-4ca8-806f-7779d19d797f · outbound

This paper cites DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence

Reference 15

Resolution: unresolved; no resolver link, observed 2026-08-08T16:31:38.554014Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 743708d9-63f8-4b77-991f-d47bcf6ba81c · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 16

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:39.183058Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation de1a3493-6bd5-436c-b038-af8b2bd7d887 · outbound

This paper cites Large Language Models for Software Engineering: A Systematic Literature Review.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Large Language Models for Software Engineering: A Systematic Literature Review

Reference 17

Resolution: unresolved; no resolver link, observed 2026-08-08T16:31:38.559051Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 49dcfa92-8cb7-4efb-9907-ca65ed383115 · outbound

This paper cites Qwen2.5-Coder Technical Report.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Qwen2.5-Coder Technical Report

Reference 18

Resolution: unresolved; no resolver link, observed 2026-08-08T16:31:38.561533Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation cc199193-66b6-482e-be04-863e10348b9d · outbound

This paper cites CodeSearchNet Challenge: Evaluating the State of Semantic Code Search.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering CodeSearchNet Challenge: Evaluating the State of Semantic Code Search

Reference 19

Resolution: unresolved; no resolver link, observed 2026-08-08T16:31:38.563930Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 904d21e8-0574-47e2-a38a-40349a14cc5b · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 20

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:39.174756Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 9ff7e2cb-85be-4c34-8efd-055bb94520bd · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 21

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:39.164109Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 683d7c4b-09d5-45cd-aac8-4830e0839131 · outbound

This paper cites Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models

Reference 22

Resolution: unresolved; no resolver link, observed 2026-08-08T16:31:38.573942Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1d6c4246-09cc-420c-9377-60b00d940dd2 · outbound

This paper cites Kusner, Yu Sun, Nicholas I.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Kusner, Yu Sun, Nicholas I

Reference 23

Resolution: verified fuzzy; raw_fallback, observed 2026-08-08T16:31:39.154619Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 625fb6f0-0ab3-477d-bec9-7e1a3d57aae5 · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 24

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:39.145883Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation ddba5173-d099-45cd-a9b4-952d08f55a54 · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 25

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:39.137467Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 2d175b8f-c051-48ec-96ef-c7f3a9aab792 · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 26

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:39.129921Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 851d245e-b47b-4b7a-9f1e-3d80fac4e74e · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 27

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:39.120619Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation dd6b44d1-7f41-4933-8eed-74cefb1c3116 · outbound

This paper cites Split and Merge: Aligning Position Biases in LLM-based Evaluators.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Split and Merge: Aligning Position Biases in LLM-based Evaluators

Reference 28

Resolution: unresolved; no resolver link, observed 2026-08-08T16:31:38.593488Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e4d2f7a9-d5c3-473b-ae23-b0814c4176df · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 29

Resolution: unresolved; no resolver link, observed 2026-08-08T16:31:38.596966Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3741a2cb-f73d-4e7b-83da-6c2bdb6ecad0 · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 30

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:39.105980Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation ec961d15-95b5-46f2-96e3-7adba1e94f81 · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 31

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:39.097227Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 54930fdc-081a-40d5-a139-25abe9c6deba · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 32

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:39.086782Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation f15b9b71-4f8a-46c1-a300-50b8cb1232da · outbound

This paper cites GPT-4 Technical Report.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering GPT-4 Technical Report

Reference 33

Resolution: unresolved; no resolver link, observed 2026-08-08T16:31:38.610051Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c71b889e-340d-4c67-81f3-622e4ff6a820 · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 34

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:39.080053Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation fd462a9e-f213-4fd4-8569-a3c7508c3a0f · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 35

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:39.072503Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation e185c660-b8b8-43b1-8383-3040f4d310a5 · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 36

Resolution: unresolved; no resolver link, observed 2026-08-08T16:31:38.619860Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 397d1667-4f3e-4c19-a58f-c657d54b7ae8 · outbound

This paper cites Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

Reference 37

Resolution: unresolved; no resolver link, observed 2026-08-08T16:31:38.623144Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c8e03438-6b55-474e-8f5e-5b208c04e4a8 · outbound

This paper cites CodeBLEU: a Method for Automatic Evaluation of Code Synthesis.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering CodeBLEU: a Method for Automatic Evaluation of Code Synthesis

Reference 38

Resolution: unresolved; no resolver link, observed 2026-08-08T16:31:38.626832Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3062d33b-5d31-4d8e-9838-71c482ab3be6 · outbound

This paper cites Code Llama: Open Foundation Models for Code.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Code Llama: Open Foundation Models for Code

Reference 39

Resolution: unresolved; no resolver link, observed 2026-08-08T16:31:38.630490Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a74136e1-ddbe-4979-8751-a2538862a7ad · outbound

This paper cites Verbosity Bias in Preference Labeling by Large Language Models.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Verbosity Bias in Preference Labeling by Large Language Models

Reference 40

Resolution: unresolved; no resolver link, observed 2026-08-08T16:31:38.634452Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 712d1f2d-f67c-4ef0-9661-92e09ac45324 · outbound

This paper cites NoFunEval: Funny How Code LMs Falter on Requirements Beyond Functional Correctness.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering NoFunEval: Funny How Code LMs Falter on Requirements Beyond Functional Correctness

Reference 41

Resolution: unresolved; no resolver link, observed 2026-08-08T16:31:38.639436Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 07ce0fdd-0c28-4312-b72f-6fabc4b445cd · outbound

This paper cites Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters

Reference 42

Resolution: unresolved; no resolver link, observed 2026-08-08T16:31:38.643324Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 09a9a8ba-6507-4bc9-8474-ea8107c38b1a · outbound

This paper cites Llama 2: Open Foundation and Fine-Tuned Chat Models.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Llama 2: Open Foundation and Fine-Tuned Chat Models

Reference 43

Resolution: unresolved; no resolver link, observed 2026-08-08T16:31:38.646275Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f02908e9-591c-4cd7-9609-c75c34de2fc5 · outbound

This paper cites Gomez, Lukasz Kaiser, and Illia Polosukhin.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Gomez, Lukasz Kaiser, and Illia Polosukhin

Reference 44

Resolution: verified fuzzy; raw_fallback, observed 2026-08-08T16:31:39.057626Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation bb3c7809-27f8-4148-80a9-857657c5d296 · outbound

This paper cites Learning Personalized Alignment for Evaluating Open-ended Text Generation.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Learning Personalized Alignment for Evaluating Open-ended Text Generation

Reference 45

Resolution: unresolved; no resolver link, observed 2026-08-08T16:31:38.651288Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 128287bf-556e-4083-af8e-bbf2711cf374 · outbound

This paper cites Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering

Reference 46

Resolution: verified fuzzy; raw_fallback, observed 2026-08-08T16:31:39.047541Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 5fe2802b-a071-4324-b8af-ed5358711612 · outbound

This paper cites Joty, and Steven C.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Joty, and Steven C

Reference 47

Resolution: verified fuzzy; raw_fallback, observed 2026-08-08T16:31:39.038228Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 0ea2c352-6e87-4bf7-a24c-75d043bee362 · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 48

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:39.020527Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 9cc4ecad-3d9e-4a3d-886c-59118aa037a1 · outbound

This paper cites Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus

Reference 49

Resolution: verified fuzzy; raw_fallback, observed 2026-08-08T16:31:39.011635Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 77714668-0593-4495-9211-c02e02f2474f · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 50

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:39.028850Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation a45db5d7-53cc-4652-a2bd-cbfa9e67f7f0 · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 51

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:38.997099Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 6b31838a-276d-4e0b-a872-5ebca17882ce · outbound

This paper cites CodeUltraFeedback: An LLM-as-a-Judge Dataset for Aligning Large Language Models to Coding Preferences.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering CodeUltraFeedback: An LLM-as-a-Judge Dataset for Aligning Large Language Models to Coding Preferences

Reference 52

Resolution: unresolved; no resolver link, observed 2026-08-08T16:31:38.677847Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 5af6ff84-71ef-498b-96e8-9a26d7517236 · outbound

This paper cites Chi, Quoc V.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Chi, Quoc V

Reference 53

Resolution: unresolved; no resolver link, observed 2026-08-08T16:31:38.671920Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8e6f94c8-1105-43d0-8855-2bfdbd7709c8 · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 54

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:38.989091Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 48d51f9a-d463-42da-8277-229b8b289b23 · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 55

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:38.978907Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 7a245692-579b-40b4-8cdd-82ee9e57bae4 · outbound

This paper cites HuggingFace's Transformers: State-of-the-art Natural Language Processing.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering HuggingFace's Transformers: State-of-the-art Natural Language Processing

Reference 56

Resolution: unresolved; no resolver link, observed 2026-08-08T16:31:38.680884Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 20b34a92-8a61-4467-b013-4aaf9d9661ab · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 57

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:38.961533Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation c50c506b-7e68-4874-9c53-550c8e21a2da · outbound

This paper cites Weinberger, and Yoav Artzi.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Weinberger, and Yoav Artzi

Reference 58

Resolution: verified fuzzy; raw_fallback, observed 2026-08-08T16:31:38.951345Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation d623d1c9-a276-4439-9a33-7cfe21876ff0 · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 59

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:38.970306Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 10399c38-2c81-4c96-9631-8d37f115350c · outbound

This paper cites Xing, Hao Zhang, Joseph E.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Xing, Hao Zhang, Joseph E

Reference 60

Resolution: verified fuzzy; raw_fallback, observed 2026-08-08T16:31:38.931257Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 35433146-82a4-41f6-a277-f6dfd458f085 · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 61

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:38.923243Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 93b47eda-514b-42c9-8afa-ff8154dc4212 · outbound

This paper cites Meyer, and Steffen Eger.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Meyer, and Steffen Eger

Reference 62

Resolution: verified fuzzy; raw_fallback, observed 2026-08-08T16:31:38.941241Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 909abd9b-028c-42f7-92fe-b16f2773e1c3 · outbound

This paper cites an unresolved cited work.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Unresolved cited work

Reference 65

Resolution: unresolved; raw_fallback, observed 2026-08-08T16:31:38.914535Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 21e5fe0a-f968-4da0-aae1-d8e8ec1fde5a · outbound

This paper cites Evaluating Large Language Models Trained on Code.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Evaluating Large Language Models Trained on Code

Reference 2021

Resolution: unresolved; no resolver link, observed 2026-08-08T16:31:38.514729Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 81b21440-3ee7-4b66-a13d-09cd647d13cc · outbound

This paper cites Mixtral of Experts.

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering Mixtral of Experts

Reference 2024

Resolution: unresolved; no resolver link, observed 2026-08-08T16:31:38.570816Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Pith citing papers

Observation b0bf9b93-84a3-4696-8c9a-dc533fb2cb45 · inbound

Larger Is Not Always Better: Exploring Small Open-source Language Models in Logging Statement Generation cites this paper.

Larger Is Not Always Better: Exploring Small Open-source Language Models in Logging Statement Generation Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering

Reference 75

Resolution: unresolved; no resolver link, observed 2026-08-07T15:02:22.560982Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ac31a24c-c423-4aa8-bf6f-cc421c3367c4 · inbound

CRScore++: Reinforcement Learning with Verifiable Tool and AI Feedback for Code Review cites this paper.

CRScore++: Reinforcement Learning with Verifiable Tool and AI Feedback for Code Review Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering

Reference 38

Resolution: unresolved; no resolver link, observed 2026-08-07T12:13:31.407675Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1a0d2f36-7f99-4b90-863c-425c93e235a5 · inbound

Querying Large Automotive Software Models: Agentic vs. Direct LLM Approaches cites this paper.

Querying Large Automotive Software Models: Agentic vs. Direct LLM Approaches Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering

Reference 22

Resolution: unresolved; no resolver link, observed 2026-08-07T00:40:17.022076Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a2e97899-30c0-4865-ad11-be9784b177b8 · inbound

Evaluating the Use of LLMs for Documentation to Code Traceability cites this paper.

Evaluating the Use of LLMs for Documentation to Code Traceability Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering

Reference 81

Resolution: unresolved; no resolver link, observed 2026-08-15T19:32:36.028070Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f548eb6d-8b09-4ea9-83a0-367b254dba3d · inbound

Spiritual-LLM : Gita Inspired Mental Health Therapy In the Era of LLMs cites this paper.

Spiritual-LLM : Gita Inspired Mental Health Therapy In the Era of LLMs Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering

Reference 49

Resolution: unresolved; no resolver link, observed 2026-08-06T23:12:14.277116Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9462f634-0751-458c-984c-38e6b37bddfb · inbound

CCISolver: End-to-End Detection and Repair of Method-Level Code-Comment Inconsistency cites this paper.

CCISolver: End-to-End Detection and Repair of Method-Level Code-Comment Inconsistency Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering

Reference 77

Resolution: verified exact; local_arxiv, observed 2026-08-06T22:55:22.740075Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.