Pith. sign in

Paper Citation Record · LEDGER

Evaluating Large Language Models as Expert Annotators

As of 20 August 2026, this Paper Citation Record lists 52 of 52 outbound references and 2 inbound Pith citation observations for arXiv:2508.07827.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2508.07827 v1

Coverage vector

measured 52 of 52 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-05T21:58:03.190450Z

measured 54 of 54 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-19T06:32:44.657259+00:00

measured 2 of 2 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-06-29T21:36:22.389047Z

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: arxiv_reference, observed 2026-06-29T21:43:59.674035Z

Reference resolution

52 of 52 outbound references displayed

  • verified exact0
  • verified fuzzy18
  • unresolved34
  • parse uncertain0
  • malformed identifier0
  • metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation 6b936667-020d-4e5e-a448-86eb1033055c · outbound

This paper cites write newline.

Evaluating Large Language Models as Expert Annotators write newline

Reference 1

Resolution
unresolved
no resolver link, observed 2026-08-05T21:57:58.492704Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:57:58.492704Z digest=sha256:c53c2fc7ee606a9305e705dea08783f3b7c72045a33f48350658d20343745732

Observation b4c63d06-2427-4046-be84-810b6c61a8cf · outbound

This paper cites GPT-4 Technical Report.

Evaluating Large Language Models as Expert Annotators GPT-4 Technical Report

Reference 2

Resolution
unresolved
no resolver link, observed 2026-08-05T21:57:58.634448Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:57:58.634448Z digest=sha256:8d41aba0a408ef76d529e399aa7990c74f6e7bee3c3532132eac0acd943693d5

Observation 1d58986c-7e14-4e38-8c82-ee71a1672364 · outbound

This paper cites Open-Source LLMs for Text Annotation: A Practical Guide for Model Setting and Fine-Tuning.

Evaluating Large Language Models as Expert Annotators Open-Source LLMs for Text Annotation: A Practical Guide for Model Setting and Fine-Tuning

Reference 3

Resolution
unresolved
no resolver link, observed 2026-08-05T21:57:58.829185Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:57:58.829185Z digest=sha256:d29bbdeb06a3c5170f79956ac132e6893cd1612d66d13fc947f4f2509bfac720

Observation b0c71ad9-7577-4825-b493-a4d9c14aa089 · outbound

This paper cites The claude 3 model family: Opus, sonnet, haiku.

Evaluating Large Language Models as Expert Annotators The claude 3 model family: Opus, sonnet, haiku

Reference 4

Resolution
unresolved
no resolver link, observed 2026-08-05T21:57:58.988358Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:57:58.988358Z digest=sha256:7249691921e330d1671eb3e030b18256b8d9d37e09a973eb3086fd780deb43ac

Observation 529cf28d-af15-4939-80ac-bf7d8d69c3d7 · outbound

This paper cites Claude 3.7 sonnet and claude code.

Evaluating Large Language Models as Expert Annotators Claude 3.7 sonnet and claude code

Reference 5

Resolution
verified fuzzy
raw_fallback, observed 2026-08-05T21:58:05.436139Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

source=arxiv_source observed=2026-08-05T21:57:59.152154Z digest=sha256:dfe1fd5ef9467cd9b08b22275b740838e7c0f3ef26bfb27ea0c4fa991c8549ed

Observation fb27085f-bb3b-4086-a198-57b7f2157896 · outbound

This paper cites Large Language Models as Annotators: Enhancing Generalization of NLP Models at Minimal Cost.

Evaluating Large Language Models as Expert Annotators Large Language Models as Annotators: Enhancing Generalization of NLP Models at Minimal Cost

Reference 6

Resolution
unresolved
no resolver link, observed 2026-08-05T21:57:59.306767Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:57:59.306767Z digest=sha256:408becb1eeae6128ac600029b602e6d07857b4e505d6f0a53a3fc2076d022acc

Observation f06ed9c3-bca1-4c83-b6eb-db6b4708b7c8 · outbound

This paper cites Must read: A systematic survey of computational persuasion.

Evaluating Large Language Models as Expert Annotators Must read: A systematic survey of computational persuasion

Reference 7

Resolution
unresolved
no resolver link, observed 2026-08-05T21:57:59.405114Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:57:59.405114Z digest=sha256:d9de173e622b4f93f094f2de1ae3e8ebbdb2fc6c7dfe55997a26c69dab19207f

Observation bae1414c-9156-4b55-b678-7dd155b038f4 · outbound

This paper cites Can GPT models be Financial Analysts? An Evaluation of ChatGPT and GPT-4 on mock CFA Exams.

Evaluating Large Language Models as Expert Annotators Can GPT models be Financial Analysts? An Evaluation of ChatGPT and GPT-4 on mock CFA Exams

Reference 8

Resolution
unresolved
no resolver link, observed 2026-08-05T21:57:59.514351Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:57:59.514351Z digest=sha256:66f21323cb8348278c2eb8cd42c7088a95128c4323a3ad2ee9aafcfc0766ec6b

Observation 1a55427f-961a-4075-a876-0c55903314d1 · outbound

This paper cites ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs.

Evaluating Large Language Models as Expert Annotators ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs

Reference 9

Resolution
unresolved
no resolver link, observed 2026-08-05T21:57:59.640281Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:57:59.640281Z digest=sha256:9fbb8e4488fd5143089c1603d10b392d85d59a5866e6a82ea9a9490a3ebd1b80

Observation 5ae76b19-00d9-4629-9bc3-94eaa1c256df · outbound

This paper cites Evaluating Large Language Models Trained on Code.

Evaluating Large Language Models as Expert Annotators Evaluating Large Language Models Trained on Code

Reference 10

Resolution
unresolved
no resolver link, observed 2026-08-05T21:57:59.744177Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:57:59.744177Z digest=sha256:d57657861daa17ae09c0d41fb17c0852a86b53061b7b4241487542971c59a244

Observation 517ec177-3791-4aa5-8cf0-ef478a09746b · outbound

This paper cites an unresolved cited work.

Evaluating Large Language Models as Expert Annotators Unresolved cited work

Reference 11

Resolution
unresolved
raw_fallback, observed 2026-08-05T21:58:05.427175Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

source=arxiv_source observed=2026-08-05T21:57:59.875292Z digest=sha256:f94ba53258725e3458e85a4f34ebc13653b82024da9995ceec87313648e50348

Observation b4fce70c-4c5d-4064-8b83-7f70e29dbcd3 · outbound

This paper cites Chatgpt goes to law school.

Evaluating Large Language Models as Expert Annotators Chatgpt goes to law school

Reference 12

Resolution
verified fuzzy
raw_fallback, observed 2026-08-05T21:58:05.417901Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

source=arxiv_source observed=2026-08-05T21:57:59.945208Z digest=sha256:8f6a855d5c43d7cb74a15a5c4e86cff86d9226c65f7f674cb54f8410abbd6e94

Observation 6872d044-9693-4b34-a660-6621fcd6be4b · outbound

This paper cites GPTs Are Multilingual Annotators for Sequence Generation Tasks.

Evaluating Large Language Models as Expert Annotators GPTs Are Multilingual Annotators for Sequence Generation Tasks

Reference 13

Resolution
unresolved
no resolver link, observed 2026-08-05T21:58:00.004961Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:58:00.004961Z digest=sha256:698f50404165f1c1483f3b032a9131df3b253c9b781733cad2142cef932d2f99

Observation f02c00c8-f313-47e0-b984-c949688251ec · outbound

This paper cites Is gpt-3 a good data annotator? In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.\ 11173--11195, 2023.

Evaluating Large Language Models as Expert Annotators Is gpt-3 a good data annotator? In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.\ 11173--11195, 2023

Reference 14

Resolution
verified fuzzy
raw_fallback, observed 2026-08-05T21:58:05.408117Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

source=arxiv_source observed=2026-08-05T21:58:00.061528Z digest=sha256:d872206ef131312e92eb62f7d0c631eeeb9869da395bf863f6156d05e6b9afd0

Observation a7c5c73a-8d72-48e5-9517-8f6c6df3b268 · outbound

This paper cites Improving Factuality and Reasoning in Language Models through Multiagent Debate.

Evaluating Large Language Models as Expert Annotators Improving Factuality and Reasoning in Language Models through Multiagent Debate

Reference 15

Resolution
unresolved
no resolver link, observed 2026-08-05T21:58:00.167703Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:58:00.167703Z digest=sha256:075d56fcca25459a87bada410c8b57f2ac9960caf1940ace8e8c331b0a6e2be7

Observation 17132b9e-4e9c-4631-bf10-a58a9008b6d5 · outbound

This paper cites Measuring the persuasiveness of language models, 2024.

Evaluating Large Language Models as Expert Annotators Measuring the persuasiveness of language models, 2024

Reference 16

Resolution
unresolved
no resolver link, observed 2026-08-05T21:58:00.256478Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:58:00.256478Z digest=sha256:e3fab751670dc04a839ee6010f8beaaac24782aad3be774ef857ff97ae138ec4

Observation de7121dc-276f-4728-ab52-9717e23e1123 · outbound

This paper cites Measuring nominal scale agreement among many raters.

Evaluating Large Language Models as Expert Annotators Measuring nominal scale agreement among many raters

Reference 17

Resolution
verified fuzzy
raw_fallback, observed 2026-08-05T21:58:05.392774Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

source=arxiv_source observed=2026-08-05T21:58:00.323738Z digest=sha256:d03a073fae0033641e7bfe902b99e587f11fc789c8cd55ff58642bfab1816846

Observation e9c6e715-aa56-4a68-95f0-99378478ba55 · outbound

This paper cites Chatgpt outperforms crowd workers for text-annotation tasks.

Evaluating Large Language Models as Expert Annotators Chatgpt outperforms crowd workers for text-annotation tasks

Reference 18

Resolution
verified fuzzy
raw_fallback, observed 2026-08-05T21:58:05.383309Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

source=arxiv_source observed=2026-08-05T21:58:00.395214Z digest=sha256:af0934ae19bc74b080e96fccb2e5b4d9422d3f43e7f0efbe0358db4019a3fb8c

Observation 56a7c1a1-538c-4428-95bd-b23114d5e49b · outbound

This paper cites Introducing gemini 2.0: our new ai model for the agentic era.

Evaluating Large Language Models as Expert Annotators Introducing gemini 2.0: our new ai model for the agentic era

Reference 19

Resolution
verified fuzzy
raw_fallback, observed 2026-08-05T21:58:05.373126Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

source=arxiv_source observed=2026-08-05T21:58:00.441674Z digest=sha256:30f95f87465478f00a1283f47f89978ee2f10dc77d99dfd792ffe88149736dad

Observation 986d6739-07a7-47d7-b125-516761e6fd95 · outbound

This paper cites Legalbench: A collaboratively built benchmark for measuring legal reasoning in large language models.

Evaluating Large Language Models as Expert Annotators Legalbench: A collaboratively built benchmark for measuring legal reasoning in large language models

Reference 20

Resolution
verified fuzzy
raw_fallback, observed 2026-08-05T21:58:05.363839Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

source=arxiv_source observed=2026-08-05T21:58:00.556464Z digest=sha256:3ff761a51148dba3c3607522828c6825af31147f1c0eee5f896b47c323593c93

Observation b75d865b-3672-4cb4-b829-b76d31677352 · outbound

This paper cites Large Language Model based Multi-Agents: A Survey of Progress and Challenges.

Evaluating Large Language Models as Expert Annotators Large Language Model based Multi-Agents: A Survey of Progress and Challenges

Reference 21

Resolution
unresolved
no resolver link, observed 2026-08-05T21:58:00.622649Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:58:00.622649Z digest=sha256:3d5ec97df447cef96763762dd362daa976c0f1be38024c979fff270cb970584a

Observation b214e203-5069-48d5-a24d-1bd72469d499 · outbound

This paper cites AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators.

Evaluating Large Language Models as Expert Annotators AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators

Reference 22

Resolution
unresolved
no resolver link, observed 2026-08-05T21:58:00.717175Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:58:00.717175Z digest=sha256:0acaaebb085fd2fb1495b7ed02abbe98de916a40a6b0b98ac8e58a35229b9fa5

Observation de5df38a-bde8-4855-9660-b973c203d59e · outbound

This paper cites Measuring massive multitask language understanding.

Evaluating Large Language Models as Expert Annotators Measuring massive multitask language understanding

Reference 23

Resolution
unresolved
no resolver link, observed 2026-08-05T21:58:00.790500Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:58:00.790500Z digest=sha256:e3e1649f4bb82cab5ef7f778e6a3a914436086c5b540a038ff5998f08bef3a74

Observation 4c4a376f-c68b-4e41-8805-cc10a7c880d8 · outbound

This paper cites CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review.

Evaluating Large Language Models as Expert Annotators CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review

Reference 24

Resolution
unresolved
no resolver link, observed 2026-08-05T21:58:00.863174Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:58:00.863174Z digest=sha256:e69e4e686fd061e50a0069ae8b8515dcc2ae54c2fd8e1e75327fc7670d5ead8d

Observation 4f533cea-5aac-4b70-9d42-58f50de4ed87 · outbound

This paper cites Coda-19: Using a non-expert crowd to annotate research aspects on 10,000+ abstracts in the covid-19 open research dataset.

Evaluating Large Language Models as Expert Annotators Coda-19: Using a non-expert crowd to annotate research aspects on 10,000+ abstracts in the covid-19 open research dataset

Reference 25

Resolution
verified fuzzy
raw_fallback, observed 2026-08-05T21:58:05.347992Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

source=arxiv_source observed=2026-08-05T21:58:00.938390Z digest=sha256:289f4629a17853eab5c087da1a006305a786b5828d5fe19ffe9eac82e836d777

Observation fad2b1ea-507f-4d2e-a830-d3b9e44fa91f · outbound

This paper cites Pubmedqa: A dataset for biomedical research question answering.

Evaluating Large Language Models as Expert Annotators Pubmedqa: A dataset for biomedical research question answering

Reference 26

Resolution
verified fuzzy
raw_fallback, observed 2026-08-05T21:58:05.234433Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

source=arxiv_source observed=2026-08-05T21:58:01.028600Z digest=sha256:360d16ccd090dbe338fa8b2bbf1329c61e4ce4487090448ec238b074ad25aacc

Observation b92f1980-6828-457c-bc87-e8a9b0266c36 · outbound

This paper cites Gpt-4 passes the bar exam.

Evaluating Large Language Models as Expert Annotators Gpt-4 passes the bar exam

Reference 27

Resolution
verified fuzzy
raw_fallback, observed 2026-08-05T21:58:05.085182Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

source=arxiv_source observed=2026-08-05T21:58:01.127299Z digest=sha256:36a118ca7349fb1efaf7da796e34c06afd000a7f1e5e485237a2441188273b4f

Observation f50825cc-d97b-4807-baac-a61adcee6c86 · outbound

This paper cites Refind: Relation extraction financial dataset.

Evaluating Large Language Models as Expert Annotators Refind: Relation extraction financial dataset

Reference 28

Resolution
verified fuzzy
raw_fallback, observed 2026-08-05T21:58:04.966746Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

source=arxiv_source observed=2026-08-05T21:58:01.208891Z digest=sha256:f6f2f47479eb6d9185322b7cd3162923cbb76b975bc91809a51f1cbb2ecadbde

Observation 48a1eac5-5a42-4160-a50b-fd833586a3fb · outbound

This paper cites Large language models are zero-shot reasoners.

Evaluating Large Language Models as Expert Annotators Large language models are zero-shot reasoners

Reference 29

Resolution
unresolved
no resolver link, observed 2026-08-05T21:58:01.287975Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:58:01.287975Z digest=sha256:95d911a7b5bcf2982602f85b8cfd4ff3248bb0ae27db1f04fcf2580907043e17

Observation 75c933e7-86b6-4513-8c8e-6d0ebec54055 · outbound

This paper cites Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate.

Evaluating Large Language Models as Expert Annotators Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate

Reference 30

Resolution
unresolved
no resolver link, observed 2026-08-05T21:58:01.346989Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:58:01.346989Z digest=sha256:853f2159a285621035b3f7887e31a6be46a4be6bd3f7241cdf9e1dd1d1f7c13b

Observation c6e66b29-847b-4193-bdea-3249e44aa8c9 · outbound

This paper cites Self-refine: Iterative refinement with self-feedback.

Evaluating Large Language Models as Expert Annotators Self-refine: Iterative refinement with self-feedback

Reference 31

Resolution
unresolved
no resolver link, observed 2026-08-05T21:58:01.425203Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:58:01.425203Z digest=sha256:2251c4cd7fe8263a9638cbc14f4bc8f4fdd2e311ff46e4189c52eca440aa696a

Observation ac125cd0-dd4a-4c59-9341-46d9049aed29 · outbound

This paper cites Note on the sampling error of the difference between correlated proportions or percentages.

Evaluating Large Language Models as Expert Annotators Note on the sampling error of the difference between correlated proportions or percentages

Reference 32

Resolution
unresolved
no resolver link, observed 2026-08-05T21:58:01.502147Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:58:01.502147Z digest=sha256:165e2e4dc23715bb15d498134476f9ba2ee54be78d8afc0a5c75d845d5b7ec84

Observation 015ce650-1ee1-475e-ab60-f723fe4f1b30 · outbound

This paper cites Hello gpt4-o.

Evaluating Large Language Models as Expert Annotators Hello gpt4-o

Reference 33

Resolution
verified fuzzy
raw_fallback, observed 2026-08-05T21:58:04.825842Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

source=arxiv_source observed=2026-08-05T21:58:01.606341Z digest=sha256:c539752ff6be40c692def730a28bd34bf00f126d7f13fb9e3ece90cee2408158

Observation 6380658b-753d-4b7e-a4a2-483069c51e97 · outbound

This paper cites Openai o3-mini.

Evaluating Large Language Models as Expert Annotators Openai o3-mini

Reference 34

Resolution
verified fuzzy
raw_fallback, observed 2026-08-05T21:58:04.688878Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

source=arxiv_source observed=2026-08-05T21:58:01.663633Z digest=sha256:4886661d78ecddf7c3ac1121ad798d5d7ff76c8c92448dffb31e4a99dd7ea6d6

Observation bd27a646-139c-455a-a90f-9e1b16a8886b · outbound

This paper cites Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context.

Evaluating Large Language Models as Expert Annotators Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

Reference 35

Resolution
unresolved
no resolver link, observed 2026-08-05T21:58:01.771395Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:58:01.771395Z digest=sha256:1ab3e7fd42d35c509f98d37ea3a420c5498966ffd9ac73ff3954bf065b460b78

Observation 39af5a28-0b05-4d68-a5d0-524cab284565 · outbound

This paper cites GPQA: A Graduate-Level Google-Proof Q&A Benchmark.

Evaluating Large Language Models as Expert Annotators GPQA: A Graduate-Level Google-Proof Q&A Benchmark

Reference 36

Resolution
unresolved
no resolver link, observed 2026-08-05T21:58:01.848446Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:58:01.848446Z digest=sha256:5bdf01425636bdbbea73a9386a93b633559a24a6c7b26ad0908659f2f8889cf5

Observation 5f4f6826-db1a-4fe3-bf6e-c0ffabb20674 · outbound

This paper cites Trillion dollar words: A new financial dataset, task & market analysis.

Evaluating Large Language Models as Expert Annotators Trillion dollar words: A new financial dataset, task & market analysis

Reference 37

Resolution
verified fuzzy
raw_fallback, observed 2026-08-05T21:58:04.560419Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

source=arxiv_source observed=2026-08-05T21:58:01.919674Z digest=sha256:2c3082726c9917c576daf22500d79ead6d92bba2f9f5b4d928887380e953e4e6

Observation 8d3760b4-75fb-4393-b1ec-f8a4ad59da4e · outbound

This paper cites Large language models encode clinical knowledge.

Evaluating Large Language Models as Expert Annotators Large language models encode clinical knowledge

Reference 38

Resolution
verified fuzzy
raw_fallback, observed 2026-08-05T21:58:04.399509Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

source=arxiv_source observed=2026-08-05T21:58:02.006829Z digest=sha256:f67e5cae52e30c4cf3421334fb9c5b6174c22a2221bfaf0a5ba2033b970730d8

Observation e6f28250-193d-4583-8c0e-52a1c449e99c · outbound

This paper cites Towards Expert-Level Medical Question Answering with Large Language Models.

Evaluating Large Language Models as Expert Annotators Towards Expert-Level Medical Question Answering with Large Language Models

Reference 39

Resolution
unresolved
no resolver link, observed 2026-08-05T21:58:02.067890Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:58:02.067890Z digest=sha256:fc37949d88cd994a6a6dd95249cda66e14d8d12171c75036550b64a8b826a987

Observation b23d2c8a-924e-4b8e-b24b-ea4a1e70e351 · outbound

This paper cites Large Language Models for Data Annotation and Synthesis: A Survey.

Evaluating Large Language Models as Expert Annotators Large Language Models for Data Annotation and Synthesis: A Survey

Reference 40

Resolution
unresolved
no resolver link, observed 2026-08-05T21:58:02.162770Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:58:02.162770Z digest=sha256:ef483e96771e4fd39d158a4405ac3f8efd7a938eddd647f95926c94cd584c33b

Observation ae8488ec-0c4b-4283-b3ae-f7adc6aeb2bd · outbound

This paper cites Are Expert-Level Language Models Expert-Level Annotators?.

Evaluating Large Language Models as Expert Annotators Are Expert-Level Language Models Expert-Level Annotators?

Reference 41

Resolution
unresolved
no resolver link, observed 2026-08-05T21:58:02.244277Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:58:02.244277Z digest=sha256:b13f85ea2df92918760dcedd40f94ed976d3bee7e13001e3a043fe76c5abeb35

Observation 6bf80663-a3d0-4615-ab02-2226a7e484fe · outbound

This paper cites Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization.

Evaluating Large Language Models as Expert Annotators Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization

Reference 42

Resolution
unresolved
no resolver link, observed 2026-08-05T21:58:02.325509Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:58:02.325509Z digest=sha256:1925e567fdfa7db46ed72e00d23b45b5a62d3799e150e877b380383b6ed47318

Observation 666ab889-8119-4572-9afa-ea7a8261fe26 · outbound

This paper cites Foundational autoraters: Taming large language models for better automatic evaluation.

Evaluating Large Language Models as Expert Annotators Foundational autoraters: Taming large language models for better automatic evaluation

Reference 43

Resolution
verified fuzzy
raw_fallback, observed 2026-08-05T21:58:04.187402Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

source=arxiv_source observed=2026-08-05T21:58:02.380263Z digest=sha256:1163ad48501ba092484c970b2effd8c2f11da3c7dbf50ecf92cb23d4ced28800

Observation dab46878-2ba9-4c4a-a4cc-256cdbdafad0 · outbound

This paper cites Cord-19: The covid-19 open research dataset.

Evaluating Large Language Models as Expert Annotators Cord-19: The covid-19 open research dataset

Reference 44

Resolution
verified fuzzy
raw_fallback, observed 2026-08-05T21:58:03.971639Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

source=arxiv_source observed=2026-08-05T21:58:02.438362Z digest=sha256:5df1f6967b9c09c05873e2123598c26b0bca22646cfc3a625a94ab3429c54b78

Observation 1d12a2d4-d2c7-4b0b-ba50-5b488092f484 · outbound

This paper cites Self-consistency improves chain of thought reasoning in language models.

Evaluating Large Language Models as Expert Annotators Self-consistency improves chain of thought reasoning in language models

Reference 45

Resolution
unresolved
no resolver link, observed 2026-08-05T21:58:02.534703Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:58:02.534703Z digest=sha256:2a26f5049d00fbc93c396d8c637570f2f04696a47633cbf05792e1a0f4688716

Observation 70c0cfab-0624-4c8e-896e-26cd8cd03869 · outbound

This paper cites Chain-of-thought prompting elicits reasoning in large language models.

Evaluating Large Language Models as Expert Annotators Chain-of-thought prompting elicits reasoning in large language models

Reference 46

Resolution
unresolved
no resolver link, observed 2026-08-05T21:58:02.595024Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:58:02.595024Z digest=sha256:63db9d8ff66aed91b404afd16484cae8b2185f9f59d78e140404ed630fb6da44

Observation e80120f2-9e6a-4fc6-a9ad-4e0b6f294264 · outbound

This paper cites The rise and potential of large language model based agents: A survey.

Evaluating Large Language Models as Expert Annotators The rise and potential of large language model based agents: A survey

Reference 47

Resolution
verified fuzzy
raw_fallback, observed 2026-08-05T21:58:03.823825Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

source=arxiv_source observed=2026-08-05T21:58:02.612999Z digest=sha256:2e71cb7fe7f4ba6f0b4e16f7052e5d4392c22dd241645202573b087b4058ccdf

Observation f22bc1e0-d7fa-4bd8-9ba3-941f36e36b57 · outbound

This paper cites LLMaAA: Making Large Language Models as Active Annotators.

Evaluating Large Language Models as Expert Annotators LLMaAA: Making Large Language Models as Active Annotators

Reference 48

Resolution
unresolved
no resolver link, observed 2026-08-05T21:58:02.659226Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:58:02.659226Z digest=sha256:b6fa20e0c5bfdf7992ca6cc7d7fd4d836d3facab7ccfb00e1b54407807727551

Observation 687ed8e3-6ff3-4836-bc6f-f9702943e27c · outbound

This paper cites Can ChatGPT Reproduce Human-Generated Labels? A Study of Social Computing Tasks.

Evaluating Large Language Models as Expert Annotators Can ChatGPT Reproduce Human-Generated Labels? A Study of Social Computing Tasks

Reference 49

Resolution
unresolved
no resolver link, observed 2026-08-05T21:58:02.738693Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:58:02.738693Z digest=sha256:514d5487aae34b4b0706f9ac1c82a80acdc3a204af295ee486de0675757355db

Observation cc5ebd11-3889-42b7-a92d-ea7c3b9c517f · outbound

This paper cites @esa (Ref.

Evaluating Large Language Models as Expert Annotators @esa (Ref

Reference 50

Resolution
unresolved
no resolver link, observed 2026-08-05T21:58:02.878965Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:58:02.878965Z digest=sha256:0f77e503ae1f47d7f0caeed53c204d45c1aa35f41d01b120a9cf2ea3faa705f0

Observation 97540c71-885f-451f-a6d0-8d9139f62244 · outbound

This paper cites an unresolved cited work.

Evaluating Large Language Models as Expert Annotators Unresolved cited work

Reference 51

Resolution
unresolved
no resolver link, observed 2026-08-05T21:58:03.021696Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:58:03.021696Z digest=sha256:fb17acd1672513638e066ecf7ebe97a12fec12b85ec0ddadf44f075b06ea6ec4

Observation 5e82cdd5-a4b0-4c2c-8998-871c749e6f46 · outbound

This paper cites By leveraging additional inference-time compute, we explore whether individual LLMs can serve as a direct alternative to expert data annotators.

Evaluating Large Language Models as Expert Annotators By leveraging additional inference-time compute, we explore whether individual LLMs can serve as a direct alternative to expert data annotators

Reference 52

Resolution
unresolved
no resolver link, observed 2026-08-05T21:58:03.190450Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T21:58:03.190450Z digest=sha256:3b9670a78126af00e582fe0042dc54444b629c241a6695c9f30574a1ea075872

Pith citing papers

Observation c8a8e6be-c643-4a74-8401-59d26806e229 · inbound

Agentic-imodels: Evolving agentic interpretability tools via autoresearch cites this paper.

Agentic-imodels: Evolving agentic interpretability tools via autoresearch Evaluating Large Language Models as Expert Annotators

Reference 39

Resolution
verified exact
arxiv_id, observed 2026-05-11T23:36:36.282145Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

source=pdf_text observed=2026-05-07T16:37:43.371592Z digest=sha256:098e888a666d563522ce0d46f220a4e701067ef1645553d07b83d9274c4f909b

Observation 7de7f1b9-5adf-4e10-b065-2fa6ed303ab6 · inbound

Double Triangle Annotation: A Scalable Human-in-the-Loop Framework for High-Precision Historical Document Annotation cites this paper.

Double Triangle Annotation: A Scalable Human-in-the-Loop Framework for High-Precision Historical Document Annotation Evaluating Large Language Models as Expert Annotators

Reference 3

Resolution
verified exact
arxiv_id, observed 2026-06-29T21:43:59.676021Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

source=pdf_text observed=2026-06-29T21:36:22.389047Z digest=sha256:5812adf353d51adeaca2718f362d20a321c6f172dc85168af9a9fd57c8a2ffdb