Pith. sign in

Paper Citation Record · LEDGER

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs

As of 9 August 2026, this Paper Citation Record lists 54 of 54 outbound references and 0 inbound Pith citation observations for arXiv:2507.15715.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2507.15715 v2

Coverage vector

measured 54 of 54 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-06T15:29:42.348441Z

measured 54 of 54 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-09T06:31:02.800959+00:00

measured 0 of 0 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: cited_works

Reference resolution

54 of 54 outbound references displayed

  • verified exact4
  • verified fuzzy9
  • unresolved40
  • parse uncertain0
  • malformed identifier1
  • metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation 27c96f69-cd38-4040-a144-6b5fa93f0f7c · outbound

This paper cites Kurtz, Edwin A.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Kurtz, Edwin A

Reference 1

Resolution
verified fuzzy
raw_fallback, observed 2026-08-06T15:29:46.780248Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

source=arxiv_source observed=2026-08-06T15:29:37.327954Z digest=sha256:d0599d91475f904d23f65e3d058a3490efaad13cb0d695f58ae3dbc933559c5c

Observation e857067f-c455-44c1-8017-c896aefbafe8 · outbound

This paper cites Henneken, Carolyn S.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Henneken, Carolyn S

Reference 2

Resolution
verified fuzzy
raw_fallback, observed 2026-08-06T15:29:46.590827Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

source=arxiv_source observed=2026-08-06T15:29:37.353537Z digest=sha256:4a6684c88f830b5057022be6ba66c430291ba590f1357f83a470d73dad66be50

Observation 16e9845e-ce20-4707-ab47-19d22826bbe0 · outbound

This paper cites OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMs.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMs

Reference 3

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:37.422914Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:37.422914Z digest=sha256:9b39be022b305daee966ecc852185b9ec40e18db7bcc2f21a0a4a2b6741da285

Observation 7b3b0184-0ec3-48f0-bbe2-5efbd72a176d · outbound

This paper cites Search B ots: User engagement with ChatBots during collaborative search.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Search B ots: User engagement with ChatBots during collaborative search

Reference 4

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:37.501898Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:37.501898Z digest=sha256:61b0c23fb908127046ba0b6f71481cd52d618bd1290e90aeb0e6be1727cdc027

Observation 9b256564-1ae3-4dc9-ac4e-53ef4e34de2a · outbound

This paper cites Embedding search into a conversational platform to support collaborative search.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Embedding search into a conversational platform to support collaborative search

Reference 5

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:37.620210Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:37.620210Z digest=sha256:b6353b4618ba450ef53acf34414069fc71e9114ef972ff732f4750b9fd209eb3

Observation 857cfe90-fb18-485b-8def-5517ff44a169 · outbound

This paper cites The effects of system initiative during conversational collaborative search.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs The effects of system initiative during conversational collaborative search

Reference 6

Resolution
verified exact
doi, observed 2026-08-06T15:29:43.102800Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

source=arxiv_source observed=2026-08-06T15:29:37.707455Z digest=sha256:8da49053aff0d1d8396dd4b568a07683f7b90407682fe8f23228ab2b0c086208

Observation f7e1caf3-44f0-4c07-86a0-d19a3b528224 · outbound

This paper cites Towards Reproducible LLM Evaluation: Quantifying Uncertainty in LLM Benchmark Scores.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Towards Reproducible LLM Evaluation: Quantifying Uncertainty in LLM Benchmark Scores

Reference 7

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:37.798046Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:37.798046Z digest=sha256:2db6ea8c7b0a00d7c4e96fc24b41b2604104fcd3393c8c0d6f686f6b450be86b

Observation 874cf6a4-dc96-4f38-9d90-68df6f0ff5e0 · outbound

This paper cites Bowman and George Dahl.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Bowman and George Dahl

Reference 8

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:37.857577Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:37.857577Z digest=sha256:c12ebc020820ce18dc0e8b5233e0c5a8d479ec442b751ce3339ec106c328e9c7

Observation 49e8f034-f8d8-4ee5-8e89-cc9f404679d3 · outbound

This paper cites Toward algorithmic accountability in public services: A qualitative study of affected community perspectives on algorithmic decision-making in child welfare services.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Toward algorithmic accountability in public services: A qualitative study of affected community perspectives on algorithmic decision-making in child welfare services

Reference 9

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:37.975077Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:37.975077Z digest=sha256:dc700990361841be5e41c283c08c18d5be1c20800c5ae756e13ca80cfb6fc206

Observation 74b9716d-7221-49e5-aa41-abd3c7533141 · outbound

This paper cites an unresolved cited work.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Unresolved cited work

Reference 10

Resolution
unresolved
raw_fallback, observed 2026-08-06T15:29:46.374142Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

source=arxiv_source observed=2026-08-06T15:29:38.086420Z digest=sha256:78318a416281a788a286c02fbb253b88147faed61134d4acc57d474ecc4e3152

Observation 043b933d-e42e-4251-ad1e-b29e00ad287c · outbound

This paper cites Fabbri, Wojciech Kry \'s ci \'n ski, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir Radev.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Fabbri, Wojciech Kry \'s ci \'n ski, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir Radev

Reference 11

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:38.206098Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:38.206098Z digest=sha256:41b2417754fe8f0b8a189c7fe6338eadf6036a15b75014f1289f8e56e8cf8aff

Observation 6570ef50-b6b5-4676-b30c-279824ba7822 · outbound

This paper cites an unresolved cited work.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Unresolved cited work

Reference 12

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:38.333778Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:38.333778Z digest=sha256:c50f9f495d778fc5123ca3a9faefbe86b7fa547598143a58d1371c0b152a8895

Observation 266e21d3-c56b-44e6-9fb6-0e17a90ecfb5 · outbound

This paper cites Enabling large language models to generate text with citations.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Enabling large language models to generate text with citations

Reference 13

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:38.435601Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:38.435601Z digest=sha256:760c2d8a366419fdcbef40bb8bab81578103744660d95682541030e902125a3b

Observation 456fe5da-355b-4c50-8d98-531913e588bb · outbound

This paper cites What can large language models do in chemistry? A comprehensive benchmark on eight tasks.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs What can large language models do in chemistry? A comprehensive benchmark on eight tasks

Reference 14

Resolution
verified fuzzy
raw_fallback, observed 2026-08-06T15:29:46.087564Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

source=arxiv_source observed=2026-08-06T15:29:38.538154Z digest=sha256:7cefc5e062978c673ccc2f922c059699102bf67e0585a0b9d8ae1e106c93737d

Observation 29d230c2-2941-40ff-8f67-5c12f3144f92 · outbound

This paper cites Measuring mathematical problem solving with the math dataset.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Measuring mathematical problem solving with the math dataset

Reference 15

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:38.632350Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:38.632350Z digest=sha256:27e11a0e36ec9bfe400ca6b8a175fe8eda0e2f5ecef1613f9803b0022715b70e

Observation bf012bce-e408-49c0-86dd-64eb54026ca1 · outbound

This paper cites Iyer, Mikaeel Yunus, Charles O’Neill, Christine Ye, Alina Hyk, Kiera McCormick, Ioana Ciucă, John F.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Iyer, Mikaeel Yunus, Charles O’Neill, Christine Ye, Alina Hyk, Kiera McCormick, Ioana Ciucă, John F

Reference 16

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:38.737079Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:38.737079Z digest=sha256:6730323a16922b5366988708a5cac79b867366bd312f3d47d021cf8095a93c17

Observation 13f31780-8ae0-4420-8746-a8aeabd65f56 · outbound

This paper cites an unresolved cited work.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Unresolved cited work

Reference 17

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:38.791678Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:38.791678Z digest=sha256:efabeb9b38f71ecf71e18574eb37b522e59a97c7f5559f80381fcf7df837a8e4

Observation 7eb8b2d2-134b-47ff-aa79-48e18b31b379 · outbound

This paper cites On scientific understanding with artificial intelligence.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs On scientific understanding with artificial intelligence

Reference 18

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:38.893061Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:38.893061Z digest=sha256:d541d7e62f5e2cf2257fb2bdb0bc2260fd1b5f5e393e6c728656fc0109d75e3e

Observation f701a313-32ec-485f-a132-4e78c62403b5 · outbound

This paper cites Wikibench: Community-driven data curation for AI evaluation on W ikipedia.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Wikibench: Community-driven data curation for AI evaluation on W ikipedia

Reference 19

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:39.005780Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:39.005780Z digest=sha256:eee357aee6037c8c18ee9a4db4afc7599b156e16f8f2dc6fd6d1da43098848c8

Observation 690c2d37-96c5-4c25-842a-3e23044ae119 · outbound

This paper cites Chapter 8 - interviews and focus groups.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Chapter 8 - interviews and focus groups

Reference 20

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:39.128709Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:39.128709Z digest=sha256:10cee8304f339c7d2a67797015918ab5e157c08bf254c1ae9d966ae56ce8ed48

Observation 0218d1d6-2b04-4562-bc11-17f6776dba69 · outbound

This paper cites H alu E val: A large-scale hallucination evaluation benchmark for large language models.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs H alu E val: A large-scale hallucination evaluation benchmark for large language models

Reference 21

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:39.222337Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:39.222337Z digest=sha256:1a9412bd639c0c5c98f301aad373b97c58352c5d7d4c3282c2a20e28d1bbcdda

Observation 04ef06a9-e284-49af-b016-443301616f0d · outbound

This paper cites an unresolved cited work.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Unresolved cited work

Reference 22

Resolution
unresolved
raw_fallback, observed 2026-08-06T15:29:45.873366Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

source=arxiv_source observed=2026-08-06T15:29:39.305544Z digest=sha256:fafa68e6a54f80e6c44a73df010b8627b76f1d7358475275ac151657954e872b

Observation 64e7b96d-dc3b-424a-bdb2-ee5f5deb5e4b · outbound

This paper cites Rethinking Model Evaluation as Narrowing the Socio-Technical Gap.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Rethinking Model Evaluation as Narrowing the Socio-Technical Gap

Reference 23

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:39.408794Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:39.408794Z digest=sha256:94d544f86e99060f57e6829a2766a8f6acb6c62f5cd920bc20c9d0ec164752eb

Observation a92f3370-7547-4c92-97ad-5a0957c5af95 · outbound

This paper cites Vera Liao, Daniel Gruen, and Sarah Miller.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Vera Liao, Daniel Gruen, and Sarah Miller

Reference 24

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:39.516481Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:39.516481Z digest=sha256:fcc5a9c52e228e6a5d35e40c13960c43836138df6bbe220c926c596db90490bd

Observation a8b993fe-9673-47a8-aa2b-6b9d3d39ce79 · outbound

This paper cites S elf C heck GPT : Zero-resource black-box hallucination detection for generative large language models.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs S elf C heck GPT : Zero-resource black-box hallucination detection for generative large language models

Reference 25

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:39.620818Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:39.620818Z digest=sha256:2ee39f337347bb6c325815f0a39f010a40f52cf166f376257e992ac227802564

Observation f96673d3-1578-4deb-aa67-acf7f07d0710 · outbound

This paper cites Mathewson, Jaylen Pittman, and Richard Evans.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Mathewson, Jaylen Pittman, and Richard Evans

Reference 26

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:39.788439Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:39.788439Z digest=sha256:f6c06b602160859d1806488eddb2ec4bece5435bde4abdd68967ab5b61be574c

Observation 23dff6a3-e837-4a45-ab51-6548962615c2 · outbound

This paper cites Wolf, Josh Andres, Michael Desmond, Narendra Nath Joshi, Zahra Ashktorab, Aabhas Sharma, Kristina Brimijoin, Qian Pan, Evelyn Duesterwald, and Casey Dugan.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Wolf, Josh Andres, Michael Desmond, Narendra Nath Joshi, Zahra Ashktorab, Aabhas Sharma, Kristina Brimijoin, Qian Pan, Evelyn Duesterwald, and Casey Dugan

Reference 27

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:39.917055Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:39.917055Z digest=sha256:c424444f55fb1d22c831b9527f850aecba13baff4356fbee49539a1d03913160

Observation 0dc0bdb0-4283-4bf8-a8fe-4d6e40d4b2e3 · outbound

This paper cites an unresolved cited work.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Unresolved cited work

Reference 28

Resolution
unresolved
raw_fallback, observed 2026-08-06T15:29:45.676716Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

source=arxiv_source observed=2026-08-06T15:29:40.039345Z digest=sha256:ec6b468cddaf108962d8fb870c3e554292c10b66566c7a2ff60e525826b88c06

Observation a529e512-12a9-4686-aae0-4702e446ffb3 · outbound

This paper cites Rodriguez Mendez, Thang Bui, Alyssa Goodman, Alberto Accomazzi, Jill Naiman, Jesse Cranney, Kevin Schawinski, and Roberta Raileanu.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Rodriguez Mendez, Thang Bui, Alyssa Goodman, Alberto Accomazzi, Jill Naiman, Jesse Cranney, Kevin Schawinski, and Roberta Raileanu

Reference 29

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:40.182493Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:40.182493Z digest=sha256:b5a36c0331679f4b4ece46168c24ebf0de670316a2e22e30546158a4cefa0159

Observation 2ed222ae-9815-4590-82e8-c0b06816356b · outbound

This paper cites emr QA : A large corpus for question answering on electronic medical records.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs emr QA : A large corpus for question answering on electronic medical records

Reference 30

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:40.252325Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:40.252325Z digest=sha256:c10c0b9cec8905e458dba8775051a1f98ae652d685b907606f4709eee62c80a7

Observation 340bbaff-2ae5-4607-9b65-9f4e20bfe3f5 · outbound

This paper cites B io R ead: A new dataset for biomedical reading comprehension.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs B io R ead: A new dataset for biomedical reading comprehension

Reference 31

Resolution
verified fuzzy
raw_fallback, observed 2026-08-06T15:29:45.468781Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

source=arxiv_source observed=2026-08-06T15:29:40.375448Z digest=sha256:055e5524f45239c257cd33d4ecd3a356641dc5a1c0a38acbbbe2d4d9161853d0

Observation e74bfa0e-789f-4503-9ab5-483edc04a799 · outbound

This paper cites B io MRC : A dataset for biomedical machine reading comprehension.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs B io MRC : A dataset for biomedical machine reading comprehension

Reference 32

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:40.489331Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:40.489331Z digest=sha256:5a4af9af65985d75656d2d60bc5842ece5e87a5339a61ce4e8a4d1752989b0cc

Observation c7ff9d22-4084-4f89-bdd1-f30853b47950 · outbound

This paper cites Exploring temperature effects on large language models across various clinical tasks.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Exploring temperature effects on large language models across various clinical tasks

Reference 33

Resolution
verified fuzzy
raw_fallback, observed 2026-08-06T15:29:45.268357Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

source=arxiv_source observed=2026-08-06T15:29:40.584441Z digest=sha256:6095a960975526288b1c9c84194e64be1c405dcc9d5b2f14b3fd1136a2bf6b14

Observation 3314d704-20c7-4a91-8284-519ee72dda91 · outbound

This paper cites Smith, Huiling Liu, Kevin Schawinski, Kartheik Iyer, Ioana Ciucă, and UniverseTBD.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Smith, Huiling Liu, Kevin Schawinski, Kartheik Iyer, Ioana Ciucă, and UniverseTBD

Reference 34

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:40.688976Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:40.688976Z digest=sha256:de852c6454297e9a59e44710dcf09b8be1006731c688d0c2fca0ac9fbc663848

Observation 4fca715d-5d5d-4316-a7e7-c0c8c5af6b18 · outbound

This paper cites Bender, Alex Hanna, and Amandalynne Paullada.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Bender, Alex Hanna, and Amandalynne Paullada

Reference 35

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:40.862940Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:40.862940Z digest=sha256:d467dcf29d8f819adcf599e5f3158cdb28431cc08f08e058683722a1055f3dbc

Observation db18018f-e43b-4cf7-8f0a-1e5d630aa9bd · outbound

This paper cites The effect of sampling temperature on problem solving in large language models.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs The effect of sampling temperature on problem solving in large language models

Reference 36

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:40.975379Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:40.975379Z digest=sha256:2d7efb4f99ea2665ac74dcd6442662d53e19df4e80a5fcf548c510d027df6137

Observation 44a2d16e-6a6f-4b4e-b5e5-73306be032ec · outbound

This paper cites Improving evidence retrieval for automated explainable fact-checking.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Improving evidence retrieval for automated explainable fact-checking

Reference 37

Resolution
verified exact
doi, observed 2026-08-06T15:29:42.860122Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

source=arxiv_source observed=2026-08-06T15:29:41.040347Z digest=sha256:e747bd5751abc25da73130fd1b783e8a0700a5dbc4c1bc7deb37b7e11e8d055f

Observation 187fe219-bbef-4d96-8112-24a864213317 · outbound

This paper cites an unresolved cited work.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Unresolved cited work

Reference 38

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:41.097841Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:41.097841Z digest=sha256:6631c982063e412a571404f0734d3899ad47934735a26d12850d4ae642d76d0c

Observation 71985320-c1e8-4ecf-a5dc-8468c119c5d6 · outbound

This paper cites The Good, The Bad, and The Greedy: Evaluation of LLMs Should Not Ignore Non-Determinism.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs The Good, The Bad, and The Greedy: Evaluation of LLMs Should Not Ignore Non-Determinism

Reference 39

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:41.138123Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:41.138123Z digest=sha256:95996e0ce870de26209226c082dc80314a5e73909194c74d73345d45d55cab18

Observation 3aa554d2-c2d9-4573-ac49-0400e7b08b79 · outbound

This paper cites Scieval: a multi-level large language model evaluation benchmark for scientific research.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Scieval: a multi-level large language model evaluation benchmark for scientific research

Reference 40

Resolution
malformed identifier
no resolver link, observed 2026-08-06T15:29:41.201818Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:41.201818Z digest=sha256:111acfa781ec2ef8bae16a1647ce2ec9c69211972528f2dac95689b594afa707

Observation 5dd25c18-98f3-4b28-91af-f17b7e0950cd · outbound

This paper cites Nestor, Ali Soroush, Pierre A.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Nestor, Ali Soroush, Pierre A

Reference 41

Resolution
verified exact
doi, observed 2026-08-06T15:29:42.679016Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

source=arxiv_source observed=2026-08-06T15:29:41.320038Z digest=sha256:e4aecb08240d6ad2b4bd0f566883a9e0092a5ffff11dd3929e6241ba361dc738

Observation 6d21ebef-9c75-4bf2-8a6d-fe3d9e583d95 · outbound

This paper cites FEVER : a large-scale dataset for fact extraction and VER ification.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs FEVER : a large-scale dataset for fact extraction and VER ification

Reference 42

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:41.418093Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:41.418093Z digest=sha256:abf1b5b114e787e48359d2f75c6757ab7596ffa6fe2290176acd62ff0087bbd3

Observation c21746c8-56d1-4ecc-b42f-5a61afa09ddd · outbound

This paper cites Group chat ecology in enterprise instant messaging: How employees collaborate through multi-user chat channels on slack.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Group chat ecology in enterprise instant messaging: How employees collaborate through multi-user chat channels on slack

Reference 43

Resolution
verified exact
doi, observed 2026-08-06T15:29:42.523254Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

source=arxiv_source observed=2026-08-06T15:29:41.479622Z digest=sha256:5caffd169f040be6d96eb655a1f46762d58843b1162b8ed235493e948d15e7f8

Observation 643ed9c8-a501-4895-bf7a-81bef5ad4cce · outbound

This paper cites Evaluating large language models on academic literature understanding and review: An empirical study among early-stage scholars.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Evaluating large language models on academic literature understanding and review: An empirical study among early-stage scholars

Reference 44

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:41.581642Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:41.581642Z digest=sha256:2c22a3110f391eac40a20ddb401e477e20cdc95b7b8192c3ff282249ea1ff28d

Observation 194c1acc-3bff-422b-b52c-c2e22d164b3c · outbound

This paper cites as an ai language model, i cannot.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs as an ai language model, i cannot

Reference 45

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:41.655835Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:41.655835Z digest=sha256:d39b3a8b376f608db2b117db38a05c328d18fe7ce6124f4ed64e77867caec9a3

Observation b5cda476-4e8d-475d-a1c2-9704ddabbf1e · outbound

This paper cites Designing an Evaluation Framework for Large Language Models in Astronomy Research.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Designing an Evaluation Framework for Large Language Models in Astronomy Research

Reference 46

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:41.746197Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:41.746197Z digest=sha256:0606e97e3f52c9cb4409eb94e48a327465bed3aab4af772533a1de28bdf857e5

Observation 347b5cc6-f90b-465a-87da-5b86b931a88a · outbound

This paper cites C-pack: Packed resources for general chinese embeddings.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs C-pack: Packed resources for general chinese embeddings

Reference 47

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:41.830639Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:41.830639Z digest=sha256:2a579d59e46c4f43bc2155eb2a8fa9b559210cbf6cf4afa36be163c3a5861e13

Observation 844b0afc-4d00-46e9-a8b5-eb4d4912d025 · outbound

This paper cites AI as an active writer: Interaction strategies with generated text in human- AI collaborative fiction writing.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs AI as an active writer: Interaction strategies with generated text in human- AI collaborative fiction writing

Reference 48

Resolution
verified fuzzy
raw_fallback, observed 2026-08-06T15:29:45.034922Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

source=arxiv_source observed=2026-08-06T15:29:41.891915Z digest=sha256:06fb9fa916c15781cca9b015ca186590d553771b41632784bb08d9561c9c810a

Observation 01bbfc41-c32a-45ea-bb2a-0b9b7c493064 · outbound

This paper cites User-Controlled Knowledge Fusion in Large Language Models: Balancing Creativity and Hallucination.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs User-Controlled Knowledge Fusion in Large Language Models: Balancing Creativity and Hallucination

Reference 49

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:41.990196Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:41.990196Z digest=sha256:791acdd15a9d72e95215a5eb537206718127082daa99b8ae28f14cdf79d0c49c

Observation b2bde327-f37d-4162-9d84-bf16ed863ac6 · outbound

This paper cites Weinberger, and Yoav Artzi.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Weinberger, and Yoav Artzi

Reference 50

Resolution
verified fuzzy
raw_fallback, observed 2026-08-06T15:29:44.830578Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

source=arxiv_source observed=2026-08-06T15:29:42.057541Z digest=sha256:ef40feaf68e02bf451ab1d64f335edd9722f49310e4bc1e32bac311c3fb56dc1

Observation 40a5499e-16cf-4445-8e8f-cd3dde3432f5 · outbound

This paper cites WildChat : 1 M C hat GPT interaction logs in the wild.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs WildChat : 1 M C hat GPT interaction logs in the wild

Reference 51

Resolution
verified fuzzy
raw_fallback, observed 2026-08-06T15:29:44.598764Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

source=arxiv_source observed=2026-08-06T15:29:42.106650Z digest=sha256:dd9be01e28dc199dd4020d810a732b96ffb7eb0401bfbae0696638623a705ffe

Observation 67552ce0-5813-426e-a34d-c33fa787ac2b · outbound

This paper cites Deconstructing NLG evaluation: Evaluation practices, assumptions, and their implications.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Deconstructing NLG evaluation: Evaluation practices, assumptions, and their implications

Reference 52

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:42.197073Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:42.197073Z digest=sha256:c1968561952a3f4e3ab993e646a13b1156bb395f4e96b8e8d841a541a6e9d1f0

Observation e98031c2-bf97-47b0-a7f9-a425c93788b5 · outbound

This paper cites Navigating the grey area: How expressions of uncertainty and overconfidence affect language models.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Navigating the grey area: How expressions of uncertainty and overconfidence affect language models

Reference 53

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:42.298617Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:42.298617Z digest=sha256:f3c8c2db415036c58dd60c7f5a8596f42ff61043f15cee7cc441bd68e14c8569

Observation fe456e7f-3a03-4119-ac9b-93293c239cc1 · outbound

This paper cites Relying on the unreliable: The impact of language models ' reluctance to express uncertainty.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Relying on the unreliable: The impact of language models ' reluctance to express uncertainty

Reference 54

Resolution
verified fuzzy
raw_fallback, observed 2026-08-06T15:29:44.375318Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

source=arxiv_source observed=2026-08-06T15:29:42.348441Z digest=sha256:64b0aace4bfb706d8c669e7fad8f5165a19849999dc2305326c08be8b30ad02f

Pith citing papers

No inbound Pith citation observations are available.