Pith. sign in

Paper Citation Record · LEDGER

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs

As of 17 August 2026, this Paper Citation Record lists 54 of 54 outbound references and 0 inbound Pith citation observations for arXiv:2507.15715.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2507.15715 v2

Coverage vector

measured 54 of 54 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-06T15:29:42.348441Z

measured 54 of 54 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-17T06:30:58.91139+00:00

measured 0 of 0 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: cited_works

Reference resolution

54 of 54 outbound references displayed

  • verified exact4
  • verified fuzzy9
  • unresolved40
  • parse uncertain0
  • malformed identifier1
  • metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation 27c96f69-cd38-4040-a144-6b5fa93f0f7c · outbound

This paper cites Kurtz, Edwin A.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Kurtz, Edwin A

Reference 1

Resolution
verified fuzzy
raw_fallback, observed 2026-08-06T15:29:46.780248Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

source=arxiv_source observed=2026-08-06T15:29:37.327954Z digest=sha256:2cc7bb2122f5a8f69be2fa065b3277c2a3457ccd02465201bbc9241a854d784b

Observation e857067f-c455-44c1-8017-c896aefbafe8 · outbound

This paper cites Henneken, Carolyn S.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Henneken, Carolyn S

Reference 2

Resolution
verified fuzzy
raw_fallback, observed 2026-08-06T15:29:46.590827Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

source=arxiv_source observed=2026-08-06T15:29:37.353537Z digest=sha256:f17b27ca95902c2854e004a6fac7cd11300b5c379ca402b8afeb27ba2bd74dee

Observation 16e9845e-ce20-4707-ab47-19d22826bbe0 · outbound

This paper cites OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMs.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMs

Reference 3

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:37.422914Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:37.422914Z digest=sha256:23a793b0e95039d9b718d93606146971ee5d32bc7a21e9214c8cbd7a18556cd4

Observation 7b3b0184-0ec3-48f0-bbe2-5efbd72a176d · outbound

This paper cites Search B ots: User engagement with ChatBots during collaborative search.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Search B ots: User engagement with ChatBots during collaborative search

Reference 4

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:37.501898Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:37.501898Z digest=sha256:61b0c23fb908127046ba0b6f71481cd52d618bd1290e90aeb0e6be1727cdc027

Observation 9b256564-1ae3-4dc9-ac4e-53ef4e34de2a · outbound

This paper cites Embedding search into a conversational platform to support collaborative search.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Embedding search into a conversational platform to support collaborative search

Reference 5

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:37.620210Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:37.620210Z digest=sha256:b6353b4618ba450ef53acf34414069fc71e9114ef972ff732f4750b9fd209eb3

Observation 857cfe90-fb18-485b-8def-5517ff44a169 · outbound

This paper cites The effects of system initiative during conversational collaborative search.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs The effects of system initiative during conversational collaborative search

Reference 6

Resolution
verified exact
doi, observed 2026-08-06T15:29:43.102800Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

source=arxiv_source observed=2026-08-06T15:29:37.707455Z digest=sha256:5a92c1a18fd8c7a3cfd41dadcaabe5df05d20326c2a04079b9a23795117eebf0

Observation f7e1caf3-44f0-4c07-86a0-d19a3b528224 · outbound

This paper cites Towards Reproducible LLM Evaluation: Quantifying Uncertainty in LLM Benchmark Scores.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Towards Reproducible LLM Evaluation: Quantifying Uncertainty in LLM Benchmark Scores

Reference 7

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:37.798046Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:37.798046Z digest=sha256:61d660d09a83abf32d5fd56f1b71a608a9f167090b4edec44261e3df3aaff536

Observation 874cf6a4-dc96-4f38-9d90-68df6f0ff5e0 · outbound

This paper cites Bowman and George Dahl.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Bowman and George Dahl

Reference 8

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:37.857577Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:37.857577Z digest=sha256:c12ebc020820ce18dc0e8b5233e0c5a8d479ec442b751ce3339ec106c328e9c7

Observation 49e8f034-f8d8-4ee5-8e89-cc9f404679d3 · outbound

This paper cites Toward algorithmic accountability in public services: A qualitative study of affected community perspectives on algorithmic decision-making in child welfare services.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Toward algorithmic accountability in public services: A qualitative study of affected community perspectives on algorithmic decision-making in child welfare services

Reference 9

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:37.975077Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:37.975077Z digest=sha256:dc700990361841be5e41c283c08c18d5be1c20800c5ae756e13ca80cfb6fc206

Observation 74b9716d-7221-49e5-aa41-abd3c7533141 · outbound

This paper cites an unresolved cited work.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Unresolved cited work

Reference 10

Resolution
unresolved
raw_fallback, observed 2026-08-06T15:29:46.374142Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

source=arxiv_source observed=2026-08-06T15:29:38.086420Z digest=sha256:f12e08454408c2bda12ea4cb6d196fa37f2ed306a1f747c08047bc4767f2e4c5

Observation 043b933d-e42e-4251-ad1e-b29e00ad287c · outbound

This paper cites Fabbri, Wojciech Kry \'s ci \'n ski, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir Radev.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Fabbri, Wojciech Kry \'s ci \'n ski, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir Radev

Reference 11

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:38.206098Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:38.206098Z digest=sha256:41b2417754fe8f0b8a189c7fe6338eadf6036a15b75014f1289f8e56e8cf8aff

Observation 6570ef50-b6b5-4676-b30c-279824ba7822 · outbound

This paper cites an unresolved cited work.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Unresolved cited work

Reference 12

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:38.333778Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:38.333778Z digest=sha256:c50f9f495d778fc5123ca3a9faefbe86b7fa547598143a58d1371c0b152a8895

Observation 266e21d3-c56b-44e6-9fb6-0e17a90ecfb5 · outbound

This paper cites Enabling large language models to generate text with citations.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Enabling large language models to generate text with citations

Reference 13

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:38.435601Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:38.435601Z digest=sha256:760c2d8a366419fdcbef40bb8bab81578103744660d95682541030e902125a3b

Observation 456fe5da-355b-4c50-8d98-531913e588bb · outbound

This paper cites What can large language models do in chemistry? A comprehensive benchmark on eight tasks.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs What can large language models do in chemistry? A comprehensive benchmark on eight tasks

Reference 14

Resolution
verified fuzzy
raw_fallback, observed 2026-08-06T15:29:46.087564Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

source=arxiv_source observed=2026-08-06T15:29:38.538154Z digest=sha256:71498eca293d4bb7b28869de59dda6f2f9eb5c993858d729917ef16224fa0700

Observation 29d230c2-2941-40ff-8f67-5c12f3144f92 · outbound

This paper cites Measuring mathematical problem solving with the math dataset.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Measuring mathematical problem solving with the math dataset

Reference 15

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:38.632350Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:38.632350Z digest=sha256:27e11a0e36ec9bfe400ca6b8a175fe8eda0e2f5ecef1613f9803b0022715b70e

Observation bf012bce-e408-49c0-86dd-64eb54026ca1 · outbound

This paper cites Iyer, Mikaeel Yunus, Charles O’Neill, Christine Ye, Alina Hyk, Kiera McCormick, Ioana Ciucă, John F.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Iyer, Mikaeel Yunus, Charles O’Neill, Christine Ye, Alina Hyk, Kiera McCormick, Ioana Ciucă, John F

Reference 16

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:38.737079Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:38.737079Z digest=sha256:6730323a16922b5366988708a5cac79b867366bd312f3d47d021cf8095a93c17

Observation 13f31780-8ae0-4420-8746-a8aeabd65f56 · outbound

This paper cites an unresolved cited work.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Unresolved cited work

Reference 17

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:38.791678Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:38.791678Z digest=sha256:efabeb9b38f71ecf71e18574eb37b522e59a97c7f5559f80381fcf7df837a8e4

Observation 7eb8b2d2-134b-47ff-aa79-48e18b31b379 · outbound

This paper cites On scientific understanding with artificial intelligence.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs On scientific understanding with artificial intelligence

Reference 18

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:38.893061Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:38.893061Z digest=sha256:d541d7e62f5e2cf2257fb2bdb0bc2260fd1b5f5e393e6c728656fc0109d75e3e

Observation f701a313-32ec-485f-a132-4e78c62403b5 · outbound

This paper cites Wikibench: Community-driven data curation for AI evaluation on W ikipedia.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Wikibench: Community-driven data curation for AI evaluation on W ikipedia

Reference 19

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:39.005780Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:39.005780Z digest=sha256:eee357aee6037c8c18ee9a4db4afc7599b156e16f8f2dc6fd6d1da43098848c8

Observation 690c2d37-96c5-4c25-842a-3e23044ae119 · outbound

This paper cites Chapter 8 - interviews and focus groups.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Chapter 8 - interviews and focus groups

Reference 20

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:39.128709Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:39.128709Z digest=sha256:10cee8304f339c7d2a67797015918ab5e157c08bf254c1ae9d966ae56ce8ed48

Observation 0218d1d6-2b04-4562-bc11-17f6776dba69 · outbound

This paper cites H alu E val: A large-scale hallucination evaluation benchmark for large language models.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs H alu E val: A large-scale hallucination evaluation benchmark for large language models

Reference 21

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:39.222337Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:39.222337Z digest=sha256:1a9412bd639c0c5c98f301aad373b97c58352c5d7d4c3282c2a20e28d1bbcdda

Observation 04ef06a9-e284-49af-b016-443301616f0d · outbound

This paper cites an unresolved cited work.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Unresolved cited work

Reference 22

Resolution
unresolved
raw_fallback, observed 2026-08-06T15:29:45.873366Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

source=arxiv_source observed=2026-08-06T15:29:39.305544Z digest=sha256:0dcd78c86ab9df1a403e634ded9151f98b9c7bbfe96f0dd3c6ff0140044acb67

Observation 64e7b96d-dc3b-424a-bdb2-ee5f5deb5e4b · outbound

This paper cites Rethinking Model Evaluation as Narrowing the Socio-Technical Gap.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Rethinking Model Evaluation as Narrowing the Socio-Technical Gap

Reference 23

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:39.408794Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:39.408794Z digest=sha256:a18616a5b314c73c4499a4df25b15f6900895509e63a9a3e250e86f02d466bcf

Observation a92f3370-7547-4c92-97ad-5a0957c5af95 · outbound

This paper cites Vera Liao, Daniel Gruen, and Sarah Miller.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Vera Liao, Daniel Gruen, and Sarah Miller

Reference 24

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:39.516481Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:39.516481Z digest=sha256:fcc5a9c52e228e6a5d35e40c13960c43836138df6bbe220c926c596db90490bd

Observation a8b993fe-9673-47a8-aa2b-6b9d3d39ce79 · outbound

This paper cites S elf C heck GPT : Zero-resource black-box hallucination detection for generative large language models.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs S elf C heck GPT : Zero-resource black-box hallucination detection for generative large language models

Reference 25

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:39.620818Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:39.620818Z digest=sha256:2ee39f337347bb6c325815f0a39f010a40f52cf166f376257e992ac227802564

Observation f96673d3-1578-4deb-aa67-acf7f07d0710 · outbound

This paper cites Mathewson, Jaylen Pittman, and Richard Evans.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Mathewson, Jaylen Pittman, and Richard Evans

Reference 26

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:39.788439Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:39.788439Z digest=sha256:f6c06b602160859d1806488eddb2ec4bece5435bde4abdd68967ab5b61be574c

Observation 23dff6a3-e837-4a45-ab51-6548962615c2 · outbound

This paper cites Wolf, Josh Andres, Michael Desmond, Narendra Nath Joshi, Zahra Ashktorab, Aabhas Sharma, Kristina Brimijoin, Qian Pan, Evelyn Duesterwald, and Casey Dugan.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Wolf, Josh Andres, Michael Desmond, Narendra Nath Joshi, Zahra Ashktorab, Aabhas Sharma, Kristina Brimijoin, Qian Pan, Evelyn Duesterwald, and Casey Dugan

Reference 27

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:39.917055Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:39.917055Z digest=sha256:c424444f55fb1d22c831b9527f850aecba13baff4356fbee49539a1d03913160

Observation 0dc0bdb0-4283-4bf8-a8fe-4d6e40d4b2e3 · outbound

This paper cites an unresolved cited work.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Unresolved cited work

Reference 28

Resolution
unresolved
raw_fallback, observed 2026-08-06T15:29:45.676716Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

source=arxiv_source observed=2026-08-06T15:29:40.039345Z digest=sha256:1eaa6bd6e9605084cac183b603119a2320d38696dfa1a13763f38e8135b70018

Observation a529e512-12a9-4686-aae0-4702e446ffb3 · outbound

This paper cites Rodriguez Mendez, Thang Bui, Alyssa Goodman, Alberto Accomazzi, Jill Naiman, Jesse Cranney, Kevin Schawinski, and Roberta Raileanu.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Rodriguez Mendez, Thang Bui, Alyssa Goodman, Alberto Accomazzi, Jill Naiman, Jesse Cranney, Kevin Schawinski, and Roberta Raileanu

Reference 29

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:40.182493Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:40.182493Z digest=sha256:b5a36c0331679f4b4ece46168c24ebf0de670316a2e22e30546158a4cefa0159

Observation 2ed222ae-9815-4590-82e8-c0b06816356b · outbound

This paper cites emr QA : A large corpus for question answering on electronic medical records.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs emr QA : A large corpus for question answering on electronic medical records

Reference 30

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:40.252325Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:40.252325Z digest=sha256:c10c0b9cec8905e458dba8775051a1f98ae652d685b907606f4709eee62c80a7

Observation 340bbaff-2ae5-4607-9b65-9f4e20bfe3f5 · outbound

This paper cites B io R ead: A new dataset for biomedical reading comprehension.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs B io R ead: A new dataset for biomedical reading comprehension

Reference 31

Resolution
verified fuzzy
raw_fallback, observed 2026-08-06T15:29:45.468781Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

source=arxiv_source observed=2026-08-06T15:29:40.375448Z digest=sha256:98c9b5aa245238d19d49584369d8c0c71c1e4c0ff354460842b9b9f1e35bfc6e

Observation e74bfa0e-789f-4503-9ab5-483edc04a799 · outbound

This paper cites B io MRC : A dataset for biomedical machine reading comprehension.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs B io MRC : A dataset for biomedical machine reading comprehension

Reference 32

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:40.489331Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:40.489331Z digest=sha256:5a4af9af65985d75656d2d60bc5842ece5e87a5339a61ce4e8a4d1752989b0cc

Observation c7ff9d22-4084-4f89-bdd1-f30853b47950 · outbound

This paper cites Exploring temperature effects on large language models across various clinical tasks.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Exploring temperature effects on large language models across various clinical tasks

Reference 33

Resolution
verified fuzzy
raw_fallback, observed 2026-08-06T15:29:45.268357Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

source=arxiv_source observed=2026-08-06T15:29:40.584441Z digest=sha256:69b86e468c6af51017e8d19a5a776f331486e6eac9169f90421e1b32911e5945

Observation 3314d704-20c7-4a91-8284-519ee72dda91 · outbound

This paper cites Smith, Huiling Liu, Kevin Schawinski, Kartheik Iyer, Ioana Ciucă, and UniverseTBD.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Smith, Huiling Liu, Kevin Schawinski, Kartheik Iyer, Ioana Ciucă, and UniverseTBD

Reference 34

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:40.688976Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:40.688976Z digest=sha256:de852c6454297e9a59e44710dcf09b8be1006731c688d0c2fca0ac9fbc663848

Observation 4fca715d-5d5d-4316-a7e7-c0c8c5af6b18 · outbound

This paper cites Bender, Alex Hanna, and Amandalynne Paullada.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Bender, Alex Hanna, and Amandalynne Paullada

Reference 35

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:40.862940Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:40.862940Z digest=sha256:d467dcf29d8f819adcf599e5f3158cdb28431cc08f08e058683722a1055f3dbc

Observation db18018f-e43b-4cf7-8f0a-1e5d630aa9bd · outbound

This paper cites The effect of sampling temperature on problem solving in large language models.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs The effect of sampling temperature on problem solving in large language models

Reference 36

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:40.975379Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:40.975379Z digest=sha256:2d7efb4f99ea2665ac74dcd6442662d53e19df4e80a5fcf548c510d027df6137

Observation 44a2d16e-6a6f-4b4e-b5e5-73306be032ec · outbound

This paper cites Improving evidence retrieval for automated explainable fact-checking.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Improving evidence retrieval for automated explainable fact-checking

Reference 37

Resolution
verified exact
doi, observed 2026-08-06T15:29:42.860122Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

source=arxiv_source observed=2026-08-06T15:29:41.040347Z digest=sha256:be57b3163e921458d3e84db3e7d6be3842b0235e34c525934a6edfc75093ea43

Observation 187fe219-bbef-4d96-8112-24a864213317 · outbound

This paper cites an unresolved cited work.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Unresolved cited work

Reference 38

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:41.097841Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:41.097841Z digest=sha256:6631c982063e412a571404f0734d3899ad47934735a26d12850d4ae642d76d0c

Observation 71985320-c1e8-4ecf-a5dc-8468c119c5d6 · outbound

This paper cites The Good, The Bad, and The Greedy: Evaluation of LLMs Should Not Ignore Non-Determinism.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs The Good, The Bad, and The Greedy: Evaluation of LLMs Should Not Ignore Non-Determinism

Reference 39

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:41.138123Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:41.138123Z digest=sha256:0878e5fde4cc03fd777d162061daa25b463f2b37540668a11062ece7e66569ae

Observation 3aa554d2-c2d9-4573-ac49-0400e7b08b79 · outbound

This paper cites Scieval: a multi-level large language model evaluation benchmark for scientific research.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Scieval: a multi-level large language model evaluation benchmark for scientific research

Reference 40

Resolution
malformed identifier
no resolver link, observed 2026-08-06T15:29:41.201818Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:41.201818Z digest=sha256:111acfa781ec2ef8bae16a1647ce2ec9c69211972528f2dac95689b594afa707

Observation 5dd25c18-98f3-4b28-91af-f17b7e0950cd · outbound

This paper cites Nestor, Ali Soroush, Pierre A.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Nestor, Ali Soroush, Pierre A

Reference 41

Resolution
verified exact
doi, observed 2026-08-06T15:29:42.679016Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

source=arxiv_source observed=2026-08-06T15:29:41.320038Z digest=sha256:aee01ba5b5f7d9fe64096165856c830cc63564c1c79f163a85f8cde927dab2cb

Observation 6d21ebef-9c75-4bf2-8a6d-fe3d9e583d95 · outbound

This paper cites FEVER : a large-scale dataset for fact extraction and VER ification.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs FEVER : a large-scale dataset for fact extraction and VER ification

Reference 42

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:41.418093Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:41.418093Z digest=sha256:abf1b5b114e787e48359d2f75c6757ab7596ffa6fe2290176acd62ff0087bbd3

Observation c21746c8-56d1-4ecc-b42f-5a61afa09ddd · outbound

This paper cites Group chat ecology in enterprise instant messaging: How employees collaborate through multi-user chat channels on slack.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Group chat ecology in enterprise instant messaging: How employees collaborate through multi-user chat channels on slack

Reference 43

Resolution
verified exact
doi, observed 2026-08-06T15:29:42.523254Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

source=arxiv_source observed=2026-08-06T15:29:41.479622Z digest=sha256:7f62b1f3d4ecda7ad6aad3c56bcc85cbaf3a88199754efca64ce812f2e64c0f2

Observation 643ed9c8-a501-4895-bf7a-81bef5ad4cce · outbound

This paper cites Evaluating large language models on academic literature understanding and review: An empirical study among early-stage scholars.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Evaluating large language models on academic literature understanding and review: An empirical study among early-stage scholars

Reference 44

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:41.581642Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:41.581642Z digest=sha256:2c22a3110f391eac40a20ddb401e477e20cdc95b7b8192c3ff282249ea1ff28d

Observation 194c1acc-3bff-422b-b52c-c2e22d164b3c · outbound

This paper cites as an ai language model, i cannot.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs as an ai language model, i cannot

Reference 45

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:41.655835Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:41.655835Z digest=sha256:d39b3a8b376f608db2b117db38a05c328d18fe7ce6124f4ed64e77867caec9a3

Observation b5cda476-4e8d-475d-a1c2-9704ddabbf1e · outbound

This paper cites Designing an Evaluation Framework for Large Language Models in Astronomy Research.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Designing an Evaluation Framework for Large Language Models in Astronomy Research

Reference 46

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:41.746197Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:41.746197Z digest=sha256:b6201e20238c2c190039cf5b4a49e74d061bafdf754d002819421e1f82ab806d

Observation 347b5cc6-f90b-465a-87da-5b86b931a88a · outbound

This paper cites C-pack: Packed resources for general chinese embeddings.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs C-pack: Packed resources for general chinese embeddings

Reference 47

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:41.830639Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:41.830639Z digest=sha256:2a579d59e46c4f43bc2155eb2a8fa9b559210cbf6cf4afa36be163c3a5861e13

Observation 844b0afc-4d00-46e9-a8b5-eb4d4912d025 · outbound

This paper cites AI as an active writer: Interaction strategies with generated text in human- AI collaborative fiction writing.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs AI as an active writer: Interaction strategies with generated text in human- AI collaborative fiction writing

Reference 48

Resolution
verified fuzzy
raw_fallback, observed 2026-08-06T15:29:45.034922Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

source=arxiv_source observed=2026-08-06T15:29:41.891915Z digest=sha256:0131436678020ad755dc1303f97e86ad6e5fa8b8af92fe403cba14a7878affa0

Observation 01bbfc41-c32a-45ea-bb2a-0b9b7c493064 · outbound

This paper cites User-Controlled Knowledge Fusion in Large Language Models: Balancing Creativity and Hallucination.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs User-Controlled Knowledge Fusion in Large Language Models: Balancing Creativity and Hallucination

Reference 49

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:41.990196Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:41.990196Z digest=sha256:4b07d87c32b61120a8b96f2a1eaf318b23c8ac4b188929370f6a09a610ef7b1f

Observation b2bde327-f37d-4162-9d84-bf16ed863ac6 · outbound

This paper cites Weinberger, and Yoav Artzi.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Weinberger, and Yoav Artzi

Reference 50

Resolution
verified fuzzy
raw_fallback, observed 2026-08-06T15:29:44.830578Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

source=arxiv_source observed=2026-08-06T15:29:42.057541Z digest=sha256:d9b9a931d18a43c386ce06e06f46f1766f34df9dda4dfbb5e943975d2ad78428

Observation 40a5499e-16cf-4445-8e8f-cd3dde3432f5 · outbound

This paper cites WildChat : 1 M C hat GPT interaction logs in the wild.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs WildChat : 1 M C hat GPT interaction logs in the wild

Reference 51

Resolution
verified fuzzy
raw_fallback, observed 2026-08-06T15:29:44.598764Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

source=arxiv_source observed=2026-08-06T15:29:42.106650Z digest=sha256:3a568f39c46e661e0f5fdd1a6dce88d76b00a3e2ef8016a50110aa75082404e4

Observation 67552ce0-5813-426e-a34d-c33fa787ac2b · outbound

This paper cites Deconstructing NLG evaluation: Evaluation practices, assumptions, and their implications.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Deconstructing NLG evaluation: Evaluation practices, assumptions, and their implications

Reference 52

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:42.197073Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:42.197073Z digest=sha256:c1968561952a3f4e3ab993e646a13b1156bb395f4e96b8e8d841a541a6e9d1f0

Observation e98031c2-bf97-47b0-a7f9-a425c93788b5 · outbound

This paper cites Navigating the grey area: How expressions of uncertainty and overconfidence affect language models.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Navigating the grey area: How expressions of uncertainty and overconfidence affect language models

Reference 53

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:42.298617Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:42.298617Z digest=sha256:f3c8c2db415036c58dd60c7f5a8596f42ff61043f15cee7cc441bd68e14c8569

Observation fe456e7f-3a03-4119-ac9b-93293c239cc1 · outbound

This paper cites Relying on the unreliable: The impact of language models ' reluctance to express uncertainty.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Relying on the unreliable: The impact of language models ' reluctance to express uncertainty

Reference 54

Resolution
verified fuzzy
raw_fallback, observed 2026-08-06T15:29:44.375318Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

source=arxiv_source observed=2026-08-06T15:29:42.348441Z digest=sha256:de836e8fc3c9d2e85704120e3babe648c5f616bbc784f62e3dee24b36bc4a3e3

Pith citing papers

No inbound Pith citation observations are available.