arxiv: 2604.02600 · v2 · submitted 2026-04-03 · 💻 cs.HC · cs.AI

Recognition: 2 theorem links

· Lean Theorem

LitPivot: Developing Well-Situated Research Ideas Through Dynamic Contextualization and Critique within the Literature Landscape

Aakanksha Naik, Andrew Head, Bhavana Dalvi Mishra, Hita Kambhamettu, Jonathan Bragg, Joseph Chee Chang, Pao Siangliulue

Authors on Pith no claims yet

Pith reviewed 2026-05-13 19:21 UTC · model grok-4.3

classification 💻 cs.HC cs.AI

keywords research ideationliterature review toolsdynamic contextualizationidea pivotinghuman-computer interactioncritique generationacademic writing support

0 comments

The pith

LitPivot helps researchers develop higher-rated ideas by dynamically retrieving relevant paper clusters and generating literature-informed critiques that prompt iterative revisions.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Developing a research idea requires keeping it distinct from prior work while building on it, which means the relevant literature shifts as the idea changes. Most existing tools treat either the idea or the literature as fixed, breaking the needed back-and-forth. LitPivot introduces literature-initiated pivots: users select text in their draft, the system pulls fresh clusters of papers, and it offers critiques that suggest how to revise the idea. A lab study with 17 researchers found that participants using this approach produced higher-rated ideas and reported stronger understanding of the surrounding literature space than controls. An open-ended study with five researchers showed them repeatedly evolving their ideas through successive pivots triggered by the retrieved literature.

Core claim

LitPivot operationalizes literature-initiated pivots so that engagement with dynamically retrieved literature prompts revision of a developing idea and each revision changes which literature becomes relevant. Researchers concurrently draft and vet the idea: selecting part of the text retrieves clusters of papers and produces literature-informed critiques that guide revision. A lab study with 17 participants showed higher-rated final ideas and stronger self-reported understanding of the literature space; an open-ended study with five participants illustrated how users iteratively evolve their own ideas through repeated cycles of retrieval and critique.

What carries the argument

Literature-initiated pivots, the mechanism in which selecting text in a draft idea triggers retrieval of paper clusters and generation of critiques that drive revisions, which then update the relevant literature.

If this is right

Idea quality improves when critiques are generated from live, context-specific paper clusters rather than a static pre-curated set.
Researchers gain stronger understanding of the literature space when drafting and vetting occur in the same interface.
Ideas can evolve iteratively through repeated literature-triggered revisions instead of one-shot evaluation.
Research support tools should combine dynamic retrieval with critique generation rather than keeping literature search and ideation as separate stages.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

If the quality gains hold in larger field trials, interfaces built around literature-initiated pivots could shorten the time from rough idea to well-situated proposal.
Retrieval or generation biases could still favor well-cited directions; measuring this effect in extended use would clarify limits.
The same pivot mechanism might apply to non-academic creative work such as product design or policy writing, where context shifts as the concept develops.

Load-bearing premise

The dynamically retrieved paper clusters and generated critiques are accurate and unbiased enough to improve idea quality rather than steer users toward popular or easily retrievable work.

What would settle it

A controlled replication in which participants using LitPivot produce ideas rated no higher, on average, than participants using conventional separate literature search plus ideation tools would falsify the claim that the integrated pivot mechanism improves outcomes.

Figures

Figures reproduced from arXiv: 2604.02600 by Aakanksha Naik, Andrew Head, Bhavana Dalvi Mishra, Hita Kambhamettu, Jonathan Bragg, Joseph Chee Chang, Pao Siangliulue.

**Figure 1.** Figure 1: LitPivot supports the co-development of a research idea and its grounding in literature. Starting with a draft idea (“dynamic highlights in a reading interface”), LitPivot organizes relevant clusters of literature, revealing that existing work on adaptive highlighting might render the idea incremental, while also identifying different clusters of literature that would be useful to build upon. LitPivot faci… view at source ↗

**Figure 2.** Figure 2: The LitPivot interface. The left pane is a document editor with a toolbar toggle to highlight facets. The right pane is a paper panel that surfaces literature clusters. The interface colors text by idea facet: problem (purple), contribution (green), and evaluation (orange; currently missing). Clusters most relevant to the selected facet (here, the contribution statement) are starred. After selecting papers… view at source ↗

**Figure 3.** Figure 3: LitPivot pipeline for literature retrieval and faceted paper clustering. (1) The researcher’s idea is used to retrieve an initial set of relevant papers via Asta PaperFinder. (2) GPT-4o extracts idea-relevant facets from each paper. (3) Sonnet-4 clusters the literature by facet, surfacing groups of papers most relevant to each part of the idea. (4) The system extracts papers that are meaningfully cited wit… view at source ↗

**Figure 4.** Figure 4: Self-reported change in idea quality Distribution of [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗

**Figure 5.** Figure 5: When a change to one idea facet might require edit [PITH_FULL_IMAGE:figures/full_fig_p018_5.png] view at source ↗

**Figure 6.** Figure 6: Baseline interface. The left pane contains is an idea [PITH_FULL_IMAGE:figures/full_fig_p018_6.png] view at source ↗

**Figure 7.** Figure 7: Design probe used in the formative study. This [PITH_FULL_IMAGE:figures/full_fig_p018_7.png] view at source ↗

read the original abstract

Developing a novel research idea is hard. It must be distinct enough from prior work to claim a contribution while also building on it. This requires iteratively reviewing literature and refining an idea based on what a researcher reads; yet when an idea changes, the literature that matters often changes with it. Most tools offer limited support for this interplay: literature tools help researchers understand a fixed body of work, while ideation tools evaluate ideas against a static, pre-curated set of papers. We introduce literature-initiated pivots, a mechanism where engagement with literature prompts revision to a developing idea, and where that revision changes which literature is relevant. We operationalize this in LitPivot, where researchers concurrently draft and vet an idea. LitPivot dynamically retrieves clusters of papers relevant to a selected part of the idea and proposes literature-informed critiques for how to revise it. A lab study ($n{=}17$) shows researchers produced higher-rated ideas with stronger self-reported understanding of the literature space; an open-ended study ($n{=}5$) reveals how researchers use LitPivot to iteratively evolve their own ideas.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

LitPivot introduces dynamic literature pivots for idea development, with small studies showing gains but lacking validation on the accuracy of its LLM-driven components.

read the letter

The main takeaway from this paper is the LitPivot system and its literature-initiated pivots. It lets users draft research ideas while the tool pulls relevant paper clusters based on selections and generates critiques to help refine the idea, with changes feeding back into new retrievals. This seems like a practical way to handle the moving target of literature as ideas evolve. The paper does a good job describing the implementation and running user studies to test it. In the controlled lab study with 17 researchers, participants came up with ideas that were rated higher and reported better understanding of the surrounding literature. The open-ended study with five users gives a sense of how the tool supports ongoing iteration on real ideas. Where it falls short is in validating the quality of the dynamic elements. There's no separate assessment of whether the paper clusters are relevant and complete or if the critiques are accurate and helpful without introducing bias. The small number of participants leaves open the possibility that the positive results stem from the structured interaction itself or from the excitement of using a new tool rather than the specific pivot feature. Long-term outcomes aren't measured either. Overall, this is relevant for researchers in human-computer interaction who are interested in building tools to aid the research ideation process. It provides a concrete example and some initial data that could spark ideas for others. I would suggest putting it through peer review. The core idea is novel enough and the prototype functional, so referees can help refine the evaluation aspects.

Referee Report

2 major / 2 minor

Summary. The paper introduces LitPivot, a system supporting research idea development via literature-initiated pivots: dynamic retrieval of paper clusters relevant to selected idea parts and generation of literature-informed critiques that prompt iterative revisions. A lab study (n=17) reports higher-rated ideas and stronger self-reported literature understanding; an open-ended study (n=5) describes iterative idea evolution through system use.

Significance. If the core mechanism holds, the work offers a concrete advance in HCI research tools by tightly coupling ideation with dynamic literature contextualization, addressing a gap between static literature tools and fixed-set ideation systems. The user studies provide direct empirical grounding for improved idea quality and process insights, though small samples constrain broader claims.

major comments (2)

[Evaluation] Evaluation section (lab study): the headline result of higher-rated ideas (n=17) rests on the assumption that retrieved clusters and generated critiques are sufficiently accurate and unbiased; no separate expert validation, precision/recall metrics, or hallucination checks are reported for these components, leaving open that gains could arise from any structured prompt rather than the claimed pivot mechanism.
[Evaluation] Open-ended study (n=5): qualitative observations of iterative evolution are presented without quantitative measures of idea change (e.g., pre/post ratings or distance metrics), weakening support for the dynamic contextualization claim relative to the lab study's quantitative results.

minor comments (2)

[Abstract] Abstract: the notation n{=}17 should be rendered as n=17 for standard readability.
[System] System description: clarify how paper clusters are formed (e.g., embedding model, clustering algorithm) to allow replication of the dynamic retrieval step.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the thoughtful review and positive assessment of the work's significance. We address the major comments on the evaluation below, providing clarifications and outlining revisions to strengthen the manuscript.

read point-by-point responses

Referee: [Evaluation] Evaluation section (lab study): the headline result of higher-rated ideas (n=17) rests on the assumption that retrieved clusters and generated critiques are sufficiently accurate and unbiased; no separate expert validation, precision/recall metrics, or hallucination checks are reported for these components, leaving open that gains could arise from any structured prompt rather than the claimed pivot mechanism.

Authors: The lab study compares the full LitPivot system to a baseline without dynamic retrieval and critiques, isolating the effect of the literature-initiated pivots. Participant feedback indicated that the retrieved clusters were relevant and critiques helpful, supporting the mechanism's validity. However, we agree that formal validation metrics are absent. In revision, we will add an analysis of cluster relevance based on user selections and a discussion of potential limitations regarding accuracy and bias, including plans for future expert validation. This will clarify that gains stem from the specific pivot mechanism rather than generic structure. revision: partial
Referee: [Evaluation] Open-ended study (n=5): qualitative observations of iterative evolution are presented without quantitative measures of idea change (e.g., pre/post ratings or distance metrics), weakening support for the dynamic contextualization claim relative to the lab study's quantitative results.

Authors: We designed the open-ended study to capture rich qualitative data on the iterative process, which is central to demonstrating dynamic contextualization. While we did not collect pre/post ratings, the study documents specific instances of idea pivots triggered by literature engagement. To strengthen this, we will revise the presentation to include quantitative summaries such as the average number of iterations and pivots per participant, and add researcher-assessed measures of idea evolution where feasible from the existing data. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical user studies with independent evaluation

full rationale

The paper introduces LitPivot as a tool for literature-initiated pivots and supports its claims exclusively through two user studies (n=17 lab study on idea ratings and understanding; n=5 open-ended study on iterative use). No equations, parameters, or derivations appear in the abstract or described structure. Central results rest on participant ratings and observed behaviors rather than any self-definitional loop, fitted-input prediction, or self-citation that defines the outcome. Any self-citations present would be non-load-bearing background and do not reduce the empirical findings to inputs by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

This is an HCI system paper with no mathematical model, so the ledger contains only the high-level design assumption that dynamic paper retrieval plus critique generation will improve idea quality.

axioms (1)

domain assumption Dynamic retrieval of paper clusters from a selected idea fragment will surface literature that is both relevant and actionable for critique.
Invoked in the system description and study design; no independent verification provided beyond the user studies.

invented entities (1)

LitPivot system no independent evidence
purpose: Operationalize literature-initiated pivots for concurrent drafting and vetting of research ideas
The system itself is the primary contribution; no external falsifiable evidence for its components is supplied.

pith-pipeline@v0.9.0 · 5522 in / 1296 out tokens · 33031 ms · 2026-05-13T19:21:43.936503+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

60 extracted references · 60 canonical work pages

[1]

Philip G Altbach and Hans De Wit. 2018. Too much academic research is being published.International Higher Education96, 2 (2018)

work page 2018
[2]

Hearst, Andrew Head, and Kyle Lo

Tal August, Lucy Lu Wang, Jonathan Bragg, Marti A. Hearst, Andrew Head, and Kyle Lo. 2022. Paper Plain: Making Medical Research Papers Approachable to Healthcare Consumers with Natural Language Processing.ACM Transactions on Computer-Human Interaction30 (2022), 1 – 38. https://api.semanticscholar.org/ CorpusID:247187606

work page 2022
[3]

2016.Qualitative HCI Research: Going Behind the Scenes

Ann Blandford, Dominic Furniss, and Stephann Makri. 2016.Qualitative HCI Research: Going Behind the Scenes. Morgan & Claypool Publishers

work page 2016
[4]

Lutz Bornmann, Rüdiger Mutz, and Hans-Dieter Daniel. 2010. A reliability- generalization study of journal peer reviews: A multilevel meta-analysis of inter-rater reliability and its determinants.PloS one5, 12 (2010), e14331

work page 2010
[5]

Kevin J Boudreau, Eva C Guinan, Karim R Lakhani, and Christoph Riedl. 2016. Looking across and looking beyond the knowledge frontier: Intellectual distance, novelty, and resource allocation in science.Management Science62, 10 (2016), 2765–2783

work page 2016
[6]

Erin A Carroll and Celine Latulipe. 2012. Triangulating the personal creative experience: self-report, external judgments, and physiology. InProceedings of Graphics Interface 2012. 53–60

work page 2012
[7]

Joel Chan, Joseph Chee Chang, Tom Hope, Dafna Shahaf, and Aniket Kittur

work page
[8]

Solvent: A mixed initiative system for finding analogies between research papers.Proceedings of the ACM on Human-Computer Interaction2, CSCW (2018), 1–21

work page 2018
[9]

Erin Cherry and Celine Latulipe. 2014. Quantifying the creativity support of digital tools through the creativity support index.ACM Transactions on Computer- Human Interaction (TOCHI)21, 4 (2014), 1–25

work page 2014
[10]

Hai Dang, Chelse Swoopes, Daniel Buschek, and Elena L Glassman. 2025. Cor- pusStudio: Surfacing Emergent Patterns In A Corpus Of Prior Work While Writ- ing. InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–19

work page 2025
[11]

Susan B Davidson and Juliana Freire. 2008. Provenance and scientific work- flows: challenges and opportunities. InProceedings of the 2008 ACM SIGMOD international conference on Management of data. 1345–1350

work page 2008
[12]

Oscar Díaz and Jeremías P Contell. 2022. Developing research questions in conversation with the literature: operationalization & tool support.Empirical Software Engineering27, 7 (2022), 174

work page 2022
[13]

Timothy J Ellis and Yair Levy. 2008. Framework of problem-based research: A guide for novice researchers on the development of a research-worthy problem. Informing Science11 (2008), 17

work page 2008
[14]

KJ Feng, Kevin Pu, Matt Latzke, Tal August, Pao Siangliulue, Jonathan Bragg, Daniel S Weld, Amy X Zhang, and Joseph Chee Chang. 2026. Cocoa: Co-planning and co-execution with ai agents.Proceedings of the ACM on Human-Computer Interaction(2026)

work page 2026
[15]

Deborah Finfgeld-Connett and E Diane Johnson. 2013. Literature search strate- gies for conducting knowledge-building and theory-generating qualitative sys- tematic reviews.Journal of advanced nursing69, 1 (2013), 194–204

work page 2013
[16]

Ronald A Finke. 1996. Imagery, creativity, and emergent structure.Consciousness and cognition5, 3 (1996), 381–393

work page 1996
[17]

1996.Creative cognition: Theory, research, and applications

Ronald A Finke, Thomas B Ward, and Steven M Smith. 1996.Creative cognition: Theory, research, and applications. MIT press

work page 1996
[18]

2017.Information seeking of scholars in the field of higher education

Sarah Rose Fitzgerald. 2017.Information seeking of scholars in the field of higher education. Michigan State University

work page 2017
[19]

Raymond Fok, Hita Kambhamettu, Luca Soldaini, Jonathan Bragg, Kyle Lo, Marti Hearst, Andrew Head, and Daniel S Weld. 2023. Scim: Intelligent skimming support for scientific papers. InProceedings of the 28th International Conference on Intelligent User Interfaces. 476–490

work page 2023
[20]

Katy Ilonka Gero, Vivian Liu, and Lydia Chilton. 2022. Sparks: Inspiration for science writing using language models. InProceedings of the 2022 ACM Designing Interactive Systems Conference. 1002–1019

work page 2022
[21]

Darcy Haag Granello. 2001. Promoting cognitive complexity in graduate written work: Using Bloom’s taxonomy as a pedagogical tool to improve literature reviews.Counselor Education and Supervision40, 4 (2001), 292–307

work page 2001
[22]

Ziwei Gu, Joyce Zhou, Ning-Er Lei, Jonathan K Kummerfeld, Mahmood Jasim, Narges Mahyar, and Elena L Glassman. 2025. AbstractExplorer: Leveraging Structure-Mapping Theory to Enhance Comparative Close Reading at Scale. In Proceedings of the 38th Annual ACM Symposium on User Interface Software and Technology. 1–25

work page 2025
[23]

Armand Hatchuel, Benoit Weil, et al . 2003. A new approach of innovative Design: an introduction to CK theory.. InDS 31: Proceedings of ICED 03, the 14th International Conference on Engineering Design, Stockholm. 109–110

work page 2003
[24]

Andrew Head, Kyle Lo, Dongyeop Kang, Raymond Fok, Sam Skjonsberg, Daniel S Weld, and Marti A Hearst. 2021. Augmenting scientific papers with just-in-time, position-sensitive definitions of terms and symbols. InProceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–18

work page 2021
[25]

Dongwei Jiang, Marcio Fonseca, and Shay B Cohen. 2024. Leanreasoner: Boosting complex logical reasoning with lean. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 7497–7510

work page 2024
[26]

Hyeonsu Kang, Joseph Chee Chang, Yongsung Kim, and Aniket Kittur. 2022. Threddy: An interactive system for personalized thread-based exploration and organization of scientific literature. InProceedings of the 35th Annual ACM Sym- posium on User Interface Software and Technology. 1–15

work page 2022
[27]

Hyeonsu B Kang, Xin Qian, Tom Hope, Dafna Shahaf, Joel Chan, and Aniket Kittur. 2022. Augmenting scientific creativity with an analogical search engine. ACM Transactions on Computer-Human Interaction29, 6 (2022), 1–36

work page 2022
[28]

Hyeonsu B Kang, Tongshuang Wu, Joseph Chee Chang, and Aniket Kittur. 2023. Synergi: A mixed-initiative system for scholarly synthesis and sensemaking. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. 1–19

work page 2023
[29]

David Klahr and Kevin Dunbar. 1988. Dual space search during scientific reason- ing.Cognitive science12, 1 (1988), 1–48

work page 1988
[30]

Jeffrey W Knopf. 2006. Doing a literature review.PS: Political Science & Politics 39, 1 (2006), 127–132

work page 2006
[31]

Christine P Lee, David Porfirio, Xinyu Jessica Wang, Kevin Chenkai Zhao, and Bilge Mutlu. 2025. Veriplan: Integrating formal verification and llms into end- user planning. InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–19

work page 2025
[32]

Yoonjoo Lee, Hyeonsu B Kang, Matt Latzke, Juho Kim, Jonathan Bragg, Joseph Chee Chang, and Pao Siangliulue. 2024. Paperweaver: Enriching topical paper alerts by contextualizing recommended papers with user-collected papers. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–19

work page 2024
[33]

Q Vera Liao, Hariharan Subramonyam, Jennifer Wang, and Jennifer Wort- man Vaughan. 2023. Designerly understanding: Information needs for model transparency to support design ideation for AI-powered user experience. In Proceedings of the 2023 CHI conference on human factors in computing systems. 1–21

work page 2023
[34]

Weng Marc Lim, Satish Kumar, and Faizan Ali. 2022. Advancing knowledge through literature reviews:‘what’, ‘why’, and ‘how to contribute’.The Service Industries Journal42, 7-8 (2022), 481–513

work page 2022
[35]

Yiren Liu, Si Chen, Haocong Cheng, Mengxia Yu, Xiao Ran, Andrew Mo, Yiliu Tang, and Yun Huang. 2024. How ai processing delays foster creativity: Exploring research question co-creation with an llm-based agent. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–25

work page 2024
[36]

OpenAI. 2026. Introducing Prism. https://openai.com/index/introducing-prism/. Accessed: 2026-02-04

work page 2026
[37]

Srishti Palani, Aakanksha Naik, Doug Downey, Amy X Zhang, Jonathan Bragg, and Joseph Chee Chang. 2023. Relatedly: Scaffolding literature reviews with existing related work sections. InProceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–20

work page 2023
[38]

Elizabeth L Pier, Markus Brauer, Anna Filut, Anna Kaatz, Joshua Raclaw, Mitchell J Nathan, Cecilia E Ford, and Molly Carnes. 2018. Low agreement among reviewers evaluating the same NIH grant applications.Proceedings of the National Academy of Sciences115, 12 (2018), 2952–2957

work page 2018
[39]

Kevin Pu, KJ Kevin Feng, Tovi Grossman, Tom Hope, Bhavana Dalvi Mishra, Matt Latzke, Jonathan Bragg, Joseph Chee Chang, and Pao Siangliulue. 2025. Ideasynth: Iterative research idea development through evolving and composing idea facets with literature-grounded feedback. InProceedings of the 2025 CHI Kambhamettu et al. Conference on Human Factors in Compu...

work page 2025
[40]

Marissa Radensky, Simra Shahid, Raymond Fok, Pao Siangliulue, Tom Hope, and Daniel S Weld. 2024. Scideator: Human-llm scientific idea generation grounded in research-paper facet recombination.arXiv preprint arXiv:2409.14634(2024)

work page arXiv 2024
[41]

Orit Shaer, Angelora Cooper, Osnat Mokryn, Andrew L Kun, and Hagit Ben Shoshan. 2024. AI-Augmented Brainwriting: Investigating the use of LLMs in group ideation. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–17

work page 2024
[42]

Nihar B Shah, Behzad Tabibian, Krikamol Muandet, Isabelle Guyon, and Ulrike Von Luxburg. 2018. Design and analysis of the NIPS 2016 review process.Journal of Machine Learning Research19, 49 (2018), 1–34

work page 2018
[43]

Zejiang Shen, Tal August, Pao Siangliulue, Kyle Lo, Jonathan Bragg, Jeff Ham- merbacher, Doug Downey, Joseph Chee Chang, and David Sontag. 2023. Beyond summarization: Designing AI support for real-world expository writing tasks. In Proceedings of the 2nd Workshop on Intelligent and Interactive Writing Assistants (In2Writing)

work page 2023
[44]

Amanpreet Singh, Joseph Chee Chang, Dany Haddad, Aakanksha Naik, Jena D Hwang, Rodney Kinney, Daniel S Weld, Doug Downey, and Sergey Feldman. 2025. Ai2 scholar qa: Organized literature synthesis with attribution. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations). 513–523

work page 2025
[45]

Sangho Suh, Meng Chen, Bryan Min, Toby Jia-Jun Li, and Haijun Xia. 2024. Lumi- nate: Structured generation and exploration of design space with large language models for human-ai co-creation. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–26

work page 2024
[46]

Sangho Suh, Bryan Min, Srishti Palani, and Haijun Xia. 2023. Sensecape: En- abling multilevel exploration and sensemaking with large language models. In Proceedings of the 36th annual ACM symposium on user interface software and technology. 1–18

work page 2023
[47]

Rosni Vasu, Chandrayee Basu, Bhavana Dalvi Mishra, Cristina Sarasua, Peter Clark, and Abraham Bernstein. 2025. HypER: Literature-grounded Hypothesis Generation and Distillation with Provenance.The Conference on Empirical Methods in Natural Language Processing (EMNLP)(2025)

work page 2025
[48]

in- terestingness

Dashun Wang, Chaoming Song, and Albert-László Barabási. 2013. Quantifying long-term scientific impact.Science342, 6154 (2013), 127–132. A Guidelines for rating idea artifacts • Focus on whether the idea description leveraged prior work to argue for its novelty and explain its relation to relevant lit- erature. For example, arguments and methods are strong...

work page 2013
[52]

evaluation

Why should we bother solving this problem? IMPORTANT: Here is an example of the output format: "evaluation" : "The current idea omits any concrete metrics or procedures, so it cannot yet demonstrate that the produced citation-hop dataset is correct or useful for verifying biomedical claims. Prior work such as “‘<Paper corpusId="268248445">Piecing It All T...

work page
[55]

evaluation

Is the solution novel? IMPORTANT: Here is an example of the output format: "evaluation" : "The current idea omits any concrete metrics or procedures, so it cannot yet demonstrate that the produced citation-hop dataset is correct or useful for verifying biomedical claims. Prior work such as “‘<Paper corpusId="268248445">Piecing It All Together</Paper>“‘ an...

work page
[57]

evidence1

Will the evaluation succeed? IMPORTANT: Your output should strictly be of the following form: Evaluation: [3 sentences of evaluation, following citing paper formats] Suggestion: full rewrite1 with concrete sugestion: "evidence1": quoted evidence from a cited paper, "paper1" : corpusId of paper1 "explanation1": explanation of why this is a good rewrite gro...

work page
[58]

We currently can’t do Y

What is the problem? This should be of the form "We currently can’t do Y" or "We want to do Y"

work page
[59]

What is the specific problem we are trying to solve?

work page
[60]

How do we know this is an actual problem?

work page
[61]

Why should we bother solving this problem?

work page
[62]

Does the solution accurately address the problem?

work page
[63]

Will the solution succeed?

work page
[64]

Is the solution novel?

work page
[65]

Will the proposed evaluation ensure the solution accurately address the problem?

work page
[66]

semi-formal proof navigation

Will the evaluation succeed? Here are previous assessments of parts of this idea. PREVIOUS_ASSESSMENTS Here is the proposal: IDEA Your assessment should be no more than 5 sentences. Be thorough. C Example of ideas fromLitPivotand the baseline from the user study LitPivot Table 1: Side-by-side comparison of baseline ideas and their LitPivot counterparts. D...

work page 2023