Pith. sign in

REVIEW 4 major objections 6 minor 50 references

Towards Temporal Knowledge-Base Creation for Fine-Grained Opinion Analysis with Language Models

T0 review · 4 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLMs can be wired as declarative annotators to build a temporal opinion knowledge base without hand-written prompts.

desk verdict Useful methodology paper with a real evaluation gap: the temporal KB is validated only by inter-LLM agreement, and the best pipeline settings are chosen on the same test samples later reported. read the letter →

arxiv 2509.02363 v1 pith:BVKXT3FP submitted 2025-09-02 cs.CL

Gaurav Negi , Atul Kr. Ojha , Omnia Zayed , Paul Buitelaar This is my paper · ORCID
classification cs.CL
keywords temporalopinionknowledgebasefine-grainedminingLLM-baseddataannotationdeclarativepromptoptimizationaspect-basedsentimentanalysisstructuredretrieval-augmentedgenerationtimestampedsocialmedia
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a temporal, fine-grained opinion knowledge base can be built at scale by using LLMs as automated annotators, as long as the annotation task is expressed through established opinion schemas and the prompting is optimized declaratively rather than by hand. The authors define three data models—aspect-category-opinion-sentiment quadruples, structured sentiment tuples, and a unified opinion-concept ontology—and use them as machine-readable output structures. A declarative compiler then tunes prompts and in-context examples for an 8-billion-parameter LLM, eliminating manual prompt engineering. On human-annotated benchmark samples the configured annotator extracts schema-conforming opinions with component-level F1 in the mid-50s range. The method is applied to timestamped political and financial corpora, yielding a time-aligned knowledge base meant for forecasting, retrieval-augmented generation, and timeline summarization.

What carries the argument

The load-bearing mechanism is the pairing of a schema with a declarative LLM program. Each of the three schemas—ACOS quadruple (aspect term, category, opinion span, sentiment polarity), structured-sentiment tuple (holder, target, expression), and Unified Opinion Concepts (adding reason, qualifier, intensity)—turns an open-ended extraction task into a constrained output structure. A compiler optimizes the prompt text and chooses in-context examples using an evaluator, so the LLM's output is marshalled directly into the schema; every tuple records the post's publication timestamp. The timestamp is what converts a static opinion database into a temporal knowledge base.

What would settle it

Take a random sample of the annotated temporal knowledge base (e.g., 200 posts from each corpus), have independent human annotators apply the same three schemas, and compute component-level agreement between human labels and the LLM output. If human-LLM agreement on free-form spans such as aspect terms and entities is close to the low inter-LLM values (roughly 14 to 30 percent), while categorical labels agree more strongly, the paper's quality proxy is too optimistic; if human agreement tracks or exceeds those inter-LLM numbers, the proxy is reasonable.

Watch

Extended reading notes

Core claim

The central claim is that a knowledge base of temporal opinions can be produced by combining (1) three established fine-grained opinion formulations used as annotation schemas, (2) a declarative LLM pipeline that automatically selects prompts and in-context examples, and (3) two open-weight 8-billion-parameter LLMs as annotators. Each extracted opinion is bound to its post's timestamp, giving the knowledge base a temporal axis absent from prior subjective knowledge bases. The authors evaluate the pipeline against human labels for each schema, achieving its strongest agreement on the quadruple and unified schemas and its weakest on structured sentiment extraction, and then annotate the tempor

Load-bearing premise

The load-bearing premise is that agreement between two LLM annotators is a valid stand-in for human annotation quality on the temporal data; if those two models share the same blind spots, their agreement does not establish that the extracted opinions are correct.

Editorial extensions

If this is right

  • Existing opinion-mining schemas can be reused as annotation targets, so new domains or opinion facets require schema definition and optimizer reruns rather than hand-crafted prompts.
  • A temporal opinion knowledge base built this way is directly consumable by retrieval-augmented generation, temporal question answering, and timeline summarization systems that need structured, dated opinions.
  • Label-wise agreement between LLM annotators gives a per-component reliability map, showing which opinion facets such as polarity and intensity are more trustworthy than free-form spans like entities and aspect terms.
  • Because the LLM weights are not trained, the pipeline stays model-agnostic: stronger or larger LLMs can be swapped in without changing the annotation setup.
  • The approach can generate opinion annotations at scale on continuous social-media streams, making periodic manual annotation unnecessary for maintaining the knowledge base.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the schema-bounded format means the resulting knowledge base's value for downstream retrieval-augmented generation will be concentrated in categorical facets; entity-level facts should be treated as candidate spans, not verified assertions, until human-validated.
  • Editorial inference: the same pipeline should transfer to other timestamped opinionated text—product reviews, public-health chatter, news comments—since none of the three schemas is domain-specific; a direct test would be running it on a small human-annotated review corpus and measuring component-level agreement.
  • Editorial inference: the high inter-LLM agreement on reason and qualifier spans in the political dataset suggests those components may be easier to extract than general entity spans, an asymmetry worth exploiting in downstream timeline summarization.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a pipeline for constructing a temporal opinion knowledge base from timestamped social media text. Three fine-grained opinion data models (ACOS, SSA, and UOC) are defined and used as output schemas. The annotation pipeline uses DSPy to automatically configure prompts and in-context examples for two 8B open-weight LLMs (Llama-3.1-8B and Ministral-8B). The pipeline is first evaluated against human-annotated source datasets (Table 3), then applied to StockTwits and Politifact data to produce the temporal KB. Since no human labels exist for the temporal data, inter-LLM F1 agreement is computed as a proxy for annotation reliability (Table 4). The authors release the annotated KB publicly.

Significance. If the central claim is secured, the paper would offer a low-cost, scalable route to building time-aligned fine-grained opinion resources, which are currently scarce. The paper's strengths include grounding the schemas in established opinion-mining formulations, using a declarative DSPy pipeline to avoid manual prompt engineering, and releasing the annotated data. The use of two open-weight LLMs with deterministic inference settings is a positive reproducibility feature. However, the evaluation protocol has several load-bearing weaknesses: the test sets are filtered by outlier exclusion, the best ICL/COT settings are selected on the same test samples that are later reported, and the temporal KB is not validated against human annotations. These issues substantially weaken the empirical support for the central claim.

major comments (4)
  1. [Sections 4.2 and 4.5, Table 3] The evaluation protocol inflates the reported pipeline performance. Section 4.2 explicitly applies outlier exclusion to the test sets, removing all instances with more than U=4 opinions before computing the scores in Table 3. This changes the evaluation population and prevents comparison with standard benchmark results. More importantly, Section 4.5 states that the same human-annotated test samples are used to select the best configuration (number of ICL examples and COT setting) and then to report final performance. With 8 configurations evaluated per model and schema, the reported F1 scores are likely optimistic due to selection on the test set. Please hold out a dedicated validation set for configuration selection and report evaluation on an untouched test set, or use nested cross-validation; also report results on the full test sets, not only the filtered inlier subset.
  2. [Section 6.1, Table 4] The central claim that the released KB contains reliable time-aligned opinions rests on using inter-LLM F1 as a proxy for annotation correctness, but this proxy is unvalidated. The paper states that no human labels exist for the temporal datasets and therefore treats agreement between Ministral-8B and Llama-3.1-8B as an IRR proxy. This is not calibrated: the same models achieve only 46–60 F1 against human labels on the source-domain test sets (Table 3), and on the temporal data several key components fall to very low agreement (e.g., Aspect Term 14.43–37.72, Entity 14.85–21.07, Sentiment Expression 20–42.98 in Table 4). Two annotators can agree without being correct, and low absolute agreement indicates that the most fine-grained fields of the KB are unstable. A human evaluation on a sample of the temporal annotations, or at minimum a calibration study comparing inter-LLM F1 to human agr
  3. [Sections 3.3 and 6.1, Tables 1 and 4] The pipeline is configured and evaluated on source domains that differ substantially from the target temporal data. ACOS is evaluated on restaurant/laptop reviews, SSA on open-domain text, and UOC on product reviews, while the KB is built for StockTwits (financial microblogging) and Politifact (political statements). No human-annotated target-domain evaluation is provided, and inter-LLM agreement alone cannot measure annotation accuracy in the new domains. The low agreement for span-based components may partly reflect domain shift. Please provide evidence of transferability, for example by annotating a small random sample of the temporal data with human labelers and reporting the pipeline's precision/recall against those labels.
  4. [Section 3.3, reference [37]] The UOC data model and its human-annotated evaluation dataset come from the authors' own prior work (Negi et al., 2025), making the UOC evaluation self-referential. The ACOS and SSA evaluations are independently grounded, so this does not invalidate the entire pipeline, but the UOC-specific results in Tables 3 and 4 should be explicitly qualified as based on the authors' own ontology and annotations, and the potential circularity should be discussed.
minor comments (6)
  1. [Section 4.3, Eq. (1)] The notation in the definition of f(o_e, o_g) is inconsistent: the numerator uses o_{g,k} but the denominator uses o_{g,k} and later o_g is used with a different index. Please clarify the alignment notation and ensure all indices are defined.
  2. [Table 3 and Table 4] There are typos in the table headers and text: 'Mininstral' and 'Ministal' in the text, 'Politifacts' in Table 4, and 'Senitment'/'Setiment' in several places. Also, the use of 'UOCE' vs 'UOC' for the same data model is inconsistent.
  3. [Section 3.1] There is a typo: 'date models' should be 'data models'.
  4. [Section 5] The paper reports temperature 0.0 and fixed hyperparameters, but DSPy optimization (MIPRO) itself is stochastic due to sampling of prompts and demonstrations. Please report the number of optimization runs or seeds, or provide variance estimates, to assess stability.
  5. [Abstract and Section 7] The claim that the KB is 'compatible with applications in RAG, temporal question answering, and timeline summarisation' is asserted but not demonstrated anywhere in the paper. Either add a small downstream-task demonstration or soften the claim.
  6. [Figure 5] Figure 5 (right) is difficult to read: the labels for datasets and the box plots are small, and the legend is unclear. Please enlarge or re-render for clarity.

Circularity Check

2 steps flagged · score 4.0 of 10

Partial circularity: UOC schema and its human-annotated evaluation derive from the authors' own prior work, and Table 3 reports F1 on the same test sample used to select the pipeline configuration; ACOS/SSA provide independent grounding for the core approach.

  1. self citation load bearing [Section 2.3, Section 3.2, Table 1; reference [37]]
    "Negi et al. [37] introduce Unified Opinion Concepts (UOC) ontology to integrate opinions within their semantic context... We use these formulations to create data models for the opinion schema of the annotated knowledge base we created. ... UOC [37] {Laptop, Restaurant, Books, Clothes, Hotels} UOC 100 N/A N/A"

    The UOC data model and its 100-sample human-annotated evaluation set both come from reference [37], authored by Negi, Zayed, and Buitelaar (overlapping authors). The paper presents UOC as one of the accepted opinion formulations and evaluates the UOC annotation branch on the authors' own prior test labels, so the UOC branch's claimed validity is supported by self-citation rather than independent evidence. The other two schemas (ACOS, SSA) are external, so the circularity is partial.

  2. fitted input called prediction [Section 4.5, Table 3]
    "The evaluations are performed on the test sample derived from the pipeline training dataset. This evaluation is against human-annotated labels... We also use these results to select the best-performing settings for applying them to the annotation of the temporal opinion datasets. These results are reported in the Table. 3."

    The DSPy configuration (number of ICL examples, COT) is chosen by maximizing F1 on the human-annotated test sample, and the same test sample is then reported as the pipeline's efficacy against human labels (Table 3). The reported F1 values are therefore the selection objective, not out-of-sample estimates; part of the apparent performance is forced by construction. This does not invalidate the pipeline's independent ACOS/SSA grounding, but it means the 'rigorous quantitative evaluation' is not a clean external benchmark.

full rationale

The paper's core idea—using DSPy with LLMs to annotate temporal text according to existing opinion schemas—is not intrinsically circular: ACOS and SSA are external benchmarks (Cai et al., Barnes et al.), and the pipeline is evaluated against those human labels in Table 3. However, two aspects are self-referential. First, the UOC schema and its evaluation set come from the authors' own prior work [37], so for one of the three schemas the validation is an internal self-consistency check rather than independent evidence. Second, the reported human-evaluation F1 (Table 3) is obtained on the same test sample used to select the best pipeline configuration, making the headline numbers a selection score. The temporal KB itself is never validated against human annotations; instead, inter-LLM F1 (Table 4) is used as a proxy, which is a limitation rather than a definitional circularity. Overall, the central method has independent content, but the UOC branch and the evaluation protocol introduce partial circularity, warranting a score of 4.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the correctness of source gold labels, the fidelity of DSPy marshalling, the validity of the component-level F1 metric, and the use of inter-LLM agreement as a quality proxy for the temporal KB. The UOC component relies on the authors' own ontology and dataset.

free parameters (1)
  • Outlier upper bound U = 4
    Chosen as max of per-dataset IQR upper bounds (2.5, 4, 4) in Section 4.2; used to exclude instances with more than 4 opinions from both training and test sets, affecting all reported F1 scores.
assumptions (5)
  • domain assumption Gold labels in the ACOS, SSA, and UOC datasets are correct
    The pipeline evaluation compares LLM outputs against these human annotations; if the gold labels are noisy, reported F1 scores are unreliable.
  • domain assumption DSPy's marshalling faithfully converts LLM outputs into the defined data models
    The pipeline relies on DSPy to enforce schema compliance; any marshalling errors would corrupt the knowledge base structure. Invoked throughout Section 4.4.
  • domain assumption Component-level F1 with one-to-one matching is a valid measure of structured annotation quality
    Section 4.3 defines the metric; its validity is assumed without comparison to human consistency or alternative metrics.
  • domain assumption Inter-LLM F1 agreement is a valid proxy for inter-annotator reliability on the temporal KB
    Section 6.1 uses F1 between two LLMs as a proxy for IRR, citing previous work; the authors themselves note the need for ensembling in limitations.
  • ad hoc to paper The UOC ontology from Negi et al. [37] is a valid opinion formulation
    The UOC data model is taken directly from the authors' prior work; its validity is not independently established in this paper, creating self-referential evaluation for one schema.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Temporal Knowledge-Base Creation for Fine-Grained Opinion Analysis with Language Models." pith.science (2026). https://pith.science/paper/BVKXT3FP

@misc{pith2026250902363,
  author       = {Pith},
  title        = {Pith review of: Towards Temporal Knowledge-Base Creation for Fine-Grained Opinion Analysis with Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BVKXT3FP}},
  note         = {Machine review of arXiv:2509.02363}
}
read the original abstract

We propose a scalable method for constructing a temporal opinion knowledge base with large language models (LLMs) as automated annotators. Despite the demonstrated utility of time-series opinion analysis of text for downstream applications such as forecasting and trend analysis, existing methodologies underexploit this potential due to the absence of temporally grounded fine-grained annotations. Our approach addresses this gap by integrating well-established opinion mining formulations into a declarative LLM annotation pipeline, enabling structured opinion extraction without manual prompt engineering. We define three data models grounded in sentiment and opinion mining literature, serving as schemas for structured representation. We perform rigorous quantitative evaluation of our pipeline using human-annotated test samples. We carry out the final annotations using two separate LLMs, and inter-annotator agreement is computed label-wise across the fine-grained opinion dimensions, analogous to human annotation protocols. The resulting knowledge base encapsulates time-aligned, structured opinions and is compatible with applications in Retrieval-Augmented Generation (RAG), temporal question answering, and timeline summarisation.

Figures

Figures reproduced from arXiv: 2509.02363 by the authors.

Figure 1
Figure 1. Aspect-Category-Opinion-Sentiment (ACOS) [32] Structured Sentiment Analysis : The data model is built upon the structured sentiment formulation aimed at performing opinion tuple extraction as dependency graph parsing, where the nodes are spans of sentiment holders, targets and expressions. The interconnection between them shows the existence of relationships among these components. The primary focus of this formulat… view at source ↗
Figure 2
Figure 2. Structured Sentiment Analysis (SSA) [41] 2Opinion in ACOS task refers to the span that expresses sentiment [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Unified Opinion Concepts [37] 3.3. Datasets We use two types of datasets in our work: (i) Data Model Datasets, (ii) Temporal Knowledge-Base Dataset. We use the former one as source datasets to train and evaluate our pipeline for the extraction of opinions adhering to the specified data models (see [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Flow diagram representing the DSPy-LLM architecture and knowledge generation pipeline. However, LLMs are known to be sensitive to the prompts [43] and exhibit a higher level of variability in performing complex tasks [44]. To minimise the spurious interaction between t…
Figure 5
Figure 5. Figure 5: Opinion per-data instance (Left), Original And Sampled Datasets (Right) [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 30 canonical work pages

  1. [1]

    Liu, Many Facets of Sentiment Analysis, in: E

    B. Liu, Many Facets of Sentiment Analysis, in: E. Cambria, D. Das, S. Bandyopadhyay, A. Feraco (Eds.), A Practical Guide to Sentiment Analysis, Springer International Publishing, Cham, 2017, pp. 11–39. URL: https://doi.org/10.1007/978-3-319-55394-8_2. doi: 10.1007/978-3-319-55394-8_ 2

    work page doi:10.1007/978-3-319-55394-8_2 2017

  2. [2]

    Zhang, X

    W. Zhang, X. Li, Y. Deng, L. Bing, W. Lam, A survey on aspect-based sentiment analysis: Tasks, methods, and challenges, IEEE Transactions on Knowledge and Data Engineering 35 (2023) 11019–11038. doi:10.1109/TKDE.2022.3230975

    arXiv 2023

  3. [3]

    Pontiki, D

    M. Pontiki, D. Galanis, J. Pavlopoulos, H. Papageorgiou, I. Androutsopoulos, S. Manandhar, Semeval-2014 task 4: Aspect based sentiment analysis, in: P. Nakov, T. Zesch (Eds.), Proceedings of the 8th International Workshop on Semantic Evaluation, SemEval@COLING 2014, Dublin, Ireland, August 23-24, 2014, The Association for Computer Linguistics, 2014, pp. 2...

    doi:10.3115/v1/s14-2004 2014

  4. [4]

    Lewis, E

    P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.- t. Yih, T. Rocktäschel, S. Riedel, D. Kiela, Retrieval-augmented generation for knowledge- intensive nlp tasks, in: H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, H. Lin (Eds.), Advances in Neural Information Processing Systems, volume 33, Curran Associates, In...

    work page 2020

  5. [5]

    K. Ning, Z. Pan, Y. Liu, Y. Jiang, J. Y. Zhang, K. Rasul, A. Schneider, L. Ma, Y. Nevmyvaka, D. Song, Ts-rag: Retrieval-augmented generation based time series foundation models are stronger zero-shot forecaster, 2025. URL: https://arxiv.org/abs/2503.07649. arXiv:2503.07649

    arXiv 2025

  6. [6]

    X. Zhu, X. Guo, S. Cao, S. Li, J. Gong, Structugraphrag: Structured document-informed knowledge graphs for retrieval-augmented generation, Proceedings of the AAAI Symposium Series 4 (2024) 242–251. URL: https://ojs.aaai.org/index.php/AAAI-SS/article/view/31798. doi:10.1609/aaaiss. v4i1.31798

    doi:10.1609/aaaiss 2024

  7. [7]

    Zhang, S

    Q. Zhang, S. Chen, Y. Bei, Z. Yuan, H. Zhou, Z. Hong, J. Dong, H. Chen, Y. Chang, X. Huang, A survey of graph retrieval-augmented generation for customized large language models, arXiv preprint arXiv:2501.13958 (2025)

    arXiv 2025

  8. [8]

    Brown, B

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, D. Amodei, Lan...

    work page 2020

Show all 50 references
  1. [9]

    O’Connor, R

    B. O’Connor, R. Balasubramanyan, B. Routledge, N. Smith, From tweets to polls: Linking text sentiment to public opinion time series, Proceedings of the International AAAI Conference on Web and Social Media 4 (2010) 122–129. URL: https://ojs.aaai.org/index.php/ICWSM/article/vie...

    2010 doi

  2. [10]

    Dermouche, J

    M. Dermouche, J. Velcin, L. Khouas, S. Loudcher, A joint model for topic-sentiment evolution over time, in: R. Kumar, H. Toivonen, J. Pei, J. Z. Huang, X. Wu (Eds.), 2014 IEEE International Conference on Data Mining, ICDM 2014, Shenzhen, China, December 14-17, 2014, IEEE Compu...

    2014 doi

  3. [11]

    Giachanou, F

    A. Giachanou, F. Crestani, Tracking sentiment by time series analysis, in: R. Perego, F. Sebastiani, J. A. Aslam, I. Ruthven, J. Zobel (Eds.), Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, SIGIR 2016, Pisa, Ita...

    2016

  4. [12]

    K. Xu, G. Qi, J. Huang, T. Wu, X. Fu, Detecting bursts in sentiment-aware topics from social media, Knowl. Based Syst. 141 (2018) 44–54. URL: https://doi.org/10.1016/j.knosys.2017.11.007. doi:10.1016/J.KNOSYS.2017.11.007

    2018 doi

  5. [13]

    Y. Xu, Y. Li, Y. Liang, L. Cai, Topic-sentiment evolution over time: a manifold learning-based model for online news, J. Intell. Inf. Syst. 55 (2020) 27–49. URL: https://doi.org/10.1007/s10844-019-00586-5. doi:10.1007/S10844-019-00586-5

    2020 doi

  6. [14]

    H. Yin, S. Yang, J. Li, Detecting topic and sentiment dynamics due to COVID-19 pandemic using social media, in: X. Yang, C. Wang, M. S. Islam, Z. Zhang (Eds.), Advanced Data Mining and Applications - 16th International Conference, ADMA 2020, Foshan, China, November 12-14, 2020...

    2020 doi

  7. [15]

    B. Zhu, X. Zheng, H. Liu, J. Li, P. Wang, Analysis of spatiotemporal characteristics of big data on social media sentiment with covid-19 epidemic topics, Chaos, Solitons and Fractals 140 (2020) 110123. URL: https://www.sciencedirect.com/science/article/pii/S0960077920305208. d...

    2020

  8. [16]

    J. Wang, Y. Fan, J. Palacios, Y. Chai, N. Guetta-Jeanrenaud, N. Obradovich, C. Zhou, S. Zheng, Global evidence of expressed sentiment alterations during the covid-19 pandemic, Nature Human Behaviour 6 (2022) 349–358

    2022

  9. [17]

    D. M. Low, L. Rumker, J. Torous, G. Cecchi, S. S. Ghosh, T. Talkar, Natural language processing reveals vulnerable mental health support groups and heightened health anxiety on reddit during covid-19: Observational study, Journal of medical Internet research 22 (2020) e22635

    2020

  10. [18]

    J. Lee, H. L. Youn, J. Poon, S. C. Han, Stockemotions: Discover investor emotions for financial sentiment analysis and multivariate time series, arXiv preprint arXiv:2301.09279 (2023)

    2023 arXiv

  11. [19]

    Cambria, A

    E. Cambria, A. Hussain, E. Cambria, A. Hussain, Senticnet, Sentic computing: a common-sense- based framework for concept-level sentiment analysis (2015) 23–71

    2015

  12. [20]

    Y. Li, A. Feng, J. Li, S. Mumick, A. Halevy, V. Li, W.-C. Tan, Subjective databases, Proc. VLDB Endow. 12 (2019) 1330–1343. URL: https://doi.org/10.14778/3342263.3342271. doi: 10.14778/3342263. 3342271

    2019

  13. [21]

    M. Hu, B. Liu, Mining and summarizing customer reviews, in: W. Kim, R. Kohavi, J. Gehrke, W. DuMouchel (Eds.), Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Seattle, Washington, USA, August 22-25, 2004, ACM, 2004, pp. 168–...

    2004

  14. [22]

    Zhuang, F

    L. Zhuang, F. Jing, X. Zhu, Movie review mining and summarization, in: P. S. Yu, V. J. Tsotras, E. A. Fox, B. Liu (Eds.), Proceedings of the 2006 ACM CIKM International Conference on Information and Knowledge Management, Arlington, Virginia, USA, November 6-11, 2006, ACM, 2006...

    2006

  15. [23]

    X. Ding, B. Liu, P. S. Yu, A holistic lexicon-based approach to opinion mining, in: M. Najork, A. Z. Broder, S. Chakrabarti (Eds.), Proceedings of the International Conference on Web Search and Web Data Mining, WSDM 2008, Palo Alto, California, USA, February 11-12, 2008, ACM, ...

    2008

  16. [24]

    B. Liu, L. Zhang, A Survey of Opinion Mining and Sentiment Analysis, in: C. C. Aggarwal, C. Zhai (Eds.), Mining Text Data, Springer US, Boston, MA, 2012, pp. 415–463. URL: https://doi.org/10. 1007/978-1-4614-3223-4_13. doi: 10.1007/978-1-4614-3223-4_13

    2012 doi

  17. [25]

    G. Qiu, B. Liu, J. Bu, C. Chen, Opinion word expansion and target extraction through double propagation, Comput. Linguistics 37 (2011) 9–27. URL: https://doi.org/10.1162/coli_a_00034. doi:10.1162/coli\_a\_00034

    2011 doi

  18. [26]

    K. Liu, H. L. Xu, Y. Liu, J. Zhao, Opinion target extraction using partially-supervised word alignment model, in: F. Rossi (Ed.), IJCAI 2013, Proceedings of the 23rd International Joint Conference on Artificial Intelligence, Beijing, China, August 3-9, 2013, IJCAI/AAAI, 2013, ...

    2013

  19. [27]

    X. Li, L. Bing, P. Li, W. Lam, Z. Yang, Aspect term extraction with history attention and selective transformation, in: J. Lang (Ed.), Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden, ...

    2018 doi

  20. [28]

    W. Wang, S. J. Pan, D. Dahlmeier, X. Xiao, Coupled multi-layer attentions for co-extraction of aspect and opinion terms, in: S. Singh, S. Markovitch (Eds.), Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, February 4-9, 2017, San Francisco, Californi...

    2017

  21. [30]

    L. Xu, H. Li, W. Lu, L. Bing, Position-aware tagging for aspect sentiment triplet extraction, in: B. Webber, T. Cohn, Y. He, Y. Liu (Eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, Associa...

    2020 doi

  22. [31]

    Z. Wu, C. Ying, F. Zhao, Z. Fan, X. Dai, R. Xia, Grid tagging scheme for aspect-oriented fine- grained opinion extraction, CoRR abs/2010.04640 (2020). URL: https://arxiv.org/abs/2010.04640. arXiv:2010.04640

    2010 arXiv

  23. [33]

    Z. Gou, Q. Guo, Y. Yang, MvP: Multi-view prompting improves aspect sentiment tuple prediction, in: A. Rogers, J. Boyd-Graber, N. Okazaki (Eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Association for Com...

    2023 doi

  24. [34]

    Xiong, Z

    H. Xiong, Z. Yan, C. Wu, G. Lu, S. Pang, Y. Xue, Q. Cai, Bart-based contrastive and retrospec- tive network for aspect-category-opinion-sentiment quadruple extraction, Int. J. Mach. Learn. Cybern. 14 (2023) 3243–3255. URL: https://doi.org/10.1007/s13042-023-01831-8. doi: 10.10...

    2023 doi

  25. [35]

    Barnes, R

    J. Barnes, R. Kurtz, S. Oepen, L. Øvrelid, E. Velldal, Structured sentiment analysis as dependency graph parsing, in: C. Zong, F. Xia, W. Li, R. Navigli (Eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International J...

    2021 doi

  26. [36]

    Barnes, L

    J. Barnes, L. Øvrelid, E. Velldal, If you’ve got it, flaunt it: Making the most of fine-grained sentiment annotations, in: P. Merlo, J. Tiedemann, R. Tsarfaty (Eds.), Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: M...

    2021 doi

  27. [37]

    G. Negi, D. Dalal, O. Zayed, P. Buitelaar, Towards semantic integration of opinions: Unified opinion concepts ontology and extraction task, 2025. URL: https://arxiv.org/abs/2505.18703. arXiv:2505.18703

    2025 arXiv

  28. [38]

    Z. Tan, D. Li, S. Wang, A. Beigi, B. Jiang, A. Bhattacharjee, M. Karami, J. Li, L. Cheng, H. Liu, Large language models for data annotation and synthesis: A survey, in: Y. Al-Onaizan, M. Bansal, Y.-N. Chen (Eds.), Proceedings of the 2024 Conference on Empirical Methods in Natu...

    2024 doi

  29. [39]

    Mirzakhmedova, M

    N. Mirzakhmedova, M. Gohsen, C. Chang, B. Stein, Are large language models reliable argument quality annotators?, in: P. Cimiano, A. Frank, M. Kohlhase, B. Stein (Eds.), Robust Argumentation Machines - First International Conference, RATIO 2024, Bielefeld, Germany, June 5-7, 2...

    2024 doi

  30. [40]

    Kasner, V

    Z. Kasner, V. Zouhar, P. Schmidtová, I. Kartác, K. Onderková, O. Plátek, D. Gkatzia, S. Ma- hamood, O. Dusek, S. Balloccu, Large language models as span annotators, CoRR abs/2504.08697 (2025). URL: https://doi.org/10.48550/arXiv.2504.08697. doi: 10.48550/ARXIV. 2504.08697. arX...

    2025 doi

  31. [41]

    Barnes, L

    J. Barnes, L. Oberländer, E. Troiano, A. Kutuzov, J. Buchmann, R. Agerri, L. Øvrelid, E. Velldal, Semeval 2022 task 10: Structured sentiment analysis, in: G. Emerson, N. Schluter, G. Stanovsky, R. Kumar, A. Palmer, N. Schneider, S. Singh, S. Ratan (Eds.), Proceedings of the 16...

    2022 doi

  32. [42]

    Misra, J

    R. Misra, J. Grover, Do not ‘fake it till you make it’! synopsis of trending fake news detection methodologies using deep learning, in: Deep Learning for Social Media Data Analytics, Springer, 2022, pp. 213–235

    2022

  33. [43]

    J. Zhuo, S. Zhang, X. Fang, H. Duan, D. Lin, K. Chen, ProSA: Assessing and understanding the prompt sensitivity of LLMs, in: Y. Al-Onaizan, M. Bansal, Y.-N. Chen (Eds.), Findings of the Asso- ciation for Computational Linguistics: EMNLP 2024, Association for Computational Ling...

    2024 doi

  34. [44]

    J. J. Wang, V. X. Wang, Assessing consistency and reproducibility in the outputs of large language models: Evidence across diverse finance and accounting tasks, CoRR abs/2503.16974 (2025). URL: https://doi.org/10.48550/arXiv.2503.16974. doi: 10.48550/ARXIV. 2503.16974. arXiv:2...

    2025 doi arXiv

  35. [45]

    Khattab, A

    O. Khattab, A. Singhvi, P. Maheshwari, Z. Zhang, K. Santhanam, S. Vardhamanan, S. Haq, A. Sharma, T. T. Joshi, H. Moazam, H. Miller, M. Zaharia, C. Potts, Dspy: Compiling declarative language model calls into self-improving pipelines, in: ICLR, 2024

    2024

  36. [46]

    Schaeffer, B

    R. Schaeffer, B. Miranda, S. Koyejo, Are emergent abilities of large language models a mirage?, in: A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, S. Levine (Eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing ...

    2023

  37. [47]

    Opsahl-Ong, M

    K. Opsahl-Ong, M. J. Ryan, J. Purtell, D. Broman, C. Potts, M. Zaharia, O. Khattab, Optimizing instructions and demonstrations for multi-stage language model programs, in: Y. Al-Onaizan, M. Bansal, Y.-N. Chen (Eds.), Proceedings of the 2024 Conference on Empirical Methods in N...

    2024 doi

  38. [48]

    Toprak, N

    C. Toprak, N. Jakob, I. Gurevych, Sentence and expression level annotation of opinions in user-generated discourse, in: Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, ACL ’10, Association for Computational Linguistics, USA, 2010, p. 575–584

    2010

  39. [49]

    Wilson, J

    T. Wilson, J. Wiebe, P. Hoffmann, Recognizing contextual polarity in phrase-level sentiment analysis, in: R. Mooney, C. Brew, L.-F. Chien, K. Kirchhoff (Eds.), Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processin...

    2005

  40. [50]

    A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. de Las Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, L. R. Lavaud, M. Lachaux, P. Stock, T. L. Scao, T. Lavril, T. Wang, T. Lacroix, W. E. Sayed, Mistral 7b, CoRR abs/2310.06825 (2023). URL: https...

    2023 doi arXiv

  41. [51]

    Touvron, T

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, G. Lample, Llama: Open and efficient foundation language models, CoRR abs/2302.13971 (2023). URL: https://doi.org/10.48550/arXi...

    2023 doi arXiv

  42. [52]

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. H. Chi, Q. V. Le, D. Zhou, Chain- of-thought prompting elicits reasoning in large language models, in: S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, A. Oh (Eds.), Advances in Neural Information Processin...

    2022

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.