Pith. sign in

REVIEW 3 major objections 6 minor 41 references

EventFull: Complete and Consistent Event Relation Annotation

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read EventFull is a web-based annotation tool that guarantees complete and consistent temporal, causal, and coreference labels for every pair of targeted event mentions, while cutting manual judgments by 41–96 percent.

desk verdict A genuinely useful annotation tool whose 'completeness' guarantee is scoped to the supplied mention set—the paper should say so more loudly. read the letter →

arxiv 2412.12733 v1 pith:KI5MBVVL submitted 2024-12-17 cs.CL

Alon Eirew , Eviatar Nachshoni , Aviv Slobodkin , Ido Dagan This is my paper · ORCID
classification cs.CL
keywords eventrelationannotationtemporalrelationscoreferencecausaltooltransitiveclosureinter-annotatoragreementNLPdatasetcreation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Event relation datasets usually omit many pairs because annotating every pair of events costs quadratic time and effort. The paper introduces EventFull, a web-based annotation tool that claims to guarantee complete and consistent annotation of temporal, causal, and coreference relations over any given set of targeted event mentions. EventFull orders the work so each relation type narrows the next: temporal labels are assigned first, coreference is then judged only for temporally co-occurring mentions, and causality is judged only for events that lie before the target. Automated transitive closure fills in deducible temporal links and consistency checks catch contradictions, so fewer pairs need manual judgment. A pilot with three non-expert annotators on six news documents reports high inter-annotator agreement and a 41–96 percent reduction in the number of pairs that annotators must actively judge.

What carries the argument

The load-bearing mechanism is the layered temporal-first workflow combined with an automated transitive closure and consistency checking over a four-label temporal scheme (before, after, equal, uncertain). Because temporal order constrains the other relations, completing the temporal graph first lets the tool restrict coreference judgements to temporally equal mentions and causal judgements to preceding events, and the transitive closure (computed with Warshall's algorithm over Allen-style interval constraints) auto-annotates pairs whose relation is logically forced from already-annotated paths. This is what converts the quadratic pair count into a smaller set of genuinely informative judgments.

What would settle it

Annotate a three-event text with A before B and B before C, then export the result: if the pair A-C does not already carry the label before, or if a conflicting label can be saved without a warning, the completeness and consistency guarantee fails.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that a dataset creator who supplies any set of targeted event mentions can use EventFull to obtain an annotation in which every mention pair is classified for coreference, every pair of coreference clusters is classified for temporal and causal relations, and no transitive contradiction survives. The tool achieves this by annotating temporal relations first over all mention pairs, applying a Warshall-style transitive closure with Allen-style interval constraints so that any pair whose relation is already implied is auto-labeled, and flagging any direct annotation that conflicts with an inferred path. The subsequent coreference step only considers mentions whose temporal relation is equal, which follows from the earlier step, and the causal step only considers events that temporally precede the target, following the EventStoryLine flow. The pilot reports agreement comparable to MAVEN-ERE, with an average total annotation time of roughly 69 minutes per 500-word document.

Load-bearing premise

The load-bearing premise is that completeness is defined with respect to the event-mention list supplied to the tool, not with respect to all events a reader would identify; the pilot compounds this by keeping only 16-18 salient events per document.

Editorial extensions

If this is right

  • Dataset builders can produce relation annotations that cover all pairs of their selected mentions, without restricting to adjacent sentences or heuristic subsets.
  • Models trained or evaluated on EventFull-produced datasets can be tested on long-range and cross-sentence relations that current datasets omit.
  • Because the same workflow handles temporal, coreference, and causal relations together, the output supports joint training of multi-relation models without alignment mismatches.
  • The relatively simple scheme and built-in guidelines make non-expert annotators usable, so new datasets in other genres or languages are easier to create.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The completeness guarantee is relative to the input event mention set; EventFull does not solve event detection, so if a mention set misses or mis-selects events the annotation is complete only within that set.
  • A temporal labeling error early in the workflow could cascade: because coreference candidates are pruned to equal-time mentions and causal candidates to preceding ones, a wrong 'after' label could hide a coreference or cause link that a human would see.
  • The pilot's 16-18 salient events per document leave the tool's efficiency claims untested at the 35 anchorable-event scale; at roughly 595 pairs per document, the reduction factor might shrink.
  • The same prioritized transitive-closure interface could be repurposed for other transitive relation types, such as containment or part-of hierarchies, in annotation tools outside temporal reasoning.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces EventFull, a web-based annotation tool for inter-annotating temporal, coreference, and causal relations between event mentions. The tool takes a text with a pre-marked set of targeted event mentions and leads annotators through three stages: temporal relation annotation with transitive closure, depth-first-style pair prioritization, and inconsistency checks; coreference annotation restricted to temporally equal mentions; and causal annotation over preceding event clusters using an EventStoryLine-style flow. The central claim is that this process guarantees complete and consistent annotation for the three relation types over the targeted event set while minimizing manual effort. A pilot study with three non-expert annotators, six news documents, and about 17 events per document reports inter-annotator agreement comparable to MAVEN-ERE (temporal kappa 0.72, coreference B3 0.96, causal kappa 0.78) and large reductions in the number of manually judged pairs. The paper includes appendices describing the transitive closure, prioritization, consistency checking, annotation screenshots, and customizable guidelines.

Significance. If the claims hold, EventFull addresses a real bottleneck in event-relation dataset creation: exhaustive pairwise annotation is quadratic, and prior datasets either restrict the pairs considered or lack a systematic protocol guaranteeing completeness. The tool's design is internally coherent and grounded in known constraints: temporal transitivity, coreference requiring temporal co-occurrence, and causation requiring temporal precedence. The paper ships the tool and code publicly, the algorithms contain no fitted parameters, and the transitivity rules are taken from prior work (Allen, Verhagen, Ning, Caselli and Vossen), so the central mechanism is not circular. The pilot's agreement numbers are encouraging and compare favorably with MAVEN-ERE. The main weaknesses are evaluation-related: the efficiency evidence is indirect, one reduction metric uses an inconsistent baseline, and the event-selection reconciliation is not specified. The completeness guarantee is also stated in places more strongly than the input-dependent design warrants.

major comments (3)
  1. [Section 4.2, Table 3] The 41% reduction reported for causal annotation is computed against 136 event-mention pairs, but causal relations are annotated over coreference clusters, not over individual mentions (§3.2.3). The appropriate baseline is the number of cluster pairs (or a full causal protocol such as RED over the same units), so the reported reduction is not directly interpretable and may be overstated. In addition, the paper only reports reductions in judged pairs and absolute times, with no control condition (e.g., the same tool with auto-completion disabled, or a generic annotation tool); this does not by itself substantiate the 'accelerates and simplifies' claim in the abstract and §4. Please report the cluster-level baseline and either add a comparison condition or soften the efficiency claim.
  2. [Section 3, opening paragraph; §3.1; §4.1] The opening sentence of §3 says EventFull 'guarantees complete and consistent annotation ... between events mentioned in an input text,' and the abstract uses 'complete annotation' without qualification. However, §3.1 explicitly leaves event-mention detection and selection orthogonal to EventFull, and the pilot further restricts the set to the '16–18 most salient events' (§4.1). The guarantee is therefore complete only over the externally supplied targeted mention set, not over all events a reader might identify. The paper should consistently phrase the guarantee as 'complete over the given targeted event mention set' and explicitly state that the tool does not evaluate or ensure completeness of the mention set itself.
  3. [Section 4.1, annotation procedure] The paper does not explain how the three annotators' individual selections of 16–18 salient events were reconciled into the final set of 102 event mentions (17 per document). If annotators annotated different event sets, the pairwise agreement values in Table 1 are not computed over the same pair universe, which would complicate their interpretation. Please specify whether the event set was unified across annotators (e.g., by consensus, by one annotator's selection, or by intersection/union) and, ideally, report agreement on the event mention selection step.
minor comments (6)
  1. [Table 1 caption] The caption cites 'Fleiss and Cohen, 1973' for the kappa coefficient, but the pairwise kappa values between two annotators are Cohen's kappa; please cite Cohen (1960) or clarify that the Fleiss and Cohen reference pertains to weighted kappa or the intraclass correlation.
  2. [§3.2.1, Table 4, Appendix B] The annotation label is called 'uncertain' in the main text but 'VAGUE' in Table 4 and Appendix B; please unify the terminology across the paper.
  3. [Appendix B] The algorithm's name is misspelled as 'Warsheall's algorithm'; it should be Warshall's algorithm, matching the cited Warshall (1962) reference.
  4. [Appendix F heading] The heading 'Printscreens of EventFull' should be 'Screenshots of EventFull'; also, the reference list contains several instances of 'V ossen' with an internal space that should be 'Vossen'.
  5. [Appendix A] The phrase 'around 0.2kappa' should read 'around 0.2 kappa' for readability.
  6. [Abstract and §1] The claim that EventFull is 'the first tool' supporting these features is hard to verify and may be read as too strong given prior tools such as NarrativeTime; consider writing 'the first tool we are aware of' or specifying the exact combination of features being claimed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: EventFull's completeness and consistency claims are implemented workflow guarantees, and the transitivity and cross-relation constraints are cited from external prior work, with pilot results reported as direct measurements.

full rationale

The paper makes no fitted predictions and contains no derivation chain that reduces to its own inputs. EventFull's guarantee that every pair of targeted mentions is classified is a bookkeeping property of the tool's workflow: the temporal step iterates until all pairs are annotated or transitively derived, and the coreference and causal steps rely on the externally cited constraints that coreferring mentions co-occur and causes precede their effects. The transitive closure and consistency-checking rules in Table 4 are attributed to Allen (1984), Warshall (1962), Verhagen (2005), Ning et al. (2018a), Caselli and Vossen (2017), and Cybulska and Vossen (2014), not to the authors' own prior results. The only self-citation in the load path is Cattan et al. (2021), which includes two EventFull authors and is used only to produce the initial candidate event mentions in the pilot; the paper's central tool claim does not rest on that detector's correctness, so this is not load-bearing self-citation. The explicit scoping in Section 3.1, where event mention detection and selection are left orthogonal to EventFull, and the pilot's restriction to 16–18 salient events in Section 4.1, are honest limitations on the scope of the completeness claim rather than a circular reduction: they define what 'targeted events' means without smuggling the conclusion into the premise. Overall, the paper's contributions are concrete engineering and measurement results, and no circular step was found.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

EventFull introduces no fitted model parameters or invented entities. Its guarantees rely on standard domain assumptions about temporal transitivity, coreference implying temporal co-occurrence, and cause preceding effect, plus the paper-specific assumption that completeness is relative to an externally supplied mention set. These assumptions are stated in the paper but are load-bearing for the 'complete and consistent' claim.

assumptions (5)
  • domain assumption Temporal relations before, after, and equal are transitive as specified in Table 4 (for example, before plus before yields before, and equal plus before yields before).
    Used by the transitive closure and consistency checking algorithms in §3.2.1 and Appendix E; adopted from Allen interval algebra and prior temporal annotation work.
  • domain assumption Coreferring event mentions must stand in the equal temporal relation.
    Used to restrict coreference annotation to co-occurring mentions (§3.2.2), citing Cybulska and Vossen 2014.
  • domain assumption A cause must temporally precede its effect.
    Used to restrict causal annotation to before-related event pairs (§3.2.3), citing Caselli and Vossen 2017.
  • ad hoc to paper The externally supplied event mention set is the correct universe for completeness.
    EventFull takes detected or selected mentions as input and guarantees completeness only over that set (§3.1); a biased or incomplete mention set makes the annotation incomplete relative to the actual events in the text.
  • domain assumption The coarse labels before, after, equal, uncertain for temporal relations and a single cause relation are sufficient for annotation.
    Adopted from MATRES, ECB+, and prior causal work; coarse labels are a modeling choice that may discard finer distinctions such as Allen interval relations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EventFull: Complete and Consistent Event Relation Annotation." pith.science (2026). https://pith.science/paper/KI5MBVVL

@misc{pith2026241212733,
  author       = {Pith},
  title        = {Pith review of: EventFull: Complete and Consistent Event Relation Annotation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KI5MBVVL}},
  note         = {Machine review of arXiv:2412.12733}
}
read the original abstract

Event relation detection is a fundamental NLP task, leveraged in many downstream applications, whose modeling requires datasets annotated with event relations of various types. However, systematic and complete annotation of these relations is costly and challenging, due to the quadratic number of event pairs that need to be considered. Consequently, many current event relation datasets lack systematicity and completeness. In response, we introduce \textit{EventFull}, the first tool that supports consistent, complete and efficient annotation of temporal, causal and coreference relations via a unified and synergetic process. A pilot study demonstrates that EventFull accelerates and simplifies the annotation process while yielding high inter-annotator agreement.

Figures

Figures reproduced from arXiv: 2412.12733 by the authors.

Figure 1
Figure 1. The EventFull annotation pipeline begins with a document containing marked targeted events and [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. A simple example illustrating the prioritiza [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Event Selection Annotation Step: This optional step aims to refine the set of events (detailed in §3.2) by selecting the events to be considered in subsequent steps. Annotators can access guidelines by clicking the “Event Selection Instruction” button. After categorizing all events as either event or no-event , they can proceed to the next annotation task by clicking the “Next Task” button. 11 [PITH_FULL_IMAGE:figu… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Temporal Relation Annotation Step: Annotators determine the temporal relations for each candidate pair based on the starting point of the events by selecting the appropriate radio button option (detailed in §3.2). Events requiring annotation are highlighted in green an…
Figure 5
Figure 5. Figure 5: Coreference Relation Annotation Step: Annotators determine coreference relations among all candidates annotated in the temporal step as having an equal time relation. The event mention representing the event cluster is highlighted in green , while all candidate event m…
Figure 6
Figure 6. Figure 6: Causal Relation Annotation Step: Annotators determine causal relations among all candidates annotated in the temporal step as having a before time relation. The event mention in focus is highlighted in green , while all preceding event mentions are highlighted in red .…
Figure 7
Figure 7. Figure 7: An example of the guidelines used in the pilot study, accessible by clicking the “Coreference Relation [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 25 canonical work pages

  1. [1]

    James F. Allen. 1983. https://doi.org/10.1145/182.358434 Maintaining knowledge about temporal intervals . Commun. ACM, 26(11):832–843

    doi:10.1145/182.358434 1983

  2. [2]

    James F. Allen. 1984. https://doi.org/10.1016/0004-3702(84)90008-0 Towards a general theory of action and time . Artificial Intelligence, 23(2):123--154

    work page doi:10.1016/0004-3702(84)90008-0 1984

  3. [3]

    Amit Bagga and Breck Baldwin. 1998. https://doi.org/10.3115/980845.980859 Entity-based cross-document coreferencing using the vector space model . In 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Volume 1, pages 79--85, Montreal, Quebec, Canada. Association for Computat...

    arXiv 1998

  4. [4]

    Valentina Bartalesi Lenzi, Giovanni Moretti, and Rachele Sprugnoli. 2012. http://www.lrec-conf.org/proceedings/lrec2012/pdf/216_Paper.pdf CAT : the CELCT annotation tool . In Proceedings of the Eighth International Conference on Language Resources and Evaluation ( LREC '12) , pages 333--338, Istanbul, Turkey. European Language Resources Association (ELRA)

    work page 2012

  5. [5]

    Ari Bornstein, Arie Cattan, and Ido Dagan. 2020. https://doi.org/10.18653/v1/2020.emnlp-demos.27 C o R efi: A crowd sourcing suite for coreference annotation . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 205--215, Online. Association for Computational Linguistics

    work page doi:10.18653/v1/2020.emnlp-demos.27 2020

  6. [6]

    Lars Buitinck, Gilles Louppe, Mathieu Blondel, Fabian Pedregosa, Andreas Mueller, Olivier Grisel, Vlad Niculae, Peter Prettenhofer, Alexandre Gramfort, Jaques Grobler, Robert Layton, Jake Vanderplas, Arnaud Joly, Brian Holt, and Gaël Varoquaux. 2013. https://arxiv.org/abs/1309.0238 Api design for machine learning software: experiences from the scikit-lear...

    arXiv 2013

  7. [7]

    Tommaso Caselli and Piek Vossen. 2017. https://doi.org/10.18653/v1/W17-2711 The event S tory L ine corpus: A new benchmark for causal and temporal relation extraction . In Proceedings of the Events and Stories in the News Workshop, pages 77--86, Vancouver, Canada. Association for Computational Linguistics

    doi:10.18653/v1/w17-2711 2017

  8. [8]

    Arie Cattan, Alon Eirew, Gabriel Stanovsky, Mandar Joshi, and Ido Dagan. 2021. https://doi.org/10.18653/v1/2021.findings-acl.453 Cross-document coreference resolution over predicted mentions . In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 5100--5107, Online. Association for Computational Linguistics

    doi:10.18653/v1/2021.findings-acl.453 2021

Show all 41 references
  1. [9]

    Nathanael Chambers, Taylor Cassidy, Bill McDowell, and Steven Bethard. 2014. https://doi.org/10.1162/tacl_a_00182 Dense Event Ordering with a Multi-Pass Architecture . Transactions of the Association for Computational Linguistics, 2:273--284

    2014 doi

  2. [10]

    Agata Cybulska and Piek Vossen. 2014. http://www.lrec-conf.org/proceedings/lrec2014/pdf/840_Paper.pdf Using a sledgehammer to crack a nut? lexical diversity and event coreference resolution . In Proceedings of the Ninth International Conference on Language Resources and Evalua...

    2014

  3. [11]

    Greenwood, and Manuel Jung

    Leon Derczynski, Jannik Str \"o tgen, Diana Maynard, Mark A. Greenwood, and Manuel Jung. 2016. https://aclanthology.org/L16-1587 GATE -time: Extraction of temporal expressions and events . In Proceedings of the Tenth International Conference on Language Resources and Evaluatio...

    2016

  4. [12]

    Quang Do, Yee Seng Chan, and Dan Roth. 2011. https://aclanthology.org/D11-1027 Minimally supervised event causality identification . In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 294--303, Edinburgh, Scotland, UK. Association ...

    2011

  5. [13]

    Quang Do, Wei Lu, and Dan Roth. 2012. https://aclanthology.org/D12-1062 Joint inference for event timeline construction . In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 677--687...

    2012

  6. [14]

    Fleiss and Jacob Cohen

    Joseph L. Fleiss and Jacob Cohen. 1973. https://doi.org/10.1177/001316447303300309 The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability . Educational and Psychological Measurement, 33(3):613--619

    1973 doi

  7. [15]

    Goran Glava s , Jan S najder, Marie-Francine Moens, and Parisa Kordjamshidi. 2014. http://www.lrec-conf.org/proceedings/lrec2014/pdf/1023_Paper.pdf H i E ve: A corpus for extracting event hierarchies from news stories . In Proceedings of the Ninth International Conference on L...

    2014

  8. [16]

    Pan, Jiaoyan Chen, and Freddy Lecue

    Yong Guan, Xiaozhi Wang, Lei Hou, Juanzi Li, Jeff Z. Pan, Jiaoyan Chen, and Freddy Lecue. 2024. https://api.semanticscholar.org/CorpusID:269757769 Tacoere: Cluster-aware compression for event relation extraction . ArXiv, abs/2405.06890

    2024 arXiv

  9. [17]

    Yu Hong, Tongtao Zhang, Tim O ' Gorman, Sharone Horowit-Hendler, Heng Ji, and Martha Palmer. 2016. https://doi.org/10.18653/v1/W16-1701 Building a cross-document event-event relation corpus . In Proceedings of the 10th Linguistic Annotation Workshop held in conjunction with AC...

    2016 doi

  10. [18]

    Guo, and Xueqi Cheng

    Zhilei Hu, Zixuan Li, Daozhu Xu, Long Bai, Cheng Jin, Xiaolong Jin, J. Guo, and Xueqi Cheng. 2023. https://api.semanticscholar.org/CorpusID:262217343 Protoem: A prototype-enhanced matching framework for event relation extraction . ArXiv, abs/2309.12892

    2023 arXiv

  11. [19]

    Sindhu Kishore and Hangfeng He. 2024. https://doi.org/10.18653/v1/2024.naacl-short.20 Unveiling divergent inductive biases of LLM s on temporal data . In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human La...

    2024 doi

  12. [20]

    Yuanyuan Lei and Ruihong Huang. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.656 Identifying conspiracy theories news based on event relation graph . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 9811--9822, Singapore. Association for...

    2023 doi

  13. [21]

    Yunshan Ma, Chenchen Ye, Zijian Wu, Xiang Wang, Yixin Cao, and Tat seng Chua. 2023. https://api.semanticscholar.org/CorpusID:260500190 Context-aware event forecasting via graph disentanglement . Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

    2023

  14. [22]

    Aman Madaan and Yiming Yang. 2021. https://doi.org/10.18653/v1/2021.naacl-main.67 Neural language modeling for contextualized temporal graph generation . In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human...

    2021 doi

  15. [23]

    Anne-Lyse Minard, Manuela Speranza, Ruben Urizar, Bego \ n a Altuna, Marieke van Erp, Anneleen Schoen, and Chantal van Son. 2016. https://aclanthology.org/L16-1699 MEANTIME , the N ews R eader multilingual event and time corpus . In Proceedings of the Tenth International Confe...

    2016

  16. [24]

    Paramita Mirza, Rachele Sprugnoli, Sara Tonelli, and Manuela Speranza. 2014. https://doi.org/10.3115/v1/W14-0702 Annotating causality in the T emp E val-3 corpus . In Proceedings of the EACL 2014 Workshop on Computational Approaches to Causality in Language ( CA to CL ) , page...

    2014 doi

  17. [25]

    Aakanksha Naik, Luke Breitfeller, and Carolyn Rose. 2019. https://doi.org/10.18653/v1/W19-5929 TDD iscourse: A dataset for discourse-level temporal ordering of events . In Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue, pages 239--249, Stockholm, Swed...

    2019 doi

  18. [26]

    Qiang Ning, Zhili Feng, Hao Wu, and Dan Roth. 2018 a . https://doi.org/10.18653/v1/P18-1212 Joint reasoning for temporal and causal relations . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2278--2288,...

    2018 doi

  19. [27]

    Qiang Ning, Hao Wu, and Dan Roth. 2018 b . https://doi.org/10.18653/v1/P18-1122 A multi-axis annotation scheme for event temporal relations . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1318--1328, M...

    2018 doi

  20. [28]

    Tim O ' Gorman, Kristin Wright-Bettner, and Martha Palmer. 2016. https://doi.org/10.18653/v1/W16-5706 Richer event description: Integrating event coreference with temporal, causal and bridging annotation . In Proceedings of the 2nd Workshop on Computing News Storylines ( CNS 2...

    2016 doi

  21. [29]

    James Pustejovsky, Jos \'e M Castano, Robert Ingria, Roser Sauri, Robert J Gaizauskas, Andrea Setzer, Graham Katz, and Dragomir R Radev. 2003 a . Timeml: Robust specification of event and temporal expressions in text. New directions in question answering, 3:28--34

    2003

  22. [30]

    James Pustejovsky, Patrick Hanks, Roser Saurí, Andrew See, Rob Gaizauskas, Andrea Setzer, Dragomir Radev, Beth Sundheim, David Day, Lisa Ferro, and Marcia Lazo. 2003 b . The timebank corpus. Proceedings of Corpus Linguistics

    2003

  23. [31]

    James Pustejovsky and Amber Stubbs. 2011. https://aclanthology.org/W11-0419 Increasing informativeness in temporal annotation . In Proceedings of the 5th Linguistic Annotation Workshop, pages 152--160, Portland, Oregon, USA. Association for Computational Linguistics

    2011

  24. [32]

    Karthik Raghunathan, Heeyoung Lee, Sudarshan Rangarajan, Nathanael Chambers, Mihai Surdeanu, Dan Jurafsky, and Christopher Manning. 2010. https://aclanthology.org/D10-1048 A multi-pass sieve for coreference resolution . In Proceedings of the 2010 Conference on Empirical Method...

    2010

  25. [33]

    Anna Rogers, Marzena Karpinska, Ankita Gupta, Vladislav Lialin, Gregory Smelkov, and Anna Rumshisky. 2024. https://aclanthology.org/2024.lrec-main.1054 Narrativetime: Dense temporal annotation on a timeline . In Proceedings of the 2024 Joint International Conference on Computa...

    2024

  26. [34]

    Pontus Stenetorp, Sampo Pyysalo, Goran Topi \'c , Tomoko Ohta, Sophia Ananiadou, and Jun ' ichi Tsujii. 2012. https://aclanthology.org/E12-2021 brat: a web-based tool for NLP -assisted text annotation . In Proceedings of the Demonstrations at the 13th Conference of the E urope...

    2012

  27. [35]

    Xingwei Tan, Yuxiang Zhou, Gabriele Pergola, and Yulan He. 2024. https://doi.org/10.18653/v1/2024.naacl-long.214 Set-aligning framework for auto-regressive event temporal graph generation . In Proceedings of the 2024 Conference of the North American Chapter of the Association ...

    2024 doi

  28. [36]

    Marc Verhagen. 2005. http://www.jstor.org/stable/30200551 Temporal closure in an annotation environment . Language Resources and Evaluation, 39(2/3):211--241

    2005

  29. [37]

    Xiaozhi Wang, Yulin Chen, Ning Ding, Hao Peng, Zimu Wang, Yankai Lin, Xu Han, Lei Hou, Juanzi Li, Zhiyuan Liu, Peng Li, and Jie Zhou. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.60 MAVEN - ERE : A unified large-scale dataset for event coreference, temporal, causal, and s...

    2022 doi

  30. [38]

    Stephen Warshall. 1962. https://doi.org/10.1145/321105.321107 A theorem on boolean matrices . J. ACM, 9(1):11–12

    1962

  31. [39]

    Jiarui Yao, Harry Hochheiser, WonJin Yoon, Eli Goldner, and Guergana Savova. 2024. https://doi.org/10.18653/v1/2024.clinicalnlp-1.53 Overview of the 2024 shared task on chemotherapy treatment timeline extraction . In Proceedings of the 6th Clinical Natural Language Processing ...

    2024 doi

  32. [40]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  33. [41]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.