ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPT
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:EQCZ6K6Brecord.jsonopen to challenge →
read the original abstract
Zero-shot information extraction (IE) aims to build IE systems from the unannotated text. It is challenging due to involving little human intervention. Challenging but worthwhile, zero-shot IE reduces the time and effort that data labeling takes. Recent efforts on large language models (LLMs, e.g., GPT-3, ChatGPT) show promising performance on zero-shot settings, thus inspiring us to explore prompt-based methods. In this work, we ask whether strong IE models can be constructed by directly prompting LLMs. Specifically, we transform the zero-shot IE task into a multi-turn question-answering problem with a two-stage framework (ChatIE). With the power of ChatGPT, we extensively evaluate our framework on three IE tasks: entity-relation triple extract, named entity recognition, and event extraction. Empirical results on six datasets across two languages show that ChatIE achieves impressive performance and even surpasses some full-shot models on several datasets (e.g., NYT11-HRL). We believe that our work could shed light on building IE models with limited resources.
This paper has not been read by Pith yet.
Forward citations
Cited by 14 Pith papers
-
Fast LLM-Based Semantic Filtering: From a Unified Framework to an Adaptive Two-Phase Method
An adaptive two-phase semantic filter using clustering then a hybrid proxy trained on LLM confidence achieves 1.6-2.0x speedup over prior methods at 90% accuracy on 10K document corpora.
-
K12-KGraph: A Curriculum-Aligned Knowledge Graph for Benchmarking and Training Educational LLMs
K12-KGraph is a textbook-derived knowledge graph that powers a new benchmark revealing LLMs' poor curriculum cognition and a small training corpus that outperforms general instruction data on educational tasks.
-
LC-ICL: Label-Guided Contrastive In-Context Learning for Robust Information Extraction
LC-ICL improves few-shot NER and RE by using label-guided contrastive demonstrations that pair positive samples with error-annotated negative samples.
-
BCL: Bayesian In-Context Learning Framework for Information Extraction
BCL introduces a particle-filtering Bayesian update framework to systematically refine label representations in in-context learning for information extraction, claiming consistent gains over prior methods.
-
Relational Probing: LM-to-Graph Adaptation for Financial Prediction
Relational Probing replaces the LM output head with a trainable relation head that induces graphs from hidden states and optimizes them end-to-end for stock trend prediction, showing gains over co-occurrence baselines.
-
EXCEEDS: Extracting Complex Events via Nugget-based Grid Modeling in Scientific Domain
Introduces the SciEvents dataset for scientific event extraction and the EXCEEDS nugget-grid modeling framework that reports state-of-the-art results on the new data.
-
Cross-lingual Relation Extraction with Large Language Models: Zero-Shot, Few-Shot, and Fine-Tuned Evaluation on Romanian
Translates SemEval-2010 Task 8 to Romanian and evaluates Gemma 31B prompting and QLoRA fine-tuning against encoder baselines, finding fine-tuning reduces the cross-lingual gap to 1.4pp while smaller models perform wit...
-
SSDAU: Structured Semantic Data Augmentation for Joint Entity and Relation Extraction
SSDAU is a new data augmentation method for JERE that uses entity-based segmentation, context-aware semantic encoding, entity restructuring, and BERTTopic filtering to produce more robust augmented data than prior baselines.
-
SSDAU: Structured Semantic Data Augmentation for Joint Entity and Relation Extraction
SSDAU is a new structured semantic data augmentation method that produces more robust training data for JERE models, showing smaller performance drops under ambiguity than seven baselines.
-
Task Decomposition for Efficient Annotation
Decomposing annotation tasks using centers from centering theory reduces aggregate inferential load via a degrees-of-freedom model and enables better sub-task allocation.
-
TCMIIES: A Browser-Based LLM-Powered Intelligent Information Extraction System for Academic Literature
TCMIIES is a zero-install browser platform with schema-guided LLM prompting that achieves over 94% structured output compliance for academic information extraction, including support for Chinese databases.
-
SchemaRAG: Dynamic Large Schema Reduction for LLM-driven Structured Information Extraction
SchemaRAG dynamically reduces large schemas via RAG for LLM information extraction, reporting up to 8.8% micro-F1 gain, 47% latency cut, and 48% token cost reduction on healthcare and e-commerce data.
-
LLMs Underperform Graph-Based Parsers on Supervised Relation Extraction for Complex Graphs
Graph-based parsers outperform LLMs on supervised relation extraction as linguistic graph complexity grows with more relations per document.
-
Automated Construction of a Knowledge Graph of Nuclear Fusion Energy for Effective Elicitation and Retrieval of Information
A multi-step LLM-based pipeline constructs the first knowledge graph for nuclear fusion energy and enables RAG for multi-hop queries.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.