An Empirical Study on Information Extraction using Large Language Models
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:N6CLQI7Rrecord.jsonopen to challenge →
read the original abstract
Human-like large language models (LLMs), especially the most powerful and popular ones in OpenAI's GPT family, have proven to be very helpful for many natural language processing (NLP) related tasks. Therefore, various attempts have been made to apply LLMs to information extraction (IE), which is a fundamental NLP task that involves extracting information from unstructured plain text. To demonstrate the latest representative progress in LLMs' information extraction ability, we assess the information extraction ability of GPT-4 (the latest version of GPT at the time of writing this paper) from four perspectives: Performance, Evaluation Criteria, Robustness, and Error Types. Our results suggest a visible performance gap between GPT-4 and state-of-the-art (SOTA) IE methods. To alleviate this problem, considering the LLMs' human-like characteristics, we propose and analyze the effects of a series of simple prompt-based methods, which can be generalized to other LLMs and NLP tasks. Rich experiments show our methods' effectiveness and some of their remaining issues in improving GPT-4's information extraction ability.
This paper has not been read by Pith yet.
Forward citations
Cited by 5 Pith papers
-
Semantic Reranking at Inference Time for Hard Examples in Rhetorical Role Labeling
RISE is an inference-time semantic reranking framework that refines low-confidence predictions in rhetorical role labeling using contrastively learned label representations, delivering an average +9.15 macro-F1 gain o...
-
DiffuSent: Towards a Unified Diffusion Framework for Aspect-Based Sentiment Analysis
DiffuSent unifies ABSA subtasks via diffusion-based boundary denoising with contrastive training, reporting +2.48 F1 gains on multi-word triplets and up to 181x faster inference across 28 settings.
-
Task Decomposition for Efficient Annotation
Decomposing annotation tasks using centers from centering theory reduces aggregate inferential load via a degrees-of-freedom model and enables better sub-task allocation.
-
JD Oxygen AI Item Center (Oxygen AIIC) V1: An Industrial-Scale LLM/VLM-Centric Solution for Item Understanding, Management, and Applications
Oxygen AIIC is an industrial platform using LLMs and VLMs for scalable item knowledge production and service at JD.com, reporting 94.2% precision and 82.8% recall along with business metric improvements.
-
JD Oxygen AI Item Center (Oxygen AIIC) V1: An Industrial-Scale LLM/VLM-Centric Solution for Item Understanding, Management, and Applications
JD.com's Oxygen AIIC applies LLMs/VLMs with a Semantic Search then Discrimination architecture and self-evolving models to produce item knowledge at industrial scale, reporting 94.2% precision and 82.8% recall.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.