REVIEW 5 cited by
UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Structured knowledge grounding (SKG) leverages structured knowledge to complete user requests, such as semantic parsing over databases and question answering over knowledge bases. Since the inputs and outputs of SKG tasks are heterogeneous, they have been studied separately by different communities, which limits systematic and compatible research on SKG. In this paper, we overcome this limitation by proposing the UnifiedSKG framework, which unifies 21 SKG tasks into a text-to-text format, aiming to promote systematic SKG research, instead of being exclusive to a single task, domain, or dataset. We use UnifiedSKG to benchmark T5 with different sizes and show that T5, with simple modifications when necessary, achieves state-of-the-art performance on almost all of the 21 tasks. We further demonstrate that multi-task prefix-tuning improves the performance on most tasks, largely improving the overall performance. UnifiedSKG also facilitates the investigation of zero-shot and few-shot learning, and we show that T0, GPT-3, and Codex struggle in zero-shot and few-shot learning for SKG. We also use UnifiedSKG to conduct a series of controlled experiments on structured knowledge encoding variants across SKG tasks. UnifiedSKG is easily extensible to more tasks, and it is open-sourced at https://github.com/hkunlp/unifiedskg.
Forward citations
Cited by 5 Pith papers
-
Utilizing Training Data to Improve LLM Reasoning for Tabular Understanding
LRTab retrieves error-avoiding prompt conditions learned from incorrect chain-of-thought traces on training tables to improve LLM tabular reasoning, achieving modest gains on WikiTQ and TabFact.
-
Can the Rookies Cut the Tough Cookie? Exploring the Use of LLMs for SQL Equivalence Checking
LLMs, especially GPT-4, can classify SQL query equivalence on complex real-world assignment queries far beyond formal tools' coverage, but they systematically over-predict equivalence.
-
LLaSA: Large Language and Structured Data Assistant
A single hypergraph encoder and G-Former, pretrained on 25 million tables, can be appended to different LLMs to improve their performance on table, knowledge graph, and database structured knowledge grounding tasks.
-
Fast Think-on-Graph: Wider, Deeper and Faster Reasoning of Large Language Model on Knowledge Graph
FastToG lets LLMs reason 'community by community' over knowledge graphs, reporting higher accuracy and faster reasoning than Think-on-Graph.
-
Way to Specialist: Closing Loop Between Specialized LLM and Evolving Domain Knowledge Graph
WTS couples retrieval-augmented generation with an LLM-built, evolving domain knowledge graph and reports SOTA gains, but its main experiments use test-set gold answers to construct the graph.
Discussion (0). Continue with ORCID to comment.