REVIEW 15 cited by
Is ChatGPT a General-Purpose Natural Language Processing Task Solver?
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Spurred by advancements in scale, large language models (LLMs) have demonstrated the ability to perform a variety of natural language processing (NLP) tasks zero-shot -- i.e., without adaptation on downstream data. Recently, the debut of ChatGPT has drawn a great deal of attention from the natural language processing (NLP) community due to the fact that it can generate high-quality responses to human input and self-correct previous mistakes based on subsequent conversations. However, it is not yet known whether ChatGPT can serve as a generalist model that can perform many NLP tasks zero-shot. In this work, we empirically analyze the zero-shot learning ability of ChatGPT by evaluating it on 20 popular NLP datasets covering 7 representative task categories. With extensive empirical studies, we demonstrate both the effectiveness and limitations of the current version of ChatGPT. We find that ChatGPT performs well on many tasks favoring reasoning capabilities (e.g., arithmetic reasoning) while it still faces challenges when solving specific tasks such as sequence tagging. We additionally provide in-depth analysis through qualitative case studies.
Forward citations
Cited by 15 Pith papers
-
SVAgent: AI Agent for Hardware Security Verification Assertion
SVAgent is a prompt-engineering framework that decomposes security requirements into sub-questions to generate SystemVerilog assertions with higher reported accuracy and consistency than direct LLM generation.
-
Visibility vs. Engagement: How Two Indian News Websites Reported on LGBTQ+ Individuals and Communities during the Pandemic
Indian news websites gave LGBTQ+ communities visibility during the pandemic but often with little depth, and Times of India's language was at times transphobic.
-
Calibrating Pre-trained Language Classifiers on LLM-generated Noisy Labels via Iterative Refinement
SiDyP improves classifiers trained on LLM-generated noisy labels by retrieving likely true labels from embedding-space neighbors and iteratively refining them with a simplex diffusion model, reporting average gains of...
-
Occ-LLM: Enhancing Autonomous Driving with Occupancy-Based Large Language Models
Occ-LLM tokenizes 4D occupancy with a motion/static separation VAE and uses Llama-2 to forecast occupancy, plan ego motion, and answer scene questions, reporting state-of-the-art results on nuScenes.
-
Retraction-Free Optimization over the Stiefel Manifold for the LoRA Fine-Tuning
A retraction-free Stiefel manifold optimization algorithm with a fixed penalty parameter is proposed and applied to LoRA fine-tuning, claiming faster convergence and better downstream performance.
-
TCAR-Gen: Temporal Graph Retrieval with Evidence Fusion for Knowledge-Grounded Generation
A query-conditioned temporal graph RAG with chain-of-trees fusion reaches 0.3738 Recall@5 on a Victorian crime diaries QA set, beating standard and graph RAG baselines.
-
Parameter-Efficient Multi-Task Fine-Tuning in Code-Related Tasks
Multi-task QLoRA on Qwen2.5-Coder matches or beats single-task QLoRA and full fine-tuning for code generation and Python summarization, but lags in Java-to-C# translation.
-
MEKiT: Multi-source Heterogeneous Knowledge Injection Method via Instruction Tuning for Emotion-Cause Pair Extraction
MEKiT improves LLM emotion-cause pair extraction by adding emotional label knowledge to instruction prompts and mixing causal examples into training data, achieving 61.49 F1 on NTCIR-13.
-
Beyond English: The Impact of Prompt Translation Strategies across Languages and Tasks in Multilingual LLMs
Selective pre-translation, translating only some prompt components into English, generally outperforms both full prompt translation and direct inference across tasks and languages, with the largest gains for low-resou...
-
Distilled Large Language Model in Confidential Computing Environment for System-on-Chip Design
Running lightweight distilled LLMs inside Intel TDX secure VMs reportedly gives higher tokens per second than plain CPU execution for sub-3B models, with Q4 quantization reaching about 3x FP16 throughput.
-
HausaNLP: Current Status, Challenges and Future Directions for Hausa Natural Language Processing
A review of Hausa NLP that catalogs existing datasets and tools and launches the HausaNLP Catalogue as a central access point.
-
Scaling Public Health Text Annotation: Zero-Shot Learning vs. Crowdsourcing for Improved Efficiency and Labeling Accuracy
On a 12,000-tweet public health dataset, zero-shot GPT-4 Turbo matched overall crowdworker accuracy but missed many positive cases, especially sleep-related tweets, while being far faster and cheaper.
-
Learning Text Styles: A Study on Transfer, Attribution, and Verification
A thesis compiles published work claiming that lightweight adapters, contrastive disentanglement, and instruction tuning improve text style transfer, authorship attribution, and authorship verification.
-
The Science of Evaluating Foundation Models
A survey-and-checklist proposal that organizes LLM evaluation into an ABCD framework (Algorithm, Big Data, Computation, Domain Expertise) for context-aware, documented assessment.
-
Building Task Bots with Self-learning for Enhanced Adaptability, Extensibility, and Factuality
A thesis that combines self-learning from dialog logs, schema-guided prompting, and self-aligned factuality to build task bots with minimal human intervention.
Discussion (0). Continue with ORCID to comment.