PiFi adds one frozen LLM layer to an SLM and fine-tunes it, reporting consistent but modest gains across NLU and NLG tasks, with larger gains when the LLM matches the target language.
Costs to Consider in Adopting NLP for Your Business
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Recent advances in Natural Language Processing (NLP) have largely pushed deep transformer-based models as the go-to state-of-the-art technique without much regard to the production and utilization cost. Companies planning to adopt these methods into their business face difficulties because of the lack of machine, data, and human resources to build them. We compare both the performance and the cost of classical learning algorithms to the latest ones in common sequence and text labeling tasks. In our industrial datasets, we find that classical models often perform on par with deep neural ones despite the lower cost. We show the trade-off between performance gain and the cost across the models to give more insights for AI-pivoting business. Further, we call for more research into low-cost models, especially for under-resourced languages.
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Plug-in and Fine-tuning: Bridging the Gap between Small Language Models and Large Language Models
PiFi adds one frozen LLM layer to an SLM and fine-tunes it, reporting consistent but modest gains across NLU and NLG tasks, with larger gains when the LLM matches the target language.