{"work":{"id":"a9dbcb7a-e48d-42a4-8d60-a8f723751a97","openalex_id":null,"doi":null,"arxiv_id":"1602.02410","raw_key":null,"title":"Exploring the Limits of Language Modeling","authors":null,"authors_text":"J \\' o zefowicz, Rafal, Vinyals, Oriol, Schuster, Mike, Shazeer, Noam, and Wu, Yonghui","year":2016,"venue":"cs.CL","abstract":"In this work we explore recent advances in Recurrent Neural Networks for large scale Language Modeling, a task central to language understanding. We extend current models to deal with two key challenges present in this task: corpora and vocabulary sizes, and complex, long term structure of language. We perform an exhaustive study on techniques such as character Convolutional Neural Networks or Long-Short Term Memory, on the One Billion Word Benchmark. Our best single model significantly improves state-of-the-art perplexity from 51.3 down to 30.0 (whilst reducing the number of parameters by a factor of 20), while an ensemble of models sets a new record by improving perplexity from 41.0 down to 23.7. We also release these models for the NLP and ML community to study and improve upon.","external_url":"https://arxiv.org/abs/1602.02410","cited_by_count":null,"metadata_source":"pith","metadata_fetched_at":"2026-05-14T21:18:00.231158+00:00","pith_arxiv_id":"1602.02410","created_at":"2026-05-09T03:35:49.600855+00:00","updated_at":"2026-05-14T21:18:00.231158+00:00","title_quality_ok":true,"display_title":"Exploring the limits of language modeling","render_title":"Exploring the limits of language modeling"},"hub":{"state":{"work_id":"a9dbcb7a-e48d-42a4-8d60-a8f723751a97","tier":"hub","tier_reason":"10+ Pith inbound or 1,000+ external citations","pith_inbound_count":13,"external_cited_by_count":null,"distinct_field_count":6,"first_pith_cited_at":"2016-05-27T21:24:32+00:00","last_pith_cited_at":"2026-05-03T07:22:55+00:00","author_build_status":"not_needed","summary_status":"needed","contexts_status":"needed","graph_status":"needed","ask_index_status":"not_needed","reader_status":"not_needed","recognition_status":"not_needed","updated_at":"2026-05-15T01:26:19.440072+00:00","tier_text":"hub"},"tier":"hub","role_counts":[{"context_role":"background","n":2}],"polarity_counts":[{"context_polarity":"background","n":2}],"runs":{},"summary":{},"graph":{},"authors":[]}}