{"paper":{"title":"What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers","license":"http://creativecommons.org/licenses/by/4.0/","headline":"","cross_cats":[],"primary_cat":"cs.CL","authors_text":"Boseop Kim, Donghoon Ham, Dong Hyeon Jeon, Donghyun Kwak, Dongju Park, Dongpil Seo, Gichang Lee, Heungsub Lee, Hiun Kim, HyoungSeok Kim, Inho Kang, Jaewook Kang, Jinseong Park, Jinuk Kim, Jisu Jeong, Jung-Woo Ha, Kang Min Yoo, KyungDuk Kim, Minsub Kim, Minsuk Chang, Minyoung Jeong, Min Young Lee, Na-Hyeon Ryu, Nako Sung, Sang-Woo Lee, Seokhun Kim, Seonhoon Kim, Soobin Suh, Sookyo In, Soyoung Kang, Suk Hyun Ko, Sunghyun Park, Sungjae Lee, Sungju Kim, Taeyong Park, Woomyoung Park, Yong Goo Yeo","submitted_at":"2021-09-10T03:32:19Z","abstract_excerpt":"GPT-3 shows remarkable in-context learning ability of large-scale language models (LMs) trained on hundreds of billion scale data. Here we address some remaining issues less reported by the GPT-3 paper, such as a non-English LM, the performances of different sized models, and the effect of recently introduced prompt optimization on in-context learning. To achieve this, we introduce HyperCLOVA, a Korean variant of 82B GPT-3 trained on a Korean-centric corpus of 560B tokens. Enhanced by our Korean-specific tokenization, HyperCLOVA with our training configuration shows state-of-the-art in-context"},"claims":{"count":0,"items":[],"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"source":{"id":"2109.04650","kind":"arxiv","version":2},"verdict":{"id":null,"model_set":{},"created_at":null,"strongest_claim":"","one_line_summary":"","pipeline_version":null,"weakest_assumption":"","pith_extraction_headline":""},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2109.04650/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"}