REVIEW 3 cited by
Thinking Tokens for Language Modeling
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
How much is 56 times 37? Language models often make mistakes in these types of difficult calculations. This is usually explained by their inability to perform complex reasoning. Since language models rely on large training sets and great memorization capability, naturally they are not equipped to run complex calculations. However, one can argue that humans also cannot perform this calculation immediately and require a considerable amount of time to construct the solution. In order to enhance the generalization capability of language models, and as a parallel to human behavior, we propose to use special 'thinking tokens' which allow the model to perform much more calculations whenever a complex problem is encountered.
Forward citations
Cited by 3 Pith papers
-
MUX: Continuous Reasoning via Multiplexed Tokens
MUX trains language models to reason with continuous latent tokens that encode spans of discrete reasoning as lossless weighted superpositions, improving accuracy and efficiency over latent-reasoning baselines.
-
LaRe: Latent Refocusing for Multimodal Reasoning
LaRe performs iterative visual refocusing in latent space and reports accuracy gains with fewer tokens, but its main experiments compare against baselines trained with less data.
-
Implicit Reasoning in Large Language Models: A Comprehensive Survey
A survey organizing implicit (silent) reasoning in LLMs into three execution paradigms, plus evidence, benchmarks, and challenges.
Discussion (0). Sign in to comment.