REVIEW 5 cited by
Assessing Look-Ahead Bias in Stock Return Predictions Generated By GPT Sentiment Analysis
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (LLMs), including ChatGPT, can extract profitable trading signals from the sentiment in news text. However, backtesting such strategies poses a challenge because LLMs are trained on many years of data, and backtesting produces biased results if the training and backtesting periods overlap. This bias can take two forms: a look-ahead bias, in which the LLM may have specific knowledge of the stock returns that followed a news article, and a distraction effect, in which general knowledge of the companies named interferes with the measurement of a text's sentiment. We investigate these sources of bias through trading strategies driven by the sentiment of financial news headlines. We compare trading performance based on the original headlines with de-biased strategies in which we remove the relevant company's identifiers from the text. In-sample (within the LLM training window), we find, surprisingly, that the anonymized headlines outperform, indicating that the distraction effect has a greater impact than look-ahead bias. This tendency is particularly strong for larger companies--companies about which we expect an LLM to have greater general knowledge. Out-of-sample, look-ahead bias is not a concern but distraction remains possible. Our proposed anonymization procedure is therefore potentially useful in out-of-sample implementation, as well as for de-biased backtesting.
Forward citations
Cited by 5 Pith papers
-
Talking to Digital Twins: Selective Disclosure and Belief Measurement in Financial Social Media
Daily real-time LLM digital-twin interviews of finfluencer accounts predict cross-sectional large-cap returns over the next ten trading days, mainly in the silent region with no concurrent public post.
-
HindsightBench: A Black-Box Behavioral Audit Protocol for Parametric Hindsight in Time-Indexed LLM Decision Tasks
An integrated black-box audit protocol profiles parametric hindsight in LLMs and shows the date-trigger reflex tracks training generation, not scale, while effective knowledge cutoffs span 22 months.
-
Named Entity Swapping for Metadata Anonymization in a Text Corpus
Swapping named entities between embedding-similar chunks of earnings call transcripts lowers LLM company-identification accuracy on the swapped chunks from about 90% to about 60%.
-
Scaling Point-in-Time Language Models
Scaling point-in-time LLMs to 4B parameters and 1T temporally filtered tokens narrows the gap to unrestricted models to about 8–11 average points and yields positive out-of-sample Sharpe ratios from news embeddings.
-
Reasoning or Overthinking: Evaluating Large Language Models on Financial Sentiment Analysis
On financial sentiment classification, zero-shot LLMs match human labels better without chain-of-thought reasoning than with it.
Discussion (0). Sign in to comment.