Pith. sign in

REVIEW 4 major objections 5 minor 20 references

Food for thought: How can machine learning help better predict and understand changes in food prices?

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Curated data, not bigger models, drives better Canadian food price forecasts

desk verdict A solid applied benchmark of food-price forecasting with a plausible but unproven curation advantage; needs significance tests and a clean holdout before the headline numbers are trusted. read the letter →

arxiv 2412.06472 v1 pith:OZTAVZHI submitted 2024-12-09 cs.LG

Kristina L. Kupferschmidt , James Requiema , Mya Simpson , Zohrah Varsallay , Ethan Jackson , Cody Kupferschmidt , Sara El-Shawa , Graham W. Taylor This is my paper · ORCID
classification cs.LG
keywords foodpriceforecastingConsumerIndexdatacurationexogenousregressorstimeserieslargelanguagemodelsensembleCanada'sReport
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether machine learning can forecast Canadian food inflation more accurately when the data fed to the models is curated rather than used wholesale. It tests 18-month forecasts of nine food Consumer Price Index categories under three settings: all 165 external regressors, none, and curated subsets grouped by expert themes (climate, geopolitical, economic, manufacturing) or selected by LLM personas. It reports that curated subsets, especially climate and geopolitical, often beat both baselines, that including all regressors consistently hurt, and that no single model family dominated. The paper also finds that simple complexity metrics of each CPI series track which model class performs best. The result matters because it suggests that cheap, interpretable data curation, rather than larger models, is the main lever for improving annual food price predictions.

What carries the argument

The carrying objects are the curated regressor sets: 165 monthly time series collected from public sources and grouped into four expert themes (economic, climate, geopolitical, manufacturing) plus LLM-generated selections made by persona-prompted GPT-4o. The evaluation mechanism is mean absolute percentage error (MAPE) over 18-month forecast horizons across annual windows, comparing each curation condition to an all-regressor baseline and a target-only baseline. A second mechanism, the category complexity ranking, uses five metrics computed in three-year windows to rank the nine CPI categories, showing that high-complexity categories such as Vegetables, Meat, and Fruit favor transformer and foundation models while low-complexity categories favor Exponential Smoothing or simple feed-forward networks.

What would settle it

Refit the same curation conditions using data through 2022 and evaluate only on untouched 2023, 2024, and 2025 realized CPI data; if curated climate and geopolitical sets no longer beat the all-regressor and no-regressor baselines on that holdout, the reported curation advantage is an artifact of the chosen evaluation windows.

Watch

Extended reading notes

Core claim

The paper's central claim is that simple curation techniques, such as thematic groupings of exogenous regressors, can improve forecasting performance relative to both using every available regressor and using none. Averaged over categories and models the gains are modest, but in every food category the best-performing configuration used a curated set, and climate and geopolitical regressors were the most frequent winners. It also claims that intrinsic properties of a time series, including trend and seasonality strength, residual variance, residual MAD, and Shannon entropy, predict whether a high-capacity transformer or foundation model will outperform a simple statistical baseline. In the resulting 2025 Canada's Food Price Report forecasts, most categories were served by ensembles dominated by transformer, foundation, or LLM models, with only Fish and Other using statistical models.

Load-bearing premise

The load-bearing premise is that the evaluation windows, described as 2018 to 2024 in Section 3.1 and 2018 to 2023 in Section 3.4 and including the volatile COVID and supply-shock years, represent the conditions the 2025 forecast must handle, even though the models and ensembles were selected using those same windows.

Editorial extensions

If this is right

  • Curated regressor sets, particularly climate and geopolitical proxies, consistently match or beat models that see all 165 regressors or none; no food category improved by including everything.
  • The best model family shifts with measured time-series complexity: low-complexity categories do well with Exponential Smoothing or simple networks, while high-complexity categories do best with transformers and the Chronos foundation model.
  • LLM-suggested regressors can perform as well as human-expert themes, but feeding LLMs additional future forecasts tends to degrade their output, while including the previous CFPR helped Bakery and Meat but not Vegetables.
  • For the 2025 report, the chosen ensembles were mostly ML-based, using transformer, foundation, and LLM models rather than statistical baselines, marking a shift from earlier editions.
  • The curation advantage appears in per-category results even where the averaged gains are small, suggesting that selecting the right subset of context can be more valuable than the choice of model family.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The curation effect is plausibly portable: any country's food CPI forecasting exercise could test whether climate and geopolitical subsets beat full-regressor sets before investing in larger models, which is a cheap and direct extension.
  • The complexity metrics suggest a Mixture-of-Experts style routing rule for time series, where residual variance and entropy pick the model class at inference time; this is testable on other forecasting benchmarks.
  • A stricter validation protocol, with model selection on one period and evaluation on a later untouched period, would clarify whether the curation gains persist outside the volatile 2018-2024 windows; the paper's numbers do not yet establish that.
  • LLM persona curation could be made more robust by treating personas as an ensemble rather than taking consensus at a fixed threshold of 7, a variant the paper does not run.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper reports an empirical study supporting the University of Guelph/Vector Institute contribution to the 2025 Canada's Food Price Report. It evaluates statistical, deep-learning, transformer, foundation, and LLM forecasting models on nine Canadian food CPI categories, using monthly data from 1986 onward. The central methodological contribution is data curation: exogenous regressors are grouped by human-suggested themes (economic, climate, geopolitical, manufacturing) or by LLM persona-based selection, and these curated sets are compared to using all regressors or none. Performance is measured by MAPE over 18-month forecasts for recent evaluation windows. The paper's main claim, stated in Section 5, is that simple curation techniques can improve model performance. Secondary claims concern which model families win for which categories and the relation to intrinsic time-series complexity metrics.

Significance. If the central claim is secured, the paper would be a useful practical contribution to food-price forecasting and to data-centric AI: it introduces a new multi-source dataset, a replicable LLM-based curation protocol, and a comparison of many model families in a real forecasting setting. The work also connects to a concrete deployment (the 2025 CFPR), which gives the evaluation ecological validity. The LLM-persona curation method and the public-data aggregation are the most valuable parts. However, the evidence for the central claim is currently weakened by the evaluation protocol, which selects final ensembles on the same windows used to report performance, and by the absence of significance testing or a strict holdout. These issues are fixable within the scope of the paper.

major comments (4)
  1. [Section 3.4 and Section 4.2] The final ensembles are selected by 'exploring all combinations of the top 10 performing models for each category and selecting the top-performing ensemble during the same evaluation period' (Section 3.4), and the performance of those ensembles is then reported in Table 2 and used to support the Section 5 conclusion that curation improves performance. This is a selection-on-the-evaluation-window protocol: with only six annual evaluation windows, ten candidate models, and multiple curation conditions, choosing the best combination on the same windows used to compute the reported MAPE values creates an optimistic bias. The authors should either hold out a strict temporal test period (e.g., select on 2018-2021 and test on 2022-2023/2024) or clearly report selection-aware, nested estimates. Without this, the reported curation gains are not a reliable estimate of predictive skill.
  2. [Section 3.1 vs Section 3.4] The paper gives two inconsistent descriptions of the data split. Section 3.1 states that training uses 1986-2017 and evaluation covers 18-month forecasts for 'the six most recent years (2018 to 2024)', while Section 3.4 states training uses 1986-2016 and evaluation covers 2018 to 2023. These differences change every reported number and also affect the interpretation of the context length (36 months versus 75 months for LLMs). The authors must state exactly which split and which evaluation years were used, and reconcile the 'six most recent years (2018 to 2024)' phrasing, which appears to list seven calendar years. This is load-bearing because the entire empirical comparison depends on the evaluation windows.
  3. [Table 2 and Section 4] The differences between curated subsets and the 'None' baseline are often small relative to the reported variability: for example, Bakery climate 0.039±0.00 vs None 0.044±0.01, Meat geopolitical 0.029±0.01 vs None 0.031±0.01, and Vegetables climate 0.050±0.01 vs None 0.060±0.02. No significance tests, paired comparisons, or confidence intervals are provided, and with only a handful of evaluation windows it is unclear whether these differences reflect systematic gains or noise. The paper should report per-window paired errors and at least a paired test or a bootstrap confidence interval for the curated-versus-baseline comparison, otherwise the conclusion that curation 'can improve model performance' is not statistically supported.
  4. [Section 4.1 and Table 3] The complexity metrics in Table 3 are computed using '3-year overlapping windows from 1986-2024', which includes the evaluation period (2018-2023/2024). Using a period that overlaps with the evaluation to explain which model family wins introduces leakage into the explanatory analysis: the 'intrinsic complexity' ranking is not purely intrinsic if it is partly determined by the same years used to evaluate forecasting performance. The authors should recompute these metrics using only data up to the start of the evaluation period (e.g., 1986-2017 or 1986-2016) to support the claim that intrinsic properties predict model-family success.
minor comments (5)
  1. [Throughout] The report uses both 'CPFR' and 'CFPR' inconsistently; the abbreviation for Canada's Food Price Report should be fixed (the title and abstract use CFPR, while the introduction uses CPFR).
  2. [Section 4.1] There is a typo in 'wich depend heavily on signal stationarity' (should be 'which').
  3. [Section 3.3.2] The model name 'TemporoSpatialTransformer' appears to be a misnomer; the text refers to the Temporal Fusion Transformer, and the reference [Lim et al., 2021] is correct, but the model name should match the reference.
  4. [Section 3.3] In 'LLMs are a form of foundation model that leverage extensive corpa of text-based data', 'corpa' should be 'corpora'.
  5. [Section 4.1] The sentence beginning 'Consistent with findings from similar approaches, such as those reported by Kristina L. Kupferschmidt, Cody Kupferschmidt, Joshua A. Skorburg, ...' lists authors but gives no year or reference entry, so it is impossible for readers to locate the cited work.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the curation comparison is an empirical benchmark against external Statistics Canada CPI data, and the self-citations are contextual rather than load-bearing.

full rationale

This paper is an empirical benchmark against external Statistics Canada CPI ground truth, so the derivation chain is not circular. The central claim in Section 5, that 'simple curation techniques, such as thematic groupings of exogenous regressors, can improve model performance,' is supported by Table 2, which reports MAPE against observed CPI for curated regressor subsets, an all-regressor baseline, and a no-regressor baseline under the same evaluation windows. The curated advantage is not equal by construction to any fitted parameter, any selection rule, or the paper's own conclusion; it is an empirical comparison against an external target. The self-citations present in the paper are contextual rather than load-bearing: DelphAI is cited as background for human-centric forecasting, prior Kupferschmidt and Kupferschmidt work is cited to explain the 2021-2022 volatility, and the longer Kupferschmidt et al. citation is used for a general observation about context-window dependence. None of these citations is used to forbid alternatives, to define the target, or to supply a uniqueness theorem. No ansatz is smuggled in via citation: the LLM personas, thematic groupings, and regressor sets are described as explicit experimental inputs whose effects are measured, not asserted. The selection of top-performing ensembles on the same 2018-2023/2024 evaluation period, together with the Section 3.1 versus Section 3.4 discrepancy about the evaluation window, raises a legitimate concern about optimistic bias or selection on the test period, but that is a statistical-validity issue, not a reduction of the result to its own inputs. On the circularity axis, the paper is self-contained and its central empirical conclusion is not forced by construction.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new entities. Its free parameters are experimental design choices, especially the LLM curation threshold and ensemble selection, that directly shape the reported comparisons. The axioms are domain assumptions about data validity, test-window representativeness, the validity of LLM judgments, AutoGluon fairness, and the use of full-data complexity metrics. These are mostly standard for applied ML forecasting, but the LLM rating assumption and the full-data complexity assumption are ad hoc to this paper.

free parameters (3)
  • LLM curation consensus threshold = >=7 on a 1-9 Likert scale
    Variables rated 7 or higher by GPT-4o personas were selected for LLM-curated conditions. This threshold is chosen by hand and directly determines which regressors enter those conditions.
  • Context length = 36 months for most models, 75 months for LLMs
    The context window is an experimental choice that affects all models; no sensitivity analysis is provided to justify these lengths.
  • Ensemble selection limits = top 10 models per category, ensemble size 1-3
    Final forecasts use the top 10 models per category and an ensemble size of 1 to 3 chosen combinatorially on the evaluation period. These cutoffs are arbitrary and selected on test data.
assumptions (5)
  • domain assumption Statistics Canada food CPI series are an accurate and consistent measure of Canadian food inflation from 1986 to 2024.
    The target variable and ground truth for all models is the CPI. If the index construction changes or has errors, every forecast comparison inherits them. Invoked in Section 3.1.
  • domain assumption The 2018-2024 (or 2018-2023, as stated inconsistently) evaluation windows are representative of future forecasting conditions.
    All conclusions about curation and model choice are based on a small set of annual windows that include volatile COVID and supply-shock periods. Section 3.4 defines the evaluation protocol.
  • ad hoc to paper LLM persona Likert ratings are a proxy for the predictive usefulness of regressors.
    GPT-4o personas ranked variables on a 1 to 9 scale and a threshold of 7 was used to select regressors. Section 3.2.2. This assumption is not validated against predictive performance.
  • domain assumption AutoGluon's default model implementations provide fair comparisons across model families.
    All non-LLM models were run through AutoGluonTS v1.0 with default or lightly specified settings. Section 3.3.1. Differences in tuning effort could bias model family rankings.
  • ad hoc to paper Intrinsic complexity metrics computed over 1986-2024 can be used to explain which model family wins, without leakage from including the evaluation period.
    Section 4.1 computes trend, seasonality, residual variance, MAD, and Shannon entropy on 3-year overlapping windows through 2024, which includes the test windows, then maps them to model rankings. This is post hoc.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Food for thought: How can machine learning help better predict and understand changes in food prices?." pith.science (2026). https://pith.science/paper/OZTAVZHI

@misc{pith2026241206472,
  author       = {Pith},
  title        = {Pith review of: Food for thought: How can machine learning help better predict and understand changes in food prices?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OZTAVZHI}},
  note         = {Machine review of arXiv:2412.06472}
}
read the original abstract

In this work, we address a lack of systematic understanding of fluctuations in food affordability in Canada. Canada's Food Price Report (CPFR) is an annual publication that predicts food inflation over the next calendar year. The published predictions are a collaborative effort between forecasting teams that each employ their own approach at Canadian Universities: Dalhousie University, the University of British Columbia, the University of Saskatchewan, and the University of Guelph/Vector Institute. While the University of Guelph/Vector Institute forecasting team has leveraged machine learning (ML) in previous reports, the most recent editions (2024--2025) have also included a human-in-the-loop approach. For the 2025 report, this focus was expanded to evaluate several different data-centric approaches to improve forecast accuracy. In this study, we evaluate how different types of forecasting models perform when estimating food price fluctuations. We also examine the sensitivity of models that curate time series data representing key factors in food pricing.

Figures

Figures reproduced from arXiv: 2412.06472 by the authors.

Figure 1
Figure 1. Comparison of citation trends across different domains of foundation models. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Performance changes for different context inclusion techniques – Averaged over all food categories and model [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Model rankings for each food category in CPI sorted in order of increasing complexity of model classes. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Forecasts and projected year over year (2025/2024) % change for the food CPI categories included in Canada’s [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Inclusion of specific time-series in evaluated groupings [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 10 canonical work pages

  1. [1]

    Chronos: Learning the language of time series

    Abdul Fatir Ansari, Lorenzo Stella, Caner Turkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Oleksandr Shchur, Syama Sundar Rangapuram, Sebastian Pineda Arango, Shubham Kapoor, et al. Chronos: Learning the language of time series. arXiv preprint arXiv:2403.07815, 2024

    arXiv 2024

  2. [2]

    On the opportunities and risks of foundation models

    Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021

    arXiv 2021

  3. [3]

    Did grain futures prices overreact to the russia--ukraine war due to herding? Journal of Commodity Markets, 35: 0 100422, 2024

    Colin A Carter and Sandro Steinbach. Did grain futures prices overreact to the russia--ukraine war due to herding? Journal of Commodity Markets, 35: 0 100422, 2024

    work page 2024

  4. [4]

    Digital traceability in agri-food supply chains: A comparative analysis of oecd member countries

    Sylvain Charlebois, Noor Latif, Ibrahim Ilahi, Bibhuti Sarker, Janet Music, and Janele Vezeau. Digital traceability in agri-food supply chains: A comparative analysis of oecd member countries. Foods, 13 0 (7): 0 1075, 2024 a

    work page 2024

  5. [5]

    Implications of carbon pricing on food affordability and agri-food sector in canada: A scoping review

    Sylvain Charlebois, Swati Saxena, Gumataw Abebe, Tony Walker, Janet Music, Vlado Keselj, Karim Tuffaha, Keshava Pallavi Gone, Janele Vezeau, Bibhuti Sarker, et al. Implications of carbon pricing on food affordability and agri-food sector in canada: A scoping review. Transportation Research Interdisciplinary Perspectives, 28: 0 101271, 2024 b

    work page 2024

  6. [6]

    Timegpt-1, 2023

    Azul Garza and Max Mergenthaler-Canseco. Timegpt-1, 2023

    2023

  7. [7]

    Large language models are zero-shot time series forecasters

    Nate Gruver, Marc Finzi, Shikai Qiu, and Andrew G Wilson. Large language models are zero-shot time series forecasters. Advances in Neural Information Processing Systems, 36, 2024

    2024

  8. [8]

    Forecasting: principles and practice

    Rob J Hyndman and George Athanasopoulos. Forecasting: principles and practice. OTexts, May 2018

    work page 2018

Show all 20 references
  1. [9]

    Large models for time series and spatio-temporal data: A survey and outlook

    Ming Jin, Qingsong Wen, Yuxuan Liang, Chaoli Zhang, Siqiao Xue, Xue Wang, James Zhang, Yi Wang, Haifeng Chen, Xiaoli Li, et al. Large models for time series and spatio-temporal data: A survey and outlook. arXiv preprint arXiv:2310.10196, 2023

    arXiv

  2. [10]

    Food Price Volatility and Its Implications for Food Security and Policy

    Matthias Kalkuhl, Joachim von Braun, and Maximo Torero. Food Price Volatility and Its Implications for Food Security and Policy. Springer, April 2016

    2016

  3. [11]

    Kupferschmidt, Cody Kupferschmidt , Joshua A

    Kristina L. Kupferschmidt, Cody Kupferschmidt , Joshua A. Skorburg , Branka Agic , Tara Elton-Marshall, Hayley Hamilton , Gina Stoduto , Katherine Vink, Samantha Wells , Christine Wickens, Graham W. Taylor . Predicting the surge: Forecasting ontario's changing mental health ne...

    2024

  4. [12]

    The 2021--22 surge in inflation

    Oleksiy Kryvtsov, James MacGee, and Luis Uzeda. The 2021--22 surge in inflation. Bank of Canada, 2023. URL https://www.bankofcanada.ca/2023/01/staff-discussion-paper-2023-3/

    2021

  5. [13]

    Kupferschmidt, J.A

    K.L. Kupferschmidt, J.A. Skorburg, and G.W. Taylor. Delphai: A human-centered approach to time-series forecasting. In 2022 IEEE International Conference on Big Data (Big Data), pages 4014--4020, 2022. doi:10.1109/BigData55660.2022.10020676

    2022

  6. [14]

    Enhancing food price forecasts in canada: An integration of expert-driven covariates and advanced ML approaches

    Kristina L Kupferschmidt and Cody Kupferschmidt. Enhancing food price forecasts in canada: An integration of expert-driven covariates and advanced ML approaches. In Proceedings of the Canadian AI Conference, 2024

    2024

  7. [15]

    Temporal fusion transformers for interpretable multi-horizon time series forecasting

    Bryan Lim, Sercan \"O Ar k, Nicolas Loeff, and Tomas Pfister. Temporal fusion transformers for interpretable multi-horizon time series forecasting. International Journal of Forecasting, 37 0 (4): 0 1748--1764, 2021

    2021

  8. [16]

    The m4 competition: Results, findings, conclusion and way forward

    Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos. The m4 competition: Results, findings, conclusion and way forward. International Journal of Forecasting, 34 0 (4): 0 802--808, 2018

    2018

  9. [17]

    Llm processes: Numerical predictive distributions conditioned on natural language

    James Requeima, John Bronskill, Dami Choi, Richard E Turner, and David Duvenaud. Llm processes: Numerical predictive distributions conditioned on natural language. arXiv preprint arXiv:2405.12856, 2024

    2024 arXiv

  10. [18]

    DeepAR : Probabilistic forecasting with autoregressive recurrent networks

    David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. DeepAR : Probabilistic forecasting with autoregressive recurrent networks. Int. J. Forecast., 36 0 (3): 0 1181--1191, July 2020

    2020

  11. [19]

    Generating personas using llms and assessing their viability

    Andreas Schuller, Doris Janssen, Julian Blumenr \"o ther, Theresa Maria Probst, Michael Schmidt, and Chandan Kumar. Generating personas using llms and assessing their viability. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, pages 1--7, 2024

    2024

  12. [20]

    Context is key: A benchmark for forecasting with essential textual information

    Andrew Robert Williams, Arjun Ashok, \'E tienne Marcotte, Valentina Zantedeschi, Jithendaraa Subramanian, Roland Riachi, James Requeima, Alexandre Lacoste, Irina Rish, Nicolas Chapados, et al. Context is key: A benchmark for forecasting with essential textual information. arXi...

    arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.