Pith. sign in

REVIEW 5 major objections 5 minor 19 references

Mitigating the Carbon Footprint of Chatbots as Consumers

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Resetting a chatbot conversation when its topic changes cuts prompt-token use by about one-fifth and lowers its carbon footprint, a simulation on real chat logs suggests.

desk verdict A straightforward token-saving mechanism backed by real logs, but the headline 19.1% is a conditional simulation result, not a measured fact. read the letter →

arxiv 2508.00911 v1 pith:CQFAYCRG submitted 2025-07-29 cs.CY

Boris Ruf , Marcin Detyniecki This is my paper · ORCID
classification cs.CY
keywords SustainableartificialintelligenceConversationalagentsEcologicalgesturecarbonfootprintmemorylargelanguagemodelstokenefficiencychatbotenergyuse
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper seeks to establish a user-side eco gesture for LLM-based chatbots: when the topic or purpose of a conversation changes, reset the conversation, because every new prompt re-sends the entire chat history and therefore consumes more tokens. It matters because the per-message cost of a long thread grows without the user noticing, while end users have few practical levers to reduce AI emissions. Using anonymized token logs from a real internal GPT-3.5-turbo system (40,065 cleaned threads over 30 days), the authors simulate resets at pause thresholds and report a 19.1% reduction in prompt tokens for pauses over 30 minutes. They translate that into an estimated 24.75 to 69.33 kg CO2e saved over the observation period, and argue the reset does not compromise answer quality when old context is irrelevant. The recommendation is actionable today, if pause duration is accepted as a reliable sign that the old topic is over.

What carries the argument

The load-bearing mechanism is the chat-history block that conversational agents append to each new prompt: a raw, growing transcript of all prior user prompts and model responses, re-tokenized on every turn, so marginal token cost rises with thread length. The companion mechanism in the study is the pause-duration proxy—a simulated reset is triggered when the idle time between messages crosses a chosen threshold—which lets the authors detect likely topic changes in logs that contain no message content.

What would settle it

Take a content-logged chatbot deployment and manually mark true topic boundaries, then simulate resets at those boundaries and compare prompt-token savings and answer-quality ratings with resets driven by pause thresholds. If the real saving is far below 19.1%, or if users rate answers worse after resets, the central claim is unsupported.

Watch

Extended reading notes

Core claim

The paper's central discovery is that conversational memory is a hidden cost driver: the chat history—previous user prompts, model responses, and any custom instructions—is included in raw form in every new prompt, making prompt-token counts rise monotonically as a thread grows. The proposed countermeasure is to discard that history by starting a new conversation once the old topic is no longer relevant. In the simulation, resetting threads when the pause before a message exceeded 30 minutes affected 5,073 of 40,065 threads and reduced total prompt tokens by 19.1%; thresholds of 1 hour, 2 hours, and 24 hours produced reductions of 12.8%, 8.3%, and 2.2%. Using measured energy consumption of a publicly available model scaled to GPT-3.5-turbo, the paper estimates the 30-day observation period would save between 24.75 and 69.33 kg CO2e. The paper concludes that the reset habit lowers cost without compromising answer quality because the discarded context is assumed to be irrelevant.

Load-bearing premise

The savings estimate rests on the premise that a pause between messages signals a real topic change, so clearing the chat history at that point does not discard context the user still needs; the paper itself calls pause-based topic detection not entirely reliable.

Editorial extensions

If this is right

  • Treating pauses of 30 minutes or more as topic boundaries in the 40,065 observed threads would reset 5,073 threads and cut prompt tokens by 19.1%; thresholds of 1 hour, 2 hours, and 24 hours give 12.8%, 8.3%, and 2.2% savings.
  • Because API pricing is token-based, the same reduction appears as direct financial savings for users who follow the recommendation.
  • Scaling the measured energy use of a comparable public model to GPT-3.5-turbo yields an estimated 24.75 to 69.33 kg CO2e reduction over the 30-day observation window.
  • A conversation's marginal cost grows with thread length, so resetting at topic changes avoids escalating cost without relying on summarization or fixed buffer windows.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication left implicit is that the same token-saving mechanism could be automated on the provider side: a client that detects a long pause and starts a fresh thread could capture much of the 19.1% saving without asking users to change behavior.
  • The relative saving of about one-fifth in prompt tokens may carry over to other stateless LLM chatbots, but the absolute CO2 saving would scale with model size, request volume, and the carbon intensity of the electricity used.
  • If future content-labeled logs show that pauses imperfectly track topic changes, the true saving could be either smaller when needed context is discarded or larger when irrelevant context is removed more aggressively than pause thresholds alone would catch.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes a simple end-user eco gesture: reset a chatbot conversation whenever the topic changes and the prior messages become irrelevant, because conversational memory causes prompt tokens to accumulate with every message. Using anonymized token logs from an internal GPT-3.5-turbo-based chatbot over 30 days, the authors reconstruct 40,065 conversational threads with a helper-request heuristic, remove 493 non-monotonic threads, and use pause durations as a proxy for topic changes. A simulation that drops accumulated history after pauses of 24 hours, 2 hours, 1 hour, and 30 minutes reports prompt-token reductions of 2.2%, 8.3%, 12.8%, and 19.1%, respectively, and an estimated CO2e saving of 24.75 to 69.33 kg over the observation period.

Significance. If the quantitative estimate were validated, the paper would give ordinary users a simple, actionable way to reduce the environmental and financial cost of LLM-based chatbots, an area where few recommendations exist. The paper's qualitative mechanism is clearly explained and supported by the token-accumulation structure of conversational memory, and the use of real operational logs is a strength. However, the numerical headline is not a measured effect: it is a simulation built on an explicitly conceded proxy for topic changes, an undisclosed thread-reconstruction constant, and an unmodeled treatment of completion tokens. These issues make the exact savings conditional rather than established, though they do not invalidate the underlying recommendation.

major comments (5)
  1. [Section 5.4 and Section 6] The proxy that maps pause duration to topic change is load-bearing for the headline 19.1% saving, because the simulation drops history only when pause_in_seconds exceeds a threshold. The data contain no message content (Section 5.1), so a 30-minute gap cannot be distinguished from a long intermission within the same task; the manuscript itself concedes in Section 6 that 'identifying topic changes based on pauses is not entirely reliable.' As a result, the numerical savings should be presented as a conditional simulation under an untested assumption, not as a measured effect; the authors should either validate the proxy on content-bearing data or explicitly frame all quantitative results as a sensitivity estimate under that assumption.
  2. [Section 5.1 and Section 5.4] The simulated reset subtracts only the previous record's prompt_tokens, but in the mechanism described in Section 3 and Figure 1, the assistant's completion tokens from previous turns are also included in the next prompt's chat history. Since Section 5.1 dismisses completion_tokens as 'holding no relevance,' the simulation does not exactly implement the paper's own recommendation and likely undercounts the tokens that would be retained or discarded; the authors should model the full prompt history, including completion tokens and any custom-instruction prefix, or justify why these components are negligible for the savings estimate.
  3. [Section 5.2] The deletion of the 493 non-monotonic threads is a substantive data-cleaning decision, not a cosmetic one: these threads are excluded before the pause thresholds and token savings are computed, and the manuscript reports no analysis of their token volume or of how their inclusion (or an alternative re-labeling) would change the results. The authors should report the number of tokens in those threads and run the simulation with them retained under alternative thread-assignment rules, because their exclusion could bias the estimated savings.
  4. [Section 5.2] Thread reconstruction relies on a helper-request marker defined by 'a constant value' that is not disclosed, so the reader cannot reproduce the identification of 40,065 threads or verify that the marker is stable across users and time. The authors should state the constant, describe how it was derived, and provide a robustness check (e.g., varying the constant) to show that the 19.1% figure is not an artifact of this choice.
  5. [Section 5.5] The CO2e range of 24.75 to 69.33 kg relies on extrapolating the measured energy consumption of a publicly available model to GPT-3.5-turbo by parameter count, but the manuscript gives no formula, no parameter values, and no uncertainty quantification for that extrapolation. The authors should provide the full calculation and treat the resulting range as a rough order-of-magnitude estimate rather than a precise saving.
minor comments (5)
  1. [Section 2] The sentence 'relevant sources to related research is provided' should be 'relevant sources to related research are provided.'
  2. [Figure 4] The x-axis of Figure 4 is labeled 'Elapsed time,' while the caption and text describe the length of pauses between messages; the label and caption should be aligned.
  3. [Footnote 1] The footnote renders 'Llama21' instead of 'Llama 2'.
  4. [References] References [5] and [7] are the same paper (Luccioni, Jernite, and Strubell) and should be merged or clearly distinguished.
  5. [Section 5.3] In Figure 3, the low alpha value makes individual outlier conversations difficult to distinguish; a boxplot or overlaid summary would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the savings estimate is a transparent counterfactual computed from observed token logs, not a fitted parameter or self-citation chain.

full rationale

The paper's central derivation is a simulation: it takes observed prompt-token counts from real conversation logs, applies a defined reset rule when pause durations exceed author-chosen thresholds, and sums the resulting token reductions. The 19.1% figure and the CO2e range are computed from those data and from external emission estimates, not from any parameter fitted to the target result. There are no self-citations that bootstrap the central claim, and no uniqueness theorem is imported from the authors' prior work. The pause-based proxy for topic change is an explicit assumption rather than a circular definition: the paper concedes in Section 6 that 'identifying topic changes based on pauses is not entirely reliable,' which is a validity limitation, not a circularity. The savings formula itself is an accounting identity (dropping previous history reduces prompt tokens by the dropped token count), but the empirical magnitude is data-dependent and therefore carries independent content. The main risks are modeling fidelity and external validity, not circular reasoning.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central estimate depends on several assumptions the paper does not independently verify: the chatbot resends raw history, pauses indicate topic changes, non-monotonic threads are noise, and completion tokens are irrelevant. None of these are fitted to data, but they are load-bearing for the headline numbers.

free parameters (2)
  • Pause reset thresholds = >24h, >2h, >1h, >30min
    Four thresholds are tested as proxies for topic change in Section 5.4. The headline savings range from 2.2% to 19.1% depending on the threshold, so the reported number is sensitive to a hand-chosen setting rather than a quantity derived from data.
  • Helper-request marker constant = not disclosed
    Section 5.2 detects thread starts by identifying a helper request whose prompt-token count exceeds the initial prompt by a constant value. The value is not given and the heuristic is not validated, yet it determines which rows are grouped into conversations.
assumptions (4)
  • domain assumption The internal chatbot resends the full raw chat history, including prior assistant responses, in every new prompt.
    Section 3 and Figure 1 describe this as the basic conversational-memory mechanism, and the whole token-growth model depends on it. The paper does not verify against system configuration or response-content data.
  • ad hoc to paper A pause longer than a threshold indicates that previous messages are irrelevant and can be dropped without harming answer quality.
    Section 5.4 uses pause duration as the proxy for topic change and reset need. The paper itself notes in Section 6 that this is not entirely reliable.
  • ad hoc to paper Conversations with non-monotonically increasing prompt tokens are noise from multiple simultaneous chats and can be removed.
    Section 5.2 removes 493 threads that do not show monotonically increasing prompt tokens. This assumes no legitimate chatbot behavior such as context truncation or summarization causes decreases, and the removal may bias the simulation.
  • ad hoc to paper Completion tokens are irrelevant to the prompt-token savings estimate.
    Section 5.1 dismisses completion_tokens as holding no relevance, but Section 3 says model responses are stored in chat history and resent. Previous completion tokens are part of the history that a reset saves, so ignoring them distorts the estimated saving.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Mitigating the Carbon Footprint of Chatbots as Consumers." pith.science (2026). https://pith.science/paper/CQFAYCRG

@misc{pith2026250800911,
  author       = {Pith},
  title        = {Pith review of: Mitigating the Carbon Footprint of Chatbots as Consumers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CQFAYCRG}},
  note         = {Machine review of arXiv:2508.00911}
}
read the original abstract

In the context of the high energy demand of large language models (LLMs) and growing concerns about global warming, there is significant demand for actionable recommendations that can help reduce emissions when utilizing such technologies. This paper examines the environmental impact linked to a fundamental function of LLM-based conversational systems that might be less well known to end users: the conversational memory, which enables the system to maintain context throughout the dialog. After analyzing conversation patterns using anonymized token data from a real world system, a recommendation for individuals on how they could use chatbots in a more sustainable way is derived. Based on a simulation, the savings potential resulting from the adoption of such an ecological gesture is estimated.

Figures

Figures reproduced from arXiv: 2508.00911 by the authors.

Figure 1
Figure 1. Schematic illustration of how conversational memory works in its most basic version. Notably, the chat history is included in every new prompt. The remaining parts of this paper are organized as follows: First, relevant sources to related research is provided. Next, some background information on the mechanisms of chatbots that use LLMs is presented. Subsequently, the rec￾ommendation for sustainable action is derive… view at source ↗
Figure 2
Figure 2. Exemplary token usage during a single conversation. It shows an increase in prompt tokens as the number of messages exchanged rises, while the number of com￾pletion tokens remains consistently low. 1 https://huggingface.co/docs/transformers/main/model_doc/llama2 2 https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo 3 https://www.pinecone.io/learn/series/langchain/langchain-conversational-memory [PITH_FULL… view at source ↗
Figure 3
Figure 3. Distribution of prompt tokens during the conversation. A low alpha value is used for the dots to show the distribution of the data points. 5.3 Data analysis [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Length of pauses between messages. The y-axis is in logarithmic scale. any message content. Therefore, a proxy signal for these topic changes was es￾tablished. Concretely, the duration of pauses between messages was used as an indicator, assuming that such breaks signi…
Figure 5
Figure 5. Figure 5: Simulated token saving potential. In the initial scenario (first bar), no tokens are discarded. In the subsequent scenarios, the conversational memory is reset when pauses of decreasing values are detected. potential. For the shortest pause length tested in our simulat…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 8 canonical work pages

  1. [1]

    Computational Linguistics49(2), 465–523 (06 2022).https://doi.org/10.1162/coli_a_00474, https://doi.org/ 10.1162/coli{_}a{_}00474

    Apidianaki, M.: From word types to tokens and back: A survey of approaches to word meaning representation and interpretation. Computational Linguistics49(2), 465–523 (06 2022).https://doi.org/10.1162/coli_a_00474, https://doi.org/ 10.1162/coli{_}a{_}00474

    doi:10.1162/coli_a_00474 2022

  2. [2]

    Computers in Human Behavior 85, 183– 189 (2018)

    Araujo, T.: Living up to the chatbot hype: The influence of anthropo- morphic design cues and communicative agency framing on conversational agent and company perceptions. Computers in Human Behavior 85, 183– 189 (2018). https://doi.org/https://doi.org/10.1016/j.chb.2018.03.051, https://www.sciencedirect.com/science/article/pii/S0747563218301560

    doi:10.1016/j.chb.2018.03.051 2018

  3. [3]

    Contribution of Work- ing Group III to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change

    IPCC: Climate Change 2022: Mitigation of Climate Change. Contribution of Work- ing Group III to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, Cambridge, UK and New York, NY, USA (2022). https://doi.org/10.1017/9781009157926, https://www.ipcc.ch/ report/ar6/wg3/downloads/report/IPCC{_}AR6{_}WGIII{...

    doi:10.1017/9781009157926 2022

  4. [4]

    Learning and individual differences103, 102274 (2023)

    Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al.: ChatGPT for good? on opportunities and challenges of large language models for education. Learning and individual differences103, 102274 (2023)

    work page 2023

  5. [5]

    Luccioni, A.S., Jernite, Y., Strubell, E.: Power hungry processing: Watts driving the cost of ai deployment? arXiv preprint arXiv:2311.16863 (2023)

    arXiv 2023

  6. [6]

    Journal of Machine Learning Re- search 24(253), 1–15 (2023)

    Luccioni, A.S., Viguier, S., Ligozat, A.L.: Estimating the carbon footprint of BLOOM, a 176b parameter language model. Journal of Machine Learning Re- search 24(253), 1–15 (2023)

    work page 2023

  7. [7]

    FAccT ’24, ACM (Jun 2024).https://doi.org/10.1145/ 3630106.3658542, http://dx.doi.org/10.1145/3630106.3658542

    Luccioni, S., Jernite, Y., Strubell, E.: Power hungry processing: Watts driving the cost of ai deployment? In: The 2024 ACM Conference on Fairness, Accountabil- ity, and Transparency. FAccT ’24, ACM (Jun 2024).https://doi.org/10.1145/ 3630106.3658542, http://dx.doi.org/10.1145/3630106.3658542

    arXiv 2024

  8. [8]

    International Journal of Human-Computer Studies151, 102630 (2021) Mitigating the Carbon Footprint of Chatbots as Consumers 9

    Rapp, A., Curti, L., Boldi, A.: The human side of human-chatbot interaction: A systematic literature review of ten years of research on text-based chatbots. International Journal of Human-Computer Studies151, 102630 (2021) Mitigating the Carbon Footprint of Chatbots as Consumers 9

    work page 2021

Show all 19 references
  1. [9]

    Internet of Things and Cyber-Physical Systems 3, 121–154 (2023)

    Ray, P.P.: ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet of Things and Cyber-Physical Systems 3, 121–154 (2023). https://doi.org/https: //doi.org/10.1016/j.iotcps.2023.04.003, https://www.scien...

    2023 doi

  2. [10]

    Rillig,M.C.,Ågerstrand,M.,Bi,M.,Gould,K.A.,Sauerland,U.:Risksandbenefits oflargelanguagemodelsfortheenvironment.EnvironmentalScience&Technology 57(9), 3464–3466 (2023)

    2023

  3. [11]

    Renewable and Sustainable Energy Reviews58, 674–691 (2016)

    Rong, H., Zhang, H., Xiao, S., Li, C., Hu, C.: Optimizing energy consumption for data centers. Renewable and Sustainable Energy Reviews58, 674–691 (2016)

    2016

  4. [12]

    In: Proceedings of the 28th International Conference on Intelli- gent User Interfaces

    Ross, S.I., Martinez, F., Houde, S., Muller, M., Weisz, J.D.: The programmer’s assistant: Conversational interaction with a large language model for software development. In: Proceedings of the 28th International Conference on Intelli- gent User Interfaces. p. 491–514. IUI ’23...

    2023

  5. [13]

    Schwartz, R., Dodge, J., Smith, N.A., Etzioni, O.: Green AI. Commun. ACM 63(12), 54–63 (Nov 2020). https://doi.org/10.1145/3381831, https://doi. org/10.1145/3381831

    2020 doi

  6. [14]

    Social Responsibility Journal 18(1), 19–42 (December 2020)

    Sheoran, M., Kumar, D.: Benchmarking the barriers of sustainable con- sumer behaviour. Social Responsibility Journal 18(1), 19–42 (December 2020). https://doi.org/10.1108/SRJ-05-2020-0203, https://ideas.repec. org/a/eme/srjpps/srj-05-2020-0203.html

    2020 doi

  7. [15]

    Nature medicine29(8), 1930–1940 (2023)

    Thirunavukarasu, A.J., Ting, D.S.J., Elangovan, K., Gutierrez, L., Tan, T.F., Ting, D.S.W.: Large language models in medicine. Nature medicine29(8), 1930–1940 (2023)

    2023

  8. [16]

    In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing...

    2017

  9. [17]

    In: Muresan, S., Nakov, P., Villavicencio, A

    Xu, J., Szlam, A., Weston, J.: Beyond goldfish memory: Long-term open-domain conversation. In: Muresan, S., Nakov, P., Villavicencio, A. (eds.) Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 5180–5197. Assoc...

    2022 doi

  10. [18]

    In: Carpuat, M., de Marn- effe, M.C., Meza Ruiz, I.V

    Zhong, H., Dou, Z., Zhu, Y., Qian, H., Wen, J.R.: Less is more: Learning to refine dialogue history for personalized dialogue generation. In: Carpuat, M., de Marn- effe, M.C., Meza Ruiz, I.V. (eds.) Proceedings of the 2022 Conference of the North American Chapter of the Associ...

    2022 doi

  11. [19]

    Zimmermann, S., Hein, A., Schulz, T., Gewald, H., Krcmar, H.: Digital nudging toward pro-environmental behavior: A literature review. PACIS p. 226 (2021)

    2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.