pith. sign in

arxiv: 1907.10855 · v1 · pith:UL5OTQBPnew · submitted 2019-07-25 · 💻 cs.CR · cs.CY

Machine learning and semantic analysis of in-game chat for cyberbullying

Shane Murnion , William J. Buchanan , Adrian Smales , Gordon Russell This is my paper

Pith reviewed 2026-05-24 16:30 UTC · model grok-4.3

classification 💻 cs.CR cs.CY
keywords cyberbullyingin-game chatWorld of Tankssentiment analysistoxic behaviormultiplayer gamesdata collection system
0
0 comments X

The pith

A short freeze on chat after death can cut cyberbullying in World of Tanks by a significant factor.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper builds an automated collector that pulls real-time chat logs from World of Tanks matches together with player statistics from public services. It applies a research-derived scoring scheme to label instances of cyberbullying and tests both simple SQL pattern matching and commercial sentiment-analysis services against hand-labeled examples. The resulting patterns show that toxic messages cluster after a player is killed and that brand-new accounts produce far fewer such messages than longer-term players. These observations lead directly to the proposal that a brief post-death chat lock would interrupt the cycle and lower overall incidence.

Core claim

It should be possible to reduce cyberbullying within the World of Tanks game by a significant factor by simply freezing the player's ability to communicate through the in-game chat function for a short period after the player is killed within a match. It was also shown that very new players are much less likely to engage in cyberbullying, suggesting that it may be a learned behaviour from other players.

What carries the argument

An automatic data-collection pipeline that gathers in-game chat together with a scoring scheme drawn from existing cyberbullying research to flag toxic messages.

If this is right

  • A brief post-death chat lock would interrupt the immediate trigger for many abusive messages.
  • Players with very low match counts produce markedly less toxic chat, indicating the behavior spreads through repeated exposure.
  • Simple keyword and pattern queries already catch a useful portion of bad-language and racist content without needing advanced models.
  • Commercial sentiment services underperformed basic SQL checks on this domain-specific chat corpus.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same post-kill mute tactic could be tested in other team-based shooters that share similar kill-triggered chat windows.
  • Onboarding materials or automated warnings aimed at new accounts might slow the acquisition of toxic habits before they become habitual.
  • The data pipeline itself offers a template for studying toxicity in any persistent multiplayer title that exposes chat logs.

Load-bearing premise

The scoring scheme based on current research accurately identifies cyberbullying in the collected in-game chat data.

What would settle it

Run the game with the proposed post-death chat freeze enabled for a period and measure whether the rate of manually confirmed or reported toxic messages drops compared with an otherwise identical control period.

read the original abstract

One major problem with cyberbullying research is the lack of data, since researchers are traditionally forced to rely on survey data where victims and perpetrators self-report their impressions. In this paper, an automatic data collection system is presented that continuously collects in-game chat data from one of the most popular online multi-player games: World of Tanks. The data was collected and combined with other information about the players from available online data services. It presents a scoring scheme to enable identification of cyberbullying based on current research. Classification of the collected data was carried out using simple feature detection with SQL database queries and compared to classification from AI-based sentiment text analysis services that have recently become available and further against manually classified data using a custom-built classification client built for this paper. The simple SQL classification proved to be quite useful at identifying some features of toxic chat such as the use of bad language or racist sentiments, however the classification by the more sophisticated online sentiment analysis services proved to be disappointing. The results were then examined for insights into cyberbullying within this game and it was shown that it should be possible to reduce cyberbullying within the World of Tanks game by a significant factor by simply freezing the player's ability to communicate through the in-game chat function for a short period after the player is killed within a match. It was also shown that very new players are much less likely to engage in cyberbullying, suggesting that it may be a learned behaviour from other players.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper describes an automated system for collecting in-game chat data from World of Tanks, introduces a scoring scheme for cyberbullying drawn from prior research, applies SQL-based feature detection and commercial sentiment analysis services for classification (compared against manual labels via a custom client), and analyzes the resulting patterns to conclude that temporarily disabling chat after a player death could reduce cyberbullying by a significant factor and that very new players are substantially less likely to engage in it, suggesting learned behavior.

Significance. If the classification labels prove reliable, the work would supply rare real-world chat data and observational evidence for a low-cost intervention in a popular game, addressing the field's reliance on self-report surveys. The data-collection pipeline and comparison of simple SQL features versus sentiment services are practical contributions. However, the lack of any quantitative validation of the labels means the intervention and learning-behavior claims cannot currently be evaluated.

major comments (2)
  1. [Classification and Results] Classification/results section: No accuracy, precision, recall, F1, or agreement statistics are reported for the SQL feature detection or the sentiment-analysis services against the manual labels. All downstream claims (chat-freeze intervention, experience-level differences) rest on these labels, so the absence of performance numbers is load-bearing.
  2. [Abstract and Discussion] Abstract and discussion: The assertion that freezing chat after death 'should be possible to reduce cyberbullying ... by a significant factor' is presented without any quantitative support, effect-size calculation, statistical test, or description of the timing correlation that was observed in the classified data.
minor comments (1)
  1. [Abstract] Abstract states that sentiment analysis was 'disappointing' and SQL 'quite useful' without any supporting numbers or examples, making the qualitative assessment difficult to interpret.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their thorough review and constructive comments. We address each major comment below and indicate planned revisions.

read point-by-point responses

  1. Referee: [Classification and Results] Classification/results section: No accuracy, precision, recall, F1, or agreement statistics are reported for the SQL feature detection or the sentiment-analysis services against the manual labels. All downstream claims (chat-freeze intervention, experience-level differences) rest on these labels, so the absence of performance numbers is load-bearing.

    Authors: We agree that the manuscript omits essential quantitative validation metrics (accuracy, precision, recall, F1, and agreement statistics such as Cohen's kappa) for both the SQL-based detection and commercial sentiment services against the manual labels. This is a substantive gap because the intervention and experience-level claims depend on label reliability. In the revised manuscript we will add these performance numbers, computed from the existing manual classifications, in a new subsection of the results. revision: yes

  2. Referee: [Abstract and Discussion] Abstract and discussion: The assertion that freezing chat after death 'should be possible to reduce cyberbullying ... by a significant factor' is presented without any quantitative support, effect-size calculation, statistical test, or description of the timing correlation that was observed in the classified data.

    Authors: We acknowledge that the abstract and discussion present the chat-freeze claim without quantitative support, effect sizes, statistical tests, or an explicit description of the observed timing correlation between deaths and toxic messages. While the claim rests on patterns visible in the classified data, the current text does not supply the requested details. We will revise both sections to include a description of the timing analysis performed on the classified data together with any available quantitative measures or effect-size estimates. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical classification rests on external research and manual labels

full rationale

The paper collects raw chat data from World of Tanks, applies a scoring scheme drawn from current external research, performs SQL feature detection and third-party sentiment analysis, and compares outputs to manual classification via a custom client. Central claims (chat-freeze intervention effect; new players less likely to bully) are observational patterns extracted from the resulting labels. No equations exist, no parameters are fitted then re-predicted, no self-citations bear load on uniqueness or ansatzes, and the scoring scheme is not defined in terms of the target conclusions. The derivation chain is therefore self-contained against external benchmarks and manual ground truth.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claims rest on the external scoring scheme being a valid proxy for cyberbullying and on the collected chat logs being representative of actual in-game behavior; both are domain assumptions not independently validated in the abstract.

axioms (1)
  • domain assumption The scoring scheme based on current research accurately identifies cyberbullying instances in chat logs.
    Invoked when the paper states it presents a scoring scheme to enable identification of cyberbullying.

pith-pipeline@v0.9.0 · 5792 in / 1360 out tokens · 60499 ms · 2026-05-24T16:30:08.129339+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.