{"id":"0f2dd87d-dc99-43f1-b153-c31c5d57b86b","arxiv_id":"2509.11139","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A multidimensional evaluation framework measures information cocoons through topic diversity, click repetition, network density, and community openness, benchmarked over multiple recommendation rounds.","lead":"This paper proposes a five-metric framework for measuring information cocoons in news recommenders, covering individual diversity and group network structure. It benchmarks seven recommendation models across two real-world news datasets and tests five lightweight mitigation strategies.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Multi-round click simulation is unspecified; the reported progressive deepening of the cocoon may be an artifact of the closed-loop protocol rather than a property of the recommender models.","rationale":"The reader's weakest_assumption is that the multi-round recommendation and click behavior simulation is unspecified, and that the observed deepening may be partially built into the loop. This is exactly the load-bearing concern. The paper's abstract, §4.2, Appendix B, and Figures 3–8 all depend on the multi-round protocol: without it, there is no way to interpret the reported monotone trends. My stress-test sharpens the concern by noting the most natural completion of the unspecified loop—users click the recommender's top-scored items and history accumulates—would mechanically drive R toward 1 and N/H downward, making the 'deepening' a tautology rather than an empirical finding. This does not contradict the reader's CONDITIONAL verdict; it reinforces it. The framework and metrics are clearly defined and may still be useful, so the paper should not be rejected outright, but the central empirical claim cannot be accepted without full disclosure of the simulation protocol and ideally a control condition. The reader already conditioned on these revisions, so the verdict remains CONDITIONAL/UNCHANGED. I agree with the reader's identification of the same weak spot.","tokens_in":18786,"tokens_out":4123,"duration_ms":54526,"concrete_test":"Run NRMS on MIND under two controlled user models with the same candidate regeneration and round count: (i) the authors' implicit protocol (click highest-scored items), and (ii) a random-click control (click K items uniformly at random from the top-100 each round, no retraining). Compare the slopes of N@20, H@20, R, D, O across rounds. If (ii) reproduces the monotone deepening, the trend is caused by the closed-loop simulation, not by personalization; if (i) and (ii) diverge, the authors must disclose the exact protocol and code to make the claimed benchmark interpretable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim—'the information cocoon effect deepens progressively' (Appendix B) and 'most news recommendation models exacerbate the information cocoon effect' (§4.5)—rests entirely on a multi-round simulation loop that is never specified. §3.1.3 says only 'we simulate users' click behavior'; §4.2 says experiments run 'across multiple rounds of recommendations'; Figure 4 shows the user-item network 'evolution across multiple recommendation rounds.' The paper does not state how many rounds are run, how the candidate pool is regenerated, how a user chooses which items to click, whether the recommender is retrained between rounds, or how the history is updated. Under the most natural protocol—each round the user clicks the top-scored items and the history is extended with those clicks—the click repeat rate R is forced upward and topic diversity N/H is forced downward regardless of which algorithm is being evaluated, because the loop feeds the model's own outputs back into the user profile. The reported R values (0.93–0.99) and the monotone trends in Figures 5–8 are exactly what such a mechanical loop would produce. Thus the headline conclusion would be an artifact of the evaluation setup rather than an empirical discovery about the algorithms. This is not an internal inconsistency, but it makes the benchmark non-reproducible and the central claim unverifiable as written.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a multidimensional framework for assessing the information cocoon effect in news recommendation. It defines five metrics: number of topic categories (N), category information entropy (H), click repeat rate (R) at the individual level, and network density (D) and community openness (O) at the group level. Using the MIND and Adressa datasets, the authors report multi-round experiments with seven classic news recommenders, claiming that the cocoon effect deepens progressively over rounds and that most models exacerbate it. They also propose five mitigation strategies and report their effects on the proposed metrics.","tokens_in":19132,"tokens_out":3568,"duration_ms":43188,"significance":"If the empirical claims are reproducible, the proposed framework would provide a useful unification of individual-level and group-level cocoon indicators and a benchmark for comparing recommender systems on diversity and polarization. The metric definitions (Eqs. 1–5) are explicit and self-consistent, and the use of two real-world datasets is a strength. However, the central empirical contribution is currently unverifiable: the multi-round simulation loop is never specified, the click-repeat-rate metric is partly circular when history is updated from the recommender's own outputs, and no statistical evidence (error bars, seeds, significance tests) is provided for any comparison. The paper also ships no code or detailed protocol, so the headline conclusions cannot be checked.","major_comments":[{"comment":"The central empirical claim—'the information cocoon effect deepens progressively' (Appendix B) and 'most news recommendation models exacerbate the cocoon' (§4.5)—rests on a multi-round simulation that is never specified. The paper does not state the number of rounds, how candidate pools are regenerated, how a user chooses which items to click, how history is updated between rounds, or whether models are retrained. Under the natural protocol (the user clicks top-scored items and the history is extended with those clicks), R and D are partly constructed by the feedback loop. Please provide the full protocol; without it, Figures 5–8 and Tables 2–6 are unreproducible.","section":"§3.1.3 and §4.2"},{"comment":"The click repeat rate R is defined as the proportion of clicked items whose category is in h_j, the user's historical click categories. If h_j is updated with the user's own clicks each round, any recommender that exploits historical behavior will mechanically increase R, independent of whether a genuine cocoon forms. The reported R values are 0.93–0.99, and these values are used to conclude that recommendation models deepen the cocoon. A non-personalized baseline (e.g., random or popularity-based recommendation) and a precise definition of h_j (initial history only, or cumulative over rounds) are needed to separate metric-induced drift from algorithmic behavior.","section":"Eq. (3), §3.1.3"},{"comment":"The experimental results are reported as single point values with no error bars, standard deviations, number of seeds, or significance tests. Consequently, differences between models, and the claimed 'most models exacerbate' finding, cannot be assessed statistically. Table 4 in particular reports mitigation improvements such as CDR on Adressa D (-15.55%) and LTAO on Adressa O (-19.61%) that are presented as effects but have no variance estimates. Please include multiple runs, error bars, and significance tests for both the benchmark comparisons and the mitigation evaluation.","section":"Tables 2–6 and Figures 5–8"},{"comment":"The text derives specific temporal conclusions—'dramatic changes occur in the early rounds, especially the first 10' and 'stabilizes around rounds 10–15'—from Figure 8. However, Figure 8 is described as a histogram of a scatter distribution with the x-axis showing category results and the y-axis showing subcategory results; round number is not visibly represented as a temporal axis. Please clarify how the round dimension is encoded in the figure and how the stated early/late-round dynamics follow from the displayed data.","section":"Appendix B, Figure 8"}],"minor_comments":[{"comment":"Reference [23] is duplicated as [22] in the Introduction; the citation list should be cleaned.","section":"References"},{"comment":"The piecewise expression for the epsilon-greedy strategy is syntactically unclear: 'softmax(s_ui),1-epsilon' should be written as a proper two-branch definition with probabilities summing to one.","section":"Eq. (6)"},{"comment":"The mitigation strategies introduce hyperparameters epsilon, lambda, mu, gamma, and alpha, but their values are never reported. Without these, the mitigation results in Table 4 are not reproducible.","section":"§4.6"},{"comment":"The text states that mitigation causes only a 'small fluctuation (2%) in performance metrics including AUC, MRR, NDCG@5, and NDCG@10,' but these accuracy metrics are not reported in any table. Please include them.","section":"§4.2, Table 4"},{"comment":"The table notes say bold and underline indicate the strongest and second-strongest cocoon effects, but these formatting cues are not visible in the rendered table. Please ensure they are displayed or describe the values explicitly.","section":"Appendix A.2, Table 5"},{"comment":"Louvain community detection is applied to a user-item bipartite network, but standard Louvain assumes a unipartite graph. Please state whether a bipartite modularity variant or a projection was used.","section":"§4.1.3"}],"recommendation":"major_revision","confidential_remarks":"The framework and metric definitions are a useful starting point, but the empirical core is not yet verifiable. The authors need to add a complete specification of the multi-round simulation, a non-personalized baseline to address the circularity of R, and statistical rigor (seeds, error bars, tests). These are substantial but addressable within the manuscript's scope, so I recommend major revision rather than rejection. I would also ask the authors to clarify the relation to prior multi-indicator frameworks, since the abstract's 'first unified metric framework' claim may be too strong given [2] and [63]."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the multidimensional framework (N, H, R, D, O) is a reasonable attempt to unify individual and group-level cocoon measurement, and the model comparison on two real datasets is a useful benchmarking exercise. But the paper's headline conclusion—that cocoons deepen progressively and most models exacerbate them—sits on a simulation protocol that is essentially absent. As written, the result could be an artifact of the loop rather than a property of the algorithms.\n\nWhat's genuinely useful: the metric definitions are transparent and easy to implement. Combining topic diversity, click repetition, network density, and community openness gives a more complete picture than single-metric studies. Running seven standard recommenders on MIND and Adressa is real work, and the mitigation strategies (EGS, CDR, LTAO, CCR, CPF) are sensible lightweight interventions with plausible mechanisms. The category/subcategory comparison is a nice robustness check.\n\nThe soft spots are significant. Section 3.1.3 says 'we simulate users' click behavior' and Section 4.2 says 'multiple rounds,' but the paper never states how many rounds, how the candidate pool is regenerated, how clicks are chosen, whether the recommender is retrained, or how history updates. Under the most natural protocol—user clicks top-ranked items, history extends with those clicks—R is mechanically pushed up and N/H pushed down, because the model's own outputs become the user profile. That would make the progressive deepening a design choice, not an empirical discovery. The click repeat rate definition (Equation 3) also has a built-in upward bias for any history-based recommender, so cross-model comparisons conflate algorithm behavior with the metric's mechanics.\n\nAlso missing: error bars, standard deviations, seeds, significance tests, and a non-personalized baseline. Tables 2-4 report single numbers. And the 'first unified framework' claim is overstated given prior work like Anwar et al. (WWW '24) and Michiels et al. (RecSys '23) already measure multiple dimensions.\n\nWho this is for: someone building an evaluation harness for cocoon-aware recommendation would get value from the metric definitions and the mitigation list. But the empirical ranking of models is not trustworthy until the simulation is disclosed and the loop is controlled for. The paper deserves a serious referee—the framework is worth airing—but it needs major revision before it can be cited as evidence about which models worsen cocoons.","headline":"A useful metric framework for measuring information cocoons, but the central multi-round finding is unverifiable as written because the simulation loop is never specified.","tokens_in":19554,"tokens_out":2167,"would_cite":false,"duration_ms":22635,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that information cocoons in news recommendation can be measured by five indicators spanning individual and group levels, that the cocoon deepens over repeated rounds for most recommenders, and that lightweight interventions","keywords":["information cocoon","news recommendation","filter bubble","echo chamber","topic diversity","group polarization","multi-round evaluation","recommendation mitigation"],"falsifier":"A concrete check: re-run the same recommenders under an explicitly specified user-simulation protocol—state how clicks are chosen, how history updates, and how many rounds run—and test whether N and H still fall while R and D rise when clicks are not simply the recommender's top-ranked items. If the trends disappear, the claimed progressive deepening is an artifact of the simulation.","tokens_in":18703,"feed_emoji":"🕸️","tokens_out":6076,"duration_ms":60208,"temperature":0.7,"pith_summary":"This paper tries to establish a single, multidimensional way to measure the 'information cocoon' effect in news recommender systems, and to use it to compare recommendation algorithms over multiple rounds of exposure. It argues that the cocoon is not just a matter of individual topic narrowing but also of group-level polarization, captured by five indicators: number of topic categories, category entropy, click repeat rate, network density, and community openness. Applying these metrics to seven recommenders on two real-world news datasets, the paper contends that the cocoon deepens progressively across recommendation rounds and that most models intensify it, with knowledge-graph-aware recommenders showing the mildest effect. It then shows five lightweight mitigation strategies—random exploration, diversity regularization, attention balancing, and two community-aware re-rankers—that improve diversity while keeping accuracy loss near two percent. A sympathetic reader would care because the framework could become a standard evaluation lens for ethical news recommendation.","feed_headline":"Five metrics show news recommenders deepen information cocoons","feed_subtitle":"A unified five-metric framework shows how personalized news narrows diversity and closes communities over rounds.","key_machinery":"The central machinery is the five-indicator assessment set (N, H, R, D, O) applied over multi-round recommendation loops. N and H are the average number of distinct topic categories and the category entropy of Top-K lists; R is the proportion of simulated clicks whose categories repeat the user's history; D and O are computed from a user-item bipartite graph with community detection, measuring average internal edge density and the balance of external versus internal edges. This set translates the abstract notion of an information cocoon into measurable quantities at two levels and exposes how each quantity evolves as recommendations are repeated.","core_discovery":"The central claim is that the proposed multidimensional framework—individual homogenization measured by number of topic categories N, category entropy H, and click repeat rate R, and group polarization measured by network density D and community openness O—captures the information cocoon effect in news recommendation. Using multi-round experiments on two real-world news datasets, the paper finds that the cocoon effect deepens progressively: N and H decrease, R and D increase, and O decreases. Most of the seven recommenders exacerbate the effect; the recommenders that integrate external knowledge retain the most diversity. The paper further claims that five mitigation strategies (EGS, CDR, LT","pith_inferences":["Inference: The same five indicators could be computed for short-video and social feeds, where topic labels and interaction graphs exist, effectively exporting the framework beyond news.","Inference: Because the click simulation is under-specified, a validation study using logged impressions from a deployed recommender—rather than simulated clicks—would test whether the deepening trend holds outside the lab.","Inference: The small accuracy cost of the mitigation strategies suggests diversity can be tuned as a knob; combining the strategies with fairness metrics could reveal whether broader exposure also improves source and publisher equity."],"forward_implications":["A single multi-indicator report, rather than any one metric, becomes the natural unit for comparing recommenders on cocoon risk.","Most current news recommenders would be expected to reduce topical breadth and increase network closure as they are deployed over time; knowledge-graph-aware models would be the most resilient.","The first roughly ten rounds are where the cocoon forms fastest, so early evaluation is the most informative.","The five mitigation strategies offer a practical toolkit with roughly two percent accuracy cost; community-aware re-ranking gives the largest gains in openness."],"fun_headline_variants":["News recommenders tighten cocoons across five metrics","Five metrics reveal how news algorithms seal users in bubbles","Unified five-metric framework tracks news cocoon deepening","Study: personalized news shrinks diversity, raises polarization","News recs: five metrics show cocoons tighten over rounds"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the multi-round simulated clicking faithfully reproduces real user behavior; the paper never specifies how clicks are simulated, how history updates, or how many rounds are run, so if the simulation is unrepresentative, the reported deepening of the cocoon could be an artifact.","fun_headline_variants_meta":{"raw":{"variants":["News recommenders tighten cocoons across five metrics","Five metrics reveal how news algorithms seal users in bubbles","Unified five-metric framework tracks news cocoon deepening","Study: personalized news shrinks diversity, raises polarization","News recs: five metrics show cocoons tighten over rounds"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000413,"raw_usage":{"total_tokens":1910,"prompt_tokens":621,"completion_tokens":1289,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":365,"completion_tokens_details":{"reasoning_tokens":1221}},"tokens_in":365,"tokens_out":1289,"duration_ms":12786,"temperature":1.0,"reasoning_tokens":1221,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T17:01:48.740071+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete check: re-run the same recommenders under an explicitly specified user-simulation protocol—state how clicks are chosen, how history updates, and how many rounds run—and test whether N and H still fall while R and D rise when clicks are not simply the recommender's top-ranked items. If the trends disappear, the claimed progressive deepening is an artifact of the simulation.","supporting_citations":[],"review_version":1}