{"id":"1d29648f-52ff-4544-bd96-9b5ba40e8302","arxiv_id":"1909.02638","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Field experiments on 82,890 real website visitors show that consent notice position, number of choices, and nudging materially change whether users accept tracking, with opt-in defaults cutting acceptance to under 0.1%.","lead":"Researchers tested different cookie consent notice designs on a real German website with over 80,000 visitors. They found that where the notice appears, how many choices it offers, and whether options are pre-selected strongly change whether people accept tracking.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 0.1% opt-in acceptance claim conflates default setting with click effort: opt-in category/vendor notices lacked an 'Accept all' button, so accepting all required manually checking every box, while preselected/nudging conditions required no action.","rationale":"The paper is a serious, large-scale field study with careful instrumentation, a plausible ethical review, and an external validation attempt. The reader's CONDITIONAL verdict is reasonable. My stress-test identifies a related but distinct weakness: the most quoted quantitative result is not just threatened by single-site generalizability; internally, the opt-in condition confounds the default with the interaction cost of accepting all purposes. Because the authors did not include an 'Accept all' button in the opt-in conditions, the 0.1% rate cannot be attributed solely to privacy-by-default preferences. This does not undermine the paper's central qualitative thesis that design details matter—that thesis is robust to the confound—but it does narrow the crisp policy claim in the abstract and Section 7. Since the reader already marked CONDITIONAL and the concern reinforces that condition rather than shifting the verdict, I leave the verdict unchanged. The concrete test I propose would separate default effects from effort effects in a direct, feasible field variation.","tokens_in":25532,"tokens_out":4236,"duration_ms":45226,"concrete_test":"Re-run Experiment 2's Categories–Non-Nudging condition with an added one-click 'Accept all' button while keeping all optional checkboxes unchecked by default, using the same field setup and round-robin assignment. If the fraction accepting all purposes remains below roughly 1%, the 0.1% figure stands as a genuine privacy-by-default effect; if it rises into double digits, the original figure largely reflects click effort rather than opt-in defaults, and the Section 7 policy inference must be rescaled to 'granular opt-in without an accept-all control.'","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Experiment 2, the opt-in conditions (Categories–Non-Nudging and Vendors–Non-Nudging) start with all checkboxes unchecked and require the visitor to tick each purpose or vendor individually before submitting. No 'Accept all' or 'Select all' control is described anywhere in Section 3.3 or the figures. The nudging conditions differ by pre-checking those same boxes, so the headline contrast in Section 4.3.1—less than 0.1% accepting all under opt-in versus roughly 30% (mobile) and 10% (desktop) accepting all when preselected—simultaneously varies the default and the number of clicks needed to reach 'accept all' (5–6 extra clicks versus 0). The paper's own timing data in Appendix A show median decision times of 7–8 seconds for category/vendor notices versus 4–5 seconds for binary notices, consistent with an effort effect. Section 7 then draws a broad conclusion: enforcing the GDPR's data-protection-by-default principle would lead to 'less than 0.1% of users actively consenting to the use of third-party cookies.' That inference is not warranted for opt-in notices generally; it is only demonstrated for this particular granular, high-effort opt-in design. The Cookiebot comparison in Section 4.3.3 does not resolve the confound, since those notices also differ in which categories can be deselected and in their interaction design. The central qualitative claim that small implementation decisions change consent behavior remains well supported, but the specific quantitative headline is narrower than stated.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper investigates how the user-interface design of GDPR cookie consent notices affects visitor consent behavior. The authors first analyze 1,000 consent notices collected from popular European websites and identify eight design variables. They then report three between-subjects field experiments on a German e-commerce website with 82,890 unique visitors, varying notice position (Experiment 1), choice granularity and nudging via preselection or highlighting (Experiment 2), and wording plus the presence of a privacy-policy link (Experiment 3). The main findings are that bottom-left placement yields the highest interaction, binary notices receive more acceptance than granular category/vendor notices, nudging substantially increases acceptance, and in the opt-in granular conditions fewer than 0.1% of visitors accept every purpose or vendor. The paper concludes that small implementation decisions have large effects and argues that enforcing the GDPR's data-protection-by-default principle would lead to very low active consent to third-party cookies, recommending opt-in category-based notices.","tokens_in":25813,"tokens_out":2746,"duration_ms":33438,"significance":"The study is valuable because it provides large-scale field evidence, with random assignment, on a topic previously studied mostly through surveys or small lab experiments. The external validation against Cookiebot data and the public availability of the materials are notable strengths, as are the rich interaction logs and the follow-up survey responses. If the headline results are interpreted carefully, the paper makes a strong empirical case that seemingly minor design choices in consent notices substantially change consent behavior, which is directly relevant to current regulatory and technical debates about dark patterns and meaningful consent. However, the quantitative headline about 'less than 0.1%' under GDPR-compliant opt-in notices is weakened by a confound between default selection and interaction effort, and the single-site sample limits the policy generalization.","major_comments":[{"comment":"The central quantitative claim conflates the default setting with the required interaction effort. In Experiment 2, the opt-in category and vendor conditions (Categories–Non-Nudging and Vendors–Non-Nudging) began with all checkboxes unchecked and required the visitor to tick each purpose or vendor individually before submitting; no 'Accept all' or 'Select all' control is described in Section 3.3 or in Figure 4. The preselected conditions, by contrast, allowed acceptance with zero additional clicks. Thus the contrast 'less than 0.1% accept all' versus roughly 30% of mobile and 10% of desktop users accepting all third parties simultaneously varies the default and the number of clicks needed to reach 'accept all' (five to six clicks versus none). The timing data in Appendix A, with medians of 7–8 seconds for category/vendor notices versus 4–5 seconds for binary notices, are consistent with an effort effect. Section 7's conclusion that enforcing data-protection-by-default would lead to 'less than 0.1% of users actively consenting to the use of third-party cookies' is therefore warranted only for this specific high-effort granular design, not for opt-in notices generally. The authors should either add an opt-in condition with an 'Accept all' button, reanalyze the existing data to separate the default effect from the effort effect, or substantially qualify the claim.","section":"Section 4.3.1 and Section 7"},{"comment":"The manuscript reports no confidence intervals for the main proportions, even though the headline claims rest on comparisons of very small percentages (e.g., <0.1% versus 0.16% in Table 2). For a proportion near zero with a few thousand observations, the sampling uncertainty is non-negligible, and the paper's own external validation shows differences that are attributed to context but are unquantified. Reporting Wilson or exact binomial confidence intervals for the key 'Accept all' proportions in Figures 4 and 6 and Table 2 would make the strength of the evidence transparent. This is not merely a presentation issue because the regulatory conclusion in Section 7 depends on how precisely the extremely low opt-in rate is estimated.","section":"Section 6.2 and Section 4.3.1"},{"comment":"The paper acknowledges in Section 6.2 that the sample is a German-language e-commerce website whose visitors may not be representative of the general public, and that the sample 'seems more inclined towards rejecting cookies.' Given that the policy recommendations in Sections 6.1 and 7 generalize from this single site, the authors should more explicitly condition their recommendations on this limitation and discuss which findings are likely to be site-specific. The Cookiebot comparison helps, but the Cookiebot notices differ in interaction design (some categories cannot be deselected), so it does not fully resolve external validity. This does not invalidate the qualitative conclusion that design matters, but it does limit the quantitative claims about the absolute level of consent under GDPR-compliant notices.","section":"Section 6.2"}],"minor_comments":[{"comment":"The 30-second automatic replacement of the consent notice with the survey invitation is a notable modification of the browsing experience, but the paper does not discuss whether this auto-replacement could itself affect the measured interaction rates or the interpretation of 'no action' responses. A sentence addressing this would strengthen the methodology.","section":"Section 3.1"},{"comment":"The observation that 'more visitors selected specific vendors than categories' is presented without a statistical test or confidence interval; given the small absolute numbers, a significance test or an explicit statement of the raw counts would help the reader gauge the robustness of this finding.","section":"Section 4.3.2"},{"comment":"In the 'Text: Processor' row, 'third party 2.6 &' should read 'third party 2.6 %'.","section":"Table 1"},{"comment":"The phrase 'data protection by default' is used to refer both to the GDPR's Article 25 principle and to an opt-in consent default; a brief clarification of the legal term would prevent conflation of the two concepts.","section":"Abstract and Section 2.1"},{"comment":"The use of Cramér's V is fine, but the paper should state whether the reported p-values are corrected for multiple comparisons across the many conditions and interactions tested in Experiment 1.","section":"Section 4.2.1"}],"recommendation":"major_revision","confidential_remarks":"The confound in Experiment 2 strikes me as the paper's main load-bearing issue: the quantitative headline is likely to be widely cited, and as currently presented it overstates what the data can show. That said, the central qualitative claim—design decisions matter—is well supported, and the paper contains enough transparency that the fix is a matter of reanalysis or careful rewriting rather than new data collection. I would encourage the editor to treat the added 'Accept all' condition or equivalent analysis as an expected revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"X,\n\nRead this one because it's already being cited as proof that GDPR privacy-by-default would kill cookie consent. The underlying study is genuinely good: large-N field experiments with random assignment on a real site, plus external validation against Cookiebot. The findings that position, choice granularity, and nudging shift interaction and consent rates are solid and important. The taxonomy of UI variables is useful, and the survey adds texture.\n\nBut the sharpest number, “less than 0.1% accept all under opt-in,” is over-broad. In Experiment 2, the opt-in category/vendor conditions presented all checkboxes unchecked and had no accept-all button; selecting everything required five or six clicks. The nudging conditions pre-checked the boxes, so accepting required one click on Submit. So the 0.1% versus 30% contrast varies default and effort together. The paper’s own timing data (median 7–8 seconds for category/vendor versus 4–5 for binary) is consistent with an effort effect. The conclusion in Section 7 that enforcing data-protection-by-default would lead to “less than 0.1%” consenting is therefore not warranted for opt-in notices in general; it is demonstrated for this particular granular, high-effort design. The Cookiebot comparison doesn’t fix this, since those notices also differ in interaction design.\n\nOther soft spots are the ones the authors name: single German e-commerce site, 78% mobile, short dwell times. They acknowledge this in Section 6.2, but the regulatory recommendations lean heavily on generalizability. There are also no confidence intervals on the main proportions, and the 30-second auto-replacement of the notice is a design choice that could affect interaction measures. The ethics of logging real users’ clicks without prior consent is a legitimate concern, though they do disclose after 30 seconds and pseudonymize.\n\nNone of this kills the paper. The qualitative conclusions—small UI decisions matter, nudging works, granular opt-in is very unpopular—are well supported. But the headline number is a specific design artifact, not a universal constant.\n\nFor peer review: definitely send it out. It is a serious, reproducible field study that deserves referee time, and the confound is fixable with wording changes or a follow-up experiment with an accept-all button in the opt-in conditions.","headline":"Good field experiment on consent banners, but the famous 0.1% opt-in figure conflates default setting with click effort; the qualitative results hold up.","tokens_in":26333,"tokens_out":2081,"would_cite":true,"duration_ms":22452,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Seemingly cosmetic choices in cookie consent banners—position, pre-selection, and wording—decisively shape whether users consent, with accept-all rates ranging from under 0.1% to roughly 30%.","keywords":["GDPR consent notices","cookie banners","nudging and dark patterns","privacy by default","field experiment","usable privacy","consent behavior","online tracking"],"falsifier":"Run the same nine-condition experiment on a site with a different audience, such as a desktop-heavy news site, and compare accept-all rates in the opt-in and pre-selected conditions: if the opt-in accept-all share approaches the pre-selected share, or if pre-selection raises acceptance only slightly, then the paper's central claim that design defaults dominate consent would not hold in that context.","tokens_in":25339,"feed_emoji":"🍪","tokens_out":6338,"duration_ms":68615,"temperature":0.7,"pith_summary":"This paper tries to establish that how a website asks for consent matters as much as whether it asks. Across three field experiments with 82,890 real visitors to a German e-commerce website, the authors varied the banner's position, the number and framing of choices, and the wording, then logged every click. They find that placement alone changes interaction rates from about 3% to 37%, that pre-selecting cookie categories makes roughly 30% of mobile and 10% of desktop visitors accept all third parties, and that a strict opt-in banner draws under 0.1% accept-all decisions. If true, this means the ubiquity of cookie banners is not the real problem; the design choices embedded in them are, and regulation that ignores interface design cannot deliver the informed, free consent the GDPR promises.","feed_headline":"Cookie banner design swings consent from 0.1% to 30%","feed_subtitle":"Three field experiments with 82,890 visitors show that position, pre-selection, and wording shape GDPR consent.","key_machinery":"The load-bearing object is a set of consent-notice variants built from eight user-interface variables the authors first catalogued in 1,000 real notices: position, size, blocking, choices, text, nudging, formatting, and links. The experiments manipulate three of these—position; choice type and nudging; wording and privacy-policy link—while holding the rest constant. The mechanism that carries the argument is the comparison of interaction and accept/decline rates between otherwise identical notices, especially the contrast between pre-selected checkboxes (nudging) and unchecked, privacy-by-default checkboxes. That contrast isolates the effect of default framing from everything else about the notice.","core_discovery":"The central claim is that seemingly small implementation decisions in consent-notice interfaces substantially change whether and how visitors consent, and that most current notices are built to manufacture consent rather than record it. In the sharpest result, a privacy-by-default (opt-in) notice led fewer than 0.1% of visitors to allow cookies for all purposes, while pre-selecting all checkboxes led around 30% of mobile and 10% of desktop users to accept all third parties; 1 to 4% of opt-in users still selected some parties. A dialog in the lower-left corner drew interactions from 37.1% of visitors, versus 2.9% for a top bar. More choices made visitors more likely to decline cookies, and highlighting the accept button increased acceptance even when it was the only action. The authors conclude that opt-out banners are unlikely to produce intentional consent and recommend opt-in, category-based notices as the design that matches both GDPR's purpose-specific consent and users' stated preferences.","pith_inferences":["If the same banner-design elasticity holds across other site types, then 'consent fatigue' is partly an artifact of bad default designs, and browser-level or cross-site consent tools could be built on opt-in defaults rather than repeated banners.","The study's site drew mostly mobile, short-dwell visitors; a replication on a desktop-heavy news site or a service with user accounts could show larger or smaller nudging effects, so the absolute percentages should be read as site-specific until re-tested.","A natural extension would test the same variants with users who already run ad blockers; the paper's ad-blocker subsample suggests these users engage less with banners, which may mean their consent decisions are systematically underrepresented in current consent logs.","Comparing the same consent-banner variants across languages and countries could separate left-to-right reading effects from genuine position preferences, which the current single-language, single-site design cannot do."],"forward_implications":["Enforcing the GDPR's data-protection-by-default principle would, on these numbers, cut accept-all consent to below 0.1% on sites like the one studied, a level that current behavioral-advertising business models could not absorb.","Regulators should specify consent-notice requirements—position, default states, and number of choices—rather than only requiring that consent be asked.","Category-based opt-in notices would satisfy users' desire for control without the overwhelming detail of vendor lists.","Websites serious about meaningful consent should place notices where they interrupt reading (lower left on desktop, lower part of the screen on mobile) and avoid pre-selection and highlighted accept buttons.","The common top-of-screen bar used by about a quarter of sites in the authors' corpus is the least effective at eliciting any choice, which suggests many current notices mainly train users to ignore them."],"supporting_citations":[{"why":"Supplies the prior measurement of consent-notice prevalence and the corpus of 5,087 notices from which the study's UI variables are drawn.","marker":"[12]"},{"why":"Supplies the legal standard of a clear affirmative act and informed consent that the study measures current notices against.","marker":"[6]"},{"why":"Provides the catalogue of nudging techniques (pre-checked boxes, highlighted buttons) that Experiment 2 operationalizes.","marker":"[10]"},{"why":"Defines the vendor-based consent format that the paper reproduces as one of its test conditions.","marker":"[14]"},{"why":"Establishes the baseline that most sites set cookies before any opt-out and that few offer meaningful opt-out, motivating the need for functional consent tests.","marker":"[38]"},{"why":"Supplies the external baseline of about 3% of users accepting marketing cookies, used to contextualize the paper's acceptance rates.","marker":"[34]"},{"why":"Provides prior evidence on how mentioning a privacy policy affects trust, which Experiment 3 tests and does not confirm.","marker":"[27]"},{"why":"Provides the design-space framing for privacy notices that the paper extends to cookie consent interfaces.","marker":"[39]"}],"fun_headline_variants":["Cookie banner design swings consent from 0.1% to 30%","Consent notices: design choices flip acceptance from 0.1% to 30%","Field experiment: banner layout shifts consent from 0.1% to 30%","GDPR consent: subtle UI tweaks move rates from 0.1% to 30%","How banner wording and layout turn 0.1% consent into 30%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The behavior of 82,890 visitors to a single German-language e-commerce website—78% on mobile devices and most staying only seconds—represents the broader European internet population closely enough to support the paper's regulatory recommendations.","fun_headline_variants_meta":{"raw":{"variants":["Cookie banner design swings consent from 0.1% to 30%","Consent notices: design choices flip acceptance from 0.1% to 30%","Field experiment: banner layout shifts consent from 0.1% to 30%","GDPR consent: subtle UI tweaks move rates from 0.1% to 30%","How banner wording and layout turn 0.1% consent into 30%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000634,"raw_usage":{"total_tokens":2943,"prompt_tokens":984,"completion_tokens":1959,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":600,"completion_tokens_details":{"reasoning_tokens":1847}},"tokens_in":600,"tokens_out":1959,"duration_ms":15680,"temperature":1.0,"reasoning_tokens":1847,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:44:01.979684+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same nine-condition experiment on a site with a different audience, such as a desktop-heavy news site, and compare accept-all rates in the opt-in and pre-selected conditions: if the opt-in accept-all share approaches the pre-selected share, or if pre-selection raises acceptance only slightly, then the paper's central claim that design defaults dominate consent would not hold in that context.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the prior measurement of consent-notice prevalence and the corpus of 5,087 notices from which the study's UI variables are drawn."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the legal standard of a clear affirmative act and informed consent that the study measures current notices against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the catalogue of nudging techniques (pre-checked boxes, highlighted buttons) that Experiment 2 operationalizes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the vendor-based consent format that the paper reproduces as one of its test conditions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the external baseline of about 3% of users accepting marketing cookies, used to contextualize the paper's acceptance rates."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides prior evidence on how mentioning a privacy policy affects trust, which Experiment 3 tests and does not confirm."}],"review_version":1}