{"id":"507f2bfa-1dae-4b3d-bd2a-7edfa92f79e9","arxiv_id":"2507.22019","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Following 11 million redirecting web addresses shows that half fail, most canonical redirects are HTTP-to-HTTPS, and 62,000 custom 404 pages include many soft 404s.","lead":"The paper followed 11 million redirecting web addresses and found that only half lead to a working page, while the other half end in errors, soft 404 pages, or dead ends. It maps how websites move traffic around the web, exposing patterns that affect search ranking, web archiving, and user experience.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Soft-404 headline rests on an unstated 'contains 404' classification; Table 5 false positives make the 47% soft-404 rate unverified.","rationale":"The reader's weakest-assumption pick (archive-derived sample skew) is a real external-validity concern, but it is a standard limitation of web-archival measurement and does not invalidate the internal analysis of the 11M URIs. The more load-bearing problem is internal to the paper's soft-404 analysis: the detection rule for 'custom 404 URIs' is never stated, and the paper's own Table 5 contains entries that are demonstrably not error pages (e.g., an affiliate URL returning 429). Because the abstract and conclusions advertise the 47% soft-404 figure as a headline finding, this classification flaw directly undermines a central claim. The redirect taxonomy, sink-URI analysis, and crawl methodology are otherwise useful and transparent; the 50/50 success/error split is clearly defined, though it is crawler-dependent because invalid redirects are counted as errors. The conditional verdict remains appropriate: the paper should be accepted only if the soft-404 identification is validated with content-based checks or the claims are softened accordingly.","tokens_in":18160,"tokens_out":7135,"duration_ms":75401,"concrete_test":"Obtain the list of the 62,000 target URIs classified as custom 404 pages (or a random sample of at least 1,000, powered for a 47%±3% estimate). For each, (1) fetch the URI with a browser-like client and record the HTTP status and rendered page title/body; (2) manually or heuristically label whether the page is genuinely a not-found/error page (e.g., title contains '404', 'Not Found', or equivalent), ignoring the URL string; (3) recompute the soft-404 fraction using only confirmed error pages. If the confirmed fraction drops materially from the reported 47% (or the count of custom 404 pages falls far below 62,000), the paper's soft-404 claim requires revision and the abstract/conclusions should be reworded.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's soft-404 result (62,000 custom 404 URIs; only 46% return 404, 47% return 200; abstract and §7 conclusions) depends entirely on how 'custom 404 URIs' are identified. Section 5.3.8 gives no explicit detection rule, but every example in Table 5 is a target URI containing the substring '404'. That string match is not a valid proxy for an error page: Table 5 itself lists http://chaturbate.com/affiliates/in/grq0/KZiNo/?track=404exit, an affiliate link that returns status 429 (Too Many Requests), not a 404 page; the same table includes https://www.wp.pl/?404&src01=99f53, whose '404' is a query parameter. Including such URIs in the 62,000 count means both the count and the 47% soft-404 share are inflated by non-error pages. The conclusions repeatedly cite soft 404s as a major finding (abstract, §5.3.8, §7), so this is load-bearing. A content-based confirmation of the error-page status of the 62,000 targets is required before the soft-404 claim can be accepted.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper analyzes a sample of 11 million unique redirecting URIs drawn from Internet Archive TimeMaps first archived between 1996 and 2021. The authors re-crawled these URIs in September 2023 with Heritrix, following up to 10 redirect hops, and report aggregate outcomes: roughly half of the URIs terminate in a 2xx success and half in an error; most redirects are one-hop; about 6 million canonical and 3.5 million non-canonical redirects are identified via SURT exact matching; and a small set of ``sink'' URIs account for large numbers of source URIs. The paper also claims to identify 62,000 custom 404 target URIs, of which 47% are soft 404s returning HTTP 200. The manuscript presents flow diagrams for canonical and non-canonical redirects, path-depth changes, domain-change statistics, and qualitative case studies of sink URIs including consolidation, affiliate marketing, login pages, loopback sinks, and a Rickroll sink.","tokens_in":18326,"tokens_out":10510,"duration_ms":110445,"significance":"If the quantitative claims hold, the paper would be a useful large-scale descriptive reference for redirect behavior in the archived web: it documents the prevalence of HTTP-to-HTTPS canonicalization, the dominance of single-hop redirects, the existence of high-degree sink URIs, and the frequency of invalid redirects. The public dataset and transparent crawl procedure are strengths. However, the headline 50/50 success/error split depends on how invalid redirects and >10-hop chains are counted, and the soft-404 finding rests on an unstated and demonstrably unreliable URI-string heuristic. Because the abstract and conclusions repeatedly emphasize the soft-404 result, that claim must be re-derived with a content-based validation before the paper's central contributions can be accepted.","major_comments":[{"comment":"The 62,000 custom-404 count and the 47% soft-404 figure are not supported by a reproducible detection rule. The text says only that certain sinks ``contain the characters '404' in their URIs,'' and no algorithm is given for identifying the 62,000 targets. Table 5 directly contradicts a literal ``contains 404'' definition: http://chaturbate.com/affiliates/in/grq0/KZiNo/?track=404exit is an affiliate link that returns 429, not a 404 page, and https://www.wp.pl/?404&src01=99f53 has ``404'' as a query parameter on a portal page. The soft-404 share is therefore unverified and likely inflated. The authors must either (a) specify and validate a string-based rule that excludes such false positives, or (b) re-run the soft-404 analysis using page content (e.g., title/text containing ``Not Found'') to confirm that a target is actually an error page.","section":"§5.3.8 and Tables 3–5"},{"comment":"The denominator behind the headline 50% success / 50% error split is unclear. Section 3.2 reports 9.5 million URIs terminating within 10 redirects, 5.4 million of them successful, and then states that ``5.5 million URIs resulted in an error, including 1.5 million invalid redirects.'' Those two statements cannot both describe the 9.5-million dataset, and the abstract's 50/50 ratio matches the original 11 million if the 1.5 million invalid redirects and 6,068 over-10-hop URIs are counted as errors. In the 9.5-million final dataset the success share is about 57%, not 50%. Also, Figure 2's caption says ``3 million reach the second stage (R2),'' but the §3.2 numbers imply 11M − 6.9M ≈ 4.1M reach R2. The paper should state explicitly which denominator is used for each percentage and reconcile the R2/R3 counts.","section":"§3.2 and Figure 2"},{"comment":"The canonical/non-canonical split of 6 million versus 3.5 million depends entirely on ``exact match'' of the SURT transform, but the paper never specifies what normalization the SURT form applies. Figure 1's blue SURT strings show the same key for http://ecogeneration.com.au/, https://ecogeneration.com.au/, and https://www.ecogeneration.com.au/, which means the implementation strips both scheme and ``www''. The standard SURT transform does not remove ``www''; the background text in Section 2 also says SURT handles www, trailing slashes, and schemes, which is inconsistent with standard definitions. If a custom canonicalization was used, it should be precisely defined; otherwise the canonical/non-canonical classification is not reproducible and the reported ratio cannot be interpreted.","section":"§3.3 and Figure 1"},{"comment":"The paper generalizes to ``the web'' (title, abstract, conclusions) from a sample of URIs first archived by the Internet Archive between 1996 and 2021. The paper's own sink analysis shows heavy representation of .work TLDs, free hosting providers, expired domains, and abandoned content, which suggests the sample may overrepresent low-maintenance and low-cost sites. A quantitative statement about selection bias, or at least an explicit limitations paragraph acknowledging that the 50% error rate and soft-404 proportions may not generalize to the live web at large, is needed before the headline claims can be accepted as global web phenomena.","section":"§3, first paragraph and §7"}],"minor_comments":[{"comment":"The subtraction 11.7 million − 744,244 gives about 10.956 million, but the text reports 10,975,138; please check the arithmetic or clarify the rounding.","section":"§3.1"},{"comment":"The sentence ``only 0.42% of the redirects exceeded four hops without termination'' should define the population precisely (all URIs, or only those that reach the fourth hop) so the claimed support for a five-hop crawl cap is unambiguous.","section":"§3.2"},{"comment":"The caption says the table shows ``how some of these custom error pages are soft 404s,'' but the table includes a 429 status and a URI whose ``404'' is a query parameter; either annotate the detection rule or exclude rows that are not error pages.","section":"Table 5"},{"comment":"The description of SURT as removing www and scheme differences is non-standard; please add a precise definition or a citation to the SURT specification and state the exact normalization used in the crawler.","section":"§2, Figure 1"},{"comment":"Figure 2's caption says 6068 redirects were still redirecting at hop 10, while Section 3.2 mentions ``6,000 URIs encountered more than 10 consecutive redirects''; please align these numbers and use a consistent count.","section":"§3.2 and Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is an extended version of a WebSci'25 paper, and the dataset is public, so the soft-404 and canonical/non-canonical findings are reproducible in principle. The soft-404 claim, however, is a headline contribution that currently rests on a string-match heuristic with clear false positives; this needs a content-based validation or a substantial rewrite of the claim. The arithmetic inconsistencies in Section 3.2 and Figure 2 also need to be resolved before the paper can be considered for acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper is a big empirical study of redirects: 9.5M redirecting URIs from an Internet Archive-derived sample, crawled with Heritrix, with a taxonomy of outcomes. That's genuinely new. Prior work studied redirects for archival policy or soft errors, but nobody has put this many redirects through a systematic classification and looked at sinks, invalid redirects, and canonical/non-canonical splits at this scale. The dataset is public, the methods are mostly transparent, and the sink analysis (pharm-discount affiliate sink, Rickroll, loopback, custom error pages) is a useful contribution. I'd trust the status-code distributions, hop counts, and canonical/non-canonical split as descriptive of this sample.\n\nThe soft spot that matters is the soft-404 claim. Section 5.3.8 says 'custom 404 URIs' were identified but never defines the detection rule. Table 5 makes it obvious: every example contains the substring '404', including http://chaturbate.com/affiliates/in/grq0/KZiNo/?track=404exit (returns 429, not an error page) and https://www.wp.pl/?404&src01=99f53 (a query parameter). So the 62,000 count and the 47%-are-soft-404 number are not established. The authors need to either define a content-based classifier or drop the claim. This is load-bearing because the conclusions lean on it, but it's fixable.\n\nSecond soft spot: the sample. The URIs come from IA's Zipnum index of pages first archived 1996-2021. That skews toward older, low-cost, abandoned, and spammy domains—the paper's own .work and free-hosting examples show the bias. The 50% success/error split is probably an upper bound on error rates for the live web as a whole. The paper sometimes says 'the web' when it should say 'our sample.' That's a wording problem, not a fatal one.\n\nThe reader's conditional verdict is fair. I'd add that the invalid-redirect 13.22% being counted as 'errors' in the 50/50 headline is a definitional choice worth flagging, but not wrong.\n\nVerdict: this deserves a serious referee. The core measurement is reproducible and the taxonomy is valuable; the soft-404 section needs real revision. I'd send it to review, possibly with a note that the soft-404 claim should be validated or removed.","headline":"Large, reproducible redirect measurement whose soft-404 headline rests on an unstated 'contains 404' rule; the rest of the descriptive findings are solid and worth citing.","tokens_in":18927,"tokens_out":2550,"would_cite":true,"duration_ms":28615,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that half of all redirecting URIs in the archived web end in errors, that canonical HTTP-to-HTTPS redirects dominate, and that nearly half of custom 404 pages are soft 404s returning HTTP 200.","keywords":["URI redirection","redirect chains","canonical redirect","soft 404","sink URI","web archiving","link rot","HTTP status codes"],"falsifier":"Re-run the same ten-hop redirect crawl on an independent, live-web sample—for example, a random sample of registered domains or a fresh URL index rather than an archive's historical index—and compare the success/error split and the fraction of custom 404s returning 200; if the 50/50 split and the 47% soft-404 rate do not approximately reproduce, the archive-derived dataset cannot be treated as representative of the broader web.","tokens_in":17901,"feed_emoji":"🔀","tokens_out":8475,"duration_ms":83226,"temperature":0.7,"pith_summary":"This paper tries to establish, at web scale, what actually happens when a URI redirects: whether the chain reaches a live page, where it fails, and where the traffic lands. Using 11 million redirecting URIs sampled from a large web archive and re-crawled in 2023 with a ten-hop limit, the authors find that about half of the chains terminate successfully and half end in errors, with 0.06% exceeding ten hops. They also show that canonical redirects—mostly HTTP to HTTPS transitions—dominate the landscape, that a small set of 'sink' URIs absorb a disproportionate share of redirect traffic, and that among 62,000 custom 404 pages, nearly half are 'soft 404s' returning HTTP 200 instead of 404. The result matters because redirect behavior determines whether preserved links, search-engine value, and user navigation actually reach the intended content.","feed_headline":"Half of web redirects fail, and error pages often hide it","feed_subtitle":"A 9.5-million-URI crawl finds a 50/50 success/error split and soft 404s on nearly half of custom error pages.","key_machinery":"The load-bearing object is the redirect chain, recorded hop by hop by a web crawler following each URI up to ten hops and labelling the final HTTP status as success, error, or indeterminate. The core classification device is the SURT (Sort-friendly URI Reordering Transform), a canonicalized key for URIs: when the SURT forms of source and target match exactly, the redirect is labelled canonical; otherwise it is non-canonical. Redirects the crawler cannot follow—because of invalid Location headers or client-side meta-refresh redirects—are labelled invalid and set aside as indeterminate. Sink URIs are then computed by aggregating source URIs and source domains by their shared target URI. Together these mechanisms carry the analysis: SURT enables the canonical/non-canonical split, the hop-by-hop crawl logs produce the success/error counts, and sink aggregation exposes the traffic-convergence patterns.","core_discovery":"In the paper's own terms, the central discovery is a quantitative map of redirect behaviour in the archived web. Following 9.5 million terminating redirect chains up to ten hops, the authors find a near-exact 50/50 split between successful (2xx) and failed terminations, with 0.06% of chains exceeding ten hops and 0.42% exceeding four hops. Canonical redirects account for roughly two-thirds of terminating chains (6 million of 9.5 million), and HTTP-to-HTTPS transitions alone make up about 4.6 million cases; these canonical chains resolve to a 2xx status only 48.70% of the time. Non-canonical redirects resolve to 2xx more often (about 73%) but frequently land on root pages, parked pages, or soft-error pages rather than the originally linked content. The authors further identify 'sink' URIs that collect redirects from many sources, and they find that of 62,000 custom 404 target URIs, only 46% actually return 404 while 47% return 200, meaning broken links are often masked as successes.","pith_inferences":["The 50/50 success/error split probably reflects the age and provenance of archive-derived URIs; a sample of freshly created live-web URIs would likely show a healthier split, so the paper's numbers are best read as a lower bound on current redirect health rather than a universal snapshot.","Sink analysis suggests a cheap content-decay detector: a redirect landing on a known sink page (login wall, custom 404, conglomerate homepage) can flag link rot without parsing page text; this heuristic could be tested against manual content review.","The affinity of soft 404s with custom error URIs implies a crawl-time heuristic—if the target path contains '404' or 'error' but returns 200, flag it as a likely soft 404—whose false-positive rate could be measured in a follow-up study.","The Rickrolling sink and affiliate-marketing sinks show that deliberate redirects can be distinguished from accidental ones by sink concentration and source-domain relationships, which could be used to surface prank or affiliate networks at scale."],"forward_implications":["A five-hop redirect cap—already the recommended practice—covers all but 0.42% of chains, so crawlers can budget resources with that bound in mind.","Because 47% of custom 404 pages return HTTP 200, link-rot and SEO audits that rely on status codes alone will systematically undercount broken pages; final page content must be inspected.","The concentration of traffic into sink URIs (login pages, custom 404s, parked domains, homepages) means web archives can waste storage on duplicates; archive crawlers could detect and deduplicate sinks.","The 13.22% of redirects with indeterminate termination (invalid or client-side redirects) form a class that crawler designs must handle explicitly rather than treating as ordinary 3xx chains."],"supporting_citations":[{"why":"Supplies the 27.3-million-URI archived sample from which the 11 million redirecting URIs are drawn.","marker":"[19]"},{"why":"Describes the index structure used to sample the archived URIs.","marker":"[41]"},{"why":"The crawler used to follow redirects and record hop-by-hop status codes.","marker":"[5]"},{"why":"Defines the TimeMap/SURT structure used to canonicalize URIs for the canonical versus non-canonical split.","marker":"[44]"},{"why":"Introduces the soft-error concept that underlies the paper's soft-404 analysis.","marker":"[33]"},{"why":"The five-hop redirect recommendation against which the hop-count results are compared.","marker":"[42]"},{"why":"Prior result that URI canonicalization distorts archival memento counts, motivating the canonical/non-canonical distinction.","marker":"[26]"}],"fun_headline_variants":["Redirects: half fail, and 47% of error pages return 200","Half of web redirects fail; soft 404s hide broken links","9.5M redirect chains: 50% fail, some end in Rickrolls"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole measurement rests on the assumption that URIs first archived between 1996 and 2021 and drawn from one archive's index are representative of redirect behavior on the web at large; if the sample skews toward abandoned or low-cost sites, the error and soft-404 rates will be overstated.","fun_headline_variants_meta":{"raw":{"variants":["Redirects: half fail, and 47% of error pages return 200","Half of web redirects fail; soft 404s hide broken links","9.5M redirect chains: 50% fail, some end in Rickrolls"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000819,"raw_usage":{"total_tokens":3656,"prompt_tokens":1085,"completion_tokens":2571,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":701,"completion_tokens_details":{"reasoning_tokens":2503}},"tokens_in":701,"tokens_out":2571,"duration_ms":23125,"temperature":1.0,"reasoning_tokens":2503,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T12:07:18.533869+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same ten-hop redirect crawl on an independent, live-web sample—for example, a random sample of registered domains or a fresh URL index rather than an archive's historical index—and compare the success/error split and the fraction of custom 404s returning 200; if the 50/50 split and the 47% soft-404 rate do not approximately reproduce, the archive-derived dataset cannot be treated as representative of the broader web.","supporting_citations":[{"cited_title":"2014–2021","cited_arxiv_id":null,"evidence_quote":"Describes the index structure used to sample the archived URIs."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The crawler used to follow redirects and record hop-by-hop status codes."},{"cited_title":"Nelson, and Robert Sanderson","cited_arxiv_id":null,"evidence_quote":"Defines the TimeMap/SURT structure used to canonicalize URIs for the canonical versus non-canonical split."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the soft-error concept that underlies the paper's soft-404 analysis."},{"cited_title":"Southern","cited_arxiv_id":null,"evidence_quote":"The five-hop redirect recommendation against which the hop-count results are compared."},{"cited_title":"Alkwai, Sawood Alam, Michael L","cited_arxiv_id":null,"evidence_quote":"Prior result that URI canonicalization distorts archival memento counts, motivating the canonical/non-canonical distinction."}],"review_version":1}