{"id":"fec3f066-c748-468a-995d-95e813548374","arxiv_id":"2502.09726","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"An experimental comparison finds that encrypted DNS protocols such as DoH and DoT add latency and traffic versus plain DNS, with DoH the slowest and most secure in this testbed.","lead":"This paper measures how much slower and heavier five DNS configurations (plain DNS, DNSSEC, DoT, DoH, and DoT plus DNSSEC) become when run through an edge-based DNS resolver meant to serve IoT devices. It reports that more encryption means more latency and traffic, and recommends that constrained devices use lighter protocols despite the security trade-off.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The IoT relevance claim rests on server-side VNF measurements with no constrained client, so the reported latency/traffic ordering may not transfer to IoT devices.","rationale":"The reader's weakest assumption identifies exactly the load-bearing threat: VNF/DNSPerf measurements are treated as representative of resource-constrained IoT clients. I agree. The measured server-side ordering is internally plausible and the paper is transparent about many limitations, so the conditional verdict is appropriate. Secondary reporting issues (the contradictory request-overhead sentence in Section III.A.3 and the unsupported pre-cache latency claim in the conclusions) are real but are readability/data-presentation defects rather than independent validity threats; they can be fixed with figures and tables. No fabricated results are apparent, so no rejection is warranted.","tokens_in":9621,"tokens_out":6716,"duration_ms":66175,"concrete_test":"Run an actual constrained IoT client (e.g., ESP32 or Raspberry Pi Zero W) against the same edge BIND9 VNF and measure end-to-end resolution latency, client CPU/memory, and per-query energy for DNS, DNSSEC, DoT, DoT+DNSSEC, and DoH. If the post-cache ordering and the 'least suitable' conclusion remain unchanged, the server-side proxy is validated; if gaps shrink or reverse, the central claim overreaches.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central contribution is to help IoT stakeholders choose DNS protocols, but the experimental setup uses DNSPerf to query a BIND9 VNF at the edge. DNSPerf is a server load generator; it measures resolver response time and server-side traffic, not the client-side cost on a resource-constrained IoT device. DoT/DoH and DoT+DNSSEC impose their heaviest costs on the client: TLS/HTTPS/QUIC handshakes and per-query cryptographic record processing on a 100-400 MHz MCU with kilobytes of RAM. Section II.B itself describes these constraints, yet no constrained device appears in the measurements. Thus the claimed ordering (DNS < DNSSEC < DoT < DoT+DNSSEC < DoH) and the conclusion that DoH is least suitable for resource-constrained IoT are assertions about server performance, not about IoT behavior. Section IV lists energy consumption and scalability as future work, confirming this gap. This is a validity threat to the central claim, not a minor scope note, because the paper's stated purpose is to guide IoT protocol selection.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper compares the performance of traditional DNS, DNSSEC, DoT, DoT+DNSSEC, and DoH using a BIND9 DNS resolver deployed as a Virtual Network Function (VNF) at the edge. It reports average post-cache latency, relative latency increases, and request/response traffic sizes, and it introduces a 0-4 security scoring rubric to quantify the security benefit of each protocol. The stated goal is to help consumers, manufacturers, and researchers choose robust DNS protocols for resource-constrained IoT devices. The main empirical findings are that traditional DNS has the lowest latency and traffic overhead, DoH has the highest latency and request overhead, and DoT+DNSSEC has the highest response overhead.","tokens_in":9780,"tokens_out":3316,"duration_ms":34387,"significance":"If the measurements are valid, the qualitative ordering of post-cache latency (DNS < DNSSEC < DoT < DoT+DNSSEC < DoH) is plausible and consistent with the protocol overheads imposed by TLS/HTTPS session establishment and DNSSEC record processing. The paper also provides a simple, transparent security rubric and a reproducible measurement pattern using widely available tools (BIND9, DNSPerf), which could be useful to edge DNS operators. However, the significance for IoT specifically is currently limited because the experiments evaluate a server-side resolver with a load generator, not a resource-constrained client stack; the paper's own Section IV lists energy consumption and scalability as future work. The contribution is therefore best assessed as a preliminary edge-resolver performance comparison rather than a direct IoT-device evaluation.","major_comments":[{"comment":"The central IoT-facing claim is not supported by the experimental setup. Section I.A states that the edge DNS service emulates resource-constrained devices, and Section II.B enumerates the constraints of such devices (100-400 MHz CPUs, kilobytes to megabytes of RAM, low-power links), but the experiments in Section III use DNSPerf to query a BIND9 VNF, which measures server-side resolver latency and aggregate network traffic, not client-side computation, memory use, energy consumption, or protocol behavior on constrained hardware. DNSPerf is a load generator; it does not emulate an IoT client. Section IV confirms that energy consumption and scalability are left to future work. Consequently, the conclusion that DoH is 'least suitable' for resource-constrained IoT environments (Section III.A.2) and the protocol-choice guidance in Section V are not directly derivable from the presented data. This can be addressed either by repeating or extending the measurements on IoT-class hardware (or a constrained client stack), or by explicitly reframing the paper's claims to edge DNS resolver performance and removing or softening the IoT-suitability conclusions.","section":"Sections I.A, II.B, III, and IV"},{"comment":"The security score for DoT+DNSSEC is internally inconsistent. The rubric assigns one point per property (encryption, authentication, integrity, advanced privacy), so the total must be an integer. Table I lists DoT+DNSSEC as 3.5, yet the text in Section III.A.1 says 'the overall security score remains in the three-point range.' Since DoT+DNSSEC provides encryption (DoT), authentication and integrity (TLS plus DNSSEC), but no advanced privacy protection, the correct score under the rubric should be 3. This inconsistency affects the security-benefit axis of the advertised security/performance trade-off and must be corrected.","section":"Table I and Section III.A.1"},{"comment":"Important experimental details are missing, and one reported claim has no corresponding data. Table I reports only 'Average Latency after Cache,' but the conclusion in Section V states that before caching, traditional DNS, DNSSEC, and DoT had comparable latency while DoT+DNSSEC and DoH had significantly higher latency. No pre-cache latency values are presented anywhere in the paper. In addition, the paper does not report the number of trials, standard deviations or confidence intervals, hardware specifications of the VNF host, BIND9 version, or DNSPerf parameters, making it impossible to assess whether the observed differences (e.g., 0.199 ms vs. 0.103 ms) are statistically meaningful. These details should be added.","section":"Section III.A and Section V"},{"comment":"The traffic results are stated inconsistently. The text says 'DoT+DNSSEC has the highest total overhead when sending requests, while DoH has the highest overhead for requests,' which is contradictory, and it does not match the conclusion in Section V that 'DoH has the highest overhead when sending requests' while 'DoT+DNSSEC causes the highest overhead' in response. The manuscript should clearly distinguish request overhead from response overhead and align the text, Figure 2, and the conclusions.","section":"Section III.A.3 and Figure 2"}],"minor_comments":[{"comment":"The phrase 'performance loss' for DNSSEC (93.2%) is actually a relative increase in latency compared to traditional DNS; consider using 'latency increase' or 'relative latency overhead' to avoid implying that a larger percentage is always a security benefit.","section":"Section III.A.2"},{"comment":"The enumeration of strategies for limitations repeats '(ii)' twice; the items should be numbered (i) through (iv).","section":"Section II.C"},{"comment":"Table II includes DoQ and DoC with qualitative ratings, but no experiments for these protocols are reported in the paper. It should be stated explicitly that these entries are qualitative assessments based on the literature, not measured results.","section":"Table II"},{"comment":"Reference [5], the Name:Wreck report, is cited without the authors' full publication venue; the Forescout Research Labs report is mentioned in the text but the reference entry is incomplete.","section":"References"},{"comment":"Figure 2 would benefit from explicit axis labels and a legend specifying which bars are requests and which are responses, since the text's description of the figure is currently ambiguous.","section":"Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The paper has a useful and plausibly correct server-side comparison, but the gap between the measurements and the IoT-specific conclusions is the main risk. A revision that either adds constrained-client measurements or carefully reframes the claims to edge-resolver performance could make the paper acceptable. The security-score inconsistency in Table I should be fixed as part of that revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a small, internally consistent measurement set that adds a modest new data point to the DNS security literature, but the IoT framing overreaches what the testbed can actually support. Worth a serious referee, with revisions.\n\nThe genuinely new thing is the specific combination: DNSSEC, DoT, DoT+DNSSEC, and DoH measured through an edge VNF resolver using BIND9 with three forwarders and five regional domains. I checked the cited related work, and that exact configuration is not there. The arithmetic in Table I checks out, and the latency ordering (DNS < DNSSEC < DoT < DoT+DNSSEC < DoH) is plausible and consistent with prior studies. The paper also cites the relevant DoC and DoQ work, and it is honest enough to list energy and scalability as future work. I see no fabrication or intentional distortion.\n\nThe soft spots are real but not fatal. First, the measurements are server-side: DNSPerf hammers a BIND9 VNF, and the paper draws conclusions about resource-constrained IoT devices even though no constrained hardware, no client-side TLS handshake cost, and no energy or memory measurement appear anywhere. The stress-test note is right that this is a validity threat to the central claim, not a minor scope remark. The paper's own Section II.B describes the constraints an IoT device faces, then Section III measures something else. The conclusion that DoH is \"least suitable\" for resource-constrained IoT is an overstatement when the post-cache latency differences are all sub-millisecond. Second, the paper reports averages without variance, trial counts, hardware specs, or the pre-cache latency values that the conclusions reference. That is fixable by releasing the scripts and raw data. Third, the 0-4 security score rubric is author-defined and a little arbitrary (DoT+DNSSEC gets 3.5, DoH gets 4), but it does not influence the latency or traffic numbers, so circularity is not a real issue.\n\nWho is this for? Practitioners who want a rough ordering of DNS protocol overhead at an edge resolver, and researchers doing DNS performance measurement. It is not a device-level engineering study, and it should not be read as one until constrained-device measurements exist.\n\nRecommendation: engage with it. Send it to a referee who understands both DNS measurement and IoT constraints, and ask for the missing artifacts and a re-scoped title or conclusion. The paper deserves revision rather than rejection.","headline":"Honest but modest DNS performance comparison that overclaims IoT relevance; worthwhile referee with requests for data and a re-scoped framing.","tokens_in":10352,"tokens_out":1619,"would_cite":false,"duration_ms":18313,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper compares secure DNS protocols for IoT edge resolvers and finds that each step of added security costs measurable latency.","keywords":["DNS","IoT","DNSSEC","DNS over TLS","DNS over HTTPS","edge computing","virtual network function","performance measurement"],"falsifier":"Run the same five protocol configurations on an actual constrained microcontroller (for example, an 8-bit device with a few hundred kilobytes of RAM) with a real DNS client, and measure per-query latency, memory usage, and energy consumption; if the post-cache ordering changes, or if DoH's energy cost does not exceed DoT's, then the paper's ranking for IoT suitability does not transfer to real devices.","tokens_in":9364,"feed_emoji":"📡","tokens_out":3506,"duration_ms":35464,"temperature":0.7,"pith_summary":"This paper tries to establish how much performance overhead each secure DNS protocol imposes when used by resource-constrained IoT devices. It argues that the choice of DNS protocol is a concrete trade-off between security benefits and latency, and that this trade-off can be quantified. The authors measure traditional DNS, DNSSEC, DoT, DoT+DNSSEC, and DoH on an edge DNS resolver implemented as a Virtual Network Function, using response time and network traffic as metrics. A sympathetic reader would care because IoT devices have limited processing, memory, and energy, and the paper provides a practical ordering of protocols to guide deployment decisions.","feed_headline":"Cached DNS lookups run 7 times slower under DoH, edge tests find","feed_subtitle":"Edge-resolver measurements rank plain DNS fastest and DoH slowest, with security scores to match the latency cost.","key_machinery":"The central object is a testbed built around an edge DNS resolver implemented as a Virtual Network Function using BIND9, with DNSPerf generating queries and measuring latency. The resolver uses external forwarders (Google, Cloudflare, and OpenDNS) for non-local domains and local zone files for cached responses. The key measurements are average response latency before and after caching, total network traffic in bytes for requests and responses, and a four-point security scoring scheme that credits encryption, authentication, integrity, and advanced privacy protection.","core_discovery":"The paper claims that, for an edge-hosted DNS resolver, secure DNS protocols introduce substantial latency compared with traditional DNS, and that the latency grows with the level of security. After caching, average response latencies are 0.103 ms for traditional DNS, 0.199 ms for DNSSEC, 0.600 ms for DoT, 0.657 ms for DoT+DNSSEC, and 0.728 ms for DoH, corresponding to performance losses of roughly 93%, 482%, 538%, and 607% relative to plain DNS. The paper also assigns security scores from 0 to 4, with plain DNS at 0, DNSSEC at 2, DoT at 3, DoT+DNSSEC at 3.5, and DoH at 4. In terms of network traffic, DoH has the highest overhead when sending requests and DoT+DNSSEC has the highest response overhead, while traditional DNS has the lowest in both cases. The paper concludes that secure DNS protocols carry a real performance cost that must be weighed against their privacy and integrity benefits for resource-constrained IoT environments.","pith_inferences":["The measurements were taken on a server-side VNF queried by DNSPerf, not on actual constrained IoT hardware; on real low-power microcontrollers, the computational cost of TLS and signature verification could make the latency gaps larger than reported.","Because the paper does not measure energy, memory, or CPU usage, the latency ordering alone may not predict battery life; a protocol with lower latency but higher per-packet computation could drain a battery faster than a slower protocol.","The paper leaves DoQ and DoC untested, and DoC is designed specifically for constrained devices, so its latency and traffic profile could fall below DoT and potentially alter the recommended protocol choice.","A direct extension would be to re-run the same comparison with an actual constrained device as the client, or to add energy consumption as a metric, which would make the trade-off actionable for battery-powered deployments."],"forward_implications":["If the measured ordering holds, IoT applications that need fast responses should prefer traditional DNS or DNSSEC over DoT, DoT+DNSSEC, or DoH when operating behind an edge resolver.","The latency penalty of secure DNS is large even with caching, so the common practice of caching DNS responses at the edge does not erase the cost of encryption and signing.","The traffic measurements suggest that even the most overhead-heavy secure DNS protocols keep packet exchanges below roughly 6,000 bytes, so bandwidth is less of a constraint than latency for many IoT scenarios.","The security score table gives device makers a direct way to trade off each additional security point against a measured increase in response time."],"supporting_citations":[{"why":"Frames the challenges of running DNS in IoT environments and motivates the need for constrained-device evaluation.","marker":"[3]"},{"why":"Documents Name:Wreck vulnerabilities in DNS implementations, justifying the focus on secure DNS for resource-constrained devices.","marker":"[5]"},{"why":"Prior study of DoH cost that this paper extends by including DNSSEC and DoT variants at the edge.","marker":"[11]"},{"why":"Earlier comparison of DNS, DoT, and DoH under network conditions, providing baseline context for the protocol trade-offs.","marker":"[12]"},{"why":"Introduces DNS over CoAP for constrained devices, which the paper cites as the best option for highly constrained IoT.","marker":"[13]"},{"why":"Evaluates DNS over QUIC, providing the reference point for the paper's discussion of DoQ as a future alternative.","marker":"[15]"}],"fun_headline_variants":["DoH slows cached DNS 7x, edge tests show","Secure DNS has a speed cost: DoH is 7x slower","Edge test ranks DNS: plain fastest, DoH 7x behind","For IoT, DoH costs 7x in DNS speed, edge study says","DoH scores top security but runs 7x slower on cache"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a server-side edge DNS resolver queried by a synthetic load generator (DNSPerf) captures the experience of resource-constrained IoT devices; the paper does not run any IoT-class hardware, so latency and byte-count results are treated as proxies for device-level cost.","fun_headline_variants_meta":{"raw":{"variants":["DoH slows cached DNS 7x, edge tests show","Secure DNS has a speed cost: DoH is 7x slower","Edge test ranks DNS: plain fastest, DoH 7x behind","For IoT, DoH costs 7x in DNS speed, edge study says","DoH scores top security but runs 7x slower on cache"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000644,"raw_usage":{"total_tokens":2971,"prompt_tokens":964,"completion_tokens":2007,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":580,"completion_tokens_details":{"reasoning_tokens":1926}},"tokens_in":580,"tokens_out":2007,"duration_ms":15090,"temperature":1.0,"reasoning_tokens":1926,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T20:41:18.566028+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same five protocol configurations on an actual constrained microcontroller (for example, an 8-bit device with a few hundred kilobytes of RAM) with a real DNS client, and measure per-query latency, memory usage, and energy consumption; if the post-cache ordering changes, or if DoH's energy cost does not exceed DoT's, then the paper's ranking for IoT suitability does not transfer to real devices.","supporting_citations":[{"cited_title":"The dns in iot: Opportunities, risks, and challenges,","cited_arxiv_id":null,"evidence_quote":"Frames the challenges of running DNS in IoT environments and motivates the need for constrained-device evaluation."},{"cited_title":"Name: Wreck-breaking and fixing dns implementations,","cited_arxiv_id":null,"evidence_quote":"Documents Name:Wreck vulnerabilities in DNS implementations, justifying the focus on secure DNS for resource-constrained devices."},{"cited_title":"An empirical study of the cost of dns-over-https,","cited_arxiv_id":null,"evidence_quote":"Prior study of DoH cost that this paper extends by including DNSSEC and DoT variants at the edge."},{"cited_title":"Comparing the effects of dns, dot, and doh on web performance,","cited_arxiv_id":null,"evidence_quote":"Earlier comparison of DNS, DoT, and DoH under network conditions, providing baseline context for the protocol trade-offs."},{"cited_title":"Securing name resolution in the iot: Dns over coap,","cited_arxiv_id":null,"evidence_quote":"Introduces DNS over CoAP for constrained devices, which the paper cites as the best option for highly constrained IoT."},{"cited_title":"Dns privacy with speed? evaluating dns over quic and its impact on web performance,","cited_arxiv_id":null,"evidence_quote":"Evaluates DNS over QUIC, providing the reference point for the paper's discussion of DoQ as a future alternative."}],"review_version":1}