Pith. sign in

REVIEW 4 major objections 4 minor 20 references

Vulnerabilities, Secrets and Misconfiguration in the Highest-Exposure Docker Hub Images

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Pith's one-line read The most-pulled Docker Hub images almost all carry known vulnerabilities, but which ones and how many depends heavily on which scanner you use.

desk verdict Major Docker Hub measurement with reproducible artifacts; the scanner-agreement headline needs a Trivy-failure sensitivity check before the numbers are cited. read the letter →

arxiv 2608.02669 v1 pith:COTZISY7 submitted 2026-08-02 cs.CR cs.AIcs.CYcs.OScs.SE

classification cs.CRcs.AIcs.CYcs.OScs.SE
keywords DockerHubcontainersecuritysoftwaresupplychainvulnerabilityscannersCISBenchmarksecretdetectionexposurescorelayergraph
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish what the security posture of the most-used Docker Hub images actually is, and how much that posture is an artifact of the tool used to measure it. Crawling 12.7 million repositories and reconstructing a layer inheritance graph of 54.4 million edges, the authors rank images by an exposure score and scan the top 52,895 repositories, which cover 84.7% of all recorded pulls, with six independent scanners. They find known package vulnerabilities in 96.3% of images, critical ones in 93.4%, and CIS Docker Benchmark misconfigurations in 98.0%. Yet of 80.7 million distinct (vulnerability, package) pairs, only 2.7% are flagged by all three vulnerability scanners and 66.8% by exactly one; the best single scanner recovers just 66.9%. Raw secret-detector hits are similarly misleading: TruffleHog flags 76.9% of images, but hand-labeling 1,100 detections shows 99.7% are not credentials. The paper's point is that a one-tool measurement of the container ecosystem is not reliably a measurement of the ecosystem, and that exposure-weighted reach, not image count, is what makes a single inherited vulnerability dangerous.

What carries the argument

The exposure score E(I) = p(R_I) plus the sum of p(R_D) over downstream images D owned by I, computed on a layer inheritance forest of 84.7 million nodes and 54.4 million IS_BASE_OF edges. It linearly orders the scan queue by total ecosystem reach, so the highest-exposure repositories are scanned first. The six-scanner battery (Syft, Trivy, Grype, OSV-Scanner, Dockle, TruffleHog) covers software inventory, package vulnerabilities from three databases, CIS Docker Benchmark misconfigurations, and hardcoded secrets; findings are consolidated per image but not deduplicated across scanners, which is what makes inter-scanner agreement measurable.

What would settle it

Resolve Stage II for all 12.7 million repositories, recompute exposure scores, and check whether CVE-2026-27171 still reaches images holding 47.3% of total corpus exposure and 1.13 million distinct downstream images; if the top-52,895 set changes materially or the propagation count moves, the reported exposure-weighted figures would need revision. A cheaper check is to scan a random sample of the unresolved tail and look for rarely pulled repositories that appear as base layers of high-exposure images.

Watch

Extended reading notes

Core claim

The central claim is that vulnerability prevalence among the highest-exposure Docker Hub images is near-universal, but the reported posture is largely a property of the scanner: only a small minority of distinct vulnerabilities are confirmed by more than one database-backed scanner, while the raw secret-detection prevalence collapses to roughly 0.3% plausible credentials after hand-labeling. The paper also claims that exposure, defined as an image's own pull count plus the pull counts of every downstream image that inherits its layers, ranks the damage a flaw can do: a single zlib vulnerability reaches images carrying 47.3% of total corpus exposure and propagates to 1.13 million distinct dow

Load-bearing premise

The exposure ranking and propagation counts are computed from only the 44.05% of crawled repositories that Stage II resolved, namely the popularity head down to roughly 72 pulls, so the most-consequential-image ranking and downstream reach are lower bounds that could shift if the unresolved tail contained widely reused base layers.

Editorial extensions

If this is right

  • CI and audit pipelines that trust one vulnerability scanner will miss roughly one distinct vulnerability in three; running at least two scanners backed by different databases is the direct implication.
  • Registry operators can order remediation by exposure score: fixing the base image whose flaws reach the most downstream pulls targets the largest share of inherited exposure, rather than the largest number of affected images.
  • Raw secret-detector hits should not gate deployments; with 99.7% of hits being non-credentials, a validation step must precede any action on them.
  • Misconfigurations from the CIS Docker Benchmark are cheap to check and nearly universal, so enforcing content-trust, HEALTHCHECK, and non-root checks in CI is a practical low-cost control.
  • Because exposure does not predict vulnerability, popularity is not a proxy for maintenance quality at the top of the registry.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: If scanner disagreement is as high in the broader population as in this top-exposure corpus, any single-scanner longitudinal comparison across years may be confounded by scanner changes as much as by real ecosystem change.
  • Inference: Exposure-weighted CVE ranking could be turned into a live registry-level alerting signal: recompute exposure as new pulls arrive and rank newly disclosed CVEs by exposure share, giving operators an automated triage order.
  • Inference: The 99.7% non-credential rate applies to raw detector hits; validating the residual ~0.3% with active credential-confirmation services could yield a small but nonzero validated-secret prevalence, which the paper leaves to future work.
  • Inference: Because the propagation graph covers only the 44% of repositories Stage II resolved, the 1.13 million downstream count for a single zlib CVE is a structural lower bound; resolving the remaining 56% would most likely increase it, not decrease it.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This paper presents ChimangoScan, a three-stage measurement pipeline that crawls Docker Hub, resolves 5.6M repositories into images and a layer forest (54.4M IS_BASE_OF edges), ranks images by a new exposure score E(I) that combines an image's own pulls with the pulls of its downstream subtree, and scans the 52,895 highest-exposure repositories with six open-source scanners. The central empirical claims are: (i) known vulnerabilities are near-universal (96.3% of images, 93.4% with a critical finding) and CIS misconfigurations affect 98.0%; (ii) inter-scanner agreement is very low—66.8% of 80.7M distinct (vulnerability, package) groups are flagged by only one of three vulnerability scanners and 2.7% by all three, so a single-scanner count is largely tool-dependent; (iii) TruffleHog flags secrets in 76.9% of images, but hand-labeling 1,100 detections indicates 99.7% are non-credentials; (iv) exposure-weighted CVE reach is highly concentrated, with a zlib CVE reaching 47.3% of total corpus exposure and propagating to about 1.13M downstream images; and (v) pull count/exposure do not predict vulnerability counts. The pipeline and a 283 GB dataset are released.

Significance. If the scanner-agreement and prevalence results survive the concerns below, this is a substantial empirical contribution. It is the largest multi-scanner comparison of Docker Hub images by a wide margin over prior samples, it quantifies tool-dependent measurement error in a way previous single-scanner ecosystem studies could not, and the exposure metric is a useful conceptual addition. The public release of the pipeline and dataset, the careful hand-labeling of secret detections, and the explicit lower-bound framing of propagation counts are genuine strengths. The weaknesses are localized to specific statistical and reporting decisions rather than the overall architecture, and most appear addressable by re-analysis or reframing.

major comments (4)
  1. [§IV-C and Table V] The agreement metrics (66.8% single-scanner, 2.7% all-three, 66.9% best single) are reported without stating how the 10,614 Trivy invocations that errored (Table V: Run OK 79.9%) are treated. If these images are included with Trivy treated as having no findings, then every (vulnerability, package) group detected by Grype or OSV-Scanner in those images automatically becomes a one-scanner group, inflating 66.8% and deflating 2.7%. Since the abstract's central claim—'the posture a single tool reports is largely an artifact of that tool'—rests directly on these percentages, the paper must report the same statistics on the complete-case subset (images where all three scanners succeeded) or explicitly state how failures were handled. This is not merely a robustness check; it changes the headline.
  2. [§IV-C] The agreement analysis groups findings by '(vulnerability, package)' but does not describe any canonicalization of package names or vulnerability identifiers across Trivy, Grype, and OSV-Scanner. These tools consume different databases (NVD/OSS-Index, Anchore VulnDB, OSV) and often express the same defect under different package strings (e.g., distribution package name vs. upstream name) or different advisory identifiers. Without normalization, the 66.8% 'one-scanner' share may overstate genuine disagreement because a matching defect is counted as two distinct groups. Please specify the exact grouping key, any normalization applied, and ideally a sensitivity check using CVE-only keys or an alias-matching step.
  3. [§IV-F] The claim that 'exposure does not predict how vulnerable an image is' is based on Spearman correlations computed on the scanned corpus, which is deliberately the top of the exposure distribution. The corpus is selected on the very variable whose predictive power is being tested; range restriction attenuates correlations and can produce near-zero estimates even when a relationship exists in the full population. The paper should either restrict the claim to 'among the highest-exposure images' or provide a selection-corrected estimate (e.g., using the lower-exposure images already resolved in Stage II but not scanned). As written, the abstract's 'exposure does not predict how vulnerable an image is' overreaches.
  4. [§III-B/IV-G] Only 44.05% of crawled repositories are resolved in the layer graph, down to roughly 72 pulls. While Table VIII's propagation counts are correctly framed as structural lower bounds, the exposure ranking itself—which defines the scanned 'highest-exposure' set and the exposure-weighted CVE shares in Table VII—is computed on this partial graph and is not labeled as approximate. If any omitted low-pull repositories are bases for large downstream subtrees (their descendants would be resolved, but their own top-layer image node would be absent), the queue composition and exposure shares could shift. Please add a bound or sensitivity analysis, or explicitly restate exposure-ranking claims as conditional on the resolved subgraph.
minor comments (4)
  1. [§III-A, Table II] 'Table II4' appears to be a formatting artifact; it should read 'Table II, fourth column' or similar.
  2. [§IV-D] The Wilson 95% CI for 1097/1100 non-credentials is approximately 99.1–99.8%, not 99.2–99.9% as stated; please recheck the interval.
  3. [§III-E] No scanner versions or vulnerability-feed dates are given. For reproducibility, list versions of Syft, Trivy, Grype, OSV-Scanner, Dockle, and TruffleHog, together with the date of the database snapshots.
  4. [Figure 4(a)] The Venn-like diagram's internal numbers are not all labeled by scanner subset; adding explicit subset labels would make the agreement figure easier to interpret.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: all headline quantities are direct measurements with disclosed limitations; no derivation reduces to its inputs.

full rationale

The paper is an empirical measurement study, not a derivation chain. Its central results—96.3% vulnerability prevalence, 66.8%/2.7% scanner-divergence shares, 66.9% best-single-scanner recovery, the 99.7% TruffleHog false-positive rate, and the 1.13-million-image downstream propagation counts—are all computed from scanner outputs, the reconstructed layer graph, and hand-labeled samples; none is fitted to a target and then re-derived as a prediction. The exposure score E(I) in Eq. (1) is an explicit definition combining pull counts and downstream subtree ownership, not a parameter inferred from vulnerability counts, so the later finding that exposure does not predict vulnerability (Section IV-F, ρ≈−0.02) is an independently observable correlation rather than a tautology. The paper's stated methodological debts to Dr. Docker [16]—the cap-refining crawl and ancestry-hashed layer identifier—are credited and are not used to establish the measurement conclusions. The limitations the paper itself discloses, such as Trivy completing on only 79.9% of images (Section IV-B, Table V) and Stage II resolving 44.05% of crawled repositories so that propagation counts are structural lower bounds (Sections III-B and IV-G), are completeness/robustness concerns, not circularity: even if the Trivy failure handling affected the scanner-agreement percentages, that would be a data-quality issue, not an identity between a model output and its input. There is no load-bearing self-citation, no imported uniqueness theorem, and no ansatz smuggled in via citation; the measurement claims stand on the released dataset and pipeline rather than on an argument that reduces to its own premises.

Assumptions & free parameters 6 free parameters · 5 assumptions · 2 invented entities

The paper is an empirical measurement, so the free parameters are mostly design choices for the exposure metric and scan scope, not fitted constants. The axioms are the background assumptions any such measurement makes about the Docker Hub API, scanner databases, and the partial layer graph. The central claim does not rest on a fitted model, so the circularity burden is low.

free parameters (6)
  • Stage II resolution cutoff = ~72 pulls
    Stage II resolved only the most-pulled repositories down to roughly 72 pulls; the layer graph and exposure scores are computed on this 44.05% subset.
  • Scan queue cutoff = 52,895 repositories
    The number of highest-exposure repositories scanned; covers 84.7% of recorded pulls. A practical limit, not a fitted constant.
  • Single-owner attribution rule
    Ad hoc heuristic: each image is attributed to the ancestor with the greatest own pull count, so downstream pulls are not double-counted in E(I).
  • Metadata-only layer barring rule
    Ad hoc rule: generic metadata-only layers (LABEL, CMD) are barred from owning downstream images, moving credit toward the more-pulled base.
  • Canonical most-pulled reference rule
    Byte-identical republications (bare FROM) are credited to the canonical most-pulled reference; affects exposure attribution.
  • TruffleHog severity rule
    Secrets are marked critical only when verified live, otherwise medium; affects severity distribution but not raw hits.
assumptions (5)
  • domain assumption Docker Hub API returns accurate pull counts, tags, and manifests.
    The entire crawl, pull-count distribution, and exposure ranking rest on the accuracy of the Docker Hub public API (Section III-A).
  • domain assumption Scanner vulnerability databases are the ground truth for known CVEs.
    The paper reports scanner findings as vulnerabilities without independent validation; this is standard for such measurements but is an assumption (Section III-D).
  • domain assumption Ancestry-hashed layer identifiers correctly encode layer inheritance.
    Adopted from Dr. Docker [16]; the IS_BASE_OF graph is constructed from these identifiers (Section III-B).
  • domain assumption The 1,100-detection hand-labeled sample is representative.
    Used to estimate the 99.7% secret false-positive rate; no inter-rater reliability is reported (Section IV-D).
  • domain assumption The Stage II-resolved layer graph is sufficient for exposure ranking and propagation counts.
    The omitted tail of rarely pulled repositories is assumed to have negligible impact on the top of the exposure distribution (Sections III-B, IV-G).
invented entities (2)
  • Exposure score E(I)
    purpose: Rank images by own pull count plus downstream subtree pulls; used to select the scan queue and compute exposure-weighted vulnerability shares.
    A methodological construct defined in this paper; it has no falsifiable handle outside this paper and is not an independently observable quantity.
  • Single-owner attribution
    purpose: Avoid double-counting downstream pulls when computing E(I).
    Ad hoc ownership rule; no external evidence supports it beyond the authors' design choice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Vulnerabilities, Secrets and Misconfiguration in the Highest-Exposure Docker Hub Images." pith.science (2026). https://pith.science/paper/COTZISY7

@misc{pith2026260802669,
  author       = {Pith},
  title        = {Pith review of: Vulnerabilities, Secrets and Misconfiguration in the Highest-Exposure Docker Hub Images},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/COTZISY7}},
  note         = {Machine review of arXiv:2608.02669}
}
read the original abstract

Docker Hub is the registry underneath most container deployments, and a flaw in a widely reused base image is inherited by every image built on it. Prior ecosystem-scale measurements each rely on a single detector, leaving the tool-dependence of their counts unquantified, while the studies that do compare scanners use samples of tens to hundreds of images. We present ChimangoScan, a pipeline that crawls the Docker Hub namespace (12,716,568 repositories, 663.8 billion cumulative pulls), reconstructs the image layer graph (54.4 million IS_BASE_OF edges), ranks images by an exposure score that folds an image's own pull count and those of its entire downstream subtree into one scalar, and scans the 52,895 highest-exposure repositories (84.7% of all recorded pulls) with six independent scanners, yielding 170.4 million findings. Vulnerabilities are near-universal: 96.3% of images carry a known package vulnerability, 93.4% a critical one, and 98.0% at least one CIS Docker Benchmark misconfiguration. The posture a single tool reports is largely an artifact of that tool: of 80.7 million distinct (vulnerability, package) groups, 66.8% are flagged by only one of the three vulnerability scanners and just 2.7% by all three, and the best single scanner recovers 66.9%. TruffleHog flags a secret in 76.9% of images, yet hand-labeling 1,100 random detections finds 99.7% are non-credentials. A single zlib CVE reaches images carrying 47.3% of total corpus exposure and propagates to 1.13 million distinct downstream images, but exposure does not predict how vulnerable an image is. We release the pipeline and the 283 GB dataset.

Figures

Figures reproduced from arXiv: 2608.02669 by the authors.

Figure 1
Figure 1. The three-stage pipeline. A. Stage I: Repository Discovery Stage I enumerates the Docker Hub namespace through the public Search API. Because the API caps any query at 10,000 results, a crawler must refine any keyword whose query saturates the cap (the cap-refining idea of Dr. Docker [16]); we realize this as a prefix-trie traversal of repository names, seeding 38 single-character prefixes (a to z, 0 to 9, -, _) and… view at source ↗
Figure 2
Figure 2. Per-image: (a) cumulative vulnerability distribution, (b) by scanner/severity, (c) scan time. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. (a) Components by ecosystem, (b) secrets by detector, (c) components per image. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Inter-scanner divergence: (a) groups by subset, (b) [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Secret detections per image (CDF): official vs. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Official vs. community: (a) vulnerabilities per image, (b) secret prevalence and share above median. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: (a) Vulnerabilities vs. pull count; (b) by exposure decile. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Known-vulnerability prevalence across studies, 2017 to 2026. [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: Reproductions: (a) official vs. community prevalence [10]; (b) severe findings by ecosystem [19]. [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 10
Figure 10. Figure 10: Shu et al. [17] reproduced: (a) worst severity, (b) CVEs by year, (c) vulns vs. image age. [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

20 extracted references · 19 canonical work pages

  1. [1]

    Security audit of Docker container images in cloud architecture

    Waheeda Syed Shameem Ahamed, Pavol Zavarsky, and Bobby Swar. Security audit of Docker container images in cloud architecture. InICSCCC 2021, pages 202–207. IEEE, 2021

  2. [2]

    Open Source, Open Threats? Investigating Security Challenges in Open-Source Software

    Seyed Ali Akhavani et al. Open source, open threats? investigating security challenges in open-source software. arXiv preprint arXiv:2506.12995, 2025

  3. [3]

    Vexed by VEX tools: Consistency evaluation of container vulnerability scanners

    Yekatierina Churakova et al. Vexed by VEX tools: Consistency evaluation of container vulnerability scanners. InFPS 2025, Lecture Notes in Computer Science, pages 139–156, 2025

  4. [4]

    An empirical analysis of the Docker container ecosystem on GitHub

    Jürgen Cito et al. An empirical analysis of the Docker container ecosystem on GitHub. InMSR 2017, pages 323–333. IEEE, 2017

  5. [5]

    Secrets revealed in container images: An internet-wide study on occurrence and impact

    Markus Dahlmanns et al. Secrets revealed in container images: An internet-wide study on occurrence and impact. InASIA CCS ’23, pages 797–811. ACM, 2023

  6. [6]

    Empirical study of the Docker smells impact on the image size

    Thomas Durieux. Empirical study of the Docker smells impact on the image size. InICSE 2024. ACM, 2024

  7. [7]

    Understanding the quality of container security vulnerability detection tools.arXiv preprint arXiv:2101.03844, 2021

    Omar Javed et al. Understanding the quality of container security vulnerability detection tools.arXiv preprint arXiv:2101.03844, 2021

  8. [8]

    An analysis of security vulnerabilities in container images for scientific data analysis.GigaScience, 10(6):giab025, 2021

    Bhupinder Kaur et al. An analysis of security vulnerabilities in container images for scientific data analysis.GigaScience, 10(6):giab025, 2021

Show all 20 references
  1. [9]

    SoK: Taxonomy of attacks on open-source software supply chains

    Piergiorgio Ladisa et al. SoK: Taxonomy of attacks on open-source software supply chains. InS&P 2023, pages 1509–1526. IEEE, 2023

  2. [10]

    Understanding the security risks of Docker Hub

    Peiyu Liu et al. Understanding the security risks of Docker Hub. InESORICS 2020, volume 12308 ofLecture Notes in Computer Science, pages 257–276. Springer, 2020

  3. [11]

    How bad can it git? characterizing secret leakage in public GitHub repositories

    Michael Meli et al. How bad can it git? characterizing secret leakage in public GitHub repositories. InNDSS 2019. Internet Society, 2019

  4. [12]

    Longitudinal risk-based security assessment of Docker software container images.Computers & Security, 135:103478, 2023

    Alan Mills et al. Longitudinal risk-based security assessment of Docker software container images.Computers & Security, 135:103478, 2023

  5. [13]

    The Docker Hub image inheritance network: Construction and empirical insights

    Ruben Opdebeeck et al. The Docker Hub image inheritance network: Construction and empirical insights. InSCAM. IEEE, 2023

  6. [14]

    Fixing Dockerfile smells: An empirical study

    Giovanni Rosa et al. Fixing Dockerfile smells: An empirical study. InICSME 2022. IEEE, 2022

  7. [15]

    Propagation-based vulnerability impact assessment for software supply chains

    Bonan Ruan et al. Propagation-based vulnerability impact assessment for software supply chains. InASE, 2025

  8. [16]

    Hequan Shi et al. Dr. Docker: A large-scale security measurement of Docker image ecosystem. InWWW ’25. ACM, 2025

  9. [17]

    A study of security vulnerabilities on Docker Hub

    Rui Shu et al. A study of security vulnerabilities on Docker Hub. InCODASPY ’17, pages 269–280. ACM, 2017

  10. [18]

    The ripple effect of vulnerabilities in Maven central: Prevalence, propagation, and mitigation challenges

    Ehtisham Ul Haq et al. The ripple effect of vulnerabilities in Maven central: Prevalence, propagation, and mitigation challenges. arXiv preprint arXiv:2504.04175, 2025. 5Code and dataset: https://github.com/ChimangoScan/chimangoscan 6Generative AI tools assisted with writing a...

  11. [19]

    Vulnerability analysis of 2500 Docker Hub images

    Katrine Wist et al. Vulnerability analysis of 2500 Docker Hub images. InAdvances in Security, Networks, and Internet of Things, Transactions on Computational Science and Computational Intelligence, pages 307–327. Springer, 2021

  12. [20]

    On the relation between outdated Docker containers, severity vulnerabilities, and bugs

    Ahmed Zerouali et al. On the relation between outdated Docker containers, severity vulnerabilities, and bugs. InSANER 2019, pages 491–501. IEEE, 2019

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.