Pith. sign in

REVIEW 3 major objections 7 minor 2 cited by

Detecting Human Artifacts from Text-to-Image Models

T0 review · 3 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that human-body artifacts in text-to-image output can be localized by a detector trained on many generators, and that feeding its predictions back as negative prompts reduces those artifacts.

desk verdict Useful multi-generator artifact dataset and detector, but the central generalization claim lacks quantitative out-of-domain evaluation. read the letter →

arxiv 2411.13842 v2 pith:BTICJZAT submitted 2024-11-21 cs.CV

Kaihong Wang , Lingzhi Zhang , Jianming Zhang This is my paper · ORCID
classification cs.CV
keywords humanartifactdetectiontext-to-imagegenerationdiffusionmodelslocalizationimagequalityevaluationgenerativemodelbenchmarkingnegativepromptfinetuninginpainting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the broken hands, distorted faces, and extra or missing limbs produced by text-to-image models can be treated as a detect-and-correct problem rather than an unsolved quality mystery. The authors build a dataset of over 37,000 generated images with bounding-box labels for two artifact classes: local defects in six body parts and global anatomical errors such as missing or extra parts. They train separate detectors on this data and claim the detectors find artifacts across the four generators they trained on while still working on generators never seen in training. They further claim that using detector predictions as negative-prompt feedback during diffusion-model finetuning reduces human artifacts, and that the same detector can drive an iterative inpainting loop that corrects local artifacts in arbitrary images.

What carries the argument

The load-bearing mechanism is the pairing of a labeled artifact dataset with a detection model that has seen healthy human bodies as negative examples. Local artifacts are annotated per body part and global artifacts as missing or extra parts, with a separate detector for each. Real human images with no artifact boxes are included in training so the model learns that normal anatomy is not an artifact; this is what lets it transfer to unseen generators and report fewer detections as image quality improves. The correction loop then uses detector confidence scores to decide which generations to keep: finetuning prepends artifact-type identifiers that are later used as negative prompts, and inpainting picks the candidate with the lowest artifact score.

What would settle it

Re-annotate a random sample of HAD images with multiple independent annotators and measure pairwise agreement; if agreement is near chance on images from the higher-fidelity generators, the AP50 values that support HADM's generalization are measuring label noise rather than detection ability.

Watch

Extended reading notes

Core claim

The central claim is that human artifacts in text-to-image output are localizable, and that a detector trained on diverse generator outputs generalizes better than aesthetic scorers, vision-language models, and prior artifact detectors. The authors curate the Human Artifact Dataset with 84,852 labeled instances across six local body-part classes and twelve global structural classes, train separate local and global detection models, and report that the local model performs best on generators with frequent artifacts while still catching subtle errors in higher-fidelity generators, including some not seen in training. The paper further claims that using detector-selected images with artifact-type identifiers as negative prompts during low-rank finetuning of a diffusion model reduces artifact scores, that human raters prefer the finetuned model, and that the detector selects better results in an iterative inpainting correction pipeline.

Load-bearing premise

The whole pipeline assumes that 'artifact' is a well-defined ground truth: individual human annotators decide what counts as a distorted, missing, or extra body part, and the paper reports no agreement check, so subjective and missed labels are baked into both the training signal and every evaluation number.

Editorial extensions

If this is right

  • HAD and HADM provide a reusable benchmark for ranking text-to-image models by how often they distort human bodies, not just by aesthetics.
  • Detector feedback can be folded into diffusion finetuning: artifact-type identifiers used as negative prompts lower both detector-measured and user-perceived artifact severity.
  • Because the detector is trained to treat real human images as clean, it can be pointed at new generators and will flag fewer and lower-confidence artifacts as generator quality improves.
  • The same detector can act as an automatic quality gate inside an inpainting loop, repeatedly fixing the most severe local artifact in an image.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the generalization claim holds, HADM-style detectors could serve as an automatic anatomy-specific reward for aligning future generation models, complementing coarse aesthetic preference models.
  • The acknowledged annotator ambiguity suggests the low scores on high-fidelity generators may be as much label noise as detector error; re-annotating a sample with adjudicated majority labels would clarify this.
  • A natural extension is to combine artifact boxes with pose or segmentation priors, which could resolve the ambiguous extra-limb cases the paper itself flags as corner cases.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper curates the Human Artifact Dataset (HAD), containing 37,554 images from SDXL, DALL-E 2, DALL-E 3, and Midjourney with 84,852 bounding-box annotations covering six local artifact classes (face, torso, arm, hand, leg, feet) and twelve global artifact classes (missing and extra body parts). It trains two detectors, HADM-L and HADM-G, using ViTDet with an EVA-02 backbone and a Cascade R-CNN head, reporting in-domain AP50 of 43.3 for local and 23.9 for global artifacts and an AUC-based comparison against HumanRefiner, HPS-v2, ImageReward, GPT-4o, LLaVA, and Llama on a binarized classification version of the validation set. The paper further claims strong generalization to unseen generators, including SD1.4, PixArt-Sigma, FLUX.1-dev, Sana, and the real-image dataset 300W; it presents a LoRA finetuning of SDXL that uses HADM predictions as negative-prompt feedback, and an iterative inpainting pipeline for artifact correction, supported by a 15-participant user study that prefers the finetuned model.

Significance. If the claims held as stated, HAD would be a valuable released benchmark for human-structure evaluation in text-to-image models, and the demonstration that a dedicated detector outperforms strong vision-language models and aesthetics metrics on in-domain artifact ranking is a genuinely useful result. The paper ships dataset and models on GitHub, includes ablations of backbone and training choices, and provides a user study as an independent check of the finetuning claim; these are real strengths. However, the headline claim of strong generalization to unseen generators currently rests on qualitative examples and prediction statistics rather than on any ground-truth-based metric, and the quantitative finetuning evaluation is partly circular because it relies on HADM itself; both issues must be resolved for the stated contribution to stand.

major comments (3)
  1. [Abstract, Sec. 1 contributions, and Sec. 4.2 'Performance on Unseen Domains'] The claim that HADM 'demonstrate[s] strong generalization, even on images from unseen generators' is not supported by any ground-truth-based metric on out-of-domain data. All AP50 and AUC results (Tabs. 1-2, Fig. 4) are computed on the in-domain HAD validation set; for SD1.4, PixArt-Sigma, FLUX.1-dev, Sana, and 300W the evidence is a selection of top predictions (Figs. 6-7) and prediction-count/confidence statistics (Tab. 3). Neither measures correctness, and the decreasing prediction counts on FLUX.1-dev and Sana in Tab. 3 could equally reflect confidence calibration or domain shift, while the low count on 300W could reflect a false-negative pattern; the interpretation of Tab. 3 as sensitivity to artifact severity is underdetermined. The stress-test concern lands here: even with perfectly reliable labels, the unseen-generator claim would remain unsupported without a labeled out-of-domain evaluation. I recommend annotating a labeled subset of images from at least the four unseen generators and reporting AP50/AUC there, or substantially tempering the generalization claim in the abstract and the contribution list.
  2. [Sec. 5.1 and supplementary Sec. 10 (Figs. 12-13, 14)] The primary quantitative evidence for artifact reduction uses HADM scores on images generated by the finetuned model, but HADM is also the detector that selected the finetuning training data (Sec. 3.3). A model trained to avoid detector-flagged regions is expected to obtain lower scores on that same detector even without genuine artifact reduction, so Figs. 12-13 and the hyperparameter analysis in Fig. 14 are partly circular. The user study in Tab. 4 is an independent check, but as reported it is small (15 participants, 200 pairs), no statistical test or confidence interval is given, and the stated preference (55% vs. 38.7%) is modest; the study design and inter-participant variability should be reported. I suggest evaluating the finetuned model on a fresh human-annotated sample outside the finetuning loop, or adding a second independent detector as an additional quantitative sanity check.
  3. [Sec. 3.1.2 (Annotation Pipeline) and Sec. 4.2] The ground truth is based on human annotation with no reported inter-annotator agreement, and the authors themselves acknowledge missed annotations and subjective ambiguity (Fig. 5 and the discussion of 'occasional oversight by annotators' and 'corner cases'). Because these ambiguous cases are stated to concentrate in the DALLE-3 and Midjourney domains, where AP50 is lowest (Tab. 1), the per-domain performance differences may partly reflect label noise rather than detector behavior, which in turn affects the validity of the in-domain benchmark claim. I recommend reporting annotation agreement (e.g., Cohen's kappa or IoU-based agreement on a re-annotated subset) and, if feasible, re-estimating per-domain AP50 after adjudication of the ambiguous cases.
minor comments (7)
  1. [Tabs. 1-2] Several per-category AP50 cells are computed on one or two instances (e.g., DALLE-2 torso at 100.0/1, DALLE-3 extra torso at 33.3/1, and multiple Tab. 2 cells with fewer than ten instances), so per-category values are statistically unstable; please add a caveat, report confidence intervals, or exclude categories below a minimum instance count.
  2. [Fig. 4 and Sec. 4.1] The y-axis of Fig. 4 is not labeled, and the paper should state the score convention for the quality-based baselines: for HPS-v2 and ImageReward, where higher scores indicate higher quality, an AUC below 50% may simply reflect inverse correlation rather than 'poor' performance, so the sign convention and the interpretation of below-50% AUC should be clarified.
  3. [Fig. 5 caption] The caption is ambiguous about which boxes are predictions and which constitute the reference annotation, particularly because the blue boxes are described both as detections and as context; please specify the visual encoding of red and blue boxes and what FP and FN are measured against.
  4. [Supplementary Sec. 7.4] The inpainting selection rule 'closest to half of the original score' is introduced as an empirical heuristic without supporting evidence or an ablation; please provide a justification or explicitly mark it as a limitation.
  5. [Abstract and Sec. 5.2] The inpainting pipeline is described as a 'novel application,' yet the paper itself cites iterative inpainting with confidence feedback in [65,66]; please clarify the specific novelty relative to those works and to HumanRefiner's refinement stage.
  6. [Reference list] Many reference entries carry stray trailing numbers after the year (e.g., [2] '... CVPR, pages 6154-6162, 2018. 4, 1'), which appear to be citation-page artifacts and should be cleaned up in the final version.
  7. [Ethical Considerations and Sec. 4.2] The ethical statement discourages applying the model to real-human images, but Sec. 4.2 evaluates on the real-image dataset 300W; please add one sentence clarifying that this evaluation is a controlled analysis of false-positive behavior and not an endorsement of deployment on real humans.

Circularity Check

2 steps flagged · score 6.0 of 10

Fine-tuning and inpainting results are evaluated with the same HADM detector that supplies the training/selection signal; the independent user study is small, so the artifact-reduction claim is only partially circular.

  1. fitted input called prediction [Supplementary Material, Sec. 10, Figs. 12-13; cf. Sec. 3.3]
    "We then compute the total scores per image and the average scores of the top10% predictions from HADM-L and HADM-G on the generated images, presenting the results in Fig. 12 and Fig. 13. These results show that, compared to the original SDXL, our finetuned model produces fewer artifacts in terms of both quantity (lower total scores) and quality (lower average scores), which aligns with our user studies."

    HADM provides the finetuning supervision: Sec. 3.3 states 'We then select the top k% of predicted bounding boxes from each category to identify artifacts' and prepends 'weird [ARTIFACT-TYPE]' identifiers. The finetuned model is trained with these HADM-derived negative prompts, so HADM is the instrument that defines which regions are artifacts and the target the model is taught to avoid. Re-scoring the finetuned model with the same HADM detector therefore measures success at the optimization objective rather than an independent reduction in human artifacts. Lower HADM scores are substantially forced by the training signal. The user study is the only independent check, and it is small (15 participants, 200 pairs) with no reported significance test.

  2. self definitional [Sec. 5.2 and Fig. 9 caption]
    "For each detected artifact, we apply inpainting with 20 different random seeds and select the result based on HADM-L’s confidence scores. This iterative process integrates HADM-L into the inpainting pipeline, automating artifact correction and producing a refined final image."

    The selection rule is literally 'select the result based on HADM-L's confidence scores' (with the precise rule in Sec. 7.4 being closest to half the original HADM-L score). The chosen inpainted image is thus the one that best satisfies HADM-L's own scoring among the 20 seeds. Presenting the output as a 'refined final image' and claiming 'improving image quality' in the abstract uses HADM-L as both the selector and the implicit quality measure, with no human or third-party evaluation reported for the inpainting results. The demonstration therefore reduces to the detector's own scoring definition.

full rationale

The paper's central detection contribution is self-contained: HADM is trained on HAD's human annotations and evaluated on the HAD validation set with AP50/AUC against those same human labels, so the in-domain detection numbers are not circular. The 'strong generalization to unseen generators' claim is not backed by ground-truth metrics, only by selected examples and prediction statistics (Tab. 3), but that is a missing-evaluation problem rather than a definitional circularity, so it is not scored as a circular step. No load-bearing self-citation chain appears. The circularity is confined to the two application demonstrations. In Secs. 3.3 and 5.1, HADM's own detections define the finetuning data (top-k% boxes become 'weird [ARTIFACT-TYPE]' negative prompts), and the main quantitative evidence of artifact reduction (Figs. 12-13) is HADM re-scoring the finetuned model; the finetuned model is expected to move away from HADM-flagged patterns, so lower HADM scores are substantially a consequence of the training signal. The 15-participant user preference study (Tab. 4) is an independent check and prevents a fully circular verdict, but it is small and reported without statistical testing. Similarly, the inpainting demonstration selects outputs by HADM-L confidence and then presents the chosen output as 'refined,' which is self-definitional without external evaluation. Overall, this is partial circularity in the artifact-reduction evidence, warranting a score of 6.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The work relies on human-defined artifact categories and annotation reliability, plus the representativeness of the four training generators. The finetuning evaluation additionally assumes the detector's scores are a valid outcome measure, which is partially circular. There are several fitted hyperparameters, but no single parameter fully determines the central claim.

free parameters (5)
  • k (top-k ratio for artifact box selection during finetuning) = 30%
    Selected empirically via ablation (Supplementary Fig. 14) to balance false negatives and false alarms.
  • LoRA rank = 32
    Chosen as the optimal trade-off between computational efficiency and artifact reduction (Supplementary Fig. 14).
  • Inpainting selection rule = closest to half of original artifact score
    Empirical heuristic from Supplementary Sec. 7.4; avoids inpaints that replace hands with non-hand objects or occluded hands.
  • Detection confidence threshold = 0.05
    Default inference threshold; affects precision-recall behavior and downstream finetuning selection.
  • Number of inpainting random seeds = 20
    More seeds give a broader search at higher compute cost; no principled selection.
assumptions (5)
  • domain assumption The six body parts and local/global taxonomy capture all relevant human artifacts
    Defined by the authors after visual inspection of four generators (Sec. 3.1.2), and not validated against independent taxonomies.
  • domain assumption Artifacts in the four training generators are representative of artifacts in unseen generators
    Basis for the generalization claim; only qualitatively verified (Sec. 4.2), with no quantitative detection metrics on unseen domains.
  • domain assumption HADM artifact scores are a valid proxy for human-perceived artifact severity
    Used to evaluate finetuning results (Fig. 12/13); the user study only partially corroborates and is small (Tab. 4).
  • domain assumption Real human images with empty annotations improve robustness without biasing detection
    Incorporated in training (Sec. 3.2); supported only by a single ablation (+3.5 AP50 for HADM-L).
  • standard math Standard object detection metrics (AP50, AUC) are appropriate for artifact localization
    Adopted from detection literature; assumes IoU and ranking properties are meaningful for artifact boxes.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Detecting Human Artifacts from Text-to-Image Models." pith.science (2026). https://pith.science/paper/BTICJZAT

@misc{pith2026241113842,
  author       = {Pith},
  title        = {Pith review of: Detecting Human Artifacts from Text-to-Image Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BTICJZAT}},
  note         = {Machine review of arXiv:2411.13842}
}
read the original abstract

Despite recent advancements, text-to-image generation models often produce images containing artifacts, especially in human figures. These artifacts appear as poorly generated human bodies, including distorted, missing, or extra body parts, leading to visual inconsistencies with typical human anatomy and greatly impairing overall fidelity. In this study, we address this challenge by curating Human Artifact Dataset (HAD), a diverse dataset specifically designed to localize human artifacts. HAD comprises over 37,000 images generated by several popular text-to-image models, annotated for human artifact localization. Using this dataset, we train the Human Artifact Detection Models (HADM), which can identify different artifacts across multiple generative domains and demonstrate strong generalization, even on images from unseen generators. Additionally, to further improve generators' perception of human structural coherence, we use the predictions from our HADM as feedback for diffusion model finetuning. Our experiments confirm a reduction in human artifacts in the resulting model. Furthermore, we showcase a novel application of our HADM in an iterative inpainting framework to correct human artifacts in arbitrary images directly, demonstrating its utility in improving image quality. Our dataset and detection models are available at: https://github.com/wangkaihong/HADM.

Figures

Figures reproduced from arXiv: 2411.13842 by the authors.

Figure 1
Figure 1. Comparison between our Human Artifact Detection [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Example annotations from different generators in Human Artifact Dataset. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Distribution of local (left) and global (right) artifacts by categories across four different image generators in our Human Artifact [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (17 more)
Figure 4
Figure 4. Figure 4: Comparison of the AUC scores of HADM against base [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Examples of predictions from our HADM considered mistakes during evaluation on SDXL (a), DALLE-3 (b), DALLE-2 (c), [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: (a): Top predictions on PixArt-Σ. (b), (c): Top predictions on FLUX.1-dev. (d): Top predictions on Sana. (a) Weird hand (88%) (b) Extra hand (39%) & arm (15%) [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Top predictions of HADM on real images from 300W. [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: Examples of the original SDXL (first row) vs. our finetuned model (second row). [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: Workflow illustrating the reduction of human artifacts through iterative inpainting. Starting with the initial image (left), we [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: Pipeline of our data collection process. [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: Original artifact and different inpainting results with [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]
Figure 12
Figure 12. Figure 12: Total scores per image (left) and average scores (right) of top [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]
Figure 13
Figure 13. Figure 13: Total scores per image (left) and average scores (right) of top [PITH_FULL_IMAGE:figures/full_fig_p014_13.png]
Figure 14
Figure 14. Figure 14: Comparison of detectors’ top 10% predictions on images generated by SDXL finetuned with different values of confidence ratio k (left) and LoRA rank (right). 11. Testing GPT-4o with more hints Given GPT-4o’s limited ability to identify, localize, and an￾alyze features …
Figure 15
Figure 15. Figure 15: Dialogue with GPT-4o providing bounding boxes as hints. [PITH_FULL_IMAGE:figures/full_fig_p018_15.png]
Figure 16
Figure 16. Figure 16: More examples of annotations from Human Artifact Dataset for local human artifacts. First row: SDXL. Second row: DALLE [PITH_FULL_IMAGE:figures/full_fig_p019_16.png]
Figure 17
Figure 17. Figure 17: More examples of annotations from Human Artifact Dataset for global human artifacts. First row: SDXL. Second row: [PITH_FULL_IMAGE:figures/full_fig_p020_17.png]
Figure 18
Figure 18. Figure 18: Examples of predictions from our HADM considered mistakes during evaluation on SDXL (first row), DALLE-2 (second row), [PITH_FULL_IMAGE:figures/full_fig_p021_18.png]
Figure 19
Figure 19. Figure 19: Examples of predictions from our HADM on unseen domains including SD1.4 (first row), PixArt- [PITH_FULL_IMAGE:figures/full_fig_p022_19.png]
Figure 20
Figure 20. Figure 20: More results from the original SDXL model and our finetuned SDXL model. In the first column, images generated by the [PITH_FULL_IMAGE:figures/full_fig_p024_20.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. GeneVA: A Dataset of Human Annotations for Generative Text to Video Artifacts

    cs.CV 2025-09 conditional novelty 7.0 of 10

    GeneVA is the first large-scale benchmark with human-annotated bounding boxes and text descriptions for artifacts in text-to-video generation.

  2. Unified Backbone Refinement for Diffusion Models via Internal-Latent Analysis

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Early abrupt deviations in deep diffusion latents track artifacts; EMA detection plus backbone-specific suppression (DUNE) reduces them without retraining.

Reference graph

Works this paper leans on

78 extracted references · 61 canonical work pages · cited by 2 Pith papers

  1. [1]

    Tim Brooks, Aleksander Holynski, and Alexei A. Efros. In- structpix2pix: Learning to follow image editing instructions. In CVPR, pages 18392–18402, 2023. 1

    2023

  2. [2]

    Cascade R-CNN: delv- ing into high quality object detection

    Zhaowei Cai and Nuno Vasconcelos. Cascade R-CNN: delv- ing into high quality object detection. InCVPR, pages 6154– 6162, 2018. 4, 1

    work page 2018

  3. [3]

    SynArtifact: Classifying and allevi- ating artifacts in synthetic images via vision-language model

    Bin Cao, Jianhao Yuan, Yexin Liu, Jian Li, Shuyang Sun, Jing Liu, and Bo Zhao. SynArtifact: Classifying and allevi- ating artifacts in synthetic images via vision-language model. CoRR, abs/2402.18068, 2024. 2, 3

    arXiv 2024

  4. [4]

    Pixart- Σ: Weak-to-strong training of dif- fusion transformer for 4k text-to-image generation

    Junsong Chen, Chongjian Ge, Enze Xie, Yue Wu, Lewei Yao, Xiaozhe Ren, Zhongdao Wang, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart- Σ: Weak-to-strong training of dif- fusion transformer for 4k text-to-image generation. CoRR, abs/2403.04692, 2024. 4, 6, 7

    arXiv 2024

  5. [5]

    Mark Everingham, Luc Van Gool, Christopher K. I. Williams, John M. Winn, and Andrew Zisserman. The pas- cal visual object classes (VOC) challenge. IJCV, pages 303– 338, 2010. 2

    work page 2010

  6. [6]

    HumanRefiner: Benchmarking abnormal human generation and refining with coarse-to-fine pose-reversible guidance

    Guian Fang, Wenbiao Yan, Yuanfan Guo, Jianhua Han, Zu- tao Jiang, Hang Xu, Shengcai Liao, and Xiaodan Liang. HumanRefiner: Benchmarking abnormal human generation and refining with coarse-to-fine pose-reversible guidance. In ECCV, pages 201–217, 2024. 2, 3, 5

    work page 2024

  7. [7]

    EV A-02: A visual representa- tion for neon genesis

    Yuxin Fang, Quan Sun, Xinggang Wang, Tiejun Huang, Xin- long Wang, and Yue Cao. EV A-02: A visual representa- tion for neon genesis. Image and Vision Computing , page 105171, 2024. 4, 1

    work page 2024

  8. [8]

    An introduction to ROC analysis

    Tom Fawcett. An introduction to ROC analysis. Pattern Recognition Letters, (8):861–874, 2006. 2

    work page 2006

Show all 78 references
  1. [9]

    Make-a-scene: Scene- based text-to-image generation with human priors

    Oran Gafni, Adam Polyak, Oron Ashual, Shelly Sheynin, Devi Parikh, and Yaniv Taigman. Make-a-scene: Scene- based text-to-image generation with human priors. InECCV, pages 89–106, 2022. 1, 2

    2022

  2. [10]

    Synthclip: Are we ready for a fully synthetic CLIP training? CoRR, abs/2402.01832, 2024

    Hasan Abed Al Kader Hammoud, Hani Itani, Fabio Pizzati, Philip Torr, Adel Bibi, and Bernard Ghanem. Synthclip: Are we ready for a fully synthetic CLIP training? CoRR, abs/2402.01832, 2024. 1

    2024 arXiv

  3. [11]

    Clipscore: A reference-free evaluation met- ric for image captioning

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation met- ric for image captioning. In EMNLP, pages 7514–7528,

  4. [12]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. In NeurIPS, pages 6626–6637, 2017. 2

    2017

  5. [13]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. In NeurIPS, 2020. 2

    2020

  6. [14]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In ICLR. OpenReview.net, 2022. 4

    2022

  7. [15]

    T2I-CompBench: A comprehensive benchmark for open-world compositional text-to-image generation

    Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xi- hui Liu. T2I-CompBench: A comprehensive benchmark for open-world compositional text-to-image generation. In NeurIPS, 2023. 2

    2023

  8. [16]

    Midjourney

    Midjourney Inc. Midjourney. https : / / www . midjourney.com/, 2023. 1, 2, 3

    2023

  9. [17]

    YOLOv8: State-of-the-Art Object Detection and Segmentation Models

    Glenn Jocher and Ultralytics. YOLOv8: State-of-the-Art Object Detection and Segmentation Models. https:// github.com/ultralytics/ultralytics, 2023. 2

    2023

  10. [18]

    Imagic: Text-based real image editing with diffusion models

    Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models. In CVPR, pages 6007–6017, 2023. 1

    2023

  11. [19]

    Pick-a-pic: An open dataset of user preferences for text-to-image generation

    Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Ma- tiana, Joe Penna, and Omer Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation. In NeurIPS, 2023. 2

    2023

  12. [20]

    Black Forest Lab. Flux. https://blackforestlabs. ai/, 2023. 1, 4, 6, 7

    2023

  13. [21]

    Holistic evaluation of text-to-image models

    Tony Lee, Michihiro Yasunaga, Chenlin Meng, Yifan Mai, Joon Sung Park, Agrim Gupta, Yunzhi Zhang, Deepak Narayanan, Hannah Teufel, Marco Bellagente, Minguk Kang, Taesung Park, Jure Leskovec, Jun-Yan Zhu, Fei-Fei Li, Jiajun Wu, Stefano Ermon, and Percy Liang. Holistic evaluatio...

    2023

  14. [22]

    GenAI-Bench: Evaluat- ing and improving compositional text-to-visual generation

    Baiqi Li, Zhiqiu Lin, Deepak Pathak, Jiayao Li, Yixin Fei, Kewen Wu, Tiffany Ling, Xide Xia, Pengchuan Zhang, Gra- ham Neubig, and Deva Ramanan. GenAI-Bench: Evaluat- ing and improving compositional text-to-visual generation. CoRR, abs/2406.13743, 2024. 2

    2024 arXiv

  15. [23]

    Towards real world human parsing: Multiple-human parsing in the wild

    Jianshu Li, Jian Zhao, Yunchao Wei, Congyan Lang, Yidong Li, and Jiashi Feng. Towards real world human parsing: Multiple-human parsing in the wild. CoRR, abs/1705.07206,

    arXiv

  16. [24]

    Girshick, and Kaiming He

    Yanghao Li, Hanzi Mao, Ross B. Girshick, and Kaiming He. Exploring plain vision transformer backbones for object de- tection. In ECCV, pages 280–296, 2022. 4

    2022

  17. [25]

    Kohlhoff, Deepak Ramachandran, and Vidhya Navalpakkam

    Youwei Liang, Junfeng He, Gang Li, Peizhao Li, Arseniy Klimovskiy, Nicholas Carolan, Jiao Sun, Jordi Pont-Tuset, Sarah Young, Feng Yang, Junjie Ke, Krishnamurthy Dj Dvi- jotham, Katie Collins, Yiwen Luo, Yang Li, Kai J. Kohlhoff, Deepak Ramachandran, and Vidhya Navalpakkam. Ri...

    2024

  18. [26]

    Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll ´ar, and C

    Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll ´ar, and C. Lawrence Zitnick. Microsoft COCO: common objects in context. In ECCV, pages 740–755, 2014. 4

    2014

  19. [27]

    Evaluating text-to-visual generation with image-to-text gen- eration

    Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. Evaluating text-to-visual generation with image-to-text gen- eration. In ECCV, pages 366–384, 2024. 2

    2024

  20. [28]

    Llava-next: Im- proved reasoning, ocr, and world knowledge, 2024

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Im- proved reasoning, ocr, and world knowledge, 2024. 1, 5

    2024

  21. [29]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, 2019. 8, 1

    2019

  22. [30]

    Handrefiner: Refining malformed hands in generated images by diffusion-based conditional inpainting

    Wenquan Lu, Yufei Xu, Jing Zhang, Chaoyue Wang, and Dacheng Tao. Handrefiner: Refining malformed hands in generated images by diffusion-based conditional inpainting. CoRR, abs/2311.17957, 2023. 2

    2023 arXiv

  23. [31]

    Generating images from captions with attention

    Elman Mansimov, Emilio Parisotto, Lei Jimmy Ba, and Rus- lan Salakhutdinov. Generating images from captions with attention. In ICLR, 2016. 1, 2

    2016

  24. [32]

    Llama 3.2

    Meta. Llama 3.2. https://ai.meta.com/blog/ llama - 3 - 2 - connect - 2024 - vision - edge - mobile-devices/, 2024. 5, 7, 1

    2024

  25. [33]

    Hand- iffuser: Text-to-image generation with realistic hand appear- ances

    Supreeth Narasimhaswamy, Uttaran Bhattacharya, Xiang Chen, Ishita Dasgupta, Saayan Mitra, and Minh Hoai. Hand- iffuser: Text-to-image generation with realistic hand appear- ances. In CVPR, pages 2468–2479, 2024. 2

    2024

  26. [34]

    Glide: Towards photorealis- tic image generation and editing with text-guided diffusion models

    Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealis- tic image generation and editing with text-guided diffusion models. In ICML, pages 16784–16804, 2022. 2

    2022

  27. [35]

    DALL-E 2

    OpenAI. DALL-E 2. https://openai.com/index/ dall-e-2/, 2022. 1, 2, 3

    2022

  28. [36]

    DALL-E 3

    OpenAI. DALL-E 3. https://openai.com/index/ dall-e-3/, 2023. 1, 2, 3

    2023

  29. [37]

    OpenAI. GPT-4. https://openai.com/index/ gpt-4/, 2023. 3

    2023

  30. [38]

    OpenAI. GPT-4o. https://openai.com/index/ hello-gpt-4o/, 2024. 1, 5

    2024

  31. [39]

    Toward verifiable and reproducible human evaluation for text-to-image generation

    Mayu Otani, Riku Togashi, Yu Sawai, Ryosuke Ishigami, Yuta Nakashima, Esa Rahtu, Janne Heikkil ¨a, and Shin’ichi Satoh. Toward verifiable and reproducible human evaluation for text-to-image generation. In CVPR, pages 14277–14286,

  32. [40]

    On aliased resizing and surprising subtleties in GAN evaluation

    Gaurav Parmar, Richard Zhang, and Jun-Yan Zhu. On aliased resizing and surprising subtleties in GAN evaluation. In CVPR, pages 11400–11410, 2022. 2

    2022

  33. [41]

    Giving a hand to diffusion models: A two-stage ap- proach to improving conditional human image generation

    Anton Pelykh, Ozge Mercanoglu Sincan, and Richard Bow- den. Giving a hand to diffusion models: A two-stage ap- proach to improving conditional human image generation. In FG, pages 1–10, 2024. 2

    2024

  34. [42]

    SDXL: Improving latent diffusion mod- els for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion mod- els for high-resolution image synthesis. In ICLR, 2024. 1, 2, 3, 4

    2024

  35. [43]

    HandCraft: Anatomically correct restoration of malformed hands in diffusion generated images.CoRR, abs/2403.01693,

    Zhenyue Qin, Yiqun Zhang, Yang Liu, and Dylan Campbell. HandCraft: Anatomically correct restoration of malformed hands in diffusion generated images.CoRR, abs/2403.01693,

    arXiv

  36. [44]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In ICML, 2021. 2

    2021

  37. [45]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, pages 10674– 10685, 2022. 1, 2, 4, 6, 7

    2022

  38. [46]

    300 faces in-the-wild challenge: The first facial landmark localization challenge

    Christos Sagonas, Georgios Tzimiropoulos, Stefanos Zafeiriou, and Maja Pantic. 300 faces in-the-wild challenge: The first facial landmark localization challenge. In ICCV Workshop, pages 397–403, 2013. 4, 7

    2013

  39. [47]

    Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen

    Tim Salimans, Ian J. Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. In NeurIPS, pages 2226–2234, 2016. 2

    2016

  40. [48]

    Crowdhuman: A benchmark for detecting human in a crowd

    Shuai Shao, Zijian Zhao, Boxun Li, Tete Xiao, Gang Yu, Xi- angyu Zhang, and Jian Sun. Crowdhuman: A benchmark for detecting human in a crowd. CoRR, abs/1805.00123, 2018. 4

    2018 arXiv

  41. [49]

    Objects365: A large-scale, high-quality dataset for object detection

    Shuai Shao, Zeming Li, Tianyuan Zhang, Chao Peng, Gang Yu, Xiangyu Zhang, Jing Li, and Jian Sun. Objects365: A large-scale, high-quality dataset for object detection. In ICCV, pages 8429–8438, 2019. 1

    2019

  42. [50]

    Emu edit: Precise image editing via recognition and genera- tion tasks

    Shelly Sheynin, Adam Polyak, Uriel Singer, Yuval Kirstain, Amit Zohar, Oron Ashual, Devi Parikh, and Yaniv Taigman. Emu edit: Precise image editing via recognition and genera- tion tasks. In CVPR, pages 8871–8879, 2024. 1

    2024

  43. [51]

    Codel: a human co-detection and labeling framework

    Jianping Shi, Renjie Liao, and Jiaya Jia. Codel: a human co-detection and labeling framework. In ICCV, pages 2096– 2103, 2013. 4

    2013

  44. [52]

    Is synthetic data all we need? benchmarking the robustness of models trained with synthetic images

    Krishnakant Singh, Thanush Navaratnam, Jannik Holmer, Simone Schaub-Meyer, and Stefan Roth. Is synthetic data all we need? benchmarking the robustness of models trained with synthetic images. In CVPRW, pages 2505–2515, 2024. 1

    2024

  45. [53]

    Weiss, Niru Mah- eswaranathan, and Surya Ganguli

    Jascha Sohl-Dickstein, Eric A. Weiss, Niru Mah- eswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In ICML, pages 2256–2265, 2015. 2

    2015

  46. [54]

    Facial descriptors for human interaction recognition in still images

    Gokhan Tanisik, Cemil Zalluhoglu, and Nazli Ikizler-Cinbis. Facial descriptors for human interaction recognition in still images. Pattern Recognition Letters, 73:44–51, 2016. 4

    2016

  47. [55]

    DF-GAN: A simple and effec- tive baseline for text-to-image synthesis

    Ming Tao, Hao Tang, Fei Wu, Xiaoyuan Jing, Bing-Kun Bao, and Changsheng Xu. DF-GAN: A simple and effec- tive baseline for text-to-image synthesis. In CVPR, pages 16494–16504, 2022. 1, 2

    2022

  48. [56]

    StableRep: Synthetic images from text-to- image models make strong visual representation learners

    Yonglong Tian, Lijie Fan, Phillip Isola, Huiwen Chang, and Dilip Krishnan. StableRep: Synthetic images from text-to- image models make strong visual representation learners. In NeurIPS, 2023. 1

    2023

  49. [57]

    Learning vision from models rivals learning vision from data

    Yonglong Tian, Lijie Fan, Kaifeng Chen, Dina Katabi, Dilip Krishnan, and Phillip Isola. Learning vision from models rivals learning vision from data. In CVPR, pages 15887– 15898, 2024. 1

    2024

  50. [58]

    Is this generated person existed in real-world? fine-grained detecting and calibrating abnor- mal human-body

    Zeqing Wang, Qingyang Ma, Wentao Wan, Haojie Li, Keze Wang, and Yonghong Tian. Is this generated person existed in real-world? fine-grained detecting and calibrating abnor- mal human-body. CoRR, abs/2411.14205, 2024. 2, 3

    2024 arXiv

  51. [59]

    Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis

    Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. CoRR, abs/2306.09341, 2023. 2, 5

    2023 arXiv

  52. [60]

    Human preference score: Better aligning text-to- image models with human preference

    Xiaoshi Wu, Keqiang Sun, Feng Zhu, Rui Zhao, and Hong- sheng Li. Human preference score: Better aligning text-to- image models with human preference. InICCV, pages 2096– 2105, 2023. 2

    2023

  53. [61]

    SANA: efficient high-resolution image synthesis with linear diffusion transformers

    Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Haotian Tang, Yujun Lin, Zhekai Zhang, Muyang Li, Ligeng Zhu, Yao Lu, and Song Han. SANA: efficient high-resolution image synthesis with linear diffusion transformers. CoRR, abs/2410.10629, 2024. 1, 4, 6, 7

    arXiv

  54. [62]

    Imagere- ward: Learning and evaluating human preferences for text- to-image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagere- ward: Learning and evaluating human preferences for text- to-image generation. In NeurIPS, 2023. 2, 5

    2023

  55. [63]

    Vit- pose: Simple vision transformer baselines for human pose estimation

    Yufei Xu, Jing Zhang, Qiming Zhang, and Dacheng Tao. Vit- pose: Simple vision transformer baselines for human pose estimation. In NeurIPS, 2022. 8, 1

    2022

  56. [64]

    Improving text-to-image synthesis using contrastive learning

    Hui Ye, Xiulong Yang, Martin Tak ´ac, Rajshekhar Sunderra- man, and Shihao Ji. Improving text-to-image synthesis using contrastive learning. In BMVC, page 154, 2021. 1, 2

    2021

  57. [65]

    High-resolution image inpainting with iterative confidence feedback and guided upsampling

    Yu Zeng, Zhe Lin, Jimei Yang, Jianming Zhang, Eli Shecht- man, and Huchuan Lu. High-resolution image inpainting with iterative confidence feedback and guided upsampling. In ECCV, pages 1–17, 2020. 8

    2020

  58. [66]

    Per- ceptual artifacts localization for inpainting

    Lingzhi Zhang, Yuqian Zhou, Connelly Barnes, Sohrab Amirghodsi, Zhe Lin, Eli Shechtman, and Jianbo Shi. Per- ceptual artifacts localization for inpainting. In ECCV, pages 146–164, 2022. 8

    2022

  59. [67]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, pages 3813–3824, 2023. 2

    2023

  60. [68]

    Perceptual artifacts localiza- tion for image synthesis tasks

    Lingzhi Zhang, Zhengjie Xu, Connelly Barnes, Yuqian Zhou, Qing Liu, He Zhang, Sohrab Amirghodsi, Zhe Lin, Eli Shechtman, and Jianbo Shi. Perceptual artifacts localiza- tion for image synthesis tasks. In ICCV, pages 7545–7556,

  61. [69]

    Rosin, Zixi Cai, Xi Han, Dingcheng Yang, Haozhi Huang, and Shi-Min Hu

    Song-Hai Zhang, Ruilong Li, Xin Dong, Paul L. Rosin, Zixi Cai, Xi Han, Dingcheng Yang, Haozhi Huang, and Shi-Min Hu. Pose2seg: Detection free human instance segmentation. In CVPR, pages 889–898, 2019. 4

    2019

  62. [70]

    Gordon, Ranjay Krishna, Austin Narcomey, Li Fei-Fei, and Michael S

    Sharon Zhou, Mitchell L. Gordon, Ranjay Krishna, Austin Narcomey, Li Fei-Fei, and Michael S. Bernstein. HYPE: A benchmark for human eye perceptual evaluation of genera- tive models. In NeurIPS, pages 3444–3456, 2019. 2

    2019

  63. [71]

    DM- GAN: dynamic memory generative adversarial networks for text-to-image synthesis

    Minfeng Zhu, Pingbo Pan, Wei Chen, and Yi Yang. DM- GAN: dynamic memory generative adversarial networks for text-to-image synthesis. In CVPR, pages 5802–5810, 2019. 1, 2 Detecting Human Artifacts from Text-to-Image Models Supplementary Material

    2019

  64. [72]

    weird hand

    Configuration Details 7.1. Data Generation Details For optimal image quality, both the base and refiner models are used during generation. Before annotation, the gener- ated images are filtered by ViTPose [63], a state-of-the-art human keypoint detection model, to ensure the s...

  65. [73]

    These metrics provide a comprehensive evaluation of the model’s performance in both classification and localization tasks

    Evaluation Metrics To assess the performance of our artifact detection model, we employ two widely used evaluation metrics: Area Under the Curve (AUC) of the Receiver Operating Characteristic (ROC) curve and Mean Average Precision (AP50). These metrics provide a comprehensive ...

  66. [74]

    Ablation study of key components in HADM-L

    Ablation Studies Table 5. Ablation study of key components in HADM-L. AP50 scores are reported, with each additional configuration building upon the previous one. Config. ViT-B + ViT-L + EV A02 + Real AP50 32.7 35.3 39.8 43.3 Table 6. Ablation study of key components in HADM-G...

  67. [75]

    Quantitative Results for Finetuned Diffu- sion Models To more comprehensively evaluate the performance of our finetuned SDXL model, we compare the quantitative detec- tion results from HADM on images generated by the orig- inal and finetuned SDXL models. To further demonstrate...

  68. [76]

    The corresponding dialogue is presented in Fig

    Testing GPT-4o with more hints Given GPT-4o’s limited ability to identify, localize, and an- alyze features within an image, as demonstrated in the main paper, we further evaluate its capacity to recognize arti- facts when provided with additional hints, such as bounding boxes...

  69. [77]

    16 and global human artifacts in Fig

    Additional Qualitative Examples Annotation Examples We present additional annotation examples of local human artifacts in Fig. 16 and global human artifacts in Fig. 17. These examples reveal that in domains where human artifacts are frequent, such as local artifacts from DALLE...

  70. [78]

    Last row: Midjourney

    Third row: DALLE-3. Last row: Midjourney. Figure 17. More examples of annotations from Human Artifact Dataset for global human artifacts. First row: SDXL. Second row: DALLE-2. Third row: DALLE-3. Last row: Midjourney. (a) FP hand in SDXL (b) FP leg in SDXL (c) FP extra torso i...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.