Pith. sign in

REVIEW 5 major objections 6 minor 22 references

Synthesizing Reality: Leveraging the Generative AI-Powered Platform Midjourney for Construction Worker Detection

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A detector trained only on AI-generated Midjourney images recognizes real construction workers with high accuracy.

desk verdict A reasonable idea with plausible numbers, but the headline transfer result rests on an unverifiable test set and baseline, so treat it as conditional until artifacts and benchmark details are released. read the letter →

arxiv 2507.13221 v1 pith:Q7AQ33V2 submitted 2025-07-17 cs.CV cs.AI

Hongyang Zhao , Tianyu Liang , Sina Davari , Daeho Kim This is my paper
classification cs.CVcs.AI
keywords constructionworkerdetectionsyntheticdatagenerativeAIMidjourneydiffusionmodelsYOLOv7domaintransferscarcity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that synthetic images generated by the text-to-image platform Midjourney can serve as training data for construction worker detection, replacing or supplementing real site photographs. The authors generated 12,000 images from 3,000 prompt variations, manually labeled 11,992 of them with 36,444 worker bounding boxes, and trained a deep neural network (YOLOv7) on the synthetic data alone. On a real construction image test set the model reached $AP_{0.5}=0.937$ and $AP_{0.5:0.95}=0.642$, while on a held-out synthetic test set it reached 0.994 and 0.919. If correct, this indicates that generative AI can ease the data scarcity and data-collection cost that limit computer vision in construction.

What carries the argument

The load-bearing mechanism is the prompt-driven diffusion pipeline: a manually fixed prompt template — “three construction workers at work <location>, <weather/lighting>, <camera/film type> --ar <aspect ratio>” — with 3,000 parameter combinations dispatched to the text-to-image platform Midjourney through a Discord bot, yielding four images per prompt. Those images are manually labeled with 2D boxes using MakeSense, and the resulting dataset trains a YOLOv7 detector for 800 epochs. The prompt template is what converts a generative model into a controllable data source: varying location, weather, camera style, and aspect ratio is the authors' mechanism for building diversity into the training set, while the manual labeling step is the price they pay for realism that older rendering pipelines could not achieve.

What would settle it

Evaluate the synthetic-trained model and the previous real-trained model with one shared evaluation script on a fixed, pre-announced real construction image test set; if the synthetic model's $AP_{0.5:0.95}$ falls well below the reported 0.642 or the real baseline's 0.75 changes materially under the unified protocol, the transferability claim is not supported.

Watch

Extended reading notes

Core claim

The central claim is that text-to-image diffusion can produce training data for construction worker detection that transfers to real imagery. The authors generated 12,000 images from 3,000 prompt variations, kept 11,992 after manual labeling (36,444 worker boxes), and trained YOLOv7 for 800 epochs on the synthetic set only. On a real construction test set the model reached $AP_{0.5}=0.937$ and $AP_{0.5:0.95}=0.642$, while on a held-out synthetic test set it reached 0.994 and 0.919. In the paper's comparison, a previous YOLOv7 trained on real SODA and MOCS data achieved $AP_{0.5:0.95}=0.75$ on the same reported real test set, so the authors conclude that synthetic data shows strong potential but does not yet match real training data. The takeaway is that generative AI can serve as a scalable, realistic image source, with the current limitations being manual labeling and residual differences from real site conditions.

Load-bearing premise

The central claim rests on the assumption that the real test set in this study and the earlier real-trained baseline were evaluated with the same images and the same protocol, which the paper states but does not document.

Editorial extensions

If this is right

  • A detector trained only on 11,992 Midjourney images transfers to real sites at $AP_{0.5}=0.937$, so construction teams can generate diverse training images without visiting sites.
  • The 3,000-prompt template gives control over location, weather, camera type, and aspect ratio, which the paper argues creates the variety needed to avoid overfitting.
  • Scores on synthetic test images ($AP_{0.5:0.95}=0.919$) well exceed real-image scores (0.642), defining the remaining synthetic-to-real domain gap.
  • Against a prior real-trained YOLOv7 on the same reported test set ($AP_{0.5:0.95}=0.75$), synthetic-only training is close but not superior, so the paper frames generative AI as a supplement, not a replacement.
  • Manual labeling of 36,444 instances remains necessary, and the paper identifies automating or reducing that step as key to broader use.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the transfer figure holds, a natural next experiment is mixing synthetic and real images in one training run: the paper does not test this, but its own baseline comparison suggests measuring how many synthetic images substitute for one real image would give a practical substitution rate.
  • The large gap between synthetic-test and real-test performance implies the model is partly tuned to Midjourney's image distribution; reporting failures per prompt attribute could reveal whether distance, occlusion, or unusual lighting causes the transfer gap.
  • Because prompt parameters directly control scene content, prompt sampling could be viewed as a data-distribution design tool: a practitioner might sample prompts to match the target site's conditions rather than generating a fixed pool blindly.
  • An automatic labeling or verification step for generated images would remove the current bottleneck and let generative pipelines compete with rendering-based synthetic data, which is already auto-annotated.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes using Midjourney-generated synthetic images to train a YOLOv7 detector for construction worker detection. The authors created 3,000 prompts with varied location, weather/lighting, camera/film type, and aspect ratio parameters, generating 12,000 images, of which 11,992 were manually labeled with 36,444 worker bounding boxes. A YOLOv7 model trained solely on these synthetic images (9,592 training images) reportedly achieves AP0.5 = 0.937 and AP0.5:0.95 = 0.642 on an unnamed real construction test set, and AP0.5 = 0.994 and AP0.5:0.95 = 0.919 on a held-out synthetic test set. The paper compares the real-test AP0.5:0.95 to 0.75 from a previous real-trained YOLOv7 model on the 'same real test dataset' and concludes that generative-AI-synthesized data can be a practical supplement to real training data.

Significance. If the central transferability claim is correct, this is a useful proof-of-concept for using text-to-image generative models to mitigate data scarcity in construction-site perception, with practical implications for reducing data collection cost and privacy burden. The paper has notable strengths: it transparently reports near-perfect performance on the synthetic test set (avoiding overclaiming generalization from that set alone), gives concrete numbers for generated images, prompts, labels, and training duration, and offers a falsifiable claim that can be independently audited if data and code are released. The main scientific value hinges on the credibility of the real-test evaluation and the comparability of the baseline, both of which are currently underspecified. The work is modest but potentially publishable if the evaluation gaps are resolved.

major comments (5)
  1. [Results & Validation, 'Performance on Real Construction Dataset'] The real test set is never identified. The sentence claiming that a previous YOLOv7 model 'achieved an AP of 0.75 on IoU thresholds of between 0.5 to 0.95 on the same real test dataset that was utilized in this study' gives no source, image count, class definition, or download location for this test set. Because the paper's main conclusion—that synthetic-only training transfers to real imagery—rests entirely on these numbers, the test set must be named and characterized (e.g., number of images, number of worker instances, scene diversity, whether it is a standard benchmark split). Without this, the reader cannot judge whether the reported AP values are meaningful or whether the test set is easy, small, or cherry-picked.
  2. [Results & Validation, baseline comparison] The prior real-trained baseline is uncited and its protocol is not matched. No reference is given for the 'previous study' that trained YOLOv7 on 35,017 real bounding boxes from SODA and MOCS, and no details are provided on model configuration, input resolution, training schedule, confidence threshold, or evaluation code. The comparison between the synthetic-trained model (AP0.5:0.95 = 0.642) and the real-trained baseline (AP0.5:0.95 = 0.75) is therefore not a controlled comparison. The authors should either re-run the baseline with the same evaluation script and hyperparameters, or report the exact evaluation protocol used in both cases.
  3. [Results & Validation and Discussion] No variance or repeated-run information is reported. The model was trained once for 800 epochs, and all AP numbers are point estimates without error bars, multiple seeds, or statistical significance tests. The difference of 0.108 in AP0.5:0.95 between the synthetic-trained model and the claimed real-trained baseline could easily be within run-to-run variability for YOLOv7, particularly with different random initializations. At minimum, the authors should report results over multiple seeds or provide evidence that the difference is stable.
  4. [Method, Image Generation, and Results] The paper does not address the risk that Midjourney, a closed model trained on internet-scale data, may have memorized or reproduced near-duplicates of public datasets such as SODA and MOCS—the likely source of the real test images. If the synthetic training set contains near-duplicates of images in the real test set, the reported real-test AP would be inflated by data leakage, not genuine transferability. The authors should perform and report a duplicate/near-duplicate check (e.g., image retrieval or perceptual similarity) between the 11,992 synthetic training images and the real test set, and discuss the limitations of using a closed generative model in this respect.
  5. [Results & Validation, 'Performance on Midjourney's Synthetic Dataset'] The near-perfect AP on the synthetic test set (0.994/0.919) is presented as evidence that the model is 'well-tuned' and that synthetic data 'can effectively supplement real-world data,' but this test set is drawn from the same Midjourney pipeline (and likely similar/or overlapping prompts) as the training set. High performance there reflects distribution overlap rather than generalization. The authors do disclose the numbers, which is good, but the Discussion should explicitly state that the synthetic test set does not measure transferability and should not be used to support the practical-effectiveness conclusion; that conclusion must rest solely on the real test set.
minor comments (6)
  1. [Abstract and Results] The paper uses 'mean average precision' for what is a single-class detection result; the correct term is 'average precision' (AP). Please correct this throughout.
  2. [Results & Validation, baseline sentence] The phrase 'an AP of 0.75 on IoU thresholds of between 0.5 to 0.95' is ambiguous; it should be written as AP0.5:0.95 = 0.75 to match the notation used elsewhere.
  3. [Data Labeling & DNN Model Training] The description of MakeSense as a tool that 'draws preliminary bounding boxes around workers' is inaccurate; MakeSense is a manual annotation interface that does not automatically propose boxes. Please reword.
  4. [Figure 4] The learning curve figure lacks axis labels and a legend; it is impossible to tell which curve is training loss, validation loss, or AP. Please label it.
  5. [Prompt Creation] The paper does not report how the 3,000 prompt combinations were distributed across the four parameter categories, nor any quantitative assessment of image diversity (e.g., number of distinct locations, weather conditions, worker scales). Adding a breakdown would strengthen the diversity claim.
  6. [Results & Validation, Figure 5] Figure 5 shows only nine successful and three failed detections; this is insufficient to convey the range of failure modes. The authors should provide error analysis statistics (e.g., failures by object size, occlusion level, or distance).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: real-image evaluation provides an external check on the synthetic-trained detector.

full rationale

The paper's central empirical claim is that a YOLOv7 model trained only on 11,992 manually labeled Midjourney images transfers to real construction images, with AP0.5 = 0.937 and AP0.5:0.95 = 0.642 on a real test set. This evaluation is external to the training pipeline: no model parameter is fitted to the real test set, and the real test images are not generated by the Midjourney procedure used to create the training data. The high scores on the 1,200-image Midjourney test split are an in-distribution measure because train and test share the same generator, but the paper does not rely on those scores alone to support transferability; the real-image evaluation is the load-bearing transfer test. The prior-team YOLOv7 baseline of AP0.5:0.95 = 0.75 on 'the same real test dataset' is underspecified and is a reproducibility and correctness risk, but it is not circular: it is an independent empirical comparison rather than an input that defines the current model's output. No equation defines the reported result in terms of the target, no fitted parameter is renamed as a prediction, and no load-bearing argument is secured only by a self-citation chain. Therefore, no significant circularity is present.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central claim rests on three domain assumptions: Midjourney output fidelity, manual label accuracy, and benchmark comparability. No new physical or mathematical entities are introduced, and no numeric constants are fitted beyond the hand-chosen prompt design.

free parameters (1)
  • Prompt design parameters (location, weather/lighting, camera/film type, aspect ratio, worker count) = 3,000 manually created combinations; default 'three construction workers at work'
    The diversity and realism of the synthetic dataset are produced by hand-selected prompt variations, described as set by qualitative trial and error rather than by a fixed or optimized protocol.
assumptions (3)
  • domain assumption Midjourney-generated construction worker images are realistic and diverse enough that features learned from them transfer to real photographs.
    The entire pipeline depends on this premise; the paper discusses the reality gap but never quantifies it.
  • domain assumption Manual labeling of 11,992 synthetic images provides accurate ground-truth bounding boxes without systematic bias.
    Labels were drawn by people with MakeSense assistance, but no inter-labeler agreement or quality check is reported.
  • domain assumption The unnamed real test dataset and the prior real-trained YOLOv7 baseline are directly comparable.
    The paper says the model was evaluated on 'the same real test dataset' but does not describe its composition, size, or evaluation protocol.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Synthesizing Reality: Leveraging the Generative AI-Powered Platform Midjourney for Construction Worker Detection." pith.science (2026). https://pith.science/paper/Q7AQ33V2

@misc{pith2026250713221,
  author       = {Pith},
  title        = {Pith review of: Synthesizing Reality: Leveraging the Generative AI-Powered Platform Midjourney for Construction Worker Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q7AQ33V2}},
  note         = {Machine review of arXiv:2507.13221}
}
read the original abstract

While recent advancements in deep neural networks (DNNs) have substantially enhanced visual AI's capabilities, the challenge of inadequate data diversity and volume remains, particularly in construction domain. This study presents a novel image synthesis methodology tailored for construction worker detection, leveraging the generative-AI platform Midjourney. The approach entails generating a collection of 12,000 synthetic images by formulating 3000 different prompts, with an emphasis on image realism and diversity. These images, after manual labeling, serve as a dataset for DNN training. Evaluation on a real construction image dataset yielded promising results, with the model attaining average precisions (APs) of 0.937 and 0.642 at intersection-over-union (IoU) thresholds of 0.5 and 0.5 to 0.95, respectively. Notably, the model demonstrated near-perfect performance on the synthetic dataset, achieving APs of 0.994 and 0.919 at the two mentioned thresholds. These findings reveal both the potential and weakness of generative AI in addressing DNN training data scarcity.

Figures

Figures reproduced from arXiv: 2507.13221 by the authors.

Figure 1
Figure 1. The proposed synthetic dataset development pipeline [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Examples of the synthesized construction worker images [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Examples of erroneous synthetization results [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Learning curve of the trained model In a previous study, our research team trained a YOLOv7 model with 35,017 bounding boxes on real training data from the mixture of SODA and MOCS datasets for 800 epochs and achieved an AP of 0.75 on IoU thresholds of between 0.5 to 0…
Figure 5
Figure 5. Figure 5: Performance of our model on construction worker detection. Nine successful [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Performance of our model on the test dataset created by Midjourney. Four [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

22 extracted references · 14 canonical work pages

  1. [1]

    Dataset and benchmark for detecting moving objects in construction sites,

    An, X., Zhou, L., Liu, Z., Wang, C., Li, P., Li, Z. (2021). “Dataset and benchmark for detecting moving objects in construction sites,” Automation in Construction, Volume 122, p. 103482. <https://doi.org/10.1016/j.autcon.2020.103482>

    work page arXiv 2021

  2. [2]

    Vision-based excavator pose estimation using synthetically generated datasets with domain randomization,

    Assadzadeh, A., Arashpour, M., Brilakis, I., Ngo, T., Konstantinou, E. (2022). “Vision-based excavator pose estimation using synthetically generated datasets with domain randomization,” Automation in Construction, Volume 134, p. 104089 <https://doi.org/10.1016/j.autcon.2021.104089>

    work page arXiv 2022

  3. [3]

    Artificial Intelligence and smart vision for building and Construction 4.0: Machine and Deep Learning Methods and Applications

    Baduge, S. K., Thilakarathna, S., Perera, J. S., Arashpour, M., Sharafi, P., Teodosio, B., Shringi, A., and Mendis, P. (2022). “Artificial Intelligence and smart vision for building and Construction 4.0: Machine and Deep Learning Methods and Applications.” Automation in Construction, Elsevier, <https://www.sciencedirect.com/science/article/pii/S0926580522003132>

    work page 2022

  4. [4]

    Image augmentation to improve construction resource detection using generative adversarial networks, cut-and-paste, and image transformation techniques,

    Bang, S., Baek, F., Park, S., Kim, W., & Kim, H., (2020). “Image augmentation to improve construction resource detection using generative adversarial networks, cut-and-paste, and image transformation techniques,” Automation in Construction, Volume 115, 103198. <https://doi.org/10.1016/j.autcon.2020.103198>

    work page arXiv 2020

  5. [5]

    Combining inverse photogrammetry and BIM for automated labeling of construction site images for machine learning,

    Braun, A., & Borrmann, A. (2019). “Combining inverse photogrammetry and BIM for automated labeling of construction site images for machine learning,” Automation in

    work page 2019

  6. [6]

    Realism Assessment for Synthetic Images in Robot Vision through Performance Characterization,

    Construction, Volume 106, p. 102879. <https://doi.org/10.1016/j.autcon.2019.102879>. Ceron Lopez, A. E., Ranjan, R., & Koganti, N. (2022). "Realism Assessment for Synthetic Images in Robot Vision through Performance Characterization," 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan, pp. 13089-13096, <https://i...

    work page arXiv 2022

  7. [7]

    ImageNet: A large-scale hierarchical image database

    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Li, F.F. (2009). “ImageNet: A large-scale hierarchical image database.” Conference on Computer Vision and Pattern Recognition, IEEE Xplore, <https://ieeexplore.ieee.org/document/5206848>

    arXiv 2009

  8. [8]

    SODA: A large-scale open site object detection dataset for deep learning in construction,

    Duan, R., Deng, H., Tian, M., Deng, Y., Lin, J. (2022). “SODA: A large-scale open site object detection dataset for deep learning in construction,” Automation in Construction, Volume 142, p.104499. <https://doi.org/10.1016/j.autcon.2022.104499>

    arXiv 2022

Show all 22 references
  1. [9]

    Proximity Prediction of Mobile Objects to Prevent Contact-Driven Accidents in Co-Robotic Construction

    Kim, D., Lee, S., and Kamat, V. R. (2020). “Proximity Prediction of Mobile Objects to Prevent Contact-Driven Accidents in Co-Robotic Construction.” ASCE Library, Journal of Computing in Civil Engineering, <https://ascelibrary.org/doi/abs/10.1061/%28ASCE%29CP.1943-5487.0000899>

    2020

  2. [10]

    Action recognition of earthmoving excavators based on sequential pattern analysis of visual features and Operation Cycles,

    Kim, J., Chi, S., (2019). “Action recognition of earthmoving excavators based on sequential pattern analysis of visual features and Operation Cycles,” Automation in Construction, Volume 104, Pages 255-264. <https://doi.org/10.1016/j.autcon.2019.03.025>

    2019 doi

  3. [11]

    Training a Visual Scene Understanding Model Only with Synthetic Construction Images

    Kim, J., Kim, D., Shah, J., and Lee, S. (2022). “Training a Visual Scene Understanding Model Only with Synthetic Construction Images.” Computing in Civil Engineering, ASCE Library, <https://ascelibrary.org/doi/abs/10.1061/9780784483893.028>

    2022 doi

  4. [12]

    Hybrid DNN training using both synthetic and real construction images to overcome training data shortage

    Kim, J., Kim, D., Lee, S., and Chi, S. (2023). “Hybrid DNN training using both synthetic and real construction images to overcome training data shortage.” Automation in

    2023

  5. [13]

    This work was presented at I3CE 2024 and is currently under consideration for publication in ASCE proceedings

    Construction, Elsevier, <https://www.sciencedirect.com/science/article/pii/S0926580523000316?via%3Dihub#b b0050>. This work was presented at I3CE 2024 and is currently under consideration for publication in ASCE proceedings. Page 10 of 10

    2024

  6. [14]

    Human–Robot Collaboration in Construction: Classification and Research Trends

    Liang, C.J., Wang, X., Kamat, V. R., and Menassa, C. C. (2021). “Human–Robot Collaboration in Construction: Classification and Research Trends.” Journal of Construction Engineering and Management, ASCE Library, <https://ascelibrary.org/doi/abs/10.1061/%28ASCE%29CO.1943-7862.0002154>

    2021

  7. [15]

    Microsoft Coco: Common Objects in Context

    Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollar, P., and Zitnick, C. L. (2014). “Microsoft Coco: Common Objects in Context.” SpringerLink, European Conference on Computer Vision, <https://link.springer.com/chapter/10.1007/978-3-319- 10602-1_48>

    2014 doi

  8. [16]

    Ionescu, C., Papava, D., Olaru, V., and Sminchisescu, C. (2013). Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments, IEEE, <https://ieeexplore.ieee.org/document/6682899/>

    2013

  9. [17]

    Fake it till you make it: Training Deep Neural Networks for Worker Detection using Synthetic Data

    Tohidifar, A., Saffari, S.F., Kim, D. (2022). “Fake it till you make it: Training Deep Neural Networks for Worker Detection using Synthetic Data.”29th International Workshop on Intelligent Computing in Engineering (EG-ICE), AU Library Scholarly Publishing

    2022

  10. [18]

    <https://doi.org/10.7146/aul.455.c229>

    Services, Aarhus, Denmark. <https://doi.org/10.7146/aul.455.c229>

    doi

  11. [19]

    Technologies for digital twin applications in construction

    Tuhaise, V. V., Tah, J. H. M., and Abanda, F. H. (2023). “Technologies for digital twin applications in construction.” Automation in Construction, Elsevier, <https://www.sciencedirect.com/science/article/pii/S0926580523001917>

    2023

  12. [20]

    YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors

    Wang, C.Y., Bochkovskiy, A., Liao, H.Y.M., (2022). “YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors.” arXiv:2207.02696. <https://doi.org/10.48550/arXiv.2207.02696>

    2022 doi arXiv

  13. [21]

    Investigating Prompt Engineering in Diffusion Models,

    Witteveen, S., Andrews, M., (2022). “Investigating Prompt Engineering in Diffusion Models,” arXiv:2211.15462. <https://arxiv.org/abs/2211.15462>

    2022 arXiv

  14. [22]

    Diffusion Models: A Comprehensive Survey of Methods and Applications

    Yang, L., Zhang, Z., Song, Y., Hong, S., Xu, R., Zhao, Y., Zhang, W., Cui, B., & Yang, M.H., (2023). “Diffusion Models: A Comprehensive Survey of Methods and Applications.” arXiv:2209.00796. <https://arxiv.org/abs/2209.00796>

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.