Pith. sign in

REVIEW 5 major objections 7 minor 19 references

Smart Eyes for Silent Threats: VLMs and In-Context Learning for THz Imaging

T0 review · 5 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single in-context example—one labeled crop prepended to the prompt—can substantially improve a vision-language model's classification of terahertz scans of a hidden C4, without fine-tuning, and yields natural-language justifications.

desk verdict Circular labeling makes the numbers unreliable, but the first-use result and honest reporting mean the paper deserves a serious referee despite my reject verdict. read the letter →

arxiv 2507.15576 v1 pith:PVPUJRU5 submitted 2025-07-21 cs.CL cs.CV

Nicolas Poggi , Shashank Agnihotri , Margret Keuper This is my paper
classification cs.CLcs.CV
keywords TerahertzimagingVision-languagemodelsIn-contextlearningZero-shotclassificationOne-shotSecurityscreeningInterpretability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Terahertz imaging can reveal concealed objects, but classifying the resulting low-resolution, visually ambiguous frames normally needs many labeled examples. This paper claims that a single labeled example inserted into the prompt—in-context learning—can make a general-purpose vision-language model (VLM) classify THz frames of a hidden C4 explosive without any fine-tuning, and that the model can state its reasoning in words. On their 1,400-frame dataset, one of the two tested models improves from 49.5% to 71.9% accuracy when the example is added, while the other becomes far more sensitive but less precise. The result matters because it suggests a cheap, interpretable route to decision support for security screening and material inspection where labeled data and experts are scarce.

What carries the argument

The load-bearing mechanism is the modality-aligned ICL prompt: a long task instruction (describing THz intensity and phase plots, expected metal and C4 features, return format, and warnings) followed by the demonstration set—a cropped image paired with the text "Use this Image as a reference for Classifying. It has the Class \"YES C4\""—and then the query image with its frame number. Keeping the visual order (demonstration then query) aligned with the textual order (demonstration label then query) encourages the VLM to use the example as a template. The THz frames themselves are produced by a fixed signal pipeline: squared magnitude for intensity, phase angle for phase, orientation correction, and softmax normalization for display.

What would settle it

Recompute zero-shot and one-shot accuracy on a subset of frames re-labeled by multiple annotators who never see any model output, with physical verification of C4 positions against the scan geometry (a sphere behind a metal shell). If the one-shot advantage (e.g., Mistral 0.4950→0.7193 accuracy) shrinks or reverses, the ICL gain is an annotation artifact. A simpler check: replace the "Yes C4" demonstration crop with a random "No C4" crop; if accuracy still rises, the effect is not about the labeled example's content.

Watch

Extended reading notes

Core claim

The paper introduces a modality-aligned one-shot prompting framework for THz imaging and reports that it improves classification and interpretability for open-weight VLMs in a low-data regime. In the one-shot setting, a 26×26 pixel crop of a clearly visible C4 signature, labeled "Yes C4", is prepended to each query frame, and the model is asked to classify the query. For Mistral-Small-3.1-24B-Instruct-2503, accuracy rises from 0.4950 to 0.7193 and F1 from 0.3825 to 0.4126; for Qwen2.5-VL-7B-Instruct, accuracy falls but recall jumps from 0.5000 to 0.9661. The authors interpret this as ICL improving the models' ability to detect C4 while shifting the precision-recall trade-off, and present it as the first application of ICL-enhanced VLMs to THz imaging, with the caveat that the system is not yet reliable for deployment.

Load-bearing premise

The human annotations used as ground truth were made "informed by early zero-shot predictions" of the same kind of VLM being evaluated, so the labels may not be independent of the models; if they are not, both the zero-shot baseline and the reported ICL improvement are biased.

Editorial extensions

If this is right

  • With one labeled example, a general-purpose VLM can be redirected to a new sensing modality at inference time, avoiding the cost of fine-tuning in low-data settings.
  • The natural-language justifications the models output give non-expert users a rationale to inspect, which supports the interpretability the paper claims ICL adds.
  • The effect of ICL is model-specific: it can raise accuracy substantially (Mistral) or trade precision for recall (Qwen), so deployment choices hinge on which error type a screening task tolerates.
  • Because the framework is prompt-level, the same recipe could be applied to other under-annotated imaging domains without retraining—the paper cites ultrasound as a parallel.
  • The authors state the system is not yet reliable enough for real-world deployment; the value is as a foundation for later fine-tuning or stronger models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The annotation step is the riskiest joint: labels were "informed by early zero-shot predictions" of the very model family being evaluated, so if the annotator absorbed those predictions, both the zero-shot baselines and the measured ICL improvement are partly self-confirming; re-labeling a hidden subset without any model input would test independence.
  • A negative-control experiment—prepending a "No C4" crop as the demonstration—would isolate whether the gain comes from the positive label association or merely from the presence of a second visual; the paper does not report this.
  • Qwen's one-shot behavior (recall 0.9661, low precision) reads as a triage detector rather than a final classifier; in a screening pipeline it could flag candidate frames for human review, an operational role the authors leave implicit.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper proposes using vision-language models (VLMs) with in-context learning (ICL) for frame-wise classification of terahertz (THz) images, specifically to detect the presence of a C4 explosive signature in a 1,400-frame quasi-3D scan. It evaluates Mistral-Small-3.1-24B-Instruct-2503 and Qwen2.5-VL-7B-Instruct under zero-shot and one-shot prompting, reporting accuracy, precision, recall, F1, and prediction-change counts. The authors claim that ICL improves classification and interpretability in low-data regimes and that this is the first application of ICL-enhanced VLMs to THz imaging.

Significance. If the stated claims were supported, the paper would offer a lightweight, fine-tuning-free alternative for THz security screening and material inspection, with natural-language justifications that could be useful in resource-constrained settings. The authors provide a code repository, prompt templates, and a link to the dataset videos, which are concrete reproducibility strengths. However, the central evaluation is undermined by the annotation procedure in Section 2.2, in which ground-truth labels were informed by the same models being evaluated; the paper's own results show that Qwen's accuracy drops under ICL, contradicting the abstract's blanket claim; and interpretability is asserted but never measured. As it stands, the evidence is not sufficient to establish the paper's contributions.

major comments (5)
  1. [Section 2.2] Section 2.2 states that 'A human annotator labeled all frames as Yes C4 or No C4 based on visible spectral features and informed by early zero-shot predictions.' Because those zero-shot predictions come from the same VLMs later evaluated in Tables 1 and 2, the ground truth is not independent of the models being tested. The reference to [11], a climate-change social-media dataset paper, does not provide a valid precedent for this annotation protocol in THz imaging. The paper reports no annotation protocol, no record of whether labels were revised after seeing model outputs, no inter-annotator reliability, and no independent physical ground truth. Consequently, the accuracy, precision, recall, and F1 values in Table 1 and the prediction-change counts in Table 2 may reflect agreement with model-influenced labels rather than correctness, and the abstract's claim that ICL improves classification is not supported. The authors should re-annotate without consulting model predictions, provide an objective labeling protocol, or compare against physically known ground truth.
  2. [Abstract; Table 1] The abstract claims 'ICL improves classification and interpretability in low-data regimes,' but Table 1 shows that Qwen2.5-VL-7B accuracy drops from 0.7207 in zero-shot to 0.5329 in one-shot, and Table 2 shows Qwen declines on 394 frames while improving on only 131. Mistral improves accuracy (0.4950 to 0.7193) but its recall falls from 0.9280 to 0.5847. Thus the general claim is contradicted by the paper's own results; at best the data support a model-specific and metric-specific trade-off. The abstract and the contributions should be rewritten to state the mixed outcome, or additional experiments (e.g., multiple demonstration crops, different models) must be supplied to support a general ICL improvement.
  3. [Section 3; Figure 4] The paper claims ICL improves interpretability through natural-language justifications, but no evaluation of interpretability is provided. Figure 4 shows example outputs with 'CORRECT'/'NOT CORRECT' labels, but there is no rubric, no human rating, no faithfulness measurement, and no comparison of explanation quality between zero-shot and one-shot settings. Without a defined protocol for assessing interpretability, this central claim is unsupported.
  4. [Section 2.4; Appendix A.1] The one-shot demonstration is a single 26×26 crop taken from Frame 0663, which is itself part of the 1,400-frame test set and shows the same C4 sphere scanned in the same experimental scene. The result is therefore specific to one demonstration image from the same sequence; it does not demonstrate generalization to new scenes or new objects. Appendix A.1 says the crop was 'selected from a random frame,' which conflicts with Section 2.4's claim that it is chosen from a frame where C4 is clearly visible. The paper reports no variance across different demonstration crops and no evaluation on independent scenes. The claimed ICL benefit is not robustly established.
  5. [Section 3.1; Table 1] The evaluation lacks any external baseline or uncertainty quantification. There is no majority-class baseline, no simple supervised or feature-based classifier, no comparison with fine-tuning, and no confidence intervals or significance tests for the accuracy and F1 differences. Given that the largest reported F1 gain is 0.0301 (Mistral zero-shot to one-shot), the improvement could be within noise. The authors should add baselines and report variability (e.g., across demonstration crops or repeated runs) before claiming ICL improves classification.
minor comments (7)
  1. [Section 3.2] Section 3.2 duplicates Section 3.1 almost verbatim; it should be removed or replaced with a distinct analysis.
  2. [Appendix A.1] Appendix A.1 says the one-shot crop was 'selected from a random frame,' which conflicts with Section 2.4's description of selecting a frame where C4 is clearly visible; please clarify the exact selection procedure.
  3. [Appendix C.1] The prompt text contains a typo ('inlcuding') and instructs the model to 'Analyse all frames at a time,' which conflicts with the frame-wise evaluation described in the main text; align the prompt instructions with the actual protocol.
  4. [Appendix C.1] In the 'Context Dump' section, the same sentence about 'substantial changes in intensities from one frame to another' appears under both the intensity and phase descriptions, which appears to be a copy-paste error.
  5. [Appendix B] The dataset URL is line-wrapped and is not a permanent identifier; provide a DOI or stable archive link.
  6. [Figure 3] Figure 3 labels the setting 'few-shot (one-shot)' while only one demonstration is used; the terminology should be consistent throughout.
  7. [Section 2.2] The citation to [11], a climate-change social-media dataset paper, is not relevant to the THz annotation procedure; either cite a relevant methodology or describe the protocol in full.

Circularity Check

1 steps flagged · score 6.0 of 10

Ground-truth labels were informed by the same models' zero-shot predictions, so the reported 'classification improvement' is measured against model-influenced labels rather than an external benchmark.

  1. self definitional [Section 2.2 (Data Representation and Annotation); used in Section 2.6 (Evaluation Protocol) and Table 1]
    "A human annotator labeled all frames as Yes C4 or No C4 based on visible spectral features and informed by early zero-shot predictions. These annotations are used as the ground truth for evaluating model outputs, similar to [11]."

    The evaluation's ground truth is defined with input from the very models being evaluated: the human annotations were 'informed by early zero-shot predictions', and Section 2.6 states that model outputs are compared 'against human annotations'. Consequently, the zero-shot and one-shot accuracy, precision, recall, and F1 scores in Table 1 measure agreement with labels that may already encode the evaluated VLMs' zero-shot behavior. Without a record of how much the annotator relied on those predictions, the claimed ICL improvement (e.g., Mistral accuracy 0.4950 to 0.7193) cannot be taken as validation against independent physical ground truth; part of the 'target' is derived from the system being scored.

full rationale

The central derivation chain is: label all frames using human judgment 'informed by early zero-shot predictions' -> use those labels as ground truth -> compute zero-shot and one-shot metrics -> conclude ICL improves classification and interpretability. The circularity is explicit in Section 2.2. If the early zero-shot predictions came from the same Mistral/Qwen VLMs used in the evaluation, then the 'ground truth' is partially a function of the models' outputs, so the accuracy numbers are not an external benchmark. This is the load-bearing step for the abstract's main claim. The paper provides no independent physical validation (e.g., known C4 geometry or sensor ground truth), no annotation protocol separating spectral features from model suggestions, no inter-annotator reliability, and no record of whether labels were revised after seeing model predictions. The comparison between zero-shot and one-shot still has some independent content (the demonstration crop is an additional input), which is why the circularity is partial rather than total. The self-citation to reference [11] (which shares an author with the present paper) is used only as a precedent for human-annotated ground truth and is not itself load-bearing; the defect is the annotation contamination, not the citation. Overall score 6: one or more 'predictions' (the reported classification improvements) are not forced by construction, but the evaluation target is partially defined by the system under test.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The paper relies on the plot-based visibility of C4, a single human annotator's labels (partly informed by model outputs), and binary per-frame classification. These assumptions are not independently validated.

free parameters (2)
  • In-context demonstration crop = 26x26 crop from frame 0663, mid-frequency range
    Hand-picked single positive example; choice affects one-shot results and is not varied across runs.
  • Prompt exclusion and attention ranges = frames 0 to 50 and 1350 to 1399 discouraged; focus on 250 to 1250
    Hand-coded heuristic instructions in the system prompt that steer the model's attention and may affect accuracy.
assumptions (3)
  • domain assumption THz intensity and phase visualizations contain enough signal for a VLM to distinguish C4 from metal
    The whole evaluation depends on the plots being class-discriminative; no quantitative feature analysis is provided.
  • domain assumption Human annotation from a single annotator is a reliable ground truth
    Section 2.2 uses one human annotator, whose labels are treated as truth and were partly informed by zero-shot predictions.
  • domain assumption Binary labeling (Yes C4 / No C4) is sufficient for the task
    The paper reduces a spectral, depth-varying signal to a single per-frame binary label.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Smart Eyes for Silent Threats: VLMs and In-Context Learning for THz Imaging." pith.science (2026). https://pith.science/paper/PVPUJRU5

@misc{pith2026250715576,
  author       = {Pith},
  title        = {Pith review of: Smart Eyes for Silent Threats: VLMs and In-Context Learning for THz Imaging},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PVPUJRU5}},
  note         = {Machine review of arXiv:2507.15576}
}
read the original abstract

Terahertz (THz) imaging enables non-invasive analysis for applications such as security screening and material classification, but effective image classification remains challenging due to limited annotations, low resolution, and visual ambiguity. We introduce In-Context Learning (ICL) with Vision-Language Models (VLMs) as a flexible, interpretable alternative that requires no fine-tuning. Using a modality-aligned prompting framework, we adapt two open-weight VLMs to the THz domain and evaluate them under zero-shot and one-shot settings. Our results show that ICL improves classification and interpretability in low-data regimes. This is the first application of ICL-enhanced VLMs to THz imaging, offering a promising direction for resource-constrained scientific domains. Code: \href{https://github.com/Nicolas-Poggi/Project_THz_Classification/tree/main}{GitHub repository}.

Figures

Figures reproduced from arXiv: 2507.15576 by the authors.

Figure 2
Figure 2. Example illustrating how crops were extracted from the [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. A high-level overview of our In-Context Learning implementation using a Vision Language Model for the Terahertz (THz) image [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. This figure shows example outputs from two Vision Language Models (VLMs): Qwen2.5-VL-7B-Instruct (top) and Mistral [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Example crop extracted from one of the 1,400 frames in [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

19 extracted references · 15 canonical work pages

  1. [11]

    Towards understanding climate change perceptions: A social media dataset

    Katharina Prasse, Steffen Jung, Isaac B Bravo, Stefanie Wal- ter, and Margret Keuper. Towards understanding climate change perceptions: A social media dataset. In Workshop on Tackling Climate Change with Machine Learning , pages 1–20. climatechange. ai, 2023. 2

    work page 2023

  2. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

    arXiv

  3. [2]

    Beware of aliases–signal preservation is crucial for robust image restoration

    Shashank Agnihotri, Julia Grabinski, Janis Keuper, and Margret Keuper. Beware of aliases–signal preservation is crucial for robust image restoration. arXiv preprint arXiv:2406.07435, 2024. 10

    arXiv 2024

  4. [3]

    A Survey on In-context Learning, 2024

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, Baobao Chang, Xu Sun, Lei Li, and Zhifang Sui. A Survey on In-context Learning, 2024. 1

    work page 2024

  5. [4]

    Wiest, Marta Ligero, Srividhya Sainath, Narmin Ghaffari Laleh, Omar S

    Dyke Ferber, Georg W ¨olflein, Isabella C. Wiest, Marta Ligero, Srividhya Sainath, Narmin Ghaffari Laleh, Omar S. M. El Nahhas, Gustav M¨uller-Franzes, Dirk J¨ager, Daniel Truhn, and Jakob Nikolas Kather. In-context learning en- ables multimodal large language models to classify cancer pathology images. Nature Communications, 15(1):10104,

    work page

  6. [5]

    Experimental Study of In- Context Learning for Text Classification and Its Application to Legal Document Review in Construction Delay Disputes

    Nathaniel Huber-Fliflet, Jianping Zhang, Peter Gronvall, Fusheng Wei, and Philip Spinelli. Experimental Study of In- Context Learning for Text Classification and Its Application to Legal Document Review in Construction Delay Disputes. In 2024 Institute of Electrical and Electronics Engineers In- ternational Conference on Big Data (BigData), pages 2119–

    work page 2024

  7. [6]

    Tobias Kubiczek and Jan C. Balzer. Material Classification for Terahertz Images Based on Neural Networks. Institute of Electrical and Electronics Engineers , 10:88667–88677,

    work page

  8. [7]

    U2- BENCH: Benchmarking Large Vision-Language Models on Ultrasound Understanding, 2025

    Anjie Le, Henan Liu, Yue Wang, Zhenyu Liu, Rongkun Zhu, Taohan Weng, Jinze Yu, Boyang Wang, Yalun Wu, Kai- wen Yan, Quanlin Sun, Meirui Jiang, Jialun Pei, Siya Liu, Haoyun Zheng, Zhoujun Li, Alison Noble, Jacques Sou- quet, Xiaoqing Guo, Manxi Lin, and Hongcheng Guo. U2- BENCH: Benchmarking Large Vision-Language Models on Ultrasound Understanding, 2025. 4

    work page 2025

Show all 19 references
  1. [8]

    A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challenges, 2025

    Zongxia Li, Xiyang Wu, Hongyang Du, Fuxiao Liu, Huy Nghiem, and Guangyao Shi. A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challenges, 2025. 1

    2025

  2. [9]

    THz spectroscopy and imaging in se- curity applications

    Norbert Palka, Mieczyslaw Szustakowski, Marcin Kowalski, Tomasz Trzcinski, Radoslaw Ryniec, Marek Piszczek, Wies- law Ciurapinski, Marek Zyczkowski, Przemyslaw Zagrajek, and Janusz Wrobel. THz spectroscopy and imaging in se- curity applications. In 2012 19th International Conf...

    2012

  3. [10]

    Humanity’s last exam

    Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Mohamed Shaa- ban, John Ling, Sean Shi, et al. Humanity’s last exam. arXiv preprint arXiv:2501.14249, 2025. 9

    2025 arXiv

  4. [12]

    Terahertz Imaging: A Beginner’s Guide for Engi- neers and Scientists, 2025

    seferyan. Terahertz Imaging: A Beginner’s Guide for Engi- neers and Scientists, 2025. 1

    2025

  5. [13]

    A transformer- based method for the registration of terahertz security images with visible light images

    Liujia Shen, Deliang Zhou, and Yechao Bai. A transformer- based method for the registration of terahertz security images with visible light images. In Proceedings of the 2024 6th In- ternational Symposium on Signal Processing Systems, pages 48–55, New York, NY , USA, 2024. Ass...

    2024

  6. [14]

    Xin Sun, Hongwei Xv, Junyu Dong, Huiyu Zhou, Changrui Chen, and Qiong Li. Few-Shot Learning for Domain- Specific Fine-Grained Image Classification.Institute of Elec- trical and Electronics Engineers Transactions on Industrial Electronics, 68(4):3588–3598, 2021. 1

    2021

  7. [15]

    Lehrstuhl H¨ochstfrequenztechnik und Quantenelek- tronik

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean- Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023. 1 5 A. Experimental Setu...

    2023 arXiv

  8. [18]

    Edges may appear slightly brighter due to specular highlights or slight an- gle differences

    Metal Intensity (viridis): The metal plate should appear as a bright region with fairly uniform intensity due to strong, flat-surface reflection. Edges may appear slightly brighter due to specular highlights or slight an- gle differences. This observation should remain more-or...

  9. [19]

    C4 present

    C4 Intensity (viridis): This object should have non-uniform reflections that stand out from the background. One give- away would be substantial changes in intensities from one frame to another. Phase (twilight): This object should have non-uniform reflections that stand out fr...

  10. [1400]

    END OF Terahertz Images

    and receiving the “END OF Terahertz Images” Tag, please output the findings based on the < FINAL Return Format> and <Warnings>. - Prompt Format Each Image <Sending the next Image now. Analyse it and answer with reasoning. Here is the Image Nr. [Image NR]> <BEGIN Return Format ...

  11. [2125]

    Institute of Electrical and Electronics Engineers, 2024. 1

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.