Foundation Model-Driven Framework for Human-Object Interaction Prediction with Segmentation Mask Integration

Hyung Jin Chang; Juhan Park; Jungchan Cho; Kyungjae Lee

REVIEW 3 major objections 3 minor 17 references

Foundation Model-Driven Framework for Human-Object Interaction Prediction with Segmentation Mask Integration

T0 review · 3 major / 3 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A frozen segmentation model can serve as the whole visual backbone for human-object interaction detection, while also producing interaction masks.

desk verdict A genuinely new frozen-foundation HOI setup with solid triplet numbers, but the quadruplet mask output is never quantitatively validated and the pseudo-label procedure is partly circular, so the paper's central new claim is not yet established. read the letter →

arxiv 2504.19847 v1 pith:FEUCDJOQ submitted 2025-04-28 cs.CV cs.AI

Juhan Park , Kyungjae Lee , Hyung Jin Chang , Jungchan Cho This is my paper · ORCID

classification cs.CVcs.AI

keywords Human-ObjectInteractionsegmentationfoundationmodelHOIquadrupletunionmaskintersectionpseudo-labelingzero-shotinteractive

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a frozen, pretrained segmentation-detection foundation model can serve as the entire visual backbone for human-object interaction (HOI) prediction: only a lightweight HOI decoder is trained, attached to the frozen model, and the standard triplet is extended to a quadruplet that also carries a union mask of the human-object pair. This matters because existing HOI detectors either fine-tune the whole network or run a separate detector, risking the performance of the original detection and segmentation task and adding cost. Seg2HOI reports that with the backbone frozen it matches or nearly matches state-of-the-art HOI performance on the HICO-DET and V-COCO benchmarks, including zero-shot compositions, and that the added masks improve HOI accuracy rather than hurting it. The paper's core bet is that low-level pixel features of a segmentation foundation model can be transferred to high-level relation reasoning without any foundation-model training.

What carries the argument

The load-bearing machinery is the frozen vision foundation model plus a six-head HOI decoder trained by Hungarian matching with an extended cost that includes mask terms. The decoder's two branches, object-aligned and human-aligned queries, exchange information through implicit cross-attention instead of explicit pairwise construction, so the query count stays fixed as the number of detections grows. The mask heads multiply learned HOI mask embeddings by the frozen model's pixel embedding map, and their training signal is the paper's pseudo-labeling procedure: instance masks from the frozen segmentation model are matched to ground-truth boxes by cost, then union and cropped-intersection masks are formed and used in both the Hungarian matching cost and the training loss. The same architecture can be mounted on a vision-language foundation model, where object and action classifiers are replaced by cosine similarity against text embeddings, giving zero-shot and prompt-based interactive behavior.

What would settle it

Take a sample of HICO-DET and V-COCO images, hand-annotate union and intersection masks for the ground-truth HOI pairs, and measure the IoU of Seg2HOI's predicted masks on those pairs; if rare-category pairs with small intersection regions score near zero, or if setting the mask losses to zero does not reduce triplet mAP, the paper's claim that the masks help would be contradicted.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a segmentation foundation model can be reused as a frozen backbone for HOI: the proposed Seg2HOI decoder learns implicit human-object relations through two cross-attention branches (object-aligned and human-aligned) over the frozen model's query features, then predicts the interaction verb, the interacting box, and two HOI masks, a union mask covering both human and object and an intersection mask covering their shared region. Because existing HOI datasets contain boxes but no interaction masks, the paper derives pseudo-labels from the frozen model's own instance masks, matched to ground-truth boxes by a bounding-box plus IoU cost, and uses the union of matched masks as the union label and the cropped overlap as the intersection label. Trained with these pseudo-labels and only the decoder, the model reports performance comparable to state-of-the-art two-stage HOI methods while preserving the foundation model's original segmentation and detection capabilities, and it inherits promptable and open-vocabulary behavior when built on a vision-language foundation model.

Load-bearing premise

The whole mask benefit rests on the assumption that instance masks produced by the frozen segmentation model and matched to ground-truth boxes are accurate enough to teach union and intersection masks, yet no comparison to human-annotated HOI masks is reported.

Editorial extensions

If this is right

With a frozen backbone, HOI detection can be added to a segmentation foundation model at the cost of training only the decoder, so the foundation model's original detection and segmentation performance is not degraded.
HOI quadruplets consisting of the triplet plus union and intersection masks are predicted from the same relation features, making interaction-aware masks available for downstream use without a separate segmentation model.
Mask supervision from pseudo-labels improves HOI accuracy on V-COCO and on most HICO-DET splits; the paper reports that adding the union mask alone raises V-COCO mAP by about 2 points over the no-mask baseline.
The framework generalizes to zero-shot HOI: unseen compositions and unseen objects are detected without extra zero-shot loss mechanisms, and the vision-language variant improves over its baseline on unseen-category mAP in the reported protocols.
Prompt-based interactive HOI segmentation is possible with visual or textual prompts because the HOI decoder inherits the foundation model's prompt mechanisms without prompt-specific training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the pseudo-labeling route works at scale, it removes the need for costly human annotation of interaction masks, suggesting that other relation-level tasks, such as object-object interaction, could be annotated automatically from frozen instance masks plus boxes.
A natural testable extension is to train with union-mask supervision only on larger and more diverse data, because the paper's own ablation shows that the intersection mask can hurt rare HICO-DET categories when the intersection region is small or empty.
The fixed-query implicit cross-attention design may transfer to object-object interaction with mostly architectural changes, since neither branch is tied to a human class by construction.
The frozen-backbone constraint is what makes the claimed result valuable; if the backbone were fine-tuned, the reported numbers would likely reduce to ordinary two-stage HOI performance and the preservation-of-original-tasks argument would disappear.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is the first HOI paper I know of that freezes a segmentation foundation model and attaches a decoder that outputs union and intersection masks, and the triplet numbers are solid enough to take seriously. But the central new output—the masks—is never quantitatively validated, and the pseudo-label procedure is self-referential in a way the paper doesn't fully confront. The quadruplet claim is not established even though the triplet story mostly holds.

What's genuinely new: Seg2HOI keeps Mask-DINO or SEEM frozen and trains only a lightweight HOI decoder with implicit cross-attention. That preserves the foundation model's original capabilities, which is a real practical advantage. The triplet results on HICO-DET and V-COCO are competitive with other two-stage methods that also freeze a COCO-trained detector; their re-run of PViC with frozen weights is a fair comparison, and the ablation shows the mask auxiliary losses improve triplet mAP consistently. Zero-shot numbers with SEEM are also plausible and show a consistent mask benefit. The citation pattern is normal; the distinction from C-HOI (joint training vs. frozen) is accurate.

Soft spots, in order of severity. First, HOI mask quality is only shown in figures. There is no mask IoU, mask AP, or any quantitative check against human-annotated masks, even on a proxy like COCO instance annotations. Second, the pseudo-labels are generated by the same frozen model that provides the features, so the mask supervision is circular: the decoder learns to reproduce the foundation model's masks, and then the paper shows the decoder reproducing them. That could be fine as a self-supervised auxiliary task, but it does not establish that the masks are semantically correct. Third, Eq. 28-29 define the intersection mask as a crop of the union mask by the intersection of expanded boxes. That is not a true overlap region; it is a box-bounded crop, so the intersection head is learning something odd. Fourth, beta_b, beta_u, and gamma are not reported, and no code is released, which makes the pseudo-labeling hard to reproduce. Finally, the conclusion says "state-of-the-art results on the HICO dataset," which overstates Table 4: Seg2HOI is best among COCO-frozen detectors, not overall.

The paper itself is honest about one consequence: in Sec. 6.5, intersection pseudo-labels cannot be generated for small or empty overlap, and HICO-DET Rare drops below baseline. That admission supports the reading that the pseudo-labels are noisy, though the union mask alone still helps.

Who it's for: anyone working on HOI with frozen foundation models, or on adding new tasks to frozen perception models. It deserves a serious referee, because the setup is new and the triplet evidence is real, but the reviewer should demand quantitative mask evaluation, hyperparameters, and ideally code before the quadruplet claim is accepted.

Referee Report

3 major / 3 minor

Summary. The paper proposes Seg2HOI, a two-stage human-object interaction (HOI) detector that freezes a segmentation foundation model (Mask-DINO or SEEM) and adds a lightweight HOI decoder with implicit human-object relation learning. The decoder predicts standard HOI triplets as well as union and intersection masks for human-object pairs, which the paper calls quadruplets. Because existing HOI datasets lack mask annotations, the masks are supervised by pseudo-labels generated from the frozen foundation model's instance masks. The framework is evaluated on HICO-DET and V-COCO for closed-vocabulary and zero-shot triplet detection, with mask quality and interactive prompting demonstrated qualitatively.

Significance. If the mask results were quantitatively validated, this would be a useful contribution: it shows that a frozen segmentation foundation model can serve as a backbone for HOI detection while preserving its original tasks, and it introduces a new output modality (HOI segmentation) without retraining the foundation model. The triplet results are externally benchmarked on HICO-DET and V-COCO, and the ablation in Table 6 shows consistent gains from the union-mask pseudo-label. The zero-shot results with SEEM/CLIP are also encouraging. The main weakness is that the central novel output, the HOI masks, is neither quantitatively evaluated nor grounded in human annotations, so the paper's central claim is not yet established.

major comments (3)

[Secs. 4.3 and 6.2] The central novelty of the paper is the HOI quadruplet with union and intersection masks, yet mask quality is only demonstrated qualitatively in Figs. 7-10. The pseudo-labeling procedure in Sec. 4.3 (Eqs. 24-29) is never quantitatively validated against human annotations; no mask IoU, Dice, or matching success rate is reported. Because m_U and m_I are the paper's main contribution, I request a quantitative evaluation of the predicted masks (e.g., mIoU against manual HOI masks on a HICO-DET/V-COCO subset, or at least against COCO instance masks as a proxy), together with a report of the matching failure rate in Eq. (26).
[Sec. 4.3, Eq. (28)] The proposed pseudo-labeled intersection mask is defined as crop(m_tilde_U, b_I), where b_I is the intersection of the expanded instance mask boxes, not the pixel-wise intersection of the human and object masks. Consequently the intersection head is trained to reproduce a box-bounded crop of the union mask, and its output cannot be interpreted as a semantic interaction region. The paper should either redefine m_I to be the true pixel-wise intersection of m_pred_h and m_pred_o, or refrain from calling it an intersection mask, and should report the hyperparameters beta_b, beta_u, and gamma in Eqs. (24), (25), and (29) along with a sensitivity analysis.
[Secs. 6.5 and 4.3] The ablation in Table 6 shows that training with the intersection mask alone degrades HICO-DET Rare to 22.44 from the 23.53 no-mask baseline, and the text attributes this to pseudo-label noise when b_I is small or empty. This admission, combined with the fact that the pseudo-labels are produced by the same frozen foundation model that provides the features (Sec. 4.3), makes the mask-supervision signal self-referential. The triplet claims are externally benchmarked and not affected, but the quadruplet claim requires independent validation.

minor comments (3)

[Sec. 6.2, Table 4] The claim that Seg2HOI "shows the best performance when the detector is fine-tuned only on the MS-COCO dataset" is not fully supported: among COCO-only methods, the only Swin-L entry is Seg2HOI itself, and direct comparisons with a Swin-L backbone in the COCO-only setting are missing.
[Sec. 4.4, implementation details] The hyperparameter names appear swapped: Eqs. (30)-(31) use alpha for matching costs and lambda for loss weights, but the implementation details state that the Hungarian-cost hyperparameters are lambda_v, lambda_c, ... and the loss-weight hyperparameters are alpha_v, alpha_c, ...; please correct this inconsistency.
[Sec. 3.3] The quadruplet is defined as Z = {m_s, b_h, (b_o, c_o), v} with a single mask m_s, while Sec. 4.2 introduces two masks m_U and m_I; this notation should be reconciled so the problem definition matches the method and the evaluation.

Circularity Check

1 steps flagged · score 4.0 of 10

Pseudo-labeled HOI masks form a self-referential loop with the frozen foundation model; triplet results remain externally grounded.

self definitional [Sec. 4.3 (Eqs. 24-29) with Sec. 4.2 (Eqs. 22-23) and Sec. 3.1 (Eqs. 1-2)]
"To overcome this issue, we create HOI masks by pseudo-labeling HOI union and intersection masks, leveraging the outputs of a segmentation foundation model. ... m̃_U = m_pred_h(σ_h) ∪ m_pred_o(σ_o) (Eq. 27) ... m_U = F_mlp(cat(R,Q_d))⊗f_seg (Eq. 22) ... m_inst = MLP(Q_d)⊗f_seg (Eq. 2)."

The HOI mask ground truths are not human-annotated; they are constructed from the instance masks m_pred_h and m_pred_o that the frozen foundation model itself outputs (m_inst = MLP(Q_d)⊗f_seg, Eq. 2). The HOI decoder's mask predictions are also formed by multiplying a learned embedding with the very same f_seg (Eq. 22). Thus both the supervision and the prediction are functions of one common frozen pixel embedding: the 'quadruplet' mask output is trained to reproduce a function of the same foundation-model features from which it is computed. No external HOI mask annotation enters the loop; Sec. 6.2 evaluates masks only qualitatively, and Sec.

full rationale

The triplet claim is not circular: Seg2HOI's mAP numbers on HICO-DET and V-COCO (Tables 3-4) are computed with official evaluation code against human-annotated triplets, so the central 'comparable to state-of-the-art' statement is externally grounded. The only self-referential loop is in the quadruplet mask component. The HOI union and intersection pseudo-labels of Sec. 4.3 are built from instance masks emitted by the frozen Mask-DINO/SEEM model, and the mask heads of Sec. 4.2 predict by multiplying learned embeddings with the same pixel-embedding map f_seg used to generate those instance masks (Eqs. 2 and 22). Consequently, the mask supervision and the mask prediction are functions of one common frozen feature source; no human-annotated HOI mask is used, and mask quality is only qualitatively illustrated (Figs. 7-10), not measured against external annotations. Sec. 6.5 explicitly concedes that when the box-intersection region b_I is small or empty, pseudo-labels cannot be generated and Rare-category HICO-DET performance drops. Additionally, Eq. 28 defines the 'intersection mask' target as a crop of the union mask, so that head is trained to reproduce a derived quantity rather than an independent overlap. The only author self-citation (Park et al., ViPLO) is related-work context and is not load-bearing. Overall, the triplet contribution stands independently; the circularity is confined to the mask-supervision loop, hence a moderate score.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

The central claim rests on empirically assumed transfer from frozen foundation models, pseudo-label reliability, and CLIP alignment. No new physical entities are postulated; the quadruplet is a task representation whose mask component has no external benchmark. Several hyperparameters used in pseudo-labeling are unreported.

free parameters (4)

HOI decoder loss and matching weights (lambda and alpha) = lambda_v=5.0, lambda_c=4.0, lambda_b=5.0, lambda_u=2.0, lambda_U=2.0, lambda_I=0.1; alpha_v=5.0, alpha_c=5.0…
Hand-set in Sec. 6.1 to balance action, object, box, and mask terms; no sensitivity analysis or hyperparameter search is reported.
Pseudo-label matching cost weights beta_b, beta_u = not reported
Used in Eqs. (24)-(25) to select which instance mask becomes the HOI pseudo-label; the values are omitted, making the pseudo-labeling procedure under-specified.
Intersection-box expansion gamma = not reported
Expands the intersection box in Eq. (29) before cropping the intersection mask; the value is never given, and it directly changes the intersection-mask supervision.
Inference confidence exponent lambda = 0.5
In Sec. 4.4, c_hoi = max_k c_o(k)^lambda * sigma(c_v(j)); set to 0.5 without ablation, affecting the precision/recall trade-off at inference.

assumptions (4)

domain assumption Frozen Mask-DINO and SEEM features contain enough instance-level information for HOI relation learning.
Sec. 4.1 builds all HOI queries solely on vision decoder outputs Q_d from the frozen foundation model; if these features lack object, pose, or context cues, the HOI decoder cannot recover them.
domain assumption Pseudo-labeled masks from a frozen segmenter are a valid supervision signal for HOI union and intersection masks.
Sec. 4.3 replaces missing HOI mask ground truth with masks matched from foundation model outputs; there is no validation against human-annotated HOI masks and no analysis of matching failure.
domain assumption CLIP text embeddings align with the HOI decoder's visual embedding space for open-vocabulary and prompt-based classification.
Sec. 5 computes cosine similarity between visual embeddings E_v, E_o and CLIP text embeddings without prompt-specific training; zero-shot validity depends entirely on this cross-modal alignment.
standard math DETR-style Hungarian matching and set-prediction losses are appropriate for the HOI quadruplet task.
Sec. 4.4 extends the matching of Tamura et al. (QPIC) to include mask costs; the set-prediction framework is standard practice, so this is a reasonable background assumption.

invented entities (2)

HOI quadruplet Z = {m_s, b_h, (b_o, c_o), v}
purpose: Extends the standard HOI triplet with an HOI union segmentation mask, enabling pixel-level HOI output and prompt-based interactive segmentation.
This is a task representation proposed by the paper; there is no external benchmark or human-annotated HOI mask ground truth, so the entity has no falsifiable handle outside the paper's own qualitative demos.
HOI intersection mask m_I
purpose: Captures the overlap region between human and object to sharpen relation features and support mask-guided training.
Derived from pseudo-labeled union masks via Eqs. (28)-(29); never evaluated quantitatively, and the paper itself notes it can be empty for many pairs, producing noisy supervision.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Foundation Model-Driven Framework for Human-Object Interaction Prediction with Segmentation Mask Integration." pith.science (2026). https://pith.science/paper/FEUCDJOQ

@misc{pith2026250419847,
  author       = {Pith},
  title        = {Pith review of: Foundation Model-Driven Framework for Human-Object Interaction Prediction with Segmentation Mask Integration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FEUCDJOQ}},
  note         = {Machine review of arXiv:2504.19847}
}

read the original abstract

In this work, we introduce Segmentation to Human-Object Interaction (\textit{\textbf{Seg2HOI}}) approach, a novel framework that integrates segmentation-based vision foundation models with the human-object interaction task, distinguished from traditional detection-based Human-Object Interaction (HOI) methods. Our approach enhances HOI detection by not only predicting the standard triplets but also introducing quadruplets, which extend HOI triplets by including segmentation masks for human-object pairs. More specifically, Seg2HOI inherits the properties of the vision foundation model (e.g., promptable and interactive mechanisms) and incorporates a decoder that applies these attributes to HOI task. Despite training only for HOI, without additional training mechanisms for these properties, the framework demonstrates that such features still operate efficiently. Extensive experiments on two public benchmark datasets demonstrate that Seg2HOI achieves performance comparable to state-of-the-art methods, even in zero-shot scenarios. Lastly, we propose that Seg2HOI can generate HOI quadruplets and interactive HOI segmentation from novel text and visual prompts that were not used during training, making it versatile for a wide range of applications by leveraging this flexibility.

Figures

Figures reproduced from arXiv: 2504.19847 by the authors.

**Figure 1.** In (a), the prediction set from traditional HOI detection methods, while (b) illustrates the proposed method from this paper, which includes HOI segmentation in addition to standard prediction. In (c), examples of potential vision and robotic applications are presented. models are retrained for each specific task. This approach is not only resource-intensive but also impractical when dealing with complex interaction… view at source ↗

**Figure 2.** Overview of the proposed Seg2HOI framework which consists of two steps: pretrained vision foundation model and HOI model. The black arrow represents the process of the pretrained segmentation model, the blue arrow represents the process added in this paper, and the purple arrow represents the process for pseudo-labeling. utilizing the feature map with the flattened and reduced dimension 𝑄𝑠 along with the positional … view at source ↗

**Figure 3.** Implicit pair learning in HOI deocoder. decoder, it learns the features of the counterpart involved in the relation. During the conversion of a specific instance’s query feature into a relation feature, the original feature’s positional prior may be lost. Therefore, by assigning positional embedding 𝑃𝑑 , the relation feature retains information about the original position of the query feature. Typically, positional … view at source ↗

Figures from the paper (7 more)

**Figure 4.** Figure 4: HOI decoder output heads. Next, to supplement the feature with global information, it undergoes refinement through the backbone feature, as empirically determined by (Zhang et al., 2023a). Finally, the relation feature 𝑅 is obtained through a feed-forward network, as s…

**Figure 5.** Figure 5: HOI mask pseudo labeling process 4.3. HOI Mask Pseudo Labeling The existing HOI dataset contains bounding box data and class information for humans and objects for the detection task. Due to the absence of segmentation masks, it is challenging to train the segmentation…

**Figure 6.** Figure 6: Illustration of our Seg2HOI framework architecture for HOI recognition, which consists of a segmentation foundation model and an HOI decoder, providing a methodology to obtain relation results. 5. Interactive Quadruplet Image Segmentation Extension In this section, we …

**Figure 7.** Figure 7: We showcase qualitative results proposed quadruplets. The first row represents the ground truth triplet data, the second row shows the Seg2HOI quadruplet output with HOI union mask, and the third row displays the HOI intersection mask. (a) RF-UC (b) NF-UC (c) UO [PITH…

**Figure 8.** Figure 8: Qualitative results from zero-shot dataset. The sub-captions indicate the corresponding datasets. the baseline, we can confirm that the presence of masks provides a benefit for zero-shot HOI learning. While existing methods fine-tune the entire model, resulting in high…

**Figure 9.** Figure 9: Qualitative results using various prompts from Seg2HOI with SEEM. Our framework leverages the capabilities of foundation models, enabling its use across various applications. We present the quadruplets extracted from each prompt. The red text represents the user referr…

**Figure 10.** Figure 10: Illustration of interactive HOI image segmentation in zero-shot datasets. The referred text can be seen on the masks in red. The sub-captions indicate the corresponding datasets. 6.5. Ablation Study In this section, we conducted several experiments to demonstrate the …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 6 canonical work pages

[3]

Qahoi: Query-based anchors for human-object interaction detection, in: 2023 18th International Conference on Machine Vision and Applications (MV A), IEEE. pp. 1–5. Chen, K., Pang, J., Wang, J., Xiong, Y ., Li, X., Sun, S., Feng, W., Liu, Z., Shi, J., Ouyang, W., et al.,

work page 2023
[4]

arXiv preprint arXiv:2311.00571

Llava-interactive: An all-in-one demo for image chat, segmentation, generation and editing. arXiv preprint arXiv:2311.00571 . Cheng, B., Misra, I., Schwing, A.G., Kirillov, A., Girdhar, R., 2022a. Masked-attention mask transformer for universal image segmentation, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1...

arXiv
[6]

arXiv preprint arXiv:1808.10437

ican: Instance-centric attention network for human-object interaction detection. arXiv preprint arXiv:1808.10437 . Gkioxari, G., Girshick, R., Dollár, P., He, K.,

arXiv
[8]

Visual compositional learning for human-object interaction detection, in: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XV 16, Springer. pp. 584–600. Hou, Z., Yu, B., Qiao, Y ., Peng, X., Tao, D., 2021a. Affordance transfer learning for human-object interaction detection, in: Proceedings of the IEE...

work page 2020
[11]

arXiv preprint arXiv:2305.05662

Interngpt: Solving vision-centric tasks by interacting with chatgpt beyond language. arXiv preprint arXiv:2305.05662 . Long, J., Shelhamer, E., Darrell, T.,

arXiv
[13]

arXiv preprint arXiv:2309.16524

Hoi4abot: Human-object interaction anticipation for human intention reading collaborative robots. arXiv preprint arXiv:2309.16524 . Milletari, F., Navab, N., Ahmadi, S.A.,

arXiv
[16]

19717–19728

Diffusion-guided reconstruction of everyday hand-object interaction clips, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 19717–19728. Zhang, A., Liao, Y ., Liu, S., Lu, M., Wang, Y ., Gao, C., LI, X., 2021a. Mining the benefits of two-stage and one-stage hoi detection, in: Ranzato, M., Beygelzimer, A., Dauphin, Y ., Lian...

arXiv
[17]

arXiv preprint arXiv:2010.04159

Deformable detr: Deformable transformers for end-to-end object detection. arXiv preprint arXiv:2010.04159 . Zou, C., Wang, B., Hu, Y ., Liu, J., Wu, Q., Zhao, Y ., Li, B., Zhang, C., Zhang, C., Wei, Y ., et al.,

arXiv 2010

Show all 17 references

[2014]

Microsoft coco: Common objects in context, in: Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, Springer. pp. 740–755. Liu, S., Li, F., Zhang, H., Yang, X., Qi, X., Su, H., Zhu, J., Zhang, L.,

2014
[2015]

arXiv preprint arXiv:1505.04474

Visual semantic role labeling. arXiv preprint arXiv:1505.04474 . He, K., Gkioxari, G., Dollár, P., Girshick, R.,

arXiv
[2016]

V-net: Fully convolutional neural networks for volumetric medical image segmentation, in: 2016 fourth international conference on 3D vision (3DV), Ieee. pp. 565–571. Ning, S., Qiu, L., Liu, Y ., He, X.,

2016
[2018]

Learning to detect human-object interactions, in: 2018 ieee winter conference on applications of computer vision (wacv), IEEE. pp. 381–389. Chen, J., Yanai, K.,

2018
[2020]

arXiv preprint arXiv:2010.11929

An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 . Fang, H.S., Xie, Y ., Shao, D., Lu, C.,

2010 arXiv
[2021]

Computational Intelligence and Neuroscience 2021, 9922697

Scaling human-object interaction recognition in the video through zero-shot learning. Computational Intelligence and Neuroscience 2021, 9922697. Mascaro, E.V ., Sliwowski, D., Lee, D.,

2021
[2022]

arXiv preprint arXiv:2201.12329

Dab-detr: Dynamic anchor boxes are better queries for detr. arXiv preprint arXiv:2201.12329 . Liu, Y ., Yuan, J., Chen, C.W.,

arXiv
[2023]

arXiv preprint arXiv:2303.08774

Gpt-4 technical report. arXiv preprint arXiv:2303.08774 . Athar, A., Luiten, J., V oigtlaender, P., Khurana, T., Dave, A., Leibe, B., Ramanan, D.,

arXiv
[2024]

arXiv preprint arXiv:2408.00714

Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714 . Ren, S., He, K., Girshick, R., Sun, J.,

arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [3]

Qahoi: Query-based anchors for human-object interaction detection, in: 2023 18th International Conference on Machine Vision and Applications (MV A), IEEE. pp. 1–5. Chen, K., Pang, J., Wang, J., Xiong, Y ., Li, X., Sun, S., Feng, W., Liu, Z., Shi, J., Ouyang, W., et al.,

work page 2023

[2] [4]

arXiv preprint arXiv:2311.00571

Llava-interactive: An all-in-one demo for image chat, segmentation, generation and editing. arXiv preprint arXiv:2311.00571 . Cheng, B., Misra, I., Schwing, A.G., Kirillov, A., Girdhar, R., 2022a. Masked-attention mask transformer for universal image segmentation, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1...

arXiv

[3] [6]

arXiv preprint arXiv:1808.10437

ican: Instance-centric attention network for human-object interaction detection. arXiv preprint arXiv:1808.10437 . Gkioxari, G., Girshick, R., Dollár, P., He, K.,

arXiv

[4] [8]

Visual compositional learning for human-object interaction detection, in: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XV 16, Springer. pp. 584–600. Hou, Z., Yu, B., Qiao, Y ., Peng, X., Tao, D., 2021a. Affordance transfer learning for human-object interaction detection, in: Proceedings of the IEE...

work page 2020

[5] [11]

arXiv preprint arXiv:2305.05662

Interngpt: Solving vision-centric tasks by interacting with chatgpt beyond language. arXiv preprint arXiv:2305.05662 . Long, J., Shelhamer, E., Darrell, T.,

arXiv

[6] [13]

arXiv preprint arXiv:2309.16524

Hoi4abot: Human-object interaction anticipation for human intention reading collaborative robots. arXiv preprint arXiv:2309.16524 . Milletari, F., Navab, N., Ahmadi, S.A.,

arXiv

[7] [16]

19717–19728

Diffusion-guided reconstruction of everyday hand-object interaction clips, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 19717–19728. Zhang, A., Liao, Y ., Liu, S., Lu, M., Wang, Y ., Gao, C., LI, X., 2021a. Mining the benefits of two-stage and one-stage hoi detection, in: Ranzato, M., Beygelzimer, A., Dauphin, Y ., Lian...

arXiv

[8] [17]

arXiv preprint arXiv:2010.04159

Deformable detr: Deformable transformers for end-to-end object detection. arXiv preprint arXiv:2010.04159 . Zou, C., Wang, B., Hu, Y ., Liu, J., Wu, Q., Zhao, Y ., Li, B., Zhang, C., Zhang, C., Wei, Y ., et al.,

arXiv 2010