SpatialSense: An Adversarially Crowdsourced Benchmark for Spatial Relation Recognition

Jia Deng; Kaiyu Yang; Olga Russakovsky

REVIEW 2 major objections 5 minor 1 cited by

SpatialSense: An Adversarially Crowdsourced Benchmark for Spatial Relation Recognition

T0 review · 2 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that state-of-the-art spatial relation recognition models perform no better than simple 2D and language baselines on a benchmark collected adversarially to remove easy cues.

desk verdict A valuable spatial-relations benchmark whose headline comparison is weakened by an unspecified binary decision rule; clarify the threshold and it's a solid accept. read the letter →

arxiv 1908.02660 v2 pith:YML3BA2I submitted 2019-08-07 cs.CV

Kaiyu Yang , Olga Russakovsky , Jia Deng This is my paper · ORCID

classification cs.CV

keywords spatialrelationrecognitionvisualrelationshipdetectionadversarialcrowdsourcingdatasetbiasbenchmarklanguagepriors2Dcuesreasoning

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that state-of-the-art systems for recognizing spatial relations between objects mostly exploit shallow shortcuts instead of reasoning about the scene. To prove it, the authors built SpatialSense, a benchmark of 17,498 relations on 11,569 images, collected by asking crowd workers to find relations that fool a simple robot relying only on object names and 2D bounding-box positions. On this benchmark the best published model reaches 71.3% accuracy, essentially tied with a tuned language-plus-2D baseline at 71.1%, and the models' error patterns line up with that baseline. The paper concludes that adversarial crowdsourcing can strip out dataset bias and that progress on spatial reasoning needs benchmarks of this kind.

What carries the argument

The load-bearing mechanism is an adversarial crowdsourcing protocol in which the 'robot' is an ensemble of a language-only model (word2vec embeddings of object names through a GRU, plus a predicate encoding) and a 2D-only model (linear layers over bounding-box coordinates). An annotator must propose a positive or negative spatial relation from a fixed nine-predicate vocabulary: the relation is kept only if the robot guesses wrong, with the robot explaining which cue gave it away. This filters out examples answerable from object names or box geometry alone, producing a testbed where simple priors fail (language accuracy 56.4% and 2D accuracy 65.2%, versus 69.2% and 71.3% on the non-adversarial ablation dataset).

What would settle it

Train a model on SpatialSense that exceeds the language-plus-2D baseline by a wide margin (for example, above roughly 75% accuracy) without additional supervision, or show that a stronger 2D-only heuristic - using relative box area, aspect ratio, or estimated depth - reaches or beats DRNet's 71.3%. Either result would show that current models can go beyond the robot's simple cues and that the benchmark's difficulty is not the real ceiling for spatial reasoning.

Watch

Extended reading notes

Core claim

The central discovery is that when spatial-relation examples are selected adversarially to defeat language and 2D-geometry priors, the accuracy of state-of-the-art visual relationship models collapses to the level of simple baselines. DRNet, the best model evaluated, scores 71.3%, while a weighted combination of a language-only and 2D-only baseline scores 71.1%; the error vectors of the neural models are strongly correlated with the 2D baseline (correlations of 0.31-0.60 for most models), indicating they make the same kinds of mistakes. Humans score 94.6% on the same test data, so the ceiling is not the task itself. Adversarial crowdsourcing - human annotators propose relations that a robot ensemble cannot guess - reduces both language bias and 2D bias relative to earlier datasets, as verified by lower baseline accuracies and better cross-dataset generalization.

Load-bearing premise

The adversarial robot, an ensemble of a GRU-based language model and a linear 2D model, is assumed to catch every simple statistical cue a real model could exploit; if it misses an easy regularity, SpatialSense still contains hidden bias and the conclusion that state-of-the-art models rely on shallow cues is overstated.

Editorial extensions

If this is right

The best evaluated model, DRNet, scores 71.3% while the language-plus-2D baseline scores 71.1%, implying that current visual-relationship models do not deliver genuine spatial reasoning on this benchmark.
Error correlation values of 0.31-0.60 between the neural models and the 2D-only baseline indicate that these models make predictions very similar to simple geometry, not deep scene understanding.
Adversarial crowdsourcing reduces dataset bias: baseline accuracy drops from the non-adversarial SpatialNaive dataset (69.2% language, 71.3% 2D) to SpatialSense (56.4% language, 65.2% 2D).
SpatialSense's positive-only split is less biased in language and 2D than existing visual-relationship datasets: models trained on SpatialSense generalize to other datasets with an accuracy increase of 1.8% (language) and 5.2% (2D), while models trained on the other datasets drop by roughly 26-32% when evaluated across datasets.
Humans score 94.6% on the test split, so the gap between machines and humans is large: SpatialSense offers a benchmark where further progress in spatial reasoning can be measured.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A stronger language-only prior than the GRU+word2vec model used in the robot might recover additional predictability from object names alone, suggesting that the residual language bias in SpatialSense could be underestimated.
Applying the same adversarial protocol to other relational tasks, such as compositional question answering or action prediction, could yield test sets where reported accuracy reflects genuine reasoning rather than corpus statistics.
The inclusion of 1,389 RGB-D images invites a direct test: if depth input lets a model substantially beat the 2D baseline, the remaining difficulty is genuinely 3D; if it does not, the bottleneck lies elsewhere.
Adversarially selected test sets could be used as a model-selection criterion during development, since they penalize shortcut reliance that biased benchmarks reward.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Kaiyu, this one is worth a look. The headline result—SOTA models at roughly 70% versus humans at 94.6% on spatial relations—is the kind of number that gets attention, and the dataset is a genuine contribution. But before trusting the specific claim that DRNet (71.3%) merely matches a tuned 2D+language baseline (71.1%), check how the binary decision was made. I couldn't find it in the paper, and the stress-test note is on point: the adapted 9-way detectors output logits (e.g., 'on: 3.45'), and the paper never states how a logit becomes a yes/no prediction. If the threshold was not tuned on validation, or if negative examples were not used sensibly during training, that 0.2-point tie could move. This is a fixable reporting gap, but it is load-bearing for the paper's central comparison.

What is genuinely new: the adversarial crowdsourcing protocol, borrowed from VQA, fits spatial relations well, and the paper shows it works. The ablation against SpatialNaive is the strongest evidence: language baseline drops from 69.2% to 56.4%, 2D from 71.3% to 65.2% when the robot is used, so the selection procedure did what it was designed to do. The balanced positives/negatives and the long-tail object classes are real improvements over VRD and Visual Genome. The cross-dataset generalization numbers are a nice bonus, and the human evaluation at 94.6% gives a clear target for future work.

Soft spots, in proportion: the threshold/negative-training issue is the big one; it needs a precise description and, ideally, a sensitivity analysis showing SOTA numbers are stable across thresholds. There are no error bars anywhere, so differences of 1–2% between models are probably not meaningful. The 80/20 fusion weight for the language+2D baseline was tuned on validation while the SOTA models were not given the same treatment—that asymmetry compounds the threshold concern. The adversarial robot is a weak proxy for 'all simple cues,' but the SpatialNaive comparison mitigates that concern, so I'd call it minor.

Bottom line: this is a useful benchmark and the main qualitative conclusion—current models are not doing deep spatial reasoning—is probably right. It deserves a serious referee and, with a clarified evaluation protocol, it should be accepted. I'd cite it and bring it to the reading group.

Referee Report

2 major / 5 minor

Summary. The paper presents SpatialSense, a benchmark for spatial relation recognition built through adversarial crowdsourcing: annotators propose positive or negative spatial relations that must fool a robot that uses only object names and 2D coordinates. The dataset contains 17,498 relations over 9 predicates with balanced positives and negatives. The authors show reduced language and 2D bias relative to VRD and Visual Genome, verify the effect through an ablation (SpatialNaive), benchmark five visual relationship detection models, and report that the best state-of-the-art model (DRNet, 71.3%) performs on par with a tuned language+2D baseline (71.1%), while humans achieve 94.6%.

Significance. If the experimental claims are fully supported, this is a strong contribution: it provides a reusable benchmark for spatial reasoning, demonstrates a practical adversarial crowdsourcing protocol, and offers evidence that current visual relationship models rely on shallow cues. The dataset and code release, the SpatialNaive ablation, and the cross-dataset generalization analysis are valuable. The central quantitative conclusion, however, rests on an evaluation protocol that is currently underspecified and asymmetric, so the headline result should be regarded as preliminary until the experimental details are clarified.

major comments (2)

[§5 Model architectures; Table 3] The adaptation of the five relation detectors to binary classification is not fully specified. For each relation, the model outputs scores for all nine predicates, and "the score for that particular predicate is the final binary classification score" (§5). The paper never states how that real-valued score is converted to a binary prediction (e.g., threshold at 0, or comparison with other predicate scores), nor how negative examples are used to train models originally designed for positive triplet ranking. Because the language+2D baseline is a calibrated binary classifier with its fusion weight tuned on validation, an arbitrary or untuned threshold for the SOTA models could artificially suppress or inflate their accuracies. Please specify the exact decision rule, tune the threshold on validation (or report ROC/PR curves), and describe the negative-example training protocol so that the comparison in Table 3 is meaningful.
[§5 Implementation details] The evaluation is asymmetric with respect to hyperparameter tuning: the 80/20 fusion weight of the language+2D baseline is chosen by validation accuracy, while the five SOTA models are adapted off-the-shelf with no comparable validation-based selection of thresholds or hyperparameters. This is acceptable for a lower-bound demonstration, but the claim that SOTA models "perform comparably to simple baselines" (Abstract; §6) needs to be qualified, or the SOTA models need to be given the same tuning budget, before the conclusion can be stated as a property of the models rather than of the evaluation setup.

minor comments (5)

[§4.1; Table 1] The claim that SpatialSense is "less susceptible to simple cues" should be explicitly scoped to the specific language and 2D models used; adversarial crowdsourcing guarantees hardness only against the particular robot, and a stronger language prior or geometric heuristic could conceivably still exploit regularities in the dataset.
[§5 Human evaluation] Excluding annotators who answer "yes" more than 80% of the time is a reasonable quality-control step, but it can bias the human accuracy estimate upward; please report the number of excluded annotators and test the robustness of the 94.6% figure to this exclusion rule.
[Table 3] The table reports single accuracies without confidence intervals or significance tests; the differences between DRNet (71.3%), VTransE (69.4%), and the baseline (71.1%) are small relative to likely sampling noise, and paired bootstrap intervals or McNemar tests would strengthen the comparative claims.
[§4.2; Table 2] For the SpatialSense/SpatialNaive comparison, please state the number of relations used in the subsampled SpatialSense split and whether the same training/validation/test splits were used across the two datasets, so that the accuracy drops in Table 2 are directly interpretable.
[Appendix B; Figure D] The DRNet description notes that "we do not perform iterative updates to the object name features"; it would help to state explicitly whether this deviates from the original DRNet training procedure and whether the model was pretrained on any other dataset before adaptation to the binary spatial-relation task.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the low simple-baseline scores are a transparent design feature of adversarial crowdsourcing, and the central claims about state-of-the-art models are not by construction.

full rationale

The paper's load-bearing claim is that state-of-the-art relationship detectors (DRNet, VTransE, Vip-CNN, etc.) perform no better than simple language+2D baselines on SpatialSense (Table 3), and that their error patterns align with the 2D baseline (Table 4). These claims are not circular: the SOTA models were not part of the adversarial collection robot, so their accuracy and error correlations are measured rather than imposed by the dataset definition. The fact that the language-only and 2D-only baselines score lower on SpatialSense than on VRD/VG is partly by design, since Section 3 defines SpatialSense as the set of relations that fool a robot composed of a language-only model and a 2D-only model. However, the paper openly presents this as the intended purpose of adversarial crowdsourcing, and the SpatialNaive ablation (Table 2) provides an independent control showing that the selection procedure, not merely the annotation task, drives the observed difficulty. The only self-citation is the hourglass network [25] used as an internal component of DRNet; it is an implementation detail and is not load-bearing for any benchmark conclusion. The thresholding/evaluation asymmetry concern raised by a skeptic is a measurement-protocol question, not a circularity, because it does not reduce the paper's conclusions to its inputs by construction.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The paper is a dataset paper with no mathematical derivation, so there are no physical constants or invented entities. The free parameter listed is a minor evaluation hyperparameter. The axioms are domain assumptions about label reliability, the adequacy of the adversarial robot as a proxy for simple cues, and image representativeness.

free parameters (1)

weight of 2D baseline in combined baseline = 0.8
Chosen by validation accuracy in Section 5. This is a hyperparameter of the evaluation, not a fitted constant in a physics or modeling sense, but it does affect the reported near-tie between the combined baseline and DRNet.

assumptions (3)

domain assumption Spatial relations can be reliably labeled with the chosen set of nine predicates as a binary classification task.
The paper relies on this to construct the benchmark. Human evaluation shows 94.6% agreement, so the assumption is reasonable but not perfect: some relations are inherently ambiguous.
domain assumption The language-only and 2D-only robot models sufficiently capture the simple cues that could be exploited.
Section 3 uses the robot as the gate for adversarial selection. If the robot misses some simple statistical regularities, the dataset retains bias and the main conclusion is weakened.
domain assumption Images from Flickr and NYU Depth, with annotators choosing among eight candidate images, are representative enough for a spatial reasoning benchmark.
This is implicit in the dataset design. The selection process could introduce distributional biases, but the paper does not analyze this risk.

how reviews work

0 comments

Cite this review

Pith. "Pith review of SpatialSense: An Adversarially Crowdsourced Benchmark for Spatial Relation Recognition." pith.science (2026). https://pith.science/paper/YML3BA2I

@misc{pith2026190802660,
  author       = {Pith},
  title        = {Pith review of: SpatialSense: An Adversarially Crowdsourced Benchmark for Spatial Relation Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YML3BA2I}},
  note         = {Machine review of arXiv:1908.02660}
}

read the original abstract

Understanding the spatial relations between objects in images is a surprisingly challenging task. A chair may be "behind" a person even if it appears to the left of the person in the image (depending on which way the person is facing). Two students that appear close to each other in the image may not in fact be "next to" each other if there is a third student between them. We introduce SpatialSense, a dataset specializing in spatial relation recognition which captures a broad spectrum of such challenges, allowing for proper benchmarking of computer vision techniques. SpatialSense is constructed through adversarial crowdsourcing, in which human annotators are tasked with finding spatial relations that are difficult to predict using simple cues such as 2D spatial configuration or language priors. Adversarial crowdsourcing significantly reduces dataset bias and samples more interesting relations in the long tail compared to existing datasets. On SpatialSense, state-of-the-art recognition models perform comparably to simple baselines, suggesting that they rely on straightforward cues instead of fully reasoning about this complex task. The SpatialSense benchmark provides a path forward to advancing the spatial reasoning capabilities of computer vision systems. The dataset and code are available at https://github.com/princeton-vl/SpatialSense.

Figures

Figures reproduced from arXiv: 1908.02660 by the authors.

**Figure 2.** When collecting negative examples, the annotator picks a pair of objects and lies about their spatial relation (“stove [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** (Left) The predicate distributions of frequent objects in VRD-Spatial, VG-Spatial and SpatialSense-Positive. For [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: The 2D locations of subjects relative to objects [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 5.** Figure 5: Failing examples of the language and 2D baselines (p [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Do Vision-Language Models Have Internal World Models? Towards an Atomic Evaluation
cs.CL 2025-06 conditional novelty 6.0 of 10

A new 23-dimension benchmark finds that even the best VLMs score near random on motion trajectory, temporal extension, and several prediction tasks, far below humans, suggesting weak internal world models.

Reference graph

Works this paper leans on

40 extracted references · 30 canonical work pages · cited by 1 Pith paper

[1]

Dont just assume; look and answer: Over- coming priors for visual question answering

Aishwarya Agrawal, Dhruv Batra, Devi Parikh, and Anirud- dha Kembhavi. Dont just assume; look and answer: Over- coming priors for visual question answering. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4971–4980, 2018

work page 2018
[2]

Beat the machine: Challenging workers to ﬁnd the unknown unknowns

Josh Attenberg, Panagiotis G Ipeirotis, and Foster J Provost. Beat the machine: Challenging workers to ﬁnd the unknown unknowns. Human Computation, 11(11), 2011

work page 2011
[3]

Being nega- tive but constructively: Lessons learnt from creating bet- ter visual question answering datasets

Wei-Lun Chao, Hexiang Hu, and Fei Sha. Being nega- tive but constructively: Lessons learnt from creating bet- ter visual question answering datasets. arXiv preprint arXiv:1704.07121, 2017

arXiv 2017
[4]

Learning phrase representations using rnn encoder-decoder for statistical machine translation

Kyunghyun Cho, Bart Van Merri ¨enboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078, 2014

arXiv 2014
[5]

Detecting visual rela- tionships with deep relational networks

Bo Dai, Yuqi Zhang, and Dahua Lin. Detecting visual rela- tionships with deep relational networks. InThe IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR) , July 2017

work page 2017
[6]

Making the v in vqa matter: Elevating the role of image understanding in visual question answer- ing

Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Ba- tra, and Devi Parikh. Making the v in vqa matter: Elevating the role of image understanding in visual question answer- ing. In The IEEE Conference on Computer Vision and Pat- tern Recognition (CVPR), July 2017

work page 2017
[7]

Grounding spatial relations for human-robot in- teraction

Sergio Guadarrama, Lorenzo Riano, Dave Golland, Daniel Go, Yangqing Jia, Dan Klein, Pieter Abbeel, Trevor Dar- rell, et al. Grounding spatial relations for human-robot in- teraction. In Intelligent Robots and Systems (IROS), 2013 IEEE/RSJ International Conference on , pages 1640–1647. IEEE, 2013

work page 2013
[8]

Deep residual learning for image recognition

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

2016

Show all 40 references

[9]

Batch normalization: Accelerating deep network training by reducing internal co- variate shift

Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal co- variate shift. In International Conference on Machine Learn- ing, pages 448–456, 2015

2015
[10]

Lawrence Zitnick, and Ross Girshick

Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Li Fei-Fei, C. Lawrence Zitnick, and Ross Girshick. Clevr: A diagnostic dataset for compositional language and elemen- tary visual reasoning. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017

2017
[11]

An analysis of visual question answering algorithms

Kushal Kaﬂe and Christopher Kanan. An analysis of visual question answering algorithms. In Computer Vision (ICCV), 2017 IEEE International Conference on , pages 1983–1991. IEEE, 2017

2017
[12]

Active learning with gaussian processes for object categorization

Ashish Kapoor, Kristen Grauman, Raquel Urtasun, and Trevor Darrell. Active learning with gaussian processes for object categorization. In Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on , pages 1–8. IEEE, 2007

2007
[13]

Visual genome: Connecting language and vision using crowdsourced dense image annotations

Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalan- tidis, Li-Jia Li, David A Shamma, Michael Bernstein, and Li Fei-Fei. Visual genome: Connecting language and vision using crowdsourced dense image annotations. IJCV, 2017

2017
[14]

The open images dataset v4: Uniﬁed image classiﬁcation, object detection, and visual relationship detection at scale

Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Ui- jlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali, Stefan Popov, Matteo Malloci, Tom Duerig, and Vittorio Ferrari. The open images dataset v4: Uniﬁed image classiﬁcation, object detection, and visual relationship detect...

2018 arXiv
[15]

Whence and whither in spatial language and spatial cognition? Behavioral and brain sciences, 16(2):255–265, 1993

Barbara Landau and Ray Jackendoff. Whence and whither in spatial language and spatial cognition? Behavioral and brain sciences, 16(2):255–265, 1993

1993
[16]

Vip-cnn: Visual phrase guided convolutional neural network

Yikang Li, Wanli Ouyang, Xiaogang Wang, and Xiao’ou Tang. Vip-cnn: Visual phrase guided convolutional neural network. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017

2017
[17]

Factorizable net: an efﬁcient subgraph-based framework for scene graph generation

Yikang Li, Wanli Ouyang, Bolei Zhou, Jianping Shi, Chao Zhang, and Xiaogang Wang. Factorizable net: an efﬁcient subgraph-based framework for scene graph generation. In Proceedings of the European Conference on Computer Vi- sion (ECCV), pages 335–351, 2018

2018
[18]

Scene graph generation from objects, phrases and region captions

Yikang Li, Wanli Ouyang, Bolei Zhou, Kun Wang, and Xi- aogang Wang. Scene graph generation from objects, phrases and region captions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 1261– 1270, 2017

2017
[19]

Xiaodan Liang, Lisa Lee, and Eric P. Xing. Deep variation- structured reinforcement learning for visual relationship and attribute detection. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017

2017
[20]

Microsoft coco: Common objects in context

Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European Conference on Computer Vision, pages 740–755. Springer, 2014

2014
[21]

Visual relationship detection with language priors

Cewu Lu, Ranjay Krishna, Michael Bernstein, and Li Fei- Fei. Visual relationship detection with language priors. In European Conference on Computer Vision, pages 852–869. Springer, 2016

2016
[22]

Explicit bias discovery in visual question answering models

Varun Manjunatha, Nirat Saini, and Larry S Davis. Explicit bias discovery in visual question answering models. In Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 9562–9571, 2019

2019
[23]

Distributed representations of words and phrases and their compositionality

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111–3119, 2013

2013
[24]

Rectiﬁed linear units im- prove restricted boltzmann machines

Vinod Nair and Geoffrey E Hinton. Rectiﬁed linear units im- prove restricted boltzmann machines. In Proceedings of the 27th international conference on machine learning (ICML- 10), pages 807–814, 2010

2010
[25]

Stacked hour- glass networks for human pose estimation

Alejandro Newell, Kaiyu Yang, and Jia Deng. Stacked hour- glass networks for human pose estimation. InEuropean Con- ference on Computer Vision, pages 483–499. Springer, 2016

2016
[26]

Automatic differentiation in pytorch

Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Al- ban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. In NIPS-W, 2017

2017
[27]

Weakly-supervised learning of visual relations

Julia Peyre, Josef Sivic, Ivan Laptev, and Cordelia Schmid. Weakly-supervised learning of visual relations. In The IEEE International Conference on Computer Vision (ICCV) , Oct 2017

2017
[28]

Recognition us- ing visual phrases

Mohammad Amin Sadeghi and Ali Farhadi. Recognition us- ing visual phrases. In Computer Vision and Pattern Recogni- tion (CVPR), 2011 IEEE Conference on , pages 1745–1752. IEEE, 2011

2011
[29]

Indoor segmentation and support inference from rgbd images

Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from rgbd images. In European Conference on Computer Vision, pages 746–760. Springer, 2012

2012
[30]

Unbiased look at dataset bias

Antonio Torralba and Alexei A Efros. Unbiased look at dataset bias. In ICCV, 2011

2011
[31]

Large- scale live active learning: Training object detectors with crawled data and crowds

Sudheendra Vijayanarasimhan and Kristen Grauman. Large- scale live active learning: Training object detectors with crawled data and crowds. International Journal of Computer Vision, 108(1-2):97–114, 2014

2014
[32]

Linknet: Relational embedding for scene graph

Sanghyun Woo, Dahun Kim, Donghyeon Cho, and In So Kweon. Linknet: Relational embedding for scene graph. In Advances in Neural Information Processing Systems , pages 558–568, 2018

2018
[33]

Choy, and Li Fei-Fei

Danfei Xu, Yuke Zhu, Christopher B. Choy, and Li Fei-Fei. Scene graph generation by iterative message passing. In The IEEE Conference on Computer Vision and Pattern Recogni- tion (CVPR), July 2017

2017
[34]

Shufﬂe-then- assemble: Learning object-agnostic visual relationship fea- tures

Xu Yang, Hanwang Zhang, and Jianfei Cai. Shufﬂe-then- assemble: Learning object-agnostic visual relationship fea- tures. In Proceedings of the European Conference on Com- puter Vision (ECCV), pages 36–52, 2018

2018
[35]

Morariu, and Larry S

Ruichi Yu, Ang Li, Vlad I. Morariu, and Larry S. Davis. Vi- sual relationship detection with internal and external linguis- tic knowledge distillation. In The IEEE International Con- ference on Computer Vision (ICCV), Oct 2017

2017
[36]

Semantic robot programming for goal- directed manipulation in cluttered scenes

Zhen Zeng, Zheming Zhou, Zhiqiang Sui, and Odest Chad- wicke Jenkins. Semantic robot programming for goal- directed manipulation in cluttered scenes. In 2018 IEEE In- ternational Conference on Robotics and Automation (ICRA), pages 7462–7469. IEEE, 2018

2018
[37]

Visual translation embedding network for visual relation detection

Hanwang Zhang, Zawlin Kyaw, Shih-Fu Chang, and Tat- Seng Chua. Visual translation embedding network for visual relation detection. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017

2017
[38]

Ppr-fcn: Weakly supervised visual relation detec- tion via parallel pairwise r-fcn

Hanwang Zhang, Zawlin Kyaw, Jinyang Yu, and Shih-Fu Chang. Ppr-fcn: Weakly supervised visual relation detec- tion via parallel pairwise r-fcn. In The IEEE International Conference on Computer Vision (ICCV), Oct 2017

2017
[39]

Yin and yang: Balancing and an- swering binary visual questions

Peng Zhang, Yash Goyal, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. Yin and yang: Balancing and an- swering binary visual questions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 5014–5022, 2016

2016
[40]

Towards context-aware interaction recognition for visual re- lationship detection

Bohan Zhuang, Lingqiao Liu, Chunhua Shen, and Ian Reid. Towards context-aware interaction recognition for visual re- lationship detection. In The IEEE International Conference on Computer Vision (ICCV), Oct 2017

2017

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Dont just assume; look and answer: Over- coming priors for visual question answering

Aishwarya Agrawal, Dhruv Batra, Devi Parikh, and Anirud- dha Kembhavi. Dont just assume; look and answer: Over- coming priors for visual question answering. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4971–4980, 2018

work page 2018

[2] [2]

Beat the machine: Challenging workers to ﬁnd the unknown unknowns

Josh Attenberg, Panagiotis G Ipeirotis, and Foster J Provost. Beat the machine: Challenging workers to ﬁnd the unknown unknowns. Human Computation, 11(11), 2011

work page 2011

[3] [3]

Being nega- tive but constructively: Lessons learnt from creating bet- ter visual question answering datasets

Wei-Lun Chao, Hexiang Hu, and Fei Sha. Being nega- tive but constructively: Lessons learnt from creating bet- ter visual question answering datasets. arXiv preprint arXiv:1704.07121, 2017

arXiv 2017

[4] [4]

Learning phrase representations using rnn encoder-decoder for statistical machine translation

Kyunghyun Cho, Bart Van Merri ¨enboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078, 2014

arXiv 2014

[5] [5]

Detecting visual rela- tionships with deep relational networks

Bo Dai, Yuqi Zhang, and Dahua Lin. Detecting visual rela- tionships with deep relational networks. InThe IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR) , July 2017

work page 2017

[6] [6]

Making the v in vqa matter: Elevating the role of image understanding in visual question answer- ing

Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Ba- tra, and Devi Parikh. Making the v in vqa matter: Elevating the role of image understanding in visual question answer- ing. In The IEEE Conference on Computer Vision and Pat- tern Recognition (CVPR), July 2017

work page 2017

[7] [7]

Grounding spatial relations for human-robot in- teraction

Sergio Guadarrama, Lorenzo Riano, Dave Golland, Daniel Go, Yangqing Jia, Dan Klein, Pieter Abbeel, Trevor Dar- rell, et al. Grounding spatial relations for human-robot in- teraction. In Intelligent Robots and Systems (IROS), 2013 IEEE/RSJ International Conference on , pages 1640–1647. IEEE, 2013

work page 2013

[8] [8]

Deep residual learning for image recognition

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

2016