Pith. sign in

REVIEW 3 major objections 4 minor 51 references

Reflective Decoding Network for Image Captioning

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that adding reflective attention over previously generated words and supervised relative-position targets to an LSTM image-caption decoder improves caption quality by about 2 CIDEr points, with the largest gains on…

desk verdict A plausible decoder-side mechanism with a clean ablation, but the position module's benefit isn't shown to be about position, and the SOTA claim is oversold. read the letter →

arxiv 1908.11824 v1 pith:JPRCW53M submitted 2019-08-30 cs.CV

Lei Ke , Wenjie Pei , Ruiyu Li , Xiaoyong Shen , Yu-Wing Tai This is my paper · ORCID
classification cs.CV
keywords imagecaptioningreflectivedecodingnetworklong-termdependencytextualattentionvisualrelativepositionmodelingCOCOLSTMdecoder
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Most image-captioning research improves the visual encoder; this paper argues the decoder's language side is also undertrained. It proposes a Reflective Decoding Network (RDN) that adds to a standard two-layer LSTM decoder a reflective attention head, which lets each word prediction look back at all earlier decoder hidden states, and a position module, which learns where each word sits in the sentence. On the COCO benchmark the full model beats its own LSTM baseline by about 2 CIDEr points (a standard caption-quality metric) and matches or outperforms published single-model systems, with the largest gains on images needing long, detailed captions. The authors read this as evidence that long-range word coherence and syntactic positioning are learnable decoder-side abilities, not just by-products of better visual features.

What carries the argument

The load-bearing object is the Reflective Decoding Network (RDN), a two-layer LSTM caption decoder with two attachable modules. The Reflective Attention Module (RAM) is an attention layer that takes the current hidden state $h^2_t$ as a query and scores all past hidden states $\{h^2_1,\dots,h^2_{t-1}\}$, producing a weighted combination that is used for word prediction; its role is to make long-range word-to-word dependencies explicit rather than relying on the LSTM's compressed memory. The Reflective Position Module (RPM) predicts the relative position $I^p_t$ of each word and penalizes its deviation from $t/n$ with an $\ell^2$ loss, injecting a supervised signal about syntactic order into the representation. The two modules share the same query value and are trained jointly, which the paper argues keeps the textual reflection aligned with visual attention.

What would settle it

Train the full RDN with the relative-position targets $I^r_t$ replaced by random values in $[0,1]$ while keeping every other setting identical; if CIDEr on the COCO test split does not drop below the model trained with true $t/n$ targets, the position module's claimed syntactic supervision is not what carries the improvement.

Watch

Extended reading notes

Core claim

The paper's central claim is that a caption decoder can be made better at long-range language modeling without changing the visual encoder. At each time step $t$, the Reflective Attention Module computes a weight $\alpha^{\mathrm{ref}}_{i,t}$ over every previously produced decoder hidden state $h^2_i$ and predicts the next word from the weighted sum $\hat{h}^2_t = \sum_i \alpha^{\mathrm{ref}}_{i,t} h^2_i$; this gives later words direct, visible access to earlier words such as 'bridge' when predicting 'river'. In parallel, the Reflective Position Module supervises a scalar prediction $I^p_t = \sigma(W_l \hat{h}^2_t)$ against the word's true relative position $I^r_t = t/n$ in the ground-truth caption, adding a position-perception loss $L_{\mathrm{pos}}$ to the cross-entropy captioning loss. During inference the position target is unknown, so the position module is used only as a training signal that shapes the hidden states. The authors report that the combined RDN reaches 115.3 CIDEr on the COCO Karpathy test split versus 113.2 for its LSTM baseline, and that the improvement grows when captions must be long.

Load-bearing premise

The load-bearing premise is that teaching the decoder each word's relative position inside the ground-truth caption (position $t$ in a caption of length $n$) gives it genuinely useful word-order knowledge, and that this knowledge still improves decoding when the position signal is withheld at inference time.

Editorial extensions

If this is right

  • On the COCO Karpathy test split, the single-model RDN scores 115.3 CIDEr, 2.1 points above its own LSTM baseline, and reaches 117.3 CIDEr as a six-model ensemble.
  • When the test set is ranked by average ground-truth caption length, RDN's advantage over the Up-Down baseline grows on the harder subsets, supporting the claim that reflective attention specifically helps long sentences.
  • The reflective attention weights are interpretable: when predicting a word like 'river', the model assigns large weight to earlier words such as 'bridge', revealing which past words influence the decision.
  • The position module's predicted relative positions track the actual positions closely, showing the decoder has learned a usable sense of sentence progress.
  • The total parameter increase of RDN over the baseline is under one percent, so the gains are attributed to the new decoder structure rather than capacity.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable implication not explored in the paper: the position loss may be acting mainly as a regularizer on teacher-forced training. Training RDN with shuffled or randomized position targets $I^r_t$ and checking whether the CIDEr gain survives would separate these two roles.
  • Because RAM and RPM sit entirely on the decoder side, they should combine with any stronger encoder; attaching them to a multi-feature fusion encoder such as RFNet's is a natural next step the paper itself suggests.
  • The relative-position supervision scheme is language-agnostic and could transfer to other sequence-generation tasks such as text summarization or source-code captioning; applying the same ablation on a non-visual task would isolate whether the position signal or the visual context produces the benefit.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes the Reflective Decoding Network (RDN) for image captioning. RDN extends a standard attention-based LSTM decoder with two modules: a Reflective Attention Module (RAM), which applies attention over previously generated decoder hidden states before predicting each word, and a Reflective Position Module (RPM), which is trained with an auxiliary loss to predict each word's relative position in the sentence. At inference the position signal is removed, so the RPM is intended to improve the learned hidden-state representation rather than to provide an explicit position feature. The model is evaluated on COCO using Karpathy splits and the online test server, with ablations showing each module improves over the baseline and with qualitative visualizations of the reflective attention weights and predicted relative positions.

Significance. If the reported effects are reliable, the paper makes a moderate contribution by showing that an attachable decoder-side attention module and a position-supervision auxiliary loss can improve captioning quality over a competitive bottom-up baseline. The paper includes a clear ablation, standard benchmark comparisons, and useful interpretability analysis of the textual attention. Its main claims, however, are stronger than the evidence: the state-of-the-art assertion is contradicted on several metrics by the paper's own tables, and the inference-time benefit of the position module is not causally separated from generic auxiliary-loss regularization. The central idea is defensible, but the paper needs additional experiments and a more calibrated framing.

major comments (3)
  1. [Section 3.2.2 (Eq. 8) and Section 3.3] The Reflective Position Module's contribution at inference time is not established. As defined in Eq. 8, the supervised target I_t^r = t/n is available only during training, and Section 3.3 states that the relative-position information is removed at inference. The +1.1 CIDEr improvement of RDN_pos over the baseline in Table 1 could therefore be a generic effect of the auxiliary loss (for example, smoothing the hidden-state trajectory or reducing exposure bias) rather than evidence that the model learns genuine syntactic-position knowledge. A control experiment with randomized or constant position targets, matched in loss scale, is needed to separate these explanations. The paper should also clarify how Figure 5's predicted relative positions are obtained at inference if the module is not used during decoding.
  2. [Section 4.2 / Table 1] The ablation numbers are reported inconsistently. The text states that the full RDN reaches CIDEr 115.7, BLEU-4 37.0, and BLEU-3 47.9, while Table 1 lists CIDEr 115.3, BLEU-4 36.8, and BLEU-3 47.9 for the same model. Since the improvements over the baseline are only about 1-2 CIDEr points, this discrepancy must be resolved, and the single-run results should be supplemented with variance across seeds. Without corrected tables or error bars, the claimed gains from the two proposed modules cannot be assessed for significance.
  3. [Abstract, Section 4.3.1, Tables 2 and 4] The state-of-the-art claims are not supported by the presented comparisons. In Table 2 the single RDN has METEOR 27.2, below RFNet's 27.4, and matches RFNet on SPICE, so it is not 'best across all six metrics.' In Table 4 the online result is also below RFNet on BLEU-4 (c5), METEOR (c5), and CIDEr (c5). The abstract and conclusion should be reframed to claim competitive performance or state-of-the-art on selected metrics, and the speculative paragraph explaining RFNet's offline/online inconsistency should be removed unless supporting evidence is provided.
minor comments (4)
  1. [Section 4.3.1 and Figure 3] The 'hard image captioning' analysis uses average annotation length as an indirect proxy for captioning difficulty, but no error bars or statistical tests are reported, and the comparison is only against Up-Down rather than the main competitor RFNet; the conclusion should be correspondingly qualified.
  2. [Throughout] There are several typos and inconsistencies: 'Karparthy' appears instead of 'Karpathy', 'RDN ref' is used inconsistently with 'RDN_ref', 'four folds' should be 'four-fold', and Section 3 contains 'the our RDN'.
  3. [Section 3.3, Eq. 9] The cross-entropy loss in Eq. 9 sums from t=2 to n, which appears to omit the likelihood of the first word; if s_0 is the start token, the sum should start at t=1 for consistency with the usual formulation.
  4. [Section 4.3.2 and Figure 5] The description of how predicted relative positions are displayed in Figure 5 is unclear, since Section 3.3 says the relative-position information is removed from the input at inference; please state explicitly whether the position predictor is applied post hoc and which hidden representation it uses.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's improvements are empirical, measured on held-out COCO data, and no load-bearing claim reduces to its inputs by construction.

full rationale

The paper's central claim is empirical: RDN improves captioning by adding reflective attention over previously generated decoder hidden states and a training-only relative-position loss, then evaluates on COCO Karpathy splits and the COCO test server. No equation in the paper defines the output metric in terms of the proposed modules' fitted values. The reflective attention module (Eqs. 5-7) attends to previously generated hidden states, not ground-truth words, so the improved word predictions are not constructed from the evaluation target. The Reflective Position Module (Eqs. 8 and 10) supervises a scalar relative-position prediction during training only; the paper states that during inference the relative position information is removed because final length is unknown, so the module is not used to generate test captions. The reported CIDEr/BLEU gains are therefore externally falsifiable outcomes rather than identities. The concern that the position loss may act as a generic regularizer rather than genuine syntactic-position supervision is a validity or interpretability caveat, not circularity: the target t/n is ground-truth auxiliary supervision, not a renamed evaluation score or a fitted quantity used as the predicted output. There are no load-bearing self-citations, uniqueness theorems, or ansatz-smuggling citations in the paper; references to prior work (Up-Down, RFNet, etc.) are used only as comparisons and building blocks, and the ablation establishes the components' contributions against a shared baseline on a held-out split. The minor inconsistency between the ablation text (CIDEr 115.7, BLEU-4 37.0) and Table 1 (115.3, 36.8) affects precision of reported numbers but does not indicate that any derivation reduces to its inputs. Overall, the derivation chain is self-contained against external benchmarks, and no circularity step can be exhibited.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central result rests on a pretrained object detector, standard LSTM training, and one tuned hyperparameter. No external physical constraints or invented entities are involved, and the method's internal assumptions are listed above.

free parameters (2)
  • position loss weight lambda = 0.02
    Tuned on the Karpathy validation split to maximize validation performance; it balances the cross-entropy loss and the position-perceptive loss in Eq. 11.
  • beam search size = 5
    Chosen for inference decoding; affects all reported results and is standard for captioning systems, but it is a hand-chosen constant rather than a learned parameter.
assumptions (4)
  • domain assumption COCO Karpathy splits and automatic metrics such as CIDEr, BLEU, METEOR, ROUGE-L, and SPICE are reliable proxies for caption quality.
    The paper uses these metrics to claim superiority, so if the metrics mis-rank models the conclusions would not hold.
  • domain assumption Object features from Faster R-CNN pretrained on Visual Genome transfer to COCO captioning.
    The encoder is refined on Visual Genome and then used for COCO; the captioning results depend on this transfer working.
  • ad hoc to paper The relative position target I_t^r = t/n is a meaningful supervisory signal for syntactic structure.
    This is the core premise of the Reflective Position Module in Eq. 8; the paper offers qualitative evidence but no independent validation of the premise.
  • domain assumption LSTM hidden states retain enough word-level semantic information for reflective attention to be useful.
    The Reflective Attention Module attends over previous hidden states, so if these states do not preserve word-level information, the attention weights would be uninformative.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reflective Decoding Network for Image Captioning." pith.science (2026). https://pith.science/paper/JPRCW53M

@misc{pith2026190811824,
  author       = {Pith},
  title        = {Pith review of: Reflective Decoding Network for Image Captioning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JPRCW53M}},
  note         = {Machine review of arXiv:1908.11824}
}
read the original abstract

State-of-the-art image captioning methods mostly focus on improving visual features, less attention has been paid to utilizing the inherent properties of language to boost captioning performance. In this paper, we show that vocabulary coherence between words and syntactic paradigm of sentences are also important to generate high-quality image caption. Following the conventional encoder-decoder framework, we propose the Reflective Decoding Network (RDN) for image captioning, which enhances both the long-sequence dependency and position perception of words in a caption decoder. Our model learns to collaboratively attend on both visual and textual features and meanwhile perceive each word's relative position in the sentence to maximize the information delivered in the generated caption. We evaluate the effectiveness of our RDN on the COCO image captioning datasets and achieve superior performance over the previous methods. Further experiments reveal that our approach is particularly advantageous for hard cases with complex scenes to describe by captions.

Figures

Figures reproduced from arXiv: 1908.11824 by the authors.

Figure 1
Figure 1. Top: Example captions generated by the basis decoder (using traditional LSTM) and our Reflective Decoding Network model. Bottom: The reflective attention weight distribution over the past generated hidden states is shown when predicting the word ‘river’. The thicker line indicates a relatively larger weight and the red line means the largest contribution to the prediction. inference and reason during the generation … view at source ↗
Figure 2
Figure 2. Given an input image, our model first uses Faster [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 2
Figure 2. Overview of our framework. Attref in RAM (Reflective Attention Module) is the attention layer used to selectively attend to the generated decoding hidden states, Attvis layer in Attention-based Recurrent Module determines the attention distribution over the detected image regions. I t p and I t r in RPM (Reflective Position Module) are respectively the t-th predicted and actual relative position in sentence. ture of… view at source ↗
Figures from the paper (3 more)
Figure 3
Figure 3. Figure 3: Performance comparison between our RDN model and [PITH_FULL_IMAGE:figures/full_fig_p007_3.png]
Figure 4
Figure 4. Figure 4: Examples of captions generated by our RDN compared to the basis decoder (using traditional LSTM) and their reflective attention [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Top: Ground-truth caption and the example caption gen￾erated by our RDN model. Bottom: The predicted relative po￾sition value (shown in blue) from the Reflective Position Module and the actual relative position for each word in the sentence. All value reported in integ…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 37 canonical work pages

  1. [1]

    Spice: Semantic propositional image cap- tion evaluation

    Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. Spice: Semantic propositional image cap- tion evaluation. In ECCV, 2016

    work page 2016

  2. [2]

    Bottom-up and top-down attention for image captioning and vqa

    Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. Bottom-up and top-down attention for image captioning and vqa. In CVPR, 2018

    work page 2018

  3. [3]

    Top-down versus bottom-up control of attention in the prefrontal and posterior parietal cortices

    Timothy J Buschman and Earl K Miller. Top-down versus bottom-up control of attention in the prefrontal and posterior parietal cortices. science, 2007

    work page 2007

  4. [4]

    Microsoft coco captions: Data collection and evaluation server

    Xinlei Chen, Hao Fang, Tsung-Yi Lin, Ramakrishna Vedan- tam, Saurabh Gupta, Piotr Doll ´ar, and C Lawrence Zitnick. Microsoft coco captions: Data collection and evaluation server. arXiv preprint arXiv:1504.00325, 2015

    arXiv 2015

  5. [5]

    Mind’s eye: A recur- rent visual representation for image caption generation

    Xinlei Chen and C Lawrence Zitnick. Mind’s eye: A recur- rent visual representation for image caption generation. In CVPR, 2015

    work page 2015

  6. [6]

    Regularizing rnns for caption generation by reconstruct- ing the past with the present

    Xinpeng Chen, Lin Ma, Wenhao Jiang, Jian Yao, and Wei Liu. Regularizing rnns for caption generation by reconstruct- ing the past with the present. In CVPR, 2018

    work page 2018

  7. [7]

    Learning phrase representations using rnn encoder-decoder for statistical machine translation

    Kyunghyun Cho, Bart Van Merri ¨enboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078, 2014

    arXiv 2014

  8. [8]

    Meteor universal: Lan- guage specific translation evaluation for any target language

    Michael Denkowski and Alon Lavie. Meteor universal: Lan- guage specific translation evaluation for any target language. In Proceedings of the ninth workshop on statistical machine translation, 2014

    work page 2014

Show all 51 references
  1. [9]

    Language models for image captioning: The quirks and what works

    Jacob Devlin, Hao Cheng, Hao Fang, Saurabh Gupta, Li Deng, Xiaodong He, Geoffrey Zweig, and Margaret Mitchell. Language models for image captioning: The quirks and what works. arXiv preprint arXiv:1505.01809, 2015

    2015 arXiv

  2. [10]

    Long-term recurrent convolutional net- works for visual recognition and description

    Jeffrey Donahue, Lisa Anne Hendricks, Sergio Guadarrama, Marcus Rohrbach, Subhashini Venugopalan, Kate Saenko, and Trevor Darrell. Long-term recurrent convolutional net- works for visual recognition and description. InCVPR, 2015

    2015

  3. [11]

    Convolutional sequence to sequence learning

    Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N Dauphin. Convolutional sequence to sequence learning. arXiv preprint arXiv:1705.03122, 2017

    2017 arXiv

  4. [12]

    Spatial pyramid pooling in deep convolutional networks for visual recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Spatial pyramid pooling in deep convolutional networks for visual recognition. In ECCV, 2014

    2014

  5. [13]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016

    2016

  6. [14]

    Long short-term memory

    Sepp Hochreiter and J ¨urgen Schmidhuber. Long short-term memory. Neural computation, 1997

    1997

  7. [15]

    Densely connected convolutional net- works

    Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kil- ian Q Weinberger. Densely connected convolutional net- works. In CVPR, 2017

    2017

  8. [16]

    Caffe: Convolutional architecture for fast feature embedding

    Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. Caffe: Convolutional architecture for fast feature embedding. In Proceedings of the 22nd ACM inter- national conference on Multimedia, 2014

    2014

  9. [17]

    Recurrent fusion network for image captioning

    Wenhao Jiang, Lin Ma, Yu-Gang Jiang, Wei Liu, and Tong Zhang. Recurrent fusion network for image captioning. In ECCV, 2018

    2018

  10. [18]

    Deep visual-semantic align- ments for generating image descriptions

    Andrej Karpathy and Li Fei-Fei. Deep visual-semantic align- ments for generating image descriptions. In CVPR, 2015

    2015

  11. [19]

    Visual genome: Connecting language and vision using crowdsourced dense image annotations

    Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalan- tidis, Li-Jia Li, David A Shamma, et al. Visual genome: Connecting language and vision using crowdsourced dense image annotations. IJCV, 2017

    2017

  12. [20]

    Imagenet classification with deep convolutional neural net- works

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural net- works. In NIPS, 2012

    2012

  13. [21]

    Babytalk: Understanding and generating simple image descriptions

    Girish Kulkarni, Visruth Premraj, Vicente Ordonez, Sag- nik Dhar, Siming Li, Yejin Choi, Alexander C Berg, and Tamara L Berg. Babytalk: Understanding and generating simple image descriptions. PAMI, 2013

    2013

  14. [22]

    Professor forcing: A new algorithm for training recurrent networks

    Alex M Lamb, Anirudh Goyal ALIAS PARTH GOY AL, Ying Zhang, Saizheng Zhang, Aaron C Courville, and Yoshua Bengio. Professor forcing: A new algorithm for training recurrent networks. In NIPS, 2016

    2016

  15. [23]

    Towards total scene understanding: Classification, annotation and segmen- tation in an automatic framework

    Li-Jia Li, Richard Socher, and Li Fei-Fei. Towards total scene understanding: Classification, annotation and segmen- tation in an automatic framework. In CVPR, 2009

    2009

  16. [24]

    Rouge: A package for automatic evaluation of summaries

    Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. Text Summarization Branches Out, 2004

    2004

  17. [25]

    Optimization of image description met- rics using policy gradient methods

    Siqi Liu, Zhenhai Zhu, Ning Ye, Sergio Guadarrama, and Kevin Murphy. Optimization of image description met- rics using policy gradient methods. CoRR, abs/1612.00370, 2016

    2016 arXiv

  18. [26]

    Knowing when to look: Adaptive attention via a visual sen- tinel for image captioning

    Jiasen Lu, Caiming Xiong, Devi Parikh, and Richard Socher. Knowing when to look: Adaptive attention via a visual sen- tinel for image captioning. In CVPR, 2017

    2017

  19. [27]

    Hierarchical question-image co-attention for visual question answering

    Jiasen Lu, Jianwei Yang, Dhruv Batra, and Devi Parikh. Hierarchical question-image co-attention for visual question answering. In NIPS, 2016

    2016

  20. [28]

    Neural baby talk

    Jiasen Lu, Jianwei Yang, Dhruv Batra, and Devi Parikh. Neural baby talk. In CVPR, 2018

    2018

  21. [29]

    Coherent dialogue with attention-based language models

    Hongyuan Mei, Mohit Bansal, and Matthew R Walter. Coherent dialogue with attention-based language models. In Thirty-First AAAI Conference on Artificial Intelligence , 2017

    2017

  22. [30]

    Self-attentive residual decoder for neural machine translation

    Lesly Miculicich Werlen, Nikolaos Pappas, Dhananjay Ram, and Andrei Popescu-Belis. Self-attentive residual decoder for neural machine translation. In North American Chapter of the Association for Computational Linguistics (NAACL) , 2018

    2018

  23. [31]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In ACL, 2002

    2002

  24. [32]

    A deep reinforced model for abstractive summarization

    Romain Paulus, Caiming Xiong, and Richard Socher. A deep reinforced model for abstractive summarization. arXiv:1705.04304, 2017

    2017 arXiv

  25. [33]

    Faster r-cnn: Towards real-time object detection with region proposal networks

    Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In NIPS, 2015

    2015

  26. [34]

    Self-critical sequence training for image captioning

    Steven J Rennie, Etienne Marcheret, Youssef Mroueh, Jarret Ross, and Vaibhava Goel. Self-critical sequence training for image captioning. In CVPR, 2017

    2017

  27. [35]

    Imagenet large scale visual recognition challenge

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, San- jeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. IJCV, 2015

    2015

  28. [36]

    Sequence to sequence learning with neural networks

    Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, 2014

    2014

  29. [37]

    Inception-v4, inception-resnet and the impact of residual connections on learning

    Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander A Alemi. Inception-v4, inception-resnet and the impact of residual connections on learning. In AAAI, 2017

    2017

  30. [38]

    Rethinking the inception ar- chitecture for computer vision

    Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception ar- chitecture for computer vision. In CVPR, 2016

    2016

  31. [39]

    Recurrent memory networks for language modeling

    Ke Tran, Arianna Bisazza, and Christof Monz. Recurrent memory networks for language modeling. arXiv preprint arXiv:1601.01272, 2016

    2016 arXiv

  32. [40]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NIPS, 2017

    2017

  33. [41]

    Cider: Consensus-based image description evalua- tion

    Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. Cider: Consensus-based image description evalua- tion. In CVPR, 2015

    2015

  34. [42]

    Show and Tell: A Neural Image Caption Gen- erator

    Oriol Vinyals, Alexander Toshev, Samy Bengio, and Du- mitru Erhan. Show and Tell: A Neural Image Caption Gen- erator. In CVPR, 2015

    2015

  35. [43]

    A learning algorithm for continually running fully recurrent neural networks.Neu- ral computation, 1989

    Ronald J Williams and David Zipser. A learning algorithm for continually running fully recurrent neural networks.Neu- ral computation, 1989

    1989

  36. [44]

    What value do explicit high level concepts have in vision to language problems? In CVPR, 2016

    Qi Wu, Chunhua Shen, Lingqiao Liu, Anthony Dick, and Anton van den Hengel. What value do explicit high level concepts have in vision to language problems? In CVPR, 2016

    2016

  37. [45]

    Zemel, and Yoshua Bengio

    Kelvin Xu, Jimmy Lei Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard S. Zemel, and Yoshua Bengio. Show, Attend and Tell: Neural Image Caption Generation with Visual Attention. ICML, 2015

    2015

  38. [46]

    Corpus-guided sentence generation of nat- ural images

    Yezhou Yang, Ching Lik Teo, Hal Daum ´e III, and Yian- nis Aloimonos. Corpus-guided sentence generation of nat- ural images. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2011

    2011

  39. [47]

    Review networks for caption gen- eration

    Zhilin Yang, Ye Yuan, Yuexin Wu, William W Cohen, and Ruslan R Salakhutdinov. Review networks for caption gen- eration. In NIPS, 2016

    2016

  40. [48]

    Exploring visual relationship for image captioning

    Ting Yao, Yingwei Pan, Yehao Li, and Tao Mei. Exploring visual relationship for image captioning. In ECCV, 2018

    2018

  41. [49]

    Boosting image captioning with attributes

    Ting Yao, Yingwei Pan, Yehao Li, Zhaofan Qiu, and Tao Mei. Boosting image captioning with attributes. In ICCV, 2017

    2017

  42. [50]

    Image captioning with semantic attention

    Quanzeng You, Hailin Jin, Zhaowen Wang, Chen Fang, and Jiebo Luo. Image captioning with semantic attention. In CVPR, 2016

    2016

  43. [51]

    Mattnet: Modular at- tention network for referring expression comprehension

    Licheng Yu, Zhe Lin, Xiaohui Shen, Jimei Yang, Xin Lu, Mohit Bansal, and Tamara L Berg. Mattnet: Modular at- tention network for referring expression comprehension. In CVPR, 2018

    2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.