Pith. sign in

REVIEW 1 cited by

Learning Visually Grounded Sentence Representations

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1707.06320 v2 pith:PZGCS76E submitted 2017-07-19 cs.CL cs.CV

Douwe Kiela , Alexis Conneau , Allan Jabri , Maximilian Nickel This is my paper · ORCID
classification cs.CLcs.CV
keywords imagesentencecaptionencodergroundedgroundingmodelsperformance
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

We introduce a variety of models, trained on a supervised image captioning corpus to predict the image features for a given caption, to perform sentence representation grounding. We train a grounded sentence encoder that achieves good performance on COCO caption and image retrieval and subsequently show that this encoder can successfully be transferred to various NLP tasks, with improved performance over text-only models. Lastly, we analyze the contribution of grounding, and show that word embeddings learned by this system outperform non-grounded ones.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Fusion of Detected Objects in Text for Visual Question Answering

    cs.CL 2019-08 conditional novelty 7.0 of 10

    Injecting detected object features into BERT's token embeddings (early fusion) improved visual question answering and achieved state-of-the-art results on VCR.

Pith tools