Pith. sign in

REVIEW 5 major objections 6 minor 34 references

Using Images to Find Context-Independent Word Representations in Vector Space

T0 review · 5 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that word vectors trained only on images of a word's dictionary definition terms match the quality of context-based embeddings while training in a fraction of the time.

desk verdict A genuinely new image-only word-vector pipeline that is undermined by an uncontrolled, borrowed-baseline evaluation; the idea deserves a revised look, not a desk reject. read the letter →

arxiv 2412.03592 v1 pith:JOPNQMPJ submitted 2024-11-28 cs.CL cs.AIcs.CV

Harsh Kumar This is my paper · ORCID
classification cs.CLcs.AIcs.CV
keywords wordembeddingsimage-basedrepresentationautoencoderdictionarydefinitionscontext-independentsemanticssimilarityconceptcategorizationoutlierdetection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a word can be represented in vector space without reading any text: you look up the word's dictionary definition, collect images for the word and for each term in the definition, train a small auto-encoder to compress those images into 32-number codes, and concatenate the codes in definition order to get the word's vector. This gives a context-free representation because meaning comes from definitions and pictures rather than from the word's usage in sentences. On word-similarity, outlier-detection, and concept-categorization benchmarks, the resulting vectors score comparably to established context-based embeddings such as skip-gram and GloVe, while the whole training run takes only ten hours on a single GPU. If the claim holds, it offers a cheap route to word embeddings for languages or domains that lack large text corpora but have observable images for their vocabulary.

What carries the argument

The load-bearing mechanism is the image-set auto-encoder with a 32-number latent bottle-neck. Each word's vector is the flattened concatenation of the latent codes of all images in its definition chain, so the vector's structure mirrors the recursive structure of the dictionary definition: the word's own image, then the images of the words that define it, in the same order. The reconstruction objective forces the auto-encoder to compress each 32x32 image into 32 numbers, and the paper's premise is that this compression retains enough semantic content to make the concatenated codes usable as word vectors. The five images per term are meant to cover multiple senses of a word, and empty tokens pad shorter definitions to a fixed length of 100 images.

What would settle it

Retrain the identical auto-encoder on the same images but with the mapping from words to images randomly shuffled, then compare Spearman correlations on WordSim-353; if the shuffled model scores about the same as the original, the image content itself is not carrying the semantic signal.

Watch

Extended reading notes

Core claim

The central discovery is that a purely visual pipeline can reproduce the semantic geometry of text-based word vectors. For each of 115,458 terms, the author builds an ordered image-set: five images of the term itself plus five images for each of up to 19 words in its dictionary definition, fetched from CommonCrawl and DuckDuckGo, yielding 100 images per word. A five-layer convolutional auto-encoder is trained on 577,290 resized 32x32 images to reconstruct them from a 32-dimensional latent code, and that code is treated as a meaningful representation of each image. Appending the latent codes of the image-set in the definition order produces a 3,200-dimensional word vector. The paper reports Spearman correlations on eight similarity benchmarks, accuracy on two outlier-detection sets, and v-measure on four categorization sets, and reads the results as 'comparable performance' to SGNS, CBOW, GloVe, FastText, ngram2vec, and Dict2vec, with much shorter training time.

Load-bearing premise

The result stands or falls on whether a 32-number code produced by an auto-encoder whose only job is to recreate images actually captures a word's meaning, especially for abstract or function words whose web images may be arbitrary or missing.

Editorial extensions

If this is right

  • Word vectors can be produced for any vocabulary whose terms have retrievable images, without needing a large text corpus.
  • The ten-hour training time on a single GPU makes the method accessible for low-resource settings where context-based embeddings are expensive.
  • Because the representation is built from definitions rather than usage, it may remain stable across genres, styles, or domains in ways that corpus-trained vectors do not.
  • The author suggests the approach may transfer across languages, since the same object images appear regardless of the language used to define them.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The method's ceiling likely sits at the image-retrieval step: a cleaner or curated image collection, or a higher-resolution latent code, could improve similarity scores without changing the architecture.
  • Because the latent codes are never supervised by word identity, the auto-encoder could be swapped for a contrastive model that explicitly aligns same-word images; the paper does not test this, but it is a natural extension.
  • The 'comparable' claim is made against static embeddings trained on Wiki2010; against contextual models, the image vectors would likely lose, so the practical window for this method is in lightweight or low-resource settings.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes a method for learning word vectors without using text context. For each word in a BERT-derived vocabulary, it retrieves images of the word and of the words in its dictionary definition, forming an ordered image-set of up to 100 images per word (5 images per term, up to 20 terms). A convolutional autoencoder is trained on 577,290 images to reconstruct 32x32 inputs through a 32-dimensional bottleneck; the 32-d latents for the image-set are concatenated into a 3200-d word vector. The method is evaluated on word similarity (eight datasets), outlier detection (two datasets), and concept categorization (four datasets). The paper claims the resulting vectors perform comparably to context-based embeddings while training in about ten hours.

Significance. If the central claims were established, the work would be a distinctive contribution: it would show that visual depictions of definition terms can support context-free word vectors, and the released image dataset (577,290 images for 115,458 terms) could be reused by the community. The paper is also honest about the dependence of vector quality on image selection. However, the experimental support is not at the level required for the claims. Baselines are not run under matched conditions, no uncertainty is reported, and the paper's own Table 1 shows the proposed method below common baselines on most similarity benchmarks. The significance of the direction is real, but the current evidence does not establish the headline result.

major comments (5)
  1. [Section 5, Table 1] The headline claim that the method 'performs comparably to context-based methods' is not supported by the evidence presented. All baseline rows are copied from Wang et al. (2019), whose models were trained on Wiki2010 text and evaluated with their own preprocessing; no confidence intervals, significance tests, or matched re-runs are reported. The point estimates in Table 1 show the proposed method below SGNS on six of eight datasets (e.g., SimLex-999 0.23 vs 0.39, Verb-143 0.30 vs 0.45, RW 0.36 vs 0.46) and below the best baseline on MC-30 (0.70 vs 0.85) and RG-65 (0.69 vs 0.85). The term 'comparable' is never defined; under the usual reading, the table contradicts it. A controlled comparison with identical evaluation code, matched vocabulary and dimensionality, and bootstrap or permutation intervals is required.
  2. [Section 4.1 and Section 5] The claim of 'much less training time' is not quantified against baselines. The paper reports 10 hours for the autoencoder on one NVIDIA Ampere GPU, but no wall-clock times for SGNS, CBOW, GloVe, FastText, ngram2vec, or Dict2vec are given, and the hardware and implementation for those baselines are not stated. The time spent collecting and filtering 577,290 images is also not included. Without a matched runtime comparison, the training-efficiency claim cannot be evaluated.
  3. [Section 3.2 and Section 6] No evidence is provided that the 32-d autoencoder latents encode meaning rather than low-level visual statistics. The only training objective is reconstruction on 32x32 images, and the paper's own conclusion states that vector quality depends on the selected images. Because the method is motivated by semantic independence, the authors should include direct checks such as nearest-neighbor inspection, probing on semantic relations, or an ablation in which image sets are perturbed or replaced with random images; otherwise the core mechanism is untested.
  4. [Section 5, Tables 2 and 3] The outlier-detection comparison repeats the uncontrolled-baseline problem, and the concept-categorization table reports v-measure scores for the proposed method only, with no baseline values, so 'decent v-measure' is not interpretable. Additionally, the proposed vectors are 3200-d while typical baselines are 100-300-d, and the paper does not discuss whether dimensionality or vector-length normalization affects the comparisons.
  5. [Section 3.1] The image-retrieval pipeline has no explicit quality filter, which matters for abstract words and function words. The paper notes that question words, conjunctions, and punctuation are retained, and for these categories image search can return arbitrary or empty images. This is acknowledged as a limitation in Section 6, but it also means the premise that images can represent definition terms is not validated for a large fraction of the vocabulary. The authors should report coverage statistics (how many terms had fewer than five usable images) and show that the results are robust to excluding low-coverage terms.
minor comments (6)
  1. [Title] The title contains 'V ector' with a stray space; it should read 'Vector'.
  2. [References] The reference list contains duplicate entries for Luong et al. 2013a and 2013b, which appear to be the same paper.
  3. [Table 3] In Table 3, 'ESSLI-2008' should be 'ESSLLI-2008'.
  4. [Section 4.2.3] The Almuhareb and Poesio dataset is listed without describing the clustering algorithm or hyperparameters used to compute v-measure.
  5. [Abstract and Conclusion] There are minor wording issues, including 'we finally evaluate' in the abstract and 'obtain the find word vector representation' in the conclusion.
  6. [Figures 2 and 3] Figures 2 and 3 are described but never explicitly referenced in the running text.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the derivation is self-contained, with external benchmarks and no fitted target labels.

full rationale

The proposed word vectors are constructed from latent codes of an autoencoder trained on images retrieved for each word and its dictionary definition terms. The autoencoder is optimized only with a binary cross-entropy image reconstruction loss (Section 4.1), not with any word-similarity, categorization, or outlier-detection objective. The evaluation is performed on external human-judgment benchmarks, concept-categorization datasets, and outlier-detection datasets whose labels are never used during training. Baseline scores are taken from Wang et al. (2019), an independent source; the comparison may be uncontrolled or unconvincing, but that is a correctness or experimental-validity concern, not a circular-dependency concern. No load-bearing claim is justified solely by a self-citation, no parameter is fitted to the evaluated quantities, and no equation reduces a prediction to its own input. The only self-referential element is that each word's image-set includes an image of the word itself along with images of its definition terms; this is a dataset-design choice and does not make the evaluation equivalent to the construction. Accordingly, no circularity steps are identified.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method relies on several unvalidated domain assumptions about the quality of dictionary definitions, image search results, and the semantic content of autoencoder codes. There are no invented physical entities, but the representation pipeline has several hand-chosen hyperparameters that could significantly affect results.

free parameters (4)
  • latent_size = 32
    Chosen as the dimension of each image's autoencoder code; no ablation justifies this value.
  • images_per_term = 5
    Selected to cover multiple meanings of a term; no analysis of adequacy is provided.
  • max_definition_terms = 19
    Definitions truncated to 19 terms plus the word itself yields 100 images per word; the cutoff is arbitrary.
  • image_size = 32x32
    Resized input images; a standard small size but not justified for this task.
assumptions (4)
  • domain assumption Dictionary definitions from Project Gutenberg and Wiktionary accurately capture word meaning.
    Section 3.1 builds the definition-terms set from these dictionaries without validating the definitions.
  • domain assumption Web image search results for a term are relevant depictions of that term's meaning.
    Section 3.1 uses CommonCrawl and DuckDuckGo image search; abstract words may have no consistent visual depiction.
  • domain assumption An autoencoder trained with binary cross-entropy reconstruction loss produces latent codes that preserve semantic similarity between words.
    Section 3.2 assumes the 32-dim latent code is a 'meaningful representation'; no semantic evaluation of the codes is provided.
  • domain assumption The BERT vocabulary is an appropriate base vocabulary for defining the word set.
    Section 3.1 uses it without discussion.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Using Images to Find Context-Independent Word Representations in Vector Space." pith.science (2026). https://pith.science/paper/JOPNQMPJ

@misc{pith2026241203592,
  author       = {Pith},
  title        = {Pith review of: Using Images to Find Context-Independent Word Representations in Vector Space},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JOPNQMPJ}},
  note         = {Machine review of arXiv:2412.03592}
}
read the original abstract

Many methods have been proposed to find vector representation for words, but most rely on capturing context from the text to find semantic relationships between these vectors. We propose a novel method of using dictionary meanings and image depictions to find word vectors independent of any context. We use auto-encoder on the word images to find meaningful representations and use them to calculate the word vectors. We finally evaluate our method on word similarity, concept categorization and outlier detection tasks. Our method performs comparably to context-based methods while taking much less training time.

Figures

Figures reproduced from arXiv: 2412.03592 by the authors.

Figure 2
Figure 2. The Auto-encoder architecture used in our method to get the latent representation for images. The top subscripts show the number of input channels whereas the bottom subscripts show the number of output channels. Each conv and convT layer has 3x3 kernel with stride = 1 and padding = 1 [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The auto-encoder is used to get the latent representation of images. These latent representations for definition terms are ap￾pended in a sequence to get the final representation of the original word stride = 1, and padding = 1 for the ConvTranspose2d layer. The overall architecture is shown in [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 20 canonical work pages

  1. [1]

    and Poesio, M

    Almuhareb, A. and Poesio, M. Attribute-based and value-based clustering: An evaluation. In Lin, D. and Wu, D. (eds.), Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pp.\ 158--165, Barcelona, Spain, July 2004. Association for Computational Linguistics. URL https://aclanthology.org/W04-3221

    work page 2004

  2. [2]

    An unsupervised model for instance level subcategorization acquisition

    Baker, S., Reichart, R., and Korhonen, A. An unsupervised model for instance level subcategorization acquisition. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp.\ 278--289, 2014

    work page 2014

  3. [3]

    and Lenci, A

    Baroni, M. and Lenci, A. How we BLESS ed distributional semantic evaluation. In Pado, S. and Peirsman, Y. (eds.), Proceedings of the GEMS 2011 Workshop on GE ometrical Models of Natural Language Semantics , pp.\ 1--10, Edinburgh, UK, July 2011. Association for Computational Linguistics. URL https://aclanthology.org/W11-2501

    work page 2011

  4. [4]

    Battig, W. F. and Montague, W. E. Category norms of verbal items in 56 categories a replication and extension of the connecticut category norms. Journal of Experimental Psychology, 80 0 (3, Pt.2): 0 1–46, Jun 1969. doi:10.1037/h0027577

    doi:10.1037/h0027577 1969

  5. [5]

    A neural probabilistic language model

    Bengio, Y., Ducharme, R., Vincent, P., and Janvin, C. A neural probabilistic language model. J. Mach. Learn. Res., 3 0 (null): 0 1137–1155, March 2003. ISSN 1532-4435

    work page 2003

  6. [6]

    Automated Generation of Multilingual Clusters for the Evaluation of Distributed Representations

    Blair, P., Merhav, Y., and Barry, J. Automated generation of multilingual clusters for the evaluation of distributed representations. CoRR, abs/1611.01547, 2016. URL http://arxiv.org/abs/1611.01547

    work page Pith review arXiv 2016

  7. [7]

    Enriching word vectors with subword information

    Bojanowski, P., Grave, E., Joulin, A., and Mikolov, T. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5: 0 135--146, 06 2017. ISSN 2307-387X. doi:10.1162/tacl_a_00051. URL https://doi.org/10.1162/tacl\_a\_00051

    doi:10.1162/tacl_a_00051 2017

  8. [8]

    Bullinaria, J. A. and Levy, J. P. Extracting semantic representations from word co-occurrence statistics: A computational study. Behavior Research Methods, 39: 0 510--526, 2007. URL https://api.semanticscholar.org/CorpusID:1025306

    work page 2007

Show all 34 references
  1. [9]

    and Navigli, R

    Camacho-Collados, J. and Navigli, R. Find the word that does not belong: A framework for an intrinsic evaluation of word vector representations. In Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP , pp.\ 43--50, Berlin, Germany, August 2016. A...

    2016 doi

  2. [10]

    Efficient vector representation for documents through corruption

    Chen, M. Efficient vector representation for documents through corruption. CoRR, abs/1707.02377, 2017. URL http://arxiv.org/abs/1707.02377

    2017 arXiv

  3. [11]

    and Pad \'o , S

    Erk, K. and Pad \'o , S. A structured vector space model for word meaning in context. In Lapata, M. and Ng, H. T. (eds.), Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pp.\ 897--906, Honolulu, Hawaii, October 2008. Association for Comp...

    2008

  4. [12]

    Placing search in context: T he concept revisited

    Finkelstein, L., Gabrilovich, E., Matias, Y., Rivlin, E., Solan, Z., Wolfman, G., and Ruppin, E. Placing search in context: T he concept revisited. ACM Transactions on Information Systems, 20 0 (1): 0 116--131, January 2002

    2002

  5. [13]

    Simverb-3500: A large-scale evaluation set of verb similarity

    Gerz, D., Vulic, I., Hill, F., Reichart, R., and Korhonen, A. Simverb-3500: A large-scale evaluation set of verb similarity. CoRR, abs/1608.00869, 2016. URL http://arxiv.org/abs/1608.00869

    2016 arXiv

  6. [14]

    Simlex-999: Evaluating semantic models with (genuine) similarity estimation

    Hill, F., Reichart, R., and Korhonen, A. Simlex-999: Evaluating semantic models with (genuine) similarity estimation. CoRR, abs/1408.3456, 2014. URL http://arxiv.org/abs/1408.3456

    2014 arXiv

  7. [15]

    and Muskens, R

    Icard, T. and Muskens, R. (eds.). Interfaces: Explorations in Logic, Language and Computation, ESSLLI 2008 and ESSLLI 2009 Student Sessions. Selected Papers, volume 6211 of Lecture Notes in Computer Science, 2010. Springer. ISBN 978-3-642-1472-9. doi:10.1007/978-3-642-14729-6....

    2008 doi

  8. [16]

    Associating neural word embeddings with deep image representations using fisher vectors

    Klein, B., Lev, G., Sadeh, G., and Wolf, L. Associating neural word embeddings with deep image representations using fisher vectors. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015

    2015

  9. [17]

    A low-dimensional vector representation for words using an extreme learning machine

    Lauren, P., Qu, G., Huang, G.-B., Watta, P., and Lendasse, A. A low-dimensional vector representation for words using an extreme learning machine. In 2017 International Joint Conference on Neural Networks (IJCNN), pp.\ 1817--1822, 2017. doi:10.1109/IJCNN.2017.7966071

    2017

  10. [18]

    and Lebret, R

    Lebret, R. and Lebret, R. Word emdeddings through hellinger PCA . CoRR, abs/1312.5542, 2013. URL http://arxiv.org/abs/1312.5542

    2013 arXiv

  11. [19]

    and Burgess, C

    Lund, K. and Burgess, C. Producing high-dimensional semantic space from lexical co-occurence. Behavior Research Methods Instruments & Computers, 28: 0 203--208, 06 1996. doi:10.3758/BF03204766

    1996 doi

  12. [20]

    Luong, M.-T., Socher, R., and Manning, C. D. Better word representations with recursive neural networks for morphology. In CoNLL, Sofia, Bulgaria, 2013 a

    2013

  13. [21]

    Better word representations with recursive neural networks for morphology

    Luong, T., Socher, R., and Manning, C. Better word representations with recursive neural networks for morphology. In Hockenmaier, J. and Riedel, S. (eds.), Proceedings of the Seventeenth Conference on Computational Natural Language Learning, pp.\ 104--113, Sofia, Bulgaria, Aug...

    2013

  14. [22]

    L., Daly, R

    Maas, A. L., Daly, R. E., Pham, P. T., Huang, D., Ng, A. Y., and Potts, C. Learning word vectors for sentiment analysis. In Lin, D., Matsumoto, Y., and Mihalcea, R. (eds.), Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language ...

    2011

  15. [23]

    Mikolov, T., Kopeck \'y , J., Burget, L., Glembek, O., and Cernock \'y , J. H. Neural network based language models for highly inflective languages. 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, pp.\ 4725--4728, 2009. URL https://api.semanticsc...

    2009

  16. [24]

    S., and Dean, J

    Mikolov, T., Chen, K., Corrado, G. S., and Dean, J. Efficient estimation of word representations in vector space. In International Conference on Learning Representations, 2013. URL https://api.semanticscholar.org/CorpusID:5959482

    2013

  17. [25]

    Miller, G. A. and Charles, W. G. Contextual correlates of semantic similarity. Language and Cognitive Processes, 6 0 (1): 0 1--28, 1991. doi:10.1080/01690969108406936. URL https://doi.org/10.1080/01690969108406936

    1991 doi

  18. [26]

    Word sense disambiguation based on word similarity calculation using word vector representation from a knowledge-based graph

    O, D., Kwon, S., Kim, K., and Ko, Y. Word sense disambiguation based on word similarity calculation using word vector representation from a knowledge-based graph. In Bender, E. M., Derczynski, L., and Isabelle, P. (eds.), Proceedings of the 27th International Conference on Com...

    2018

  19. [27]

    G lo V e: Global vectors for word representation

    Pennington, J., Socher, R., and Manning, C. G lo V e: Global vectors for word representation. In Moschitti, A., Pang, B., and Daelemans, W. (eds.), Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing ( EMNLP ) , pp.\ 1532--1543, Doha, Qatar, ...

    2014 doi

  20. [28]

    A word at a time: computing word relatedness using temporal semantic analysis

    Radinsky, K., Agichtein, E., Gabrilovich, E., and Markovitch, S. A word at a time: computing word relatedness using temporal semantic analysis. In Proceedings of the 20th International Conference on World Wide Web, WWW '11, pp.\ 337–346, New York, NY, USA, 2011. Association fo...

    2011

  21. [29]

    An improved method for deriving word meaning from lexical co-occurrence

    Rohde, D., Gonnerman, L., and Plaut, D. An improved method for deriving word meaning from lexical co-occurrence. Cognitive Science - COGSCI, 7, 01 2005

    2005

  22. [30]

    Roy, D., Ganguly, D., Mitra, M., and Jones, G. J. F. Representing documents and queries as sets of word embedded vectors for information retrieval. CoRR, abs/1606.07869, 2016. URL http://arxiv.org/abs/1606.07869

    2016 arXiv

  23. [31]

    and Goodenough, J

    Rubenstein, H. and Goodenough, J. B. Contextual correlates of synonymy. Commun. ACM, 8 0 (10): 0 627–633, October 1965. ISSN 0001-0782. doi:10.1145/365628.365657. URL https://doi.org/10.1145/365628.365657

    1965

  24. [32]

    and McCallum, A

    Vilnis, L. and McCallum, A. Word representations via gaussian embedding. In Bengio, Y. and LeCun, Y. (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings , 2015. URL http://arxiv.org/abs/1412.6623

    2015 arXiv

  25. [33]

    Wang, B., Wang, A., Chen, F., Wang, Y., and Kuo, C. J. Evaluating word embedding models: Methods and experimental results. CoRR, abs/1901.09785, 2019. URL http://arxiv.org/abs/1901.09785

    1901 arXiv

  26. [34]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.