Pith. sign in

REVIEW 5 major objections 7 minor 61 references

TriMod Fusion for Multimodal Named Entity Recognition in Social Media

T0 review · 5 major / 7 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper reports that adding hashtag features and transformer-attention fusion to a text-image NER model achieves F1 80.00 on the Twitter MNER benchmark, surpassing CATMNER's 78.72.

desk verdict The paper's SOTA claim is unsupported: the architecture never defines how pooled fusion becomes per-token CRF emissions, and the headline F1 is arithmetically inconsistent. read the letter →

arxiv 2501.08267 v1 pith:KMY57RUE submitted 2025-01-14 cs.IR cs.SI

Mosab Alfaqeeh This is my paper · ORCID
classification cs.IRcs.SI
keywords multimodalnamedentityrecognitionhashtagfeaturestransformerattentionfusionconditionalrandomfieldsocialmediaTwitterNERdisambiguationmodality
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that named entity recognition in tweets improves when a model combines three sources of evidence—text, images, and hashtags—instead of just text and images. The proposed TriMod architecture encodes words with embeddings and Bi-GRUs, extracts object-level image features through a captioning model, segments hashtags into words with a character-level network, then merges all three through a transformer-attention pooling step before a conditional random field labels each token. On the Twitter MNER benchmark of 8,257 tweets, it reports F1 80.00, slightly ahead of the best cited baseline CATMNER at 78.72. If that holds, hashtags supply disambiguating context that visual and textual cues alone miss, especially for organization and miscellaneous entity types.

What carries the argument

The mechanism doing the work is the transformer-attention fusion in Equations (11) and (12): each modality's feature vector is linearly transformed, scored with a learned vector via a tanh gate and exponential, and the normalized weighted sum produces a single fused feature $f_s$. This fused vector is the input to a conditional random field (CRF) that outputs the BIO2 label sequence, combining transition scores with emission scores from the encoder. Hashtag features come from a character-level CNN-BiLSTM segmentation module that splits hashtags into words before embedding, and visual features are object-level representations from a ResNet-based image captioning model.

What would settle it

A faithful reimplementation following only the paper's equations would find no defined mapping from the pooled feature $f_s$ to the per-token emission scores the CRF requires; if the authors cannot supply that mapping, the reported F1 cannot be reproduced. Separately, an ablation that removes the hashtag branch and still measures F1 80.00 would falsify the claim that hashtags drive the improvement.

Watch

Extended reading notes

Core claim

The central claim, stated in Section V-B, is that the TriMod model achieves state-of-the-art performance in multimodal named entity recognition on social media, outperforming earlier methods in precision, recall, and F1. Concretely, it reports precision 79.90%, recall 79.44%, and F1 80.00% on the Twitter test set, versus F1 78.72% for the cross-attention baseline CATMNER. The paper attributes the gain to the transformer-attention fusion of textual, object-level visual, and segmented hashtag features, which lets hashtags such as #AustonMatthews and images of a hockey rink jointly disambiguate entities like person names and organizations.

Load-bearing premise

The load-bearing premise is that the single pooled multimodal feature in Equation (12) can be expanded back into per-token features for the CRF, yet the paper never describes that expansion.

Editorial extensions

If this is right

  • TriMod's reported F1 of 80.00 exceeds CATMNER's 78.72, the closest cited baseline.
  • The model's precision (79.90) and recall (79.44) are both higher than all listed baselines, indicating no precision-recall trade-off in the reported comparison.
  • Category-level results place ORG and MISC at F1 80.3 and 80.5, the two highest among the four entity types.
  • Ablating dropout lowers F1 from 80.00 to 78.94, while L1, L2, and batch normalization each yield smaller changes around 79.1-79.4.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The fused vector $f_s$ from Equation (12) is a single pooled representation, but the CRF needs per-token emission scores; the paper never specifies how $f_s$ is expanded back to token level, so a reader cannot tell from the text alone whether the reported architecture is fully defined.
  • Because the paper does not ablate the hashtag branch, the marginal contribution of hashtags is untested; comparing TriMod to the text-image baselines in Table III implies hashtags could account for roughly one to six F1 points, but that estimate is an editorial extrapolation.
  • A natural testable variant would compute attention per token instead of one global pooling, which would supply the per-token features the CRF needs and likely remove the ambiguity in the current description.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper proposes TriMod, a multimodal named entity recognition (NER) model for Twitter that combines textual features, object-level visual features, and segmented hashtag features, then fuses them through a Transformer-attention weighted average before a CRF output layer. The authors report experiments on the Twitter multimodal NER dataset of Zhang et al., claiming an F1 score of 80.00 and stating that this outperforms the strongest cited baseline CATMNER (78.72). The manuscript describes textual Bi-GRU and character-embedding components, a ResNet-based visual feature extractor with image captioning, a CNN-BiLSTM hashtag segmentation module, an attention-pooling fusion stage, and CRF decoding.

Significance. If the architecture were fully specified and the reported numbers reproducible, the contribution would be incremental but potentially useful: it augments standard text+image MNER pipelines with hashtag features and a simple attention-based fusion. However, the paper currently provides no code or release artifacts, and the load-bearing empirical claim is clouded by an arithmetic inconsistency in the headline F1 score and by an undefined fusion-to-CRF inference pathway. These issues prevent the central claim from being verified from the manuscript as written.

major comments (5)
  1. [IV-E, IV-F, Eqs. (12)-(14)] The paper never defines how the single pooled vector fs from Eq. (12) becomes the per-token emission matrix P required by the CRF in Eq. (13). A CRF needs an emission score for each token and each label, but fs is a single weighted average over all modality features with no expansion back to n tokens. Equation (14) conditions the label sequence probability on GT, the per-token textual features, not on fs, so it is unclear how visual and hashtag features influence token-level labeling at all. This missing mapping is a load-bearing gap: without it, the model's output labels are not defined and the reported F1 of 80.00 in Table III cannot be reproduced from the described architecture.
  2. [V-B, Table III] The headline result is internally inconsistent. With Precision = 79.90% and Recall = 79.44%, the harmonic mean F1 is 2 * 79.90 * 79.44 / (79.90 + 79.44) = 79.67%, not 80.00% as reported. Since the paper's central claim is that TriMod surpasses CATMNER (78.72%), this discrepancy matters: the corrected F1 would be only 0.95 points higher, and the paper provides no error bars, multiple-run statistics, or significance tests to establish that the gap is meaningful.
  3. [IV-B and Ref. [44]] The visual feature extractor is said to be 'based on the deep learning technique introduced by Alfaqeeh et al. [44]', but the cited reference [44] is titled 'Uncovering influential users in social networks: A comparative study' and appears unrelated to image captioning or visual feature extraction. This citation does not support the claimed technique. The visual component is a core part of the multimodal model, so its provenance needs a correct and verifiable reference or a complete technical description.
  4. [IV-C and IV-D] The hashtag segmentation module is described as trained on a 'non-standard dataset' with a CNN-BiLSTM decoder, but no dataset, training procedure, evaluation, or release is provided. More importantly, the paper never explains how the segmented hashtag output is transformed into per-token features that can be fused with text and image features. Without this step, the hashtag modality's contribution to the CRF labeling is underspecified.
  5. [III] The problem statement formalizes the objective as 'Maximize f(T) + g(V,H)' with relationship constraints, but this optimization formulation is never used in the methodology or the loss functions. It is not connected to the cross-entropy objective, the attention weights, or the CRF loss in Eqs. (11)-(15), so it does not describe the actual model. This is not by itself fatal, but it is misleading about what the paper optimizes.
minor comments (7)
  1. [Table III] MNER-MA and MNER-QG are reported with identical precision, recall, and F1 (72.33, 63.51, 67.63), which is likely a copy-paste error and should be corrected.
  2. [Table III] The row for ITA reports only F1 = 75.00 with no precision or recall values; please fill in those entries or state why they are unavailable.
  3. [Eq. (13)] The notation P_{y[i]→i} for CRF emission scores is confusing; emission scores are normally written P_{i,y[i]}, and the arrow notation conflicts with transition scores T_{l[i-1]→l[i]}.
  4. [VI] The conclusion describes 'multi-headed attention mechanisms', but the methodology in Section IV-E defines only a single attention pooling mechanism (Eqs. (11)-(12)); no multi-head attention is specified.
  5. [IV-B] The text says the image encoder uses 'ResNet [45]', but reference [45] is the ViT paper 'An Image is Worth 16x16 Words'; the reference list does not match the cited architecture.
  6. [V-D, Table V] The text claims consistent improvement in precision and recall across entity types, but Table V reports only F1 per category, with no precision or recall breakdown.
  7. [VII] The limitations section discusses image-text matching and modality noise but does not acknowledge the undefined per-token fusion pathway or the need for multiple-run reporting; these limitations are more directly relevant to the validity of the empirical results.

Circularity Check

1 steps flagged · score 2.0 of 10

No derivation-level circularity; the F1 claim is empirical, but the visual extractor is sourced from a self-citation ([44]) that does not support the described captioning technique.

  1. other [Section IV-B, Visual Feature Extractor]
    "Our approach is based on the deep learning technique introduced by Alfaqeeh et al. [44] for the task of processing images to generate textual descriptions."

    This sentence makes the visual feature extractor, one of TriMod's three modalities, rest on reference [44], a self-authored paper. The reference list identifies [44] as 'Uncovering influential users in social networks: A comparative study,' which is not about image captioning. The component is therefore justified by a self-citation rather than by an independently stated technique. This is a support gap, not a constructed prediction: the reported F1 of 80.00 is an empirical result against external baselines and does not reduce to this citation.

full rationale

The paper's central claim is an empirical comparison (Section V-B, Table III): TriMod achieves F1 80.00 versus 78.72 for CATMNER. No fitted parameter is renamed as a prediction, and no equation defines a result in terms of its own conclusion. The fusion equations (11)-(12) and CRF equations (13)-(16) are incomplete (the paper never defines how the pooled fs expands to per-token emission scores), but that is an under-specification, not circularity. The objective function in Section III is tautological ('maximize f(T)+g(V,H)'), but it is a problem statement, not a derived result. The only circularity-adjacent issue is the visual extractor's reliance on self-citation [44]; because the performance claim is externally benchmarked and does not depend logically on that citation, the score is a low 2 rather than anything higher.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The paper's central performance claim rests on a dataset, a set of hyperparameters, and several underspecified modules rather than on a derivation. The model is a combination of standard components, and no new physical or conceptual entities are introduced.

free parameters (7)
  • Number of selected visual objects k = Not specified in paper
    Section V-A says the optimal k is determined through validation results, but no values or sensitivity analysis are provided.
  • Word-level Bi-GRU hidden dimension = 150
    Chosen by empirical experimentation on held-out data (Section V-A, Table II).
  • Char-level Bi-GRU hidden dimension = 30
    Reported in Table II; hand-chosen hyperparameter.
  • Character embedding dimension = 30
    Reported in Table II; hand-chosen hyperparameter.
  • Dropout rate = 0.55
    Reported in Table II; selected to mitigate overfitting.
  • Learning rate = 0.005
    Reported in Table II; selected as optimization hyperparameter.
  • Learning rate decay = 0.05
    Reported in Table II; selected as optimization hyperparameter.
assumptions (5)
  • domain assumption Each tweet in the Twitter MNER dataset has one associated image whose visual content is relevant to the entities in the text.
    This is the premise of the entire multimodal approach; the paper's own Limitations section acknowledges that strict text-image matching may not hold.
  • domain assumption The image captioning model attributed to Alfaqeeh et al. [44] provides useful visual features for NER.
    Section IV-B relies on [44], but the cited reference is about influential users in social networks, not image captioning. The assumption is unverified.
  • ad hoc to paper The hashtag segmentation module (CNN-BiLSTM decoder) is trainable and effective, but no training data, evaluation, or code is provided.
    Sections IV-C and IV-D describe the segmenter without reporting its accuracy or release, so its output quality is an unsupported premise.
  • domain assumption The reported baseline F1 values in Table III are directly comparable, obtained with the same split and evaluation protocol.
    The paper does not reproduce baselines; it cites numbers from other papers, which may use different preprocessing, object detection, or tagging conventions.
  • ad hoc to paper The optimization formulation in Section III, maximizing f(T)+g(V,H), is a valid characterization of the MNER objective.
    This formalization is never used in the methodology, imposes only trivial non-negativity constraints, and does not constrain the model. It is decorative.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TriMod Fusion for Multimodal Named Entity Recognition in Social Media." pith.science (2026). https://pith.science/paper/KMY57RUE

@misc{pith2026250108267,
  author       = {Pith},
  title        = {Pith review of: TriMod Fusion for Multimodal Named Entity Recognition in Social Media},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KMY57RUE}},
  note         = {Machine review of arXiv:2501.08267}
}
read the original abstract

Social media platforms serve as invaluable sources of user-generated content, offering insights into various aspects of human behavior. Named Entity Recognition (NER) plays a crucial role in analyzing such content by identifying and categorizing named entities into predefined classes. However, traditional NER models often struggle with the informal, contextually sparse, and ambiguous nature of social media language. To address these challenges, recent research has focused on multimodal approaches that leverage both textual and visual cues for enhanced entity recognition. Despite advances, existing methods face limitations in capturing nuanced mappings between visual objects and textual entities and addressing distributional disparities between modalities. In this paper, we propose a novel approach that integrates textual, visual, and hashtag features (TriMod), utilizing Transformer-attention for effective modality fusion. The improvements exhibited by our model suggest that named entities can greatly benefit from the auxiliary context provided by multiple modalities, enabling more accurate recognition. Through the experiments on a multimodal social media dataset, we demonstrate the superiority of our approach over existing state-of-the-art methods, achieving significant improvements in precision, recall, and F1 score.

Figures

Figures reproduced from arXiv: 2501.08267 by the authors.

Figure 1
Figure 1. Mappings between visual objects, processed hashtags, and correspond [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The fused representation fs, which integrates information from text, images, and hashtags using Transformer-attention. where T is the transition scores matrix and P is the matrix of scores obtained from the encoder network. Here, Ti→j represents the transition score from label i to label j, and Py[i]→i represents the score of the i-th word in the sentence belonging to the predicted label y[i]. The probability of the… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 55 canonical work pages

  1. [44]

    Uncovering influential users in social networks: A comparative study,

    G. Alfaqeeh, R. Alhajj et al. , “Uncovering influential users in social networks: A comparative study,” in Proceedings of the AAAI Conference on Artificial Intelligence , 2023, pp. 1–8

    work page 2023

  2. [45]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer et al. , “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929, 2020

    arXiv 2010

  3. [1]

    Community detection in social networks by spectral embedding of typed graphs,

    M. Alfaqeeh and D. Skillicorn, “Community detection in social networks by spectral embedding of typed graphs,” Social Network Analysis and Mining, vol. 14, no. 1, p. 12, 2023

    work page 2023

  4. [2]

    Social networks benchmark dataset for diseases classification,

    M. Quwaider and M. Alfaqeeh, “Social networks benchmark dataset for diseases classification,” in 2016 IEEE 4th international conference on future internet of things and cloud workshops (FiCloudW). IEEE, 2016, pp. 234–239

    work page 2016

  5. [3]

    A survey of sentiment analysis techniques: Challenges, applications, and future directions,

    L. Jiang, W. Wang, and W. Zhang, “A survey of sentiment analysis techniques: Challenges, applications, and future directions,” Information Processing & Management , vol. 59, no. 1, p. 102617, 2022

    work page 2022

  6. [4]

    Cross-lingual short-text document classification for facebook com- ments,

    M. Faqeeh, N. Abdulla, M. Al-Ayyoub, Y . Jararweh, and M. Quwaider, “Cross-lingual short-text document classification for facebook com- ments,” in 2014 International Conference on Future Internet of Things and Cloud. IEEE, 2014, pp. 573–578

    work page 2014

  7. [5]

    Real-time event detection and tracking in social media: A survey,

    X. Wei, J. Zhang, C. Wang, L. Zhang, and X. Zhang, “Real-time event detection and tracking in social media: A survey,” Information Fusion, vol. 84, pp. 57–78, 2022

    work page 2022

  8. [6]

    Content recom- mendation systems: A comprehensive survey,

    Y . Liu, Y . Zhang, Y . Li, J. Zhang, and H. Wang, “Content recom- mendation systems: A comprehensive survey,” ACM Computing Surveys (CSUR), vol. 55, no. 3, pp. 1–41, 2022

    work page 2022

Show all 61 references
  1. [7]

    Adapting named entity recognition models for social media texts,

    J. Smith and A. Johnson, “Adapting named entity recognition models for social media texts,” in Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 2022, pp. 100–110

    2022

  2. [8]

    Named entity recognition in tweets: An experimental study,

    A. Ritter, S. Clark, Mausam, and O. Etzioni, “Named entity recognition in tweets: An experimental study,” in Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing , R. Barzilay and M. Johnson, Eds. Edinburgh, Scotland, UK.: Association for Comp...

    2011

  3. [9]

    A comparison of few-shot and traditional named entity recognition models for medical text,

    Y . Ge, Y . Guo, Y .-C. Yang, M. A. Al-Garadi, and A. Sarker, “A comparison of few-shot and traditional named entity recognition models for medical text,” in 2022 IEEE 10th International Conference on Healthcare Informatics (ICHI) . IEEE, 2022, pp. 84–89

    2022

  4. [10]

    Named entity recognition for social media texts with semantic augmentation,

    Y . Nie, Y . Tian, X. Wan, Y . Song, and B. Dai, “Named entity recognition for social media texts with semantic augmentation,” arXiv preprint arXiv:2010.15458, 2020

    2010 arXiv

  5. [11]

    Improving named entity recognition for social media with data augmentation,

    W. Liu and X. Cui, “Improving named entity recognition for social media with data augmentation,” Applied Sciences , vol. 13, no. 9, p. 5360, 2023

    2023

  6. [12]

    Multimodal named entity recog- nition for short social media posts,

    S. Moon, L. Neves, and V . Carvalho, “Multimodal named entity recog- nition for short social media posts,” arXiv preprint arXiv:1802.07862 , 2018

    2018 arXiv

  7. [13]

    Named entity recognition for social media text,

    Y . Zhang, “Named entity recognition for social media text,” 2019

    2019

  8. [14]

    Natural language processing: State of the art, current trends and challenges,

    D. Khurana, A. Koli, K. Khatter, and S. Singh, “Natural language processing: State of the art, current trends and challenges,” Multimedia tools and applications , vol. 82, no. 3, pp. 3713–3744, 2023

    2023

  9. [15]

    Characterizing english variation across social media communities with bert,

    L. Lucy and D. Bamman, “Characterizing english variation across social media communities with bert,” Transactions of the Association for Computational Linguistics, vol. 9, pp. 538–556, 2021

    2021

  10. [16]

    Exploiting data sparsity in secure cross-platform social recommendation,

    J. Cui, C. Chen, L. Lyu, C. Yang, and W. Li, “Exploiting data sparsity in secure cross-platform social recommendation,” Advances in Neural Information Processing Systems , vol. 34, pp. 10 524–10 534, 2021

    2021

  11. [17]

    Rum extractor: a facebook extractor for data analysis,

    R. M. Duwairi and M. Alfaqeeh, “Rum extractor: a facebook extractor for data analysis,” in 2015 3rd International Conference on Future Internet of Things and Cloud . IEEE, 2015, pp. 709–713

    2015

  12. [18]

    Understanding social me- dia cross-modality discourse in linguistic space,

    C. Xu, H. Tan, J. Li, and P. Li, “Understanding social me- dia cross-modality discourse in linguistic space,” arXiv preprint arXiv:2302.13311, 2023

    2023 arXiv

  13. [19]

    Vtkel: a resource for visual-textual-knowledge entity linking,

    S. Dost, L. Serafini, M. Rospocher, L. Ballan, and A. Sperduti, “Vtkel: a resource for visual-textual-knowledge entity linking,” in Proceedings of the 35th Annual ACM Symposium on Applied Computing , 2020, pp. 2021–2028

    2020

  14. [20]

    Enhancing multimodal entity and relation extraction with variational information bottleneck,

    S. Cui, J. Cao, X. Cong, J. Sheng, Q. Li, T. Liu, and J. Shi, “Enhancing multimodal entity and relation extraction with variational information bottleneck,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024

    2024

  15. [21]

    A survey of multi-modal knowledge graphs: Technologies and trends,

    W. Liang, P. De Meo, Y . Tang, and J. Zhu, “A survey of multi-modal knowledge graphs: Technologies and trends,” ACM Computing Surveys, 2024

    2024

  16. [22]

    A survey of named entity recognition and classification,

    D. Nadeau and S. Sekine, “A survey of named entity recognition and classification,” Lingvisticae Investigationes , vol. 30, no. 1, pp. 3–26, 2007

    2007

  17. [23]

    Neural architectures for named entity recognition,

    G. Lample, M. Ballesteros, S. Subramanian, K. Kawakami, and C. Dyer, “Neural architectures for named entity recognition,” in Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , 2016, p...

    2016

  18. [24]

    Bidirectional lstm-crf models for sequence tagging,

    Z. Huang, W. Xu, and K. Yu, “Bidirectional lstm-crf models for sequence tagging,” in arXiv preprint arXiv:1508.01991 , 2015

    2015 arXiv

  19. [25]

    End-to-end sequence labeling via bi-directional lstm-cnns-crf,

    X. Ma and E. Hovy, “End-to-end sequence labeling via bi-directional lstm-cnns-crf,” in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2016, pp. 1064–1074

    2016

  20. [26]

    Bert: Pre- training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre- training of deep bidirectional transformers for language understanding,” Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies...

    2019

  21. [27]

    Contextual string embeddings for sequence labeling,

    A. Akbik, D. Blythe, and R. V ollgraf, “Contextual string embeddings for sequence labeling,” in Proceedings of the 27th International Conference on Computational Linguistics , 2018, pp. 1638–1649

    2018

  22. [28]

    A technical review of canonical correlation analysis for neuroscience applications,

    X. Zhuang, Z. Yang, and D. Cordes, “A technical review of canonical correlation analysis for neuroscience applications,” Human brain map- ping, vol. 41, no. 13, pp. 3807–3833, 2020

    2020

  23. [29]

    Visual relationship detection with language priors,

    C. Lu, R. Krishna, M. Bernstein, and L. Fei-Fei, “Visual relationship detection with language priors,” in Proceedings of the European confer- ence on computer vision (ECCV) , 2018, pp. 852–869

    2018

  24. [30]

    Adaptive co-attention network for named entity recognition in tweets,

    Q. Zhang, J. Fu, X. Liu, and X. Huang, “Adaptive co-attention network for named entity recognition in tweets,” in Proceedings of the AAAI conference on artificial intelligence , vol. 32, no. 1, 2018

    2018

  25. [31]

    Assisting multimodal named entity recognition by cross-modal auxiliary tasks,

    Z. Chen, Y . Zhang, and S. Mi, “Assisting multimodal named entity recognition by cross-modal auxiliary tasks,” Pattern Recognition Letters, vol. 175, pp. 52–58, 2023

    2023

  26. [32]

    Cwi: A multimodal deep learning approach for named entity recognition from social media using character, word and image features,

    M. Asgari-Chenaghlu, M. R. Feizi-Derakhshi, L. Farzinvash, M. Balafar, and C. Motamed, “Cwi: A multimodal deep learning approach for named entity recognition from social media using character, word and image features,” Neural Computing and Applications , pp. 1–18, 2022

    2022

  27. [33]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems , vol. 30, 2017

    2017

  28. [34]

    Uamner: uncertainty- aware multimodal named entity recognition in social media posts,

    L. Liu, M. Wang, M. Zhang, L. Qing, and X. He, “Uamner: uncertainty- aware multimodal named entity recognition in social media posts,” Applied Intelligence, vol. 52, no. 4, pp. 4109–4125, 2022

    2022

  29. [35]

    Named entity recognition for social media assisted by parallel entity spans with transformer,

    J. Yu, C. Meng, L. Bing, R. Xia, and X. Zhu, “Named entity recognition for social media assisted by parallel entity spans with transformer,” in Proceedings of the 28th International Conference on Computational Linguistics, 2020, pp. 3472–3483

    2020

  30. [36]

    Multi-modal named entity recognition for social media posts,

    C. Meng, J. Yu, L. Bing, R. Xia, and X. Zhu, “Multi-modal named entity recognition for social media posts,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Vo...

    2021

  31. [37]

    The graph neural network model,

    F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, “The graph neural network model,” in IEEE Transactions on Neural Networks, vol. 20, no. 1. IEEE, 2008, pp. 61–80

    2008

  32. [38]

    Vilbert: Pretraining task- agnostic visiolinguistic representations for vision-and-language tasks,

    J. Lu, D. Batra, D. Parikh, and S. Lee, “Vilbert: Pretraining task- agnostic visiolinguistic representations for vision-and-language tasks,” in Advances in Neural Information Processing Systems , 2019, pp. 13– 23

    2019

  33. [39]

    Efficient estimation of word representations in vector space,

    T. Mikolov, I. Sutskever, K. Chen, G. Corrado, and J. Dean, “Efficient estimation of word representations in vector space,” arXiv preprint arXiv:1301.3781, 2013

    2013 arXiv

  34. [40]

    State-of-the-art named entity recognition techniques,

    J. Smith, J. Doe et al. , “State-of-the-art named entity recognition techniques,” Journal of NLP Research, vol. 5, no. 2, pp. 123–145, 2018

    2018

  35. [41]

    Advances in named entity recognition using multimodal fusion,

    M. Garcia, P. Gonzalez et al. , “Advances in named entity recognition using multimodal fusion,” in Proceedings of the ACL Conference, 2021, pp. 456–478

    2021

  36. [42]

    Enhancing named entity recognition with character representations,

    X. Chen, J. Li et al., “Enhancing named entity recognition with character representations,” Journal of Natural Language Processing, vol. 24, no. 4, pp. 437–456, 2017

    2017

  37. [43]

    Improving named entity recognition with bidirectional gated recurrent units,

    J.-H. Park, S.-W. Lee et al. , “Improving named entity recognition with bidirectional gated recurrent units,” in Proceedings of the EMNLP Conference, 2020, pp. 789–799

    2020

  38. [46]

    Towards deep semi- supervised learning,

    T. Bansal, D. Belanger, and A. Mccallum, “Towards deep semi- supervised learning,” in Proceedings of the 32nd International Confer- ence on Machine Learning , 2015, pp. 1825–1834

    2015

  39. [47]

    Real-time hashtag recommenda- tion for tweets using deep recurrent neural networks,

    K. Cho, A. Courville, and Y . Bengio, “Real-time hashtag recommenda- tion for tweets using deep recurrent neural networks,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018, pp. 1203–1211

    2018

  40. [48]

    Learning phrase representations using rnn encoder-decoder for statistical machine translation,

    K. Cho, B. van Merri ¨enboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y . Bengio, “Learning phrase representations using rnn encoder-decoder for statistical machine translation,” in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Proc...

    2014

  41. [49]

    Distributed representations of words and phrases and their compositionality,

    T. Mikolov, I. Sutskever, K. Chen, G. Corrado, and J. Dean, “Distributed representations of words and phrases and their compositionality,” Ad- vances in Neural Information Processing Systems , vol. 26, 2013

    2013

  42. [50]

    Enriching word vectors with subword information,

    P. Bojanowski, E. Grave, A. Joulin, and T. Mikolov, “Enriching word vectors with subword information,” in Transactions of the Association for Computational Linguistics , vol. 5, 2017, pp. 135–146

    2017

  43. [51]

    Lxmert: Learning cross-modality encoder representations from transformers,

    H. Tan and M. Bansal, “Lxmert: Learning cross-modality encoder representations from transformers,” in Proceedings of the 2019 Con- ference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNL...

    2019

  44. [52]

    Pixel-bert: Aligning image pixels with text by deep multi- modal transformers,

    Z. Huang, Z. Zeng, Y . Liu, X. Liu, Y . Fu, G. Larsson, M. Pollefeys, and H. Bilen, “Pixel-bert: Aligning image pixels with text by deep multi- modal transformers,” arXiv preprint arXiv:2004.07492 , 2020

    2004 arXiv

  45. [53]

    Incorporating non-local information into information extraction systems by gibbs sampling,

    J. R. Finkel, T. Grenager, and C. D. Manning, “Incorporating non-local information into information extraction systems by gibbs sampling,” in Proceedings of the 43rd annual meeting of the association for computational linguistics (ACL’05) , 2005, pp. 363–370

    2005

  46. [54]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018

    2018 arXiv

  47. [55]

    Umt: A unified multi-modal transformer for grounded situation recognition,

    J. Yu, J. Qiu, S. Jiang, X. Huang, X. Xie, and Z. Luo, “Umt: A unified multi-modal transformer for grounded situation recognition,” in Proceedings of the 28th ACM International Conference on Multimedia , 2020, pp. 4326–4335

    2020

  48. [56]

    Unified multimodal graph fusion for grounded situation recognition,

    Y . Zhang, S. Jiang, X. Huang, X. Xie, and Z. Luo, “Unified multimodal graph fusion for grounded situation recognition,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Process- ing, 2021, pp. 10 723–10 734

    2021

  49. [57]

    Mner-qg: Multimodal named entity recognition with query-guided visual grounding,

    Y . Jia, X. Huang, X. Xie, and Z. Luo, “Mner-qg: Multimodal named entity recognition with query-guided visual grounding,” in Proceedings of the 13th Language Resources and Evaluation Conference , 2022, pp. 6941–6951

    2022

  50. [58]

    Relational graph convolutional network for multimodal named entity recognition,

    Y . Zhao, X. Huang, X. Xie, and Z. Luo, “Relational graph convolutional network for multimodal named entity recognition,” in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022, pp. 1–12

    2022

  51. [59]

    Interactive text-and-image attention for multimodal named entity recognition,

    Y . Wang, X. Huang, X. Xie, and Z. Luo, “Interactive text-and-image attention for multimodal named entity recognition,” arXiv preprint arXiv:2105.00034, 2021

    2021 arXiv

  52. [60]

    Catmner: Cross-attention transformer for multimodal named entity recognition,

    ——, “Catmner: Cross-attention transformer for multimodal named entity recognition,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics , 2022, pp. 1–12

    2022

  53. [61]

    More: Multimodal reasoning for named entity recognition,

    ——, “More: Multimodal reasoning for named entity recognition,” arXiv preprint arXiv:2205.00034, 2022

    2022 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.