Integrating Multimodal Information in Large Pretrained Transformers

Amir Zadeh; Chengfeng Mao; Ehsan Hoque; Louis-Philippe Morency; Md. Kamrul Hasan; Sangwu Lee; Wasifur Rahman

REVIEW 3 major objections 5 minor 31 references

Integrating Multimodal Information in Large Pretrained Transformers

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A gated shift of internal word representations lets fine-tuned BERT and XLNet integrate tone and facial expression, and on CMU-MOSI the XLNet variant matches reported human sentiment judgment.

desk verdict Useful first step for multimodal fine-tuning of BERT/XLNet, but single-run results and an over-sold human-level claim keep it from being a strong accept. read the letter →

arxiv 1908.05787 v3 pith:UYJKVUTD submitted 2019-08-15 cs.LG cs.CLstat.ML

Wasifur Rahman , Md. Kamrul Hasan , Sangwu Lee , Amir Zadeh , Chengfeng Mao , Louis-Philippe Morency , Ehsan Hoque This is my paper · ORCID

classification cs.LGcs.CLstat.ML

keywords multimodalsentimentanalysisBERTXLNetfine-tuningadaptationgatenonverbalbehaviorhuman-levelperformancetransformer

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Large pretrained language transformers only accept text, so multimodal face-to-face data has typically been handled by fusing separate modality streams after the fact. This paper proposes an attachment, called the Multimodal Adaptation Gate, that instead shifts the transformer's internal word vectors by a gated combination of acoustic and visual features during fine-tuning, leaving the pretrained architecture unchanged. The authors show that fine-tuning BERT and XLNet with this gate consistently outperforms language-only fine-tuning and previous multimodal models on CMU-MOSI and CMU-MOSEI, and that the XLNet variant reaches reported human-level performance on CMU-MOSI. If the result holds, it means pretrained text models can absorb nonverbal cues through a small, layer-local mechanism rather than needing a multimodal pretraining stage.

What carries the argument

The Multimodal Adaptation Gate (MAG), a per-word module attached to one encoder layer of BERT or XLNet. For each word it computes bimodal gates $g^v_i = R(W_{gv}[Z_i; V_i] + b_v)$ and $g^a_i = R(W_{ga}[Z_i; A_i] + b_a)$, fuses them into a displacement $H_i = g^a_i \odot (W_a A_i) + g^v_i \odot (W_v V_i) + b_H$, and shifts the internal vector to $\bar{Z}_i = Z_i + \alpha H_i$ with the scaled $\alpha$. This mechanism carries the argument because it modifies the pretrained semantic space directly, in the direction of nonverbal behavior, and the layer-ablation experiments show that early injection is what lets the shift propagate through the remaining transformer layers.

What would settle it

A decisive test is to ablate the acoustic and visual inputs inside MAG, feeding the gate only the lexical vector; if performance does not drop, the gains come from added parameters, not from nonverbal adaptation. A complementary test is to apply MAG to sentences where tone contradicts lexical sentiment, such as positive words spoken sarcastically; a model at human level should flip polarity on those items, and a failure would show the additive shift cannot capture context-dependent inversion.

Watch

Extended reading notes

Core claim

The central claim is that a single additive displacement of lexical vectors, computed from the word's own representation and its aligned acoustic and visual features, is enough to let a pretrained transformer adapt to multimodal sentiment. The update is $\bar{Z}_i = Z_i + \alpha H_i$, where $H_i$ is a gated fusion of modality features and $\alpha = \min(\|Z_i\|_2^2 / \|H_i\|_2^2 \cdot \beta, 1)$ is a norm-based scale. Injecting this gate at an early encoder layer yields the best results, with MAG-XLNet reaching binary accuracy 85.7/87.9, F1 85.6/87.9, MAE 0.675, and correlation 0.821 on CMU-MOSI, compared with human 85.7, 87.5, 0.710, 0.820. The paper interprets this as the first human-level multimodal sentiment analysis result in this community.

Load-bearing premise

The method assumes that the effect of tone of voice and facial expression on a word can be represented as a single additive displacement in the transformer's semantic space at one layer; if the true interaction is not approximately additive, the reported gains may be specific to this architecture and dataset pairing.

Editorial extensions

If this is right

Fine-tuning BERT or XLNet with MAG gives consistent accuracy and regression gains over language-only fine-tuning on both CMU-MOSI and CMU-MOSEI.
MAG-XLNet closes the gap to reported human performance on CMU-MOSI, suggesting the remaining error is not primarily a fusion problem.
Injecting the gate at early encoder layers outperforms later layers and input-level concatenation or addition, pointing to where multimodal adaptation should occur in transformer stacks.
Randomly reinitializing the transformer weights removes most of the gain, so the superior results depend on successful fine-tuning of pretrained representations rather than the gate alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural extension is to apply MAG to other transformer families or to decoder-only models, where a similar gate could shift query, key, or value representations during generation rather than only classification.
The gate could be reused as a lightweight adapter for multitask learning: different nonverbal gates could attach to different tasks while sharing the base transformer, reducing the cost of per-task fine-tuning.
Because the paper's qualitative examples include sarcasm, a targeted experiment on sarcasm-heavy utterances could reveal whether the additive shift captures pragmatic inversion or only intensity adjustment.
An interpretability check of the gate weights could show which acoustic and visual features carry the signal, and whether the gate learns feature selection or degenerates to a constant offset.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Two things to know up front. This is the first paper I know of that fine-tunes BERT and XLNet directly on multimodal data, using a small gate that shifts internal word representations based on acoustic and visual features. That part is genuinely useful. The second thing is that the evidence for the headline claims is thinner than the prose suggests: the results in Table 1 are single runs, with no standard deviations or seed counts, and the 'human-level' comparison rests on a single published human number.

What the paper does well: the method is simple and leaves the pretrained transformer intact, which makes it easy to adopt. The authors correctly credit RAVEN for the displacement idea. They also do the right thing by retraining older baselines with BERT/XLNet embeddings, so the comparison is not against stale GloVe-era numbers. The ablations—gate placement, input-level concatenation/addition, and random initialization—are informative. The random-init results make the fine-tuning argument convincing. Code is promised publicly.

The weaknesses are real and mostly about reporting. No error bars anywhere. The single t-test in §6.2 is not described well enough: if it is a paired per-sample test over the test set, it only shows the mean difference is nonzero, not that it is stable across random seeds. On MOSEI the gains shrink to a few tenths of a point, with no significance test. And the human-level claim is oversold: under the left-hand evaluation protocol, MAG-XLNet actually has a lower F1 (85.6 vs 87.5) than the reported human value. The additive-shift assumption is a crude approximation, but that is an acceptable first step rather than a flaw.

I would send this to serious peer review rather than desk reject. The method is useful, the experiments are coherent, and the central weakness—missing variance information and a too-strong claim—can be fixed in revision. A referee should ask for multiple seeds and a clearer statement of what the t-test is over, and push the authors to soften 'human-level' to 'comparable on some metrics.' If the improvements survive seed variance, this will be a standard reference for multimodal fine-tuning of pretrained transformers.

Referee Report

3 major / 5 minor

Summary. The paper proposes a Multimodal Adaptation Gate (MAG), a lightweight attachment to BERT and XLNet that injects visual and acoustic features into the transformer's internal representations during fine-tuning. For each token, MAG computes a gated displacement from nonverbal features and adds it to the lexical representation, with a norm-based scaling factor. The authors fine-tune MAG-BERT and MAG-XLNet on CMU-MOSI and CMU-MOSEI for multimodal sentiment analysis and report consistent improvements over language-only fine-tuning and over multimodal baselines that use BERT/XLNet embeddings. They further report a layer ablation, input-level fusion baselines, a fine-tuning control experiment with randomly initialized transformers, qualitative examples, and a claim that MAG-XLNet reaches human-level performance on CMU-MOSI.

Significance. If the empirical claims hold, MAG is a simple and useful contribution: it lets pretrained text-only transformers consume nonverbal modalities without architectural changes, adds minimal parameters, is evaluated on standard public benchmarks, and the authors release code. The paper also makes a useful comparison by retraining prior multimodal models with BERT/XLNet embeddings, which is fairer than comparing against GloVe-based numbers. The central weakness is that the headline claims—"consistent improvement" over language-only fine-tuning and "human-level" performance—rest on single-run results without variance estimates and on an ambiguously described significance test. The MOSEI generalization claim is also under-supported. These issues are fixable and do not, in my view, invalidate the method, but they currently prevent the paper from fully supporting its strongest conclusions.

major comments (3)

[§6.2, Table 1] The central empirical claim of consistent improvement over language-only fine-tuning is not yet statistically supported. Table 1 reports only single values for BA, F1, MAE, and Corr, with no standard deviations, confidence intervals, or number of seeds for any model, including the baselines. The only significance statement, in §6.2, reports a student t-test with p < 10e−5 but does not state whether the test is over paired test-set predictions or over repeated fine-tuning runs. If it is a per-sample paired test, it only shows that the average prediction differs from zero, not that the improvements are stable across random initializations and data orders. This distinction matters because fine-tuning large transformers is known to be seed-sensitive, and the reported MAG-XLNet gains over XLNet are small: BA +1.0/+1.2, F1 +1.0/+1.2, MAE −0.001, Corr +0.009. Please report mean and standard deviation over multiple seeds for all models and clarify the design of the significance test.
[§6.5] The claim that MAG generalizes to CMU-MOSEI is not adequately supported. The subsection reports only binary accuracy and F1 for three models, with no standard deviations, no significance test, and an explicit statement that the evaluation was limited by time constraints. The gains over language-only XLNet are 0.2–0.5 points in BA/F1, and the gains over BERT are 0.8/0.6 points. Without repeated runs or error bars, these differences are within plausible run-to-run variation. Either complete the MOSEI evaluation with variance-aware results and a significance test, or temper the generalization claim in the abstract and conclusion.
[Table 1, Human row] The 'human-level performance' claim is fragile because it is based on a single reported human value from Zadeh et al. (2018a) with no human annotation variance. MAG-XLNet's BA equals the human value (85.7), its MAE is better (0.675 vs 0.710), and its correlation is essentially identical (0.821 vs 0.820), but without human error bars or multiple human annotators it is impossible to know whether these differences are meaningful. The phrase 'achieves human-level multimodal sentiment analysis performance for the first time' should be conditioned on the uncertainty of both the model estimates and the human estimates.

minor comments (5)

[§5.4] The experimental setup should specify the number of epochs, batch size, early stopping criterion, and how the validation set was used for each model; the current text only lists learning rate and dropout ranges. Also, 'datset' is a typo in the first sentence of §5.4.
[§6.2] The notation '10e−5' is ambiguous; it should be written as 1e−5 or 10^{−5}, and the t-test should report the test statistic, degrees of freedom, and whether it is paired or unpaired.
[§6.4] The input-level concatenation and addition baselines are described too briefly for reproducibility; please report the exact input dimensions, how the visual/acoustic features were projected, and whether these variants were tuned with the same hyperparameter search as MAG.
[§4, Eq. (5)] Equation (5) is undefined when ||H_i||_2 = 0, which can occur with zero acoustic/visual features or with a zero gate output. Adding a small epsilon in the denominator or a fallback alpha would make the formulation numerically robust.
[Table 2] The layer ablation table reports single numbers without variance or significance testing; at minimum, the caption should state that these are single-run results, and the selection of the insertion layer j and beta should be described in the experimental setup.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation: MAG is an architectural ansatz evaluated on held-out benchmarks, and cited prior work is not load-bearing.

full rationale

The paper's central claim is empirical rather than derivational: fine-tuning BERT and XLNet with the Multimodal Adaptation Gate improves multimodal sentiment analysis on CMU-MOSI and CMU-MOSEI. Section 4 defines the gate as ̅Z_i = Z_i + alpha H_i, an explicit modeling choice rather than a result derived from the data; hyperparameters including beta and the insertion layer are selected on the designated validation set (Section 5.4), and Table 1 reports held-out test metrics. No equation reduces to its own input by construction, and no fitted parameter is relabeled as a prediction. The RAVEN citation (Wang et al., 2018) is used as design inspiration for the displacement idea, and the human performance value comes from the same group's earlier dataset paper (Zadeh et al., 2018a), but neither is invoked as a uniqueness theorem or as proof of MAG's effectiveness; they are external benchmarks or prior art. Concerns about missing seed variance and the ambiguous t-test are statistical-reporting risks, not circularity.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central mechanism is an architectural addition; the non-derived inputs are the beta hyperparameter, the insertion layer choice, the learned gate weights, and the assumption that an additive shift in semantic space captures nonverbal meaning. No new physical entities are postulated.

free parameters (3)

beta (scaling cap) = selected by cross-validation (value not reported)
Equation 5 uses beta to control the magnitude of the multimodal shift alpha; it is a hyperparameter tuned on the validation set, not derived from first principles.
MAG insertion layer j = 1 for XLNet in reported results (Table 2)
The layer at which MAG is inserted is chosen by validation performance (Section 6.3). This affects the outcome and is a design choice fitted to the data.
MAG gate weight matrices and biases = trained on task data (values not listed)
The displacement in Eq. 3 uses learned matrices W_gv, W_ga, W_a, W_v and biases b_v, b_a, b_H; no closed form is given, so the method depends on these being fit to the downstream training data.

assumptions (3)

domain assumption Nonverbal meaning is additive in semantic space: the multimodal representation equals the language-only representation plus a displacement vector (Section 4).
This is the core modeling assumption of MAG and is not derived; it is motivated by RAVEN and by intuition about semantic shifts.
domain assumption The aligned word-level acoustic and visual features are informative for sentiment beyond the text (Section 5.2).
The method assumes COVAREP and FACET features aligned to words carry signal that is not already fully captured by the language model; otherwise MAG would add no value.
domain assumption Pretrained BERT and XLNet weights can be fine-tuned with a small inserted module without losing the benefits of pretraining (Section 6.6).
The fine-tuning effect experiment treats random initialization as a control and assumes that pretraining is necessary for the observed gains.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Integrating Multimodal Information in Large Pretrained Transformers." pith.science (2026). https://pith.science/paper/UYJKVUTD

@misc{pith2026190805787,
  author       = {Pith},
  title        = {Pith review of: Integrating Multimodal Information in Large Pretrained Transformers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UYJKVUTD}},
  note         = {Machine review of arXiv:1908.05787}
}

read the original abstract

Recent Transformer-based contextual word representations, including BERT and XLNet, have shown state-of-the-art performance in multiple disciplines within NLP. Fine-tuning the trained contextual models on task-specific datasets has been the key to achieving superior performance downstream. While fine-tuning these pre-trained models is straightforward for lexical applications (applications with only language modality), it is not trivial for multimodal language (a growing area in NLP focused on modeling face-to-face communication). Pre-trained models don't have the necessary components to accept two extra modalities of vision and acoustic. In this paper, we proposed an attachment to BERT and XLNet called Multimodal Adaptation Gate (MAG). MAG allows BERT and XLNet to accept multimodal nonverbal data during fine-tuning. It does so by generating a shift to internal representation of BERT and XLNet; a shift that is conditioned on the visual and acoustic modalities. In our experiments, we study the commonly used CMU-MOSI and CMU-MOSEI datasets for multimodal sentiment analysis. Fine-tuning MAG-BERT and MAG-XLNet significantly boosts the sentiment analysis performance over previous baselines as well as language-only fine-tuning of BERT and XLNet. On the CMU-MOSI dataset, MAG-XLNet achieves human-level multimodal sentiment analysis performance for the first time in the NLP community.

Figures

Figures reproduced from arXiv: 1908.05787 by the authors.

**Figure 1.** Multimodal Adaptation Gate (MAG) takes as input a lexical input vector, as well as its visual and acoustic accompaniments. Subsequently, an attention over lexical and nonverbal dimensions is used to fuse the multimodal data into another vector, which is subsequently added to the input lexical vector (shifting). 4 Multimodal Adaptation Gate (MAG) In multimodal language, a lexical input is accompanied by visual and … view at source ↗

**Figure 2.** Best viewed zoomed in and in color. The Transformer architecture of BERT/XLNet with MAG applied at jth layer. We consider a total of M layers within the pretrained Transformer. MAG can be applied at different layers of the pretrained Transformers. 4.1 MAG-BERT MAG-BERT is a combination of MAG applied to a certain layer of BERT network ( [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 10 canonical work pages

[1]

Minghai Chen, Sen Wang, Paul Pu Liang, Tadas Baltru s aitis, Amir Zadeh, and Louis-Philippe Morency. 2017. Multimodal sentiment analysis with word-level fusion and reinforcement learning. In Proceedings of the 19th ACM International Conference on Multimodal Interaction, pages 163--171. ACM

2017
[2]

Andy Coenen, Emily Reif, Ann Yuan, Been Kim, Adam Pearce, Fernanda Vi \'e gas, and Martin Wattenberg. 2019. Visualizing and measuring the geometry of bert. arXiv preprint arXiv:1906.02715

arXiv 2019
[3]

Zihang Dai, Zhilin Yang, Yiming Yang, William W Cohen, Jaime Carbonell, Quoc V Le, and Ruslan Salakhutdinov. 2019. Transformer-xl: Attentive language models beyond a fixed-length context. arXiv preprint arXiv:1901.02860

arXiv 2019
[4]

Gilles Degottex, John Kane, Thomas Drugman, Tuomo Raitio, and Stefan Scherer. 2014. Covarep—a collaborative voice analysis repository for speech technologies. In 2014 ieee international conference on acoustics, speech and signal processing (icassp), pages 960--964. IEEE

work page 2014
[5]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

arXiv 2018
[6]

Devamanyu Hazarika, Soujanya Poria, Amir Zadeh, Erik Cambria, Louis-Philippe Morency, and Roger Zimmermann. 2018. Conversational memory network for emotion recognition in dyadic dialogue videos. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long ...

work page 2018
[7]

iMotions. 2017. goo.gl/1rh1JN Facial expression analysis

work page 2017
[8]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

arXiv 2014

Show all 31 references

[9]

Paul Pu Liang, Ziyin Liu, Amir Zadeh, and Louis-Philippe Morency. 2018. Multimodal language analysis with recurrent multistage fusion. arXiv preprint arXiv:1808.03920

2018 arXiv
[10]

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111--3119

2013
[11]

Sunghyun Park, Han Suk Shim, Moitreya Chatterjee, Kenji Sagae, and Louis-Philippe Morency. 2014. Computational analysis of persuasiveness in social multimedia: A novel dataset and multimodal prediction approach. In Proceedings of the 16th International Conference on Multimodal...

2014
[12]

Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532--1543

2014
[13]

Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. arXiv preprint arXiv:1802.05365

2018 arXiv
[14]

Hai Pham, Paul Pu Liang, Thomas Manzini, Louis-Philippe Morency, and Barnabas Poczos. 2019. Found in translation: Learning robust joint representations by cyclic translations between modalities. arXiv preprint arXiv:1812.07809

2019 arXiv
[15]

Soujanya Poria, Amir Hussain, and Erik Cambria. 2018. Multimodal Sentiment Analysis, volume 8. Springer

2018
[16]

Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. URL https://s3-us-west-2. amazonaws. com/openai-assets/research-covers/languageunsupervised/language understanding paper. pdf

2018
[17]

Chen Sun, Austin Myers, Carl Vondrick, Kevin Murphy, and Cordelia Schmid. 2019. Videobert: A joint model for video and language representation learning. arXiv preprint arXiv:1904.01766

2019 arXiv
[18]

Yao-Hung Hubert Tsai, Shaojie Bai, Paul Pu Liang, J Zico Kolter, Louis-Philippe Morency, and Ruslan Salakhutdinov. 2019. Multimodal transformer for unaligned multimodal language sequences. arXiv preprint arXiv:1906.00295

2019 arXiv
[19]

Yao-Hung Hubert Tsai, Paul Pu Liang, Amir Zadeh, Louis-Philippe Morency, and Ruslan Salakhutdinov. 2018. Learning factorized multimodal representations. arXiv preprint arXiv:1806.06176

2018 arXiv
[20]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, pages 5998--6008

2017
[21]

Yansen Wang, Ying Shen, Zhun Liu, Paul Pu Liang, Amir Zadeh, and Louis-Philippe Morency. 2018. Words can shift: Dynamically adjusting word representations using nonverbal behaviors. arXiv preprint arXiv:1811.09362

2018 arXiv
[22]

Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. arXiv preprint arXiv:1906.08237

2019 arXiv
[23]

Jiahong Yuan and Mark Liberman. 2008. Speaker identification on the scotus corpus. Journal of the Acoustical Society of America, 123(5):3878

2008
[24]

Amir Zadeh, Minghai Chen, Soujanya Poria, Erik Cambria, and Louis-Philippe Morency. 2017. Tensor fusion network for multimodal sentiment analysis. arXiv preprint arXiv:1707.07250

2017 arXiv
[25]

Amir Zadeh, Paul Pu Liang, Navonil Mazumder, Soujanya Poria, Erik Cambria, and Louis-Philippe Morency. 2018 a . Memory fusion network for multi-view sequential learning. In Thirty-Second AAAI Conference on Artificial Intelligence

2018
[26]

Amir Zadeh, Paul Pu Liang, Louis-Philippe Morency, Soujanya Poria, Erik Cambria, and Stefan Scherer. 2018 b . Proceedings of grand challenge and workshop on human multimodal language (challenge-hml). In Proceedings of Grand Challenge and Workshop on Human Multimodal Language (...

2018
[27]

Amir Zadeh, Paul Pu Liang, Soujanya Poria, Prateek Vij, Erik Cambria, and Louis-Philippe Morency. 2018 c . Multi-attention recurrent network for human communication comprehension. In Thirty-Second AAAI Conference on Artificial Intelligence

2018
[28]

Amir Zadeh, Rowan Zellers, Eli Pincus, and Louis-Philippe Morency. 2016. Mosi: multimodal corpus of sentiment intensity and subjectivity analysis in online opinion videos. arXiv preprint arXiv:1606.06259

2016 arXiv
[29]

AmirAli Bagher Zadeh, Paul Pu Liang, Soujanya Poria, Erik Cambria, and Louis-Philippe Morency. 2018 d . Multimodal language analysis in the wild: Cmu-mosei dataset and interpretable dynamic fusion graph. In Proceedings of the 56th Annual Meeting of the Association for Computat...

2018
[30]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...
[31]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Minghai Chen, Sen Wang, Paul Pu Liang, Tadas Baltru s aitis, Amir Zadeh, and Louis-Philippe Morency. 2017. Multimodal sentiment analysis with word-level fusion and reinforcement learning. In Proceedings of the 19th ACM International Conference on Multimodal Interaction, pages 163--171. ACM

2017

[2] [2]

Andy Coenen, Emily Reif, Ann Yuan, Been Kim, Adam Pearce, Fernanda Vi \'e gas, and Martin Wattenberg. 2019. Visualizing and measuring the geometry of bert. arXiv preprint arXiv:1906.02715

arXiv 2019

[3] [3]

Zihang Dai, Zhilin Yang, Yiming Yang, William W Cohen, Jaime Carbonell, Quoc V Le, and Ruslan Salakhutdinov. 2019. Transformer-xl: Attentive language models beyond a fixed-length context. arXiv preprint arXiv:1901.02860

arXiv 2019

[4] [4]

Gilles Degottex, John Kane, Thomas Drugman, Tuomo Raitio, and Stefan Scherer. 2014. Covarep—a collaborative voice analysis repository for speech technologies. In 2014 ieee international conference on acoustics, speech and signal processing (icassp), pages 960--964. IEEE

work page 2014

[5] [5]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

arXiv 2018

[6] [6]

Devamanyu Hazarika, Soujanya Poria, Amir Zadeh, Erik Cambria, Louis-Philippe Morency, and Roger Zimmermann. 2018. Conversational memory network for emotion recognition in dyadic dialogue videos. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long ...

work page 2018

[7] [7]

iMotions. 2017. goo.gl/1rh1JN Facial expression analysis

work page 2017

[8] [8]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

arXiv 2014