Pith. sign in

REVIEW 3 major objections 5 minor 30 references

An Entailment Tree Generation Approach for Multimodal Multi-Hop Question Answering with Mixture-of-Experts and Iterative Feedback Mechanism

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proposes treating multimodal multi-hop QA as a joint entailment tree generation and question answering problem, with a multi-task mixture-of-experts model and iterative feedback, and reports state-of-the-art results on WebQA…

desk verdict A plausible SOTA claim on WebQA with a genuinely new application of entailment trees, but the paper's account of how errors are corrected is internally inconsistent and needs scrutiny. read the letter →

arxiv 2412.05821 v2 pith:7E2AA6YR submitted 2024-12-08 cs.CL cs.AI

Qing Zhang , Haocheng Lv , Jie Liu , Zhiyun Chen , Jianyong Duan , Hao Wang , Li He , Mingying Xv This is my paper · ORCID
classification cs.CLcs.AI
keywords multimodalmulti-hopquestionansweringentailmenttreemixture-of-expertsiterativefeedbackinterpretablereasoningdecompositionWebQA
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that multimodal multi-hop question answering is best framed as a joint entailment tree generation and question answering problem, rather than feeding all retrieved evidence to a language model as flat text. The authors build a two-stage pipeline: first a large language model decomposes the question, constructs a fact base from images, tables, and text, and generates an entailment tree whose leaves are facts and whose internal nodes are intermediate conclusions; then a smaller multi-task mixture-of-experts model jointly retrieves the correct leaf facts and answers the question while attending only to the tree's description. An iterative feedback loop feeds the small model's retrieved facts and answer back to the language model to regenerate a better tree. On the WebQA benchmark the method reports state-of-the-art results with first place on the official leaderboard, and on MultimodalQA it reports competitive results, with human evaluation rating the reasoning paths correct about 86% of the time. The authors argue this makes multimodal reasoning both more accurate and more inspectable.

What carries the argument

The carrying mechanism is the entailment tree, defined as a tuple $T = (h, L, E, S)$ with the question as hypothesis $h$, leaf nodes $L$ drawn from the fact base, internal nodes $E$ as intermediate conclusions supplied by the language model, and entailment steps $S$ linking children to parents. Around this object the paper builds a two-stage pipeline: in the initialization stage, a large language model decomposes the question into sub-questions, fills a fact base using a VQA model for images and linearized tables for tables, generates the tree structure with symbolic notation ('fact1 & fact2 -> int1'), and then refines intermediate nodes; in the optimization stage, a shared encoder encodes the tree description concatenated with the question, two gating networks with top-2 token-choice routing select among a shared expert and two task-specific experts, and two decoders perform fact retrieval generation and question answering. The load-bearing design choice is that the QA decoder attends only to the entailment tree description, so the tree must be sufficient for the answer; any failure is addressed by the iterative feedback mechanism, which replaces the fact indices with the retrieved facts, appends the predicted answer, and asks the language model to regenerate the tree.

What would settle it

Take a set of questions where the initially generated entailment tree omits a fact that is present in the fact base and needed for the correct answer, run the full pipeline including iterative feedback, and compare accuracy on that subset against questions with complete initial trees. If accuracy on the incomplete-tree subset does not drop sharply, the QA decoder must be drawing information from somewhere besides the tree description, contradicting the paper's claimed information bottleneck.

Watch

Extended reading notes

Core claim

The paper's discovery is that the entailment tree, a structure borrowed from textual entailment reasoning, can serve as the information bottleneck that solves two problems at once in multimodal multi-hop QA: it filters the redundant evidence that comes from converting entire images and tables into text, and it makes the reasoning steps explicit so errors can be located and corrected. In the proposed formulation, the input to entailment tree generation is a corpus of premises (facts from a fact base) and a hypothesis (the original question); the output is a tree $T = (h, L, E, S)$ where leaf nodes $l_i \in L$ are selected facts, internal nodes $e_i \in E$ are intermediate conclusions generated by the language model, and the root is the hypothesis. Because the QA decoder is deliberately restricted to cross-attend only over the entailment tree's textual description plus the question, the tree is not just an explanation attached after the fact; it is the channel through which all answer-relevant information must pass. The paper argues that the small model's joint fact-retrieval and QA losses, combined with an iterative feedback loop that returns the predicted facts and answer to the language model, correct the leaf-selection and structural errors that the language model makes during initialization, and this is what yields the leaderboard results.

Load-bearing premise

The approach assumes that the entailment tree, as initially generated by the language model and then patched by the feedback loop, is a faithful and complete channel for every fact needed to answer the question, because the QA model never sees the raw evidence and can only recover a missing fact if the fact-retrieval decoder selects it and the feedback loop repairs the tree.

Editorial extensions

If this is right

  • On the WebQA official test set, the method reports a QA score of 0.54 with QA-FL 0.68 and QA-Acc 0.73, surpassing the previous best result of 0.44, and it holds first place on the official leaderboard since April 10, 2024.
  • On MultimodalQA, the method reports EM 69.8 on single-modal questions and 64.7 on multi-modal questions, exceeding the previous best on both splits, with an overall EM of 68.2 and F1 of 70.9.
  • Ablation on MultimodalQA shows every component contributes: removing the fact-retrieval generation decoder drops overall EM from 68.2 to 65.2, removing entailment tree generation drops it to 65.3, and removing the iterative feedback mechanism drops it to 66.1.
  • Human evaluation on the WebQA dev set rates 88.6% of the decomposed questions correct and 85.8% of the entailment-tree reasoning paths correct, which the paper offers as evidence that the reasoning paths are usable explanations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's information-bottleneck design implies a sharper comparison than the one reported: holding the tree fixed, a QA decoder with access to the full fact base should outperform one restricted to the tree if raw facts carry information the tree discards, so such a comparison would directly test whether the tree is a necessary filter or merely a helpful one.
  • Because the paper notes that only about 1% of WebQA questions require three or more reasoning hops, the reported gains may be concentrated on simple two-hop cases; a stress test on a dataset with more genuinely multi-hop multimodal questions would reveal whether the entailment-tree machinery scales to the cases it is designed for.
  • The iterative feedback cycle between a small supervised model and an LLM has a natural transfer: the same correct-then-regenerate loop could be applied to chain-of-thought refinement, program synthesis, or tool-use planning, where an LLM's initial structural errors are corrected by a small model that sees ground-truth labels.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a two-stage method for multimodal multi-hop question answering (MMQA). In the first stage, GPT-3.5 decomposes the original question into sub-questions, builds a fact base from text, image, and table evidence via VQA and table linearization, and generates an entailment tree whose internal nodes are filled by the LLM. In the second stage, a T5-based multi-task mixture-of-experts model is trained jointly on fact retrieval generation (FRG) and question answering (QA), where the QA decoder is restricted to cross-attend only over the entailment tree description plus the question. An iterative feedback mechanism feeds the retrieved facts and predicted answer back to GPT-3.5 to regenerate the entailment tree for another round. The paper reports first place on the WebQA leaderboard, competitive results on MultimodalQA, an ablation study, and a human evaluation of reasoning-path quality.

Significance. If the central claims hold, the paper makes a useful contribution by introducing entailment tree generation into multimodal multi-hop QA and showing that interpretable reasoning structures can be coupled with a smaller trained model to improve both answer accuracy and explanation quality. The use of an external leaderboard (WebQA), a human evaluation of decomposed questions and reasoning paths (Section 4, Table 6), and ablations isolating the proposed modules (Table 5) are concrete strengths. However, the claimed mechanism for correcting LLM-generated entailment tree errors rests on an internal inconsistency in the description of the FRG supervision signal; until that is resolved, the central corrective-loop claim is not supported by the evidence as written.

major comments (3)
  1. [§3.2.1, Eq. (13) and §4 Case Study] Equation (13) defines the FRG training target as 'The fact index sequence extracted from the entailment tree generated by LLM.' This means the fact retrieval decoder is trained to imitate the LLM's leaf-node selection rather than to correct it. The case study in Section 4, however, states that 'the small model can supervise the selection of evidence with detailed labels in the dataset' and thereby correct the LLM's leaf-selection errors during iterative feedback. These two statements are mutually inconsistent. If Eq. (13) is accurate, the FRG decoder has no independent supervision signal, so any initial leaf-selection error in the LLM tree is passed forward through training, and the iterative feedback loop cannot provide the correction claimed in the contributions. If the FRG actually uses gold evidence labels, then Eq. (13) is mis-specified and the true supervision source is undisclosed. Either way, the load-bearing claim that errors from the LLM-generated entailment tree are corrected by the joint learning and feedback mechanism is not supported by the current description. The authors should clarify the exact supervision source for Eq. (13), and if the LLM tree is used, they should provide evidence that the FRG can still correct leaf errors (e.g., by comparing FRG outputs against gold evidence independently of the LLM tree).
  2. [§3.2.1, Eq. (14) and §3.2.2] The QA decoder cross-attends only over the entailment tree description and the question, not over the fact base or retrieved evidence directly (Eq. 14). The design rationale is that the tree should be a sufficient and faithful channel for the answer. Given the inconsistency in the FRG supervision noted above, the paper does not currently demonstrate that the tree channel is faithful after iterative feedback. The case study in Figure 5 shows one qualitative example of correction, but no quantitative comparison is provided between (a) answers produced from tree-only cross-attention and (b) answers produced when the QA decoder also has access to the fact base or retrieved facts. Without such a comparison, the central architectural restriction is not empirically justified, and the failure mode of an unfixable leaf-selection error remains unaddressed.
  3. [§4, Tables 2 and 4] The reported results are presented as single numbers from the WebQA leaderboard and the authors' own MultimodalQA runs, with no error bars, no multiple-seed statistics, and no released code. On MultimodalQA, the Single-Modal EM improvement over some baselines is small (69.8 vs. 69.7 for PERQA and Solar), while the claimed Multi-Modal improvement is large (64.7 vs. 55.5 for Solar). Since the core claim is state-of-the-art performance, the absence of statistical significance testing or repeated-run variance makes it hard to assess whether the gains are robust. The authors should at least report multiple runs with seeds or variance across validation folds, and ideally release code to enable independent verification.
minor comments (5)
  1. [§3.1.1, Eq. (2)] Equation (2) includes 'E_k' inside the argmax over generated tokens, but E_k is a retrieved evidence item, not a decode variable; this is likely a typo that should be corrected to avoid confusion about what the LLM generates.
  2. [Figure 1 and Figure 2] The introductory example contains the typo 'churchill down truck' instead of 'Churchill Downs track', and Figure 2 has 'Gneration' in the box label 'Entailment Tree Gneration Module'. These typos should be fixed.
  3. [§4, Table 3] The rows 'VitaminC', 'CMU ITL', 'HIT TMG', and 'SDU' are not cited or described in the text; the reader cannot tell which published systems or team entries these are. Please add references or a note on the leaderboard source.
  4. [§3.2.2] The iterative feedback mechanism uses the validation set to decide whether to continue iterating. This should be described as a model-selection step, and the authors should clarify whether any validation-set accuracy was used in choosing the final test-set configuration, since repeated selection on the leaderboard can inflate reported numbers.
  5. [§4, Ablation Study] The ablation in Table 5 shows that removing the Mixture-of-Experts module changes the overall EM by only 0.9 points (68.2 to 67.3) and leaves Single-Modal EM unchanged at 69.8. This weakens the claim that the MoE component is a key contributor; the authors should discuss this more carefully or provide additional analysis.

Circularity Check

1 steps flagged · score 6.0 of 10

FRG supervision in Eq. 13 makes the iterative 'correction' of leaf-selection errors a copy of the LLM's own tree, not an independent signal.

  1. fitted input called prediction [Section 3.2.1 Eq. (13); Section 3.2.2; Section 4 Case Study]
    "where w_{t,t+} denotes the score of the target source (The fact index sequence extracted from the entailment tree generated by LLM) at time step t ... we first replace the fact indices obtained from the fact retrieval generation task in the fact base with the corresponding facts. Then we concatenate it with the final answer and input it into GPT-3.5 as additional information to correct the entailment tree ... since the small model can supervise the selection of evidence with detailed labels in the dataset"

    Eq. 13 defines the FRG decoder's training target as the fact index sequence extracted from the LLM-generated entailment tree. Section 3.2.2 then takes the FRG decoder's predicted fact indices, maps them to facts, and feeds them back to GPT-3.5 'to correct the entailment tree.' Thus the signal used to correct the LLM's leaf-selection errors is produced by a decoder trained to imitate that same LLM leaf selection. There is no independent gold-evidence supervision specified in Eq. 13; the later claim that the small model uses 'detailed labels in the dataset' is inconsistent with Eq. 13 unless gold labels are silently introduced.

full rationale

The headline numbers are externally measured leaderboard results on held-out WebQA and MultimodalQA test sets, so the SOTA/competitive claims are not circular: the pipeline is a fixed system evaluated against unseen answers, and the cited baselines are independent. No load-bearing self-citation chain is present; the key external citations (EntailmentBank, OpenMoE, etc.) do not reduce to this paper's claims. However, a supporting but load-bearing contribution—that the second stage corrects LLM leaf-selection errors via joint learning and iterative feedback—is internally self-referential. Eq. 13 trains the fact-retrieval decoder to output the LLM-generated tree's fact indices, and Section 3.2.2 feeds those same predicted indices back to the LLM as the correction signal. After training, the 'retrieved' facts are a fitted copy of the LLM's initial tree rather than an independent correction. The paper's own case study asserts the small model supervises with 'detailed labels in the dataset,' which contradicts Eq. 13 unless gold evidence labels are secretly used. This is a partial circularity in the error-correction mechanism, but it does not invalidate the external benchmark measurement, so the overall score reflects one structurally circular prediction rather than a fully circular derivation.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method is an empirical pipeline with no new mathematical or physical entities. Its correctness depends on several domain assumptions about LLM outputs being good enough to construct facts and trees, and on treating LLM-generated fact selections as training targets. The free parameters are standard training hyperparameters plus the number of feedback rounds.

free parameters (5)
  • iteration_round_k = 2
    Number of iterative feedback rounds; chosen by validation-set accuracy plateau (Section 4 Implementation Details), not fixed by theory.
  • top_k_experts = 2
    Top-2 expert selection in the MoE routing, follows OpenMoE rather than a fitted value; a hyperparameter.
  • learning_rate = 1e-4
    AdamW learning rate for T5 fine-tuning, chosen by validation.
  • batch_size_retrieval = 32
    Batch size for the retrieval/FRG training, chosen by hardware and validation.
  • batch_size_qa = 12
    Batch size for the QA training, chosen by hardware and validation.
assumptions (4)
  • domain assumption Question decomposition with GPT-3.5 partitions the original multi-hop question into sub-questions whose VQA/table answers are sufficient to construct the fact base.
    Section 3.1.1; if a sub-question is wrong or its answer is wrong, the fact base lacks the needed facts.
  • domain assumption GPT-3.5 can generate a structurally valid entailment tree from facts and question alone, without the answer, using the '&' and '->' notation from EntailmentBank.
    Section 3.1.2 and Algorithm 1; the whole pipeline depends on this generation being correctable.
  • domain assumption The T5-based QA decoder can answer the question from the serialized entailment tree plus question alone, without direct access to raw evidence.
    Section 3.2.1; the QA decoder only cross-attends with the tree description, so the tree must be a complete information channel.
  • domain assumption The FRG decoder's supervision signal, the fact index sequence extracted from the LLM-generated entailment tree, is a reliable target for training fact retrieval.
    Eq. 13; this target is LLM-generated, not gold labels, yet it is used as ground truth for cross-entropy loss.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An Entailment Tree Generation Approach for Multimodal Multi-Hop Question Answering with Mixture-of-Experts and Iterative Feedback Mechanism." pith.science (2026). https://pith.science/paper/7E2AA6YR

@misc{pith2026241205821,
  author       = {Pith},
  title        = {Pith review of: An Entailment Tree Generation Approach for Multimodal Multi-Hop Question Answering with Mixture-of-Experts and Iterative Feedback Mechanism},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7E2AA6YR}},
  note         = {Machine review of arXiv:2412.05821}
}
read the original abstract

With the rise of large-scale language models (LLMs), it is currently popular and effective to convert multimodal information into text descriptions for multimodal multi-hop question answering. However, we argue that the current methods of multi-modal multi-hop question answering still mainly face two challenges: 1) The retrieved evidence containing a large amount of redundant information, inevitably leads to a significant drop in performance due to irrelevant information misleading the prediction. 2) The reasoning process without interpretable reasoning steps makes the model difficult to discover the logical errors for handling complex questions. To solve these problems, we propose a unified LLMs-based approach but without heavily relying on them due to the LLM's potential errors, and innovatively treat multimodal multi-hop question answering as a joint entailment tree generation and question answering problem. Specifically, we design a multi-task learning framework with a focus on facilitating common knowledge sharing across interpretability and prediction tasks while preventing task-specific errors from interfering with each other via mixture of experts. Afterward, we design an iterative feedback mechanism to further enhance both tasks by feeding back the results of the joint training to the LLM for regenerating entailment trees, aiming to iteratively refine the potential answer. Notably, our method has won the first place in the official leaderboard of WebQA (since April 10, 2024), and achieves competitive results on MultimodalQA.

Figures

Figures reproduced from arXiv: 2412.05821 by the authors.

Figure 1
Figure 1. Examples of current methods that converting mul [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. In our proposed (a) Entailment Tree Initialization Stage, we build a fact base by decomposing multi-hop questions and [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The prompt template of decompose question. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The prompt template of entailment tree structure [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Case Study of Our Method. For simple inference, the entailment tree structure generated by the large model can usually be correctly predicted, thereby correctly answering the questions. However, for complex inference data, the large model may predict the entailment tre…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 15 canonical work pages

  1. [1]

    Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. 2015. Vqa: Visual question answering. In Proceedings of the IEEE international conference on computer vision . 2425–2433

    2015

  2. [2]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. , 1877–1901 pages

    work page 2020

  3. [3]

    Yingshan Chang, Mridu Narang, Hisami Suzuki, Guihong Cao, Jianfeng Gao, and Yonatan Bisk. 2022. Webqa: Multihop and multimodal qa. , 16495–16504 pages

    work page 2022

  4. [4]

    Soravit Changpinyo, Doron Kukliansky, Idan Szpektor, Xi Chen, Nan Ding, and Radu Soricut. 2022. All you may need for vqa are image captions. arXiv preprint arXiv:2205.01883 (2022)

    arXiv 2022

  5. [5]

    Wenhu Chen, Hexiang Hu, Xi Chen, Pat Verga, and William W Cohen. 2022. Murag: Multimodal retrieval-augmented generator for open question answering over images and text. arXiv preprint arXiv:2210.02928 (2022)

    arXiv 2022

  6. [6]

    Bhavana Dalvi, Peter Jansen, Oyvind Tafjord, Zhengnan Xie, Hannah Smith, Leighanna Pipatanangkura, and Peter Clark. 2021. Explaining Answers with Entailment Trees. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 7358–7370

    work page 2021

  7. [7]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding.arXiv preprint arXiv:1810.04805 (2018)

    arXiv 2018

  8. [8]

    Feng Gao, Qing Ping, Govind Thattai, Aishwarya Reganti, Ying Nian Wu, and Prem Natarajan. 2022. Transform-retrieve-generate: Natural language-centric outside-knowledge visual question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 5067–5077

    work page 2022

Show all 30 references
  1. [9]

    Shashank Gupta, Subhabrata Mukherjee, Krishan Subudhi, Eduardo Gonzalez, Damien Jose, Ahmed H Awadallah, and Jianfeng Gao. 2022. Sparsely activated mixture-of-experts are robust multi-task learners. arXiv preprint arXiv:2204.07689 (2022)

    2022 arXiv

  2. [10]

    Ruixin Hong, Hongming Zhang, Xintong Yu, and Changshui Zhang. 2022. MET- GEN: A Module-Based Entailment Tree Generation Framework for Answer Ex- planation. In Findings of the Association for Computational Linguistics: NAACL

    2022

  3. [11]

    Yushi Hu, Hang Hua, Zhengyuan Yang, Weijia Shi, Noah A Smith, and Jiebo Luo. 2023. Promptcap: Prompt-guided image captioning for vqa with gpt-3. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 2963– 2975

    2023

  4. [12]

    Aran Komatsuzaki, Joan Puigcerver, James Lee-Thorp, Carlos Riquelme Ruiz, Basil Mustafa, Joshua Ainslie, Yi Tay, Mostafa Dehghani, and Neil Houlsby

  5. [13]

    Xiaoxi Li, Yujia Zhou, and Zhicheng Dou. 2024. Unigen: A unified generative framework for retrieval and question answering with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 38. 8688–8696

    2024

  6. [14]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 26296–26306

    2024

  7. [15]

    Tengxiao Liu, Qipeng Guo, Xiangkun Hu, Yue Zhang, Xipeng Qiu, and Zheng Zhang. 2022. RLET: A Reinforcement Learning Based Approach for Explainable QA with Entailment Trees. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing . 7177–7189

    2022

  8. [16]

    Weihao Liu, Fangyu Lei, Tongxu Luo, Jiahe Lei, Shizhu He, Jun Zhao, and Kang Liu. 2023. MMHQA-ICL: Multimodal In-context Learning for Hybrid Question Answering over Text, Tables and Images. arXiv preprint arXiv:2309.04790 (2023)

    2023 arXiv

  9. [17]

    Ilya Loshchilov and Frank Hutter. 2018. Decoupled Weight Decay Regularization. In International Conference on Learning Representations

    2018

  10. [18]

    Haohao Luo, Ying Shen, and Yang Deng. 2023. Unifying Text, Tables, and Images for Multimodal Question Answering. In Findings of the Association for Computa- tional Linguistics: EMNLP 2023 . 9355–9367

    2023

  11. [19]

    Jiaqi Ma, Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, and Ed H Chi. 2018. Modeling task relationships in multi-task learning with multi-gate mixture-of- experts. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining . 1930–1939

    2018

  12. [20]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learni...

    2021

  13. [21]

    Danilo Neves Ribeiro, Shen Wang, Xiaofei Ma, Rui Dong, Xiaokai Wei, Henghui Zhu, Xinchi Chen, Peng Xu, Zhiheng Huang, Andrew Arnold, et al. 2022. Entail- ment Tree Explanations via Iterative Retrieval-Generation Reasoner. In Findings of the Association for Computational Lingui...

    2022

  14. [22]

    Alon Talmor, Ori Yoran, Amnon Catav, Dan Lahav, Yizhong Wang, Akari Asai, Gabriel Ilharco, Hannaneh Hajishirzi, and Jonathan Berant. 2021. Multimodalqa: Complex question answering over text, tables and images. arXiv preprint arXiv:2104.06039 (2021)

    2021 arXiv

  15. [23]

    Hongyan Tang, Junning Liu, Ming Zhao, and Xudong Gong. 2020. Progressive layered extraction (ple): A novel multi-task learning (mtl) model for personalized recommendations. In Proceedings of the 14th ACM Conference on Recommender Systems. 269–278

    2020

  16. [24]

    Fuzhao Xue, Zian Zheng, Yao Fu, Jinjie Ni, Zangwei Zheng, Wangchunshu Zhou, and Yang You. 2024. Openmoe: An early effort on open mixture-of-experts language models. arXiv preprint arXiv:2402.01739 (2024)

    2024 arXiv

  17. [25]

    Qian Yang, Qian Chen, Wen Wang, Baotian Hu, and Min Zhang. 2022. Enhancing Multi-modal and Multi-hop Question Answering via Structured Knowledge and Unified Retrieval-Generation. arXiv preprint arXiv:2212.08632 (2022)

    2022 arXiv

  18. [26]

    Shuwen Yang, Anran Wu, Xingjiao Wu, Luwei Xiao, Tianlong Ma, Cheng Jin, and Liang He. 2023. Progressive Evidence Refinement for Open-domain Multimodal Retrieval Question Answering. arXiv e-prints (2023), arXiv–2310

    2023

  19. [27]

    Zhengyuan Yang, Zhe Gan, Jianfeng Wang, Xiaowei Hu, Yumao Lu, Zicheng Liu, and Lijuan Wang. 2022. An empirical study of gpt-3 for few-shot knowledge- based vqa. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 36. 3081–3089

    2022

  20. [28]

    Bowen Yu, Cheng Fu, Haiyang Yu, Fei Huang, and Yongbin Li. 2023. Unified Language Representation for Question Answering over Text, Tables, and Images. In Findings of the Association for Computational Linguistics: ACL 2023. 4756–4765

    2023

  21. [29]

    Barret Zoph, Irwan Bello, Sameer Kumar, Nan Du, Yanping Huang, Jeff Dean, Noam Shazeer, and William Fedus. 2022. St-moe: Designing stable and transfer- able sparse expert models. arXiv preprint arXiv:2202.08906 (2022)

    2022 arXiv

  22. [2023]

    In The Eleventh International Conference on Learning Representations

    Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints. In The Eleventh International Conference on Learning Representations . https: //openreview.net/forum?id=T5nUQDrM4u

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.