Pith. sign in

REVIEW 4 major objections 6 minor 30 references

A Step towards Interpretable Multimodal AI Models with MultiFIX

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Pith's one-line read MultiFIX replaces the opaque parts of a multimodal deep-learning model with readable symbolic expressions, and on synthetic problems the resulting explainable model matches or beats the black box's balanced accuracy.

desk verdict The training-strategy comparison is solid, but the central claim that MultiFIX explains the DL model's features is under-supported by the missing fidelity checks. read the letter →

arxiv 2505.11262 v1 pith:XJDHT6EW submitted 2025-05-16 cs.NE

classification cs.NE
keywords geneticprogramminginterpretabilitymultimodalitymultimodalfusionsymbolicregressionexplainableAIdeeplearningGrad-CAM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MultiFIX is a pipeline for building multimodal AI models that can explain themselves. It trains deep-learning blocks to extract a deliberately narrow set of features from each modality, then replaces the tabular feature extractor and the fusion layer with symbolic expressions evolved by genetic programming, and explains the image block with Grad-CAM heatmaps. The paper's central claim is that these explainable replacements do not cost predictive performance: on synthetic problems including AND, XOR, Multifeature, and Multiclass, the explainable model's balanced accuracy stayed within 0.006 lower to 0.075 higher than the original black-box model. If true, this matters in high-stakes domains such as healthcare, where practitioners need to know how each modality contributes to a decision.

What carries the argument

The load-bearing mechanism is sparse feature engineering plus symbolic replacement. Each modality is reduced to a bottleneck of at most three real-valued features; the tabular features and the fusion block are then approximated by tiny symbolic expressions (trees of arithmetic, Boolean, and if-then-else operators) evolved by GP-GOMEA, while Grad-CAM visualizes which image regions drive each image feature. The narrow bottleneck is what makes fusion-level symbolic regression feasible and readable.

What would settle it

Run MultiFIX on a synthetic multimodal problem with a known hidden rule, give the user only the recovered expressions, and check whether they can correctly answer a question the intended rule answers. The AND and XOR results show the model can be right while describing a different, inverted logic; if a non-expert reader cannot tell whether the intended rule was actually learned, the claim that MultiFIX accurately explains the model's integration of modalities is not established.

Watch

Extended reading notes

Core claim

The paper claims that multimodal predictions can be made interpretable by design without sacrificing accuracy, provided each modality is forced through a sparse bottleneck of at most three engineered features. On problems built from synthetic images of shapes and ten tabular variables, the DL model's tabular features and its fusion were replaced with compact symbolic expressions (using GP-GOMEA), while image features were explained post-hoc with Grad-CAM. For the AND, XOR, Multifeature, and Multiclass problems, the resulting interpretable model matched or improved on the DL model's balanced accuracy, by up to 0.075. The authors also show that the recovered expressions may encode equivalent-but-inverted logic: e.g., a learned tabular feature was the complement of the intended $x_1 > x_2$, yet the model remained correct. They interpret this as evidence that component-wise explanation lets a user see what the model actually learned, not just that it predicts well.

Load-bearing premise

The claim depends on the assumption that a handful of symbolic expressions can faithfully stand in for the deep-learning feature extractors; if the extracted features are complex, the paper itself shows the recovering expressions can become hard to interpret, and the explanation then no longer explains much.

Editorial extensions

If this is right

  • Interpretable models can be obtained from DL-trained multimodal pipelines without a performance penalty, at least on problems whose true feature structure is simple and known.
  • Component-wise inspection can reveal that the model uses a different but equally valid encoding than the intended one, turning equivalent-model discovery from a surprise into a feature of the audit trail.
  • Training strategy matters less than problem structure; end-to-end, hybrid, and sequential variants were mostly not significantly different, so practitioners can pick based on available pretrained blocks.
  • The hard case is when a modality needs more than one complex feature: the Multifeature problem produced expressions that are transparent but not easy to interpret, motivating complexity penalties in future work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same replacement strategy could be applied to other modality blocks—text or time series encoders—as long as the bottleneck stays small enough for symbolic regression to fit; on current evidence there is nothing image- or table-specific about the mechanism.
  • A controlled ablation that keeps the DL fusion but uses symbolic features would separate two effects the paper conflates: the gains on XOR and Multiclass (0.035 and 0.075 BAcc) may come from GP's regularization rather than from interpretability itself.
  • The observed equivalence phenomenon suggests a concrete diagnostic test for trust: if the symbolic expressions and a known ground-truth rule are equivalent-but-different, the model's reasoning is correct but non-unique, and a deployment decision should be reviewed by a domain expert.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents MultiFIX, a multimodal fusion pipeline that first trains a deep learning model with small bottlenecks for image and tabular features, then replaces the tabular feature-extraction block and the fusion block with symbolic expressions evolved by GP-GOMEA, while explaining image features with Grad-CAM. Experiments on four synthetic problems (AND, XOR, Multifeature, Multiclass) compare balanced accuracy across six training strategies and report interpretable models for the best-performing strategy per problem. The paper claims that MultiFIX can generate multimodal models that accurately explain both the extracted features and their integration without compromising predictive performance.

Significance. If the fidelity of the GP surrogates to the DL blocks were established, the combination of a sparse feature bottleneck with GP symbolic expressions would be a genuinely useful step toward interpretable multimodal modeling, particularly because the synthetic problems have known ground truth and the paper examines several training strategies. The paper is honest about several limitations, including the difficulty of interpreting complex image features and the post-hoc nature of Grad-CAM. However, the central interpretability claim is not yet supported by the reported evidence, since no fidelity metric is provided between the GP expressions and the DL blocks they replace, and the interpretable-model results are reported for only the best training strategy without variance.

major comments (4)
  1. [3.5, 4.1.3, 4.2.3, 4.3.3, 4.4.3] The central interpretability claim is unsupported because the GP symbolic expressions are never checked for fidelity to the DL blocks they replace. The evaluation only compares the final explainable model's balanced accuracy with the DL model's balanced accuracy (e.g., AND: 0.967 vs 0.973; XOR: 0.975 vs 0.940; Multifeature: 0.884 vs 0.864; Multiclass: 0.995 vs 0.920). A high-BAcc explainable model could be explaining the data-generating rule rather than the original DL model, especially since Sections 4.1.3 and 4.2.3 report equivalent models with inverted intermediate features. The authors should quantify the agreement between each GP expression and the corresponding DL block output on held-out data (e.g., R^2 or mean absolute error for tabular features, and accuracy or BAcc for the fusion expression), and report how often the final explainable model's predictions match the DL model's predictions.
  2. [4.1-4.4] The interpretable-model results are reported only for the single best-performing training strategy per problem, with point estimates and no variance. For the AND problem, the explainable model's BAcc is reported as 0.967 vs the DL model's 0.973, but the DL results in Table 3 have standard deviations around 0.02-0.04, and no standard deviation is given for the explainable model. This selection and the lack of variance make it impossible to assess the robustness of the 'no performance compromise' claim. The authors should report the explainable model's performance across all folds and, ideally, for all training strategies, not only the best one.
  3. [4.3.3 and 5] The manuscript itself acknowledges that in the Multifeature problem the intermediate image features are 'not easy to interpret' and that the fusion expression with depth three is 'not easy to interpret' (Section 4.3.3), and that Grad-CAM's post-hoc nature limits symbolic meaning (Section 5). This directly undercuts the conclusion that MultiFIX produces interpretable models for that problem. The authors should provide a concrete interpretability metric or a clear protocol for deciding when an obtained expression counts as interpretable, and discuss how the Multifeature result is compatible with the abstract's claim.
  4. [3.6, Tables 3-7] The statistical comparison of training strategies relies on paired t-tests over 5 folds. With n=5, the tests have very low power, and the paper interprets 'not significant' as evidence of similar performance, which is not justified. The authors should either report effect sizes with confidence intervals, use a resampling-based procedure such as the bootstrap or the corrected repeated k-fold CV test, or clearly state the power limitations of the comparison.
minor comments (6)
  1. [Table 1] The operator list 'numeric[+,−,∗,/,.2,.3,]]' is malformed; the intended numeric operators and constants (e.g., 0.2 and 0.3) are unclear and should be stated without stray brackets.
  2. [3.5] Clarify whether the GP symbolic expressions for tabular features are fitted to the DL-extracted feature values using the raw tabular inputs as independent variables, and whether the fusion GP uses the DL-extracted or the GP-reconstructed features.
  3. [3.6] The text states that HPO chooses the learning rate, weight decay, and 'number of extracted features (up to three) for the image and tabular inputs,' but Table 2 lists Image Bottleneck and Tabular Bottleneck; specify whether these are the same quantities.
  4. [5] The word 'MutiFIX' appears as a typo in the first paragraph of Section 5; it should be 'MultiFIX.'
  5. [Figures 3 and 4] The symbolic expressions in the figures are too small to read in the manuscript; consider displaying them in a larger font or presenting them in the text as well.
  6. [3.7] Clarify how the 1,000 samples are reused across the four problems and whether the same image/tabular pairs are used in each problem, since this affects the comparability of the reported results.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the BAcc results are externally benchmarked against held-out labels, and the GP surrogate fidelity gap is a correctness risk rather than a definitional reduction.

full rationale

Walking the claimed derivation chain, MultiFIX trains DL feature blocks, then fits GP-GOMEA expressions to the tabular feature outputs and to the final prediction, replacing those blocks. The final explainable model is evaluated by comparing its balanced accuracy on held-out validation folds to the DL model's balanced accuracy. This evaluation is benchmarked against external ground-truth labels, not against quantities defined by the paper's own construction. The abstract's interpretability claim is not backed by a reported fidelity metric between the GP expressions and the DL blocks they replace, and Sections 4.1.3/4.2.3 explicitly acknowledge that the learned intermediate features can be inverted complements of the intended features, so the explanation may describe an equivalent model rather than the exact DL internals. These are genuine correctness and validity concerns, but they are not circularity: neither the GP expression nor the BAcc score is defined in terms of the target claim, and no equation in the paper reduces a prediction to a fitted input. The self-citations ([14] for preliminary MultiFIX, [18] for the GP-GOMEA adaptation, [24]/[25] for background) are routine method and preliminary citations and are not load-bearing; the paper's experimental results are self-contained. Therefore no significant circularity is identified.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical entities. Its contributions rest on domain assumptions about the representativeness of synthetic data and the ability of GP to approximate deep features, which are acknowledged limitations in Section 5. The free parameters are mostly model-selection hyperparameters chosen on validation data.

free parameters (5)
  • Image bottleneck size = 0-3 (selected by HPO)
    The number of features extracted from images is chosen by grid search on validation loss; the selected value changes per problem.
  • Tabular bottleneck size = 0-3 (selected by HPO)
    Same as above for tabular features.
  • GP maximum tree depth = 2 or 3
    Fixed in Table 1; limits the size of symbolic expressions and strongly influences interpretability.
  • GP population size = 64 initially (IMS)
    Fixed in Table 1; evolutionary search parameters.
  • Adam learning rate and weight decay = grid searched over {1e-3,1e-4,1e-5} and {1e-3,1e-4,0}
    HPO hyperparameters for all DL blocks.
assumptions (4)
  • domain assumption GP-GOMEA can find compact symbolic expressions that faithfully approximate the DL feature blocks and fusion block.
    The interpretability claim depends on this; the paper shows it works on simple problems but becomes fragile in the Multifeature problem.
  • domain assumption The synthetic dataset with shapes and tabular features is representative of real-world multimodal dependencies.
    The abstract and Section 4 claim generalizability; Section 5 admits real-world validation is needed.
  • domain assumption Grad-CAM heatmaps correctly indicate which image regions drive each extracted feature.
    Used in Section 3.5 without independent verification.
  • standard math A paired t-test on the 5 folds, with Bonferroni correction, is a valid statistical comparison.
    Section 3.6; the small number of folds makes normality and independence assumptions fragile.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Step towards Interpretable Multimodal AI Models with MultiFIX." pith.science (2026). https://pith.science/paper/XJDHT6EW

@misc{pith2026250511262,
  author       = {Pith},
  title        = {Pith review of: A Step towards Interpretable Multimodal AI Models with MultiFIX},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XJDHT6EW}},
  note         = {Machine review of arXiv:2505.11262}
}
read the original abstract

Real-world problems are often dependent on multiple data modalities, making multimodal fusion essential for leveraging diverse information sources. In high-stakes domains, such as in healthcare, understanding how each modality contributes to the prediction is critical to ensure trustworthy and interpretable AI models. We present MultiFIX, an interpretability-driven multimodal data fusion pipeline that explicitly engineers distinct features from different modalities and combines them to make the final prediction. Initially, only deep learning components are used to train a model from data. The black-box (deep learning) components are subsequently either explained using post-hoc methods such as Grad-CAM for images or fully replaced by interpretable blocks, namely symbolic expressions for tabular data, resulting in an explainable model. We study the use of MultiFIX using several training strategies for feature extraction and predictive modeling. Besides highlighting strengths and weaknesses of MultiFIX, experiments on a variety of synthetic datasets with varying degrees of interaction between modalities demonstrate that MultiFIX can generate multimodal models that can be used to accurately explain both the extracted features and their integration without compromising predictive performance.

Figures

Figures reproduced from arXiv: 2505.11262 by the authors.

Figure 1
Figure 1. Overview of MultiFIX. Data passes into the feature [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Representative samples for the image modality. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Interpretable Models: Grad-CAM heatmaps explain the image input contributions for each extracted feature. GP [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Interpretable Models: Grad-CAM heatmaps explain the image input contributions for each extracted feature. GP [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 21 canonical work pages

  1. [14]

    Mafalda Malafaia, Thalea Schlender, Peter AN Bosman, and Tanja Alderliesten

  2. [1]

    Johannes Allgaier, Lena Mulansky, Rachel Draelos, and Rüdiger Pryss. 2023. How does the model make predictions? A systematic literature review on the explainability power of machine learning in healthcare.Artificial Intelligence in Medicine143 (09 2023), 102616. https://doi.org/10.1016/j.artmed.2023.102616

  3. [2]

    Nitin Arora, Anupam Singh, Vivek Shahare, and Goutam Datta. 2023. Introduction to Big Data Analytics. InTowards the Integration of IoT, Cloud and Big Data: Services, Applications and Standards. Springer, 1–18

  4. [3]

    Jaume Bacardit, Alexander EI Brownlee, Stefano Cagnoni, Giovanni Iacca, John McCall, and David Walker. 2022. The intersection of evolutionary computation and explainable AI. InProceedings of the Genetic and Evolutionary Computation conference companion. 1757–1762

  5. [4]

    Richard J Chen, Ming Y Lu, Drew FK Williamson, Tiffany Y Chen, Jana Lipkova, Zahra Noor, Muhammad Shaban, Maha Shady, Mane Williams, Bumjin Joo, et al

  6. [5]

    Benjamin P Evans, Bing Xue, and Mengjie Zhang. 2019. What’s inside the black- box? a genetic programming method for interpreting complex machine learning models. InProceedings of the genetic and evolutionary computation conference. 1012–1020

  7. [6]

    Jacob Gildenblat and contributors. 2021. PyTorch library for CAM methods. https://github.com/jacobgil/pytorch-grad-cam

  8. [7]

    Valerio Guarrasi, Fatih Aksu, Camillo Maria Caruso, Francesco Di Feola, Aurora Rofena, Filippo Ruffini, and Paolo Soda. 2024. A Systematic Review of Interme- diate Fusion in Multimodal Deep Learning for Biomedical Applications.arXiv preprint arXiv:2408.02686(2024)

Show all 30 references
  1. [8]

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 770–778. https://doi.org/10.1109/CVPR.2016.90

  2. [9]

    Shih-Cheng Huang, Anuj Pareek, Saeed Seyyedi, Imon Banerjee, and Matthew P Lungren. 2020. Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines.NPJ digital medicine3, 1 (2020), 136

  3. [10]

    Gargi Joshi, Rahee Walambe, and Ketan Kotecha. 2021. A review on explainability in multimodal deep neural nets.IEEE Access9 (2021), 59800–59821

  4. [11]

    Diederik P Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Opti- mization.International Conference on Learning Representations (ICLR)(2015)

  5. [12]

    Adrienne Kline, Hanyin Wang, Yikuan Li, Saya Dennis, Meghan Hutch, Zhenxing Xu, Fei Wang, Feixiong Cheng, and Yuan Luo. 2022. Multimodal machine learning in precision health: A scoping review.npj Digital Medicine5, 1 (2022), 171

  6. [13]

    William La Cava, Patryk Orzechowski, Bogdan Burlacu, Fabricio de Franca, Marco Virgolin, Ying Jin, Michael Kommenda, and Jason Moore. 2021. Contemporary Symbolic Regression Methods and their Relative Performance. InProceedings of the Neural Information Processing Systems Track...

  7. [15]

    Anil Rahate, Rahee Walambe, Sheela Ramanna, and Ketan Kotecha. 2022. Multi- modal co-learning: Challenges, applications with datasets, recent advances and future directions.Information Fusion81 (2022), 203–239

  8. [16]

    Cynthia Rudin. 2019. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead.Nature machine intelligence1, 5 (2019), 206–215

  9. [17]

    Cynthia Rudin, Chaofan Chen, Zhi Chen, Haiyang Huang, Lesia Semenova, and Chudi Zhong. 2022. Interpretable machine learning: Fundamental principles and 10 grand challenges.Statistic Surveys16 (2022), 1–85

  10. [18]

    Thalea Schlender, Mafalda Malafaia, Tanja Alderliesten, and Peter Bosman. 2024. Improving the efficiency of GP-GOMEA for higher-arity operators. InProceedings of the Genetic and Evolutionary Computation Conference. 971–979

  11. [19]

    Daan Schouten, Giulia Nicoletti, Bas Dille, Catherine Chia, Pierpaolo Vendittelli, Megan Schuurmans, Geert Litjens, and Nadieh Khalili. 2024. Navigating the landscape of multimodal AI in medicine: a scoping review on technical challenges and clinical applications.arXiv preprin...

  12. [20]

    Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedan- tam, Devi Parikh, and Dhruv Batra

    Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedan- tam, Devi Parikh, and Dhruv Batra. 2019. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization.International Journal of Com- puter Vision128, 2 (2019), 336–359. https://doi....

  13. [21]

    William C Sleeman, Rishabh Kapoor, and Preetam Ghosh. 2022. Multimodal classification: Current landscape, taxonomy and future directions.Comput. Surveys55, 7 (2022), 1–31

  14. [22]

    Sören Richard Stahlschmidt, Benjamin Ulfenborg, and Jane Synnergren. 2022. Multimodal deep learning for biomedical data fusion: a review.Briefings in Bioinformatics23, 2 (2022), bbab569

  15. [23]

    Vinitra Swamy, Malika Satayeva, Jibril Frej, Thierry Bossy, Thijs Vogels, Martin Jaggi, Tanja Käser, and Mary-Anne Hartley. 2024. Multimodn—multimodal, multi- task, interpretable modular networks.Advances in Neural Information Processing Systems36 (2024)

  16. [24]

    Marco Virgolin, Tanja Alderliesten, and Peter AN Bosman. 2020. On explaining machine learning models by evolving crucial and compact features.Swarm and Evolutionary Computation53 (2020), 100640

  17. [25]

    Marco Virgolin, Tanja Alderliesten, Cees Witteveen, and Peter A. N. Bosman

  18. [26]

    Fei Zhao, Chengcui Zhang, and Baocheng Geng. 2024. Deep Multimodal Data Fusion.Comput. Surveys56, 9 (2024), 1–36

  19. [27]

    Ryan Zhou and Ting Hu. 2023. Evolutionary approaches to explainable machine learning. InHandbook of Evolutionary Machine Learning. Springer, 487–506. 9

  20. [2021]

    Improving model-based genetic programming for symbolic regression of small expressions.Evolutionary Computation29, 2 (2021), 211–237

  21. [2022]

    Pan-cancer integrative histology-genomic analysis via multimodal deep learning.Cancer Cell40, 8 (2022), 865–878

  22. [2024]

    MultiFIX: An XAI-friendly feature inducing approach to building models from multimodal data.arXiv preprint arXiv:2402.12183(2024)

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.