Pith. sign in

REVIEW 4 major objections 7 minor 4 cited by

Strong speaker perception in omni models does not predict natural interruption timing or phrasing in live dialogue.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 23:26 UTC pith:GRI5P6UN

load-bearing objection Solid joint who–when–how OLM benchmark with real rank inversions; the decoupling claim is directionally right but rests on a deliberately small, LLM-judged generation diagnostic. the 4 major comments →

arxiv 2603.16859 v2 pith:GRI5P6UN submitted 2026-03-17 cs.AI

SocialOmni: Benchmarking Audio-Visual Social Interactivity in Omni Models

Tianyu Xie , Jinfa Huang , Yuexiao Ma , Rongfang Luo , Yan Yang , Wang Chen , Yuhui Zeng , Yixuan Zou
show 6 more authors
Qingchuan Ma Zhiqiang Lu Ruize Fang Xiawu Zheng Jiebo Luo Rongrong Ji
This is my paper
classification cs.AI
keywords omni-modal large language modelssocial interactivityspeaker identificationturn-takinginterruption generationaudio-visual inconsistencybenchmarkperception-generation decoupling
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Omni-modal models can see, hear, and talk, but existing tests mostly check static answer accuracy. This paper argues that real conversational competence also requires knowing who is speaking, when to take a turn, and how to interrupt naturally. It introduces SocialOmni, a benchmark built around those three axes, with thousands of perception items, a controlled generation set, and audio-visual mismatch probes. Across twelve leading models, the scores vary widely by axis, and high accuracy on speaker identification often fails to line up with timely, appropriate interruptions. The result is a concrete case that understanding-centric metrics alone do not measure social interactivity, and that separate diagnostics are needed to close the perception-interaction gap.

Core claim

When twelve omni-modal models are scored jointly on speaker identification (who), turn-entry timing (when), and interruption generation (how), perceptual accuracy and generation quality systematically diverge: models that lead on who do not necessarily lead on how, and no model dominates all three axes. Controlled audio-visual inconsistency further exposes brittle cross-modal binding. The paper therefore claims that conversational social competence cannot be read off understanding accuracy alone.

What carries the argument

SocialOmni’s who–when–how evaluation: frame-level multiple-choice speaker attribution (with consistent vs inconsistent audio-visual clips), streaming turn-entry decisions scored by timing offset windows, and multi-judge scoring of generated interruptions.

Load-bearing premise

That a compact, tightly curated set of 209 generation clips with fixed one-second query steps, preset timing windows, and coarse three-judge scores is a reliable enough diagnostic of real-time social interactivity.

What would settle it

Re-run the same twelve models on a substantially larger, independently annotated generation set with human turn-boundary labels and human quality ratings; if perception and interruption quality then tightly co-rank, the claimed decoupling collapses.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper introduces SocialOmni, a benchmark for evaluating omni-modal large language models (OLMs) on audio-visual social interactivity along three axes: who (speaker identification via 2,000 MCQ perception items, including consistent/inconsistent AV splits), when (turn-entry timing on a 209-item generation set with signed offset categories), and how (natural interruption generation scored by three LLM judges on a {25,50,75,100} scale). Twelve commercial and open-source OLMs are evaluated under a dual-axis protocol with bootstrap CIs, macro-F1, E/O/L timing decompositions, precision–recall operating points, and consistency-gap diagnostics. The central empirical claim is a pronounced decoupling between perceptual accuracy and interruption-generation quality (e.g., Qwen3-Omni high who / low how; GPT-4o low who / stronger how), implying that understanding-centric metrics alone are insufficient to characterize conversational social competence.

Significance. If the decoupling result holds under stronger validation, SocialOmni fills a genuine gap: existing OLM suites (Table 1) largely measure static answer accuracy and do not jointly operationalize speaker attribution, turn-entry timing, and interruption realization under controlled AV conflict. The who design (orthogonal speaker/content distractors), Δ_cons robustness probe, and multi-judge generation protocol are concrete contributions, and the reported rank inversions and failure modes (saliency-driven attribution, premature silence-gap triggering, context–response decoupling) give actionable signals for architecture and training. Strengths include multi-model coverage, bootstrap CIs, macro-F1 against positional bias, and an appendix human check on a hard subset. The work is timely for full-duplex multi-party OLMs, provided the generation diagnostic is shown to be reliable enough to support the insufficiency claim.

major comments (4)
  1. The load-bearing claim of “pronounced decoupling” between who and how (§4.2, Table 2; Abstract; Conclusion) rests primarily on the N_g=209 generation split (§3.2, A.5) and the three-judge how protocol (§3.4, Algorithm 2, A.11). The paper itself keeps this set compact for variance control and does not report a full-benchmark human baseline or systematic human–judge correlation for how; A.17 gives human how only on a 50-item hard subset (55.15/100) selected from model failures. Rank inversions on this diagnostic are real, but without broader human calibration (or at least item-level agreement between judges and humans on a stratified sample of the 209), the inference that understanding-centric metrics are “insufficient to characterize conversational social competence” is stronger than the evidence currently warrants. Strengthen with human ratings on a larger, stratified generation sample a
  2. Limitations and A.10 acknowledge that Task II how scoring uses transcribed outputs and may underweight visual grounding and prosody. Yet the abstract and §5 frame how as “natural interruption generation” in an audio-visual social setting. If judges see only ASR + reference text, the how axis is effectively a text-continuation appropriateness score conditioned on a when decision, not a full multimodal social-response score. This weakens the claim that SocialOmni jointly evaluates the integrated who–when–how triad under multimodal cues. Either (i) make the text-only judging scope explicit in the main claims and title framing, or (ii) add a visually/prosodically grounded scoring condition (as partially sketched in A.10) and show that decoupling persists under that condition.
  3. When metrics depend on free parameters that are only partially stress-tested: streaming stride δ=1 s (A.6) and timing windows (θ1,θ2,θ3)=(1,2,5) s (A.7), with primary On-time defined as −1≤Δτ≤2 s. Table 4 and Figure 4–5 report useful E/O/L and P–R structure, and A.8 defines δ∈{0.2,0.5,1.0} binary metrics, but the main leaderboard (Table 2) and the “best when” ranking (Gemini 3 Pro Preview 67.31%) are not shown to be stable under threshold/stride sweeps in the main text. Because when is one of the three axes used to argue heterogeneous capability profiles and non-dominance (§4.2, Figure 3), report main-text sensitivity of On-time rates and model ranks to (θ,δ), or justify the windows with human timing-tolerance data rather than appendix-only rationale.
  4. Cross-task association is asserted as “pronounced decoupling” and “do not correlate” (§4.2) but is not quantified in the main results with a formal association statistic over the 12 models (e.g., Spearman ρ between who Acc and how score, with CI). Appendix A.16 defines correlation machinery largely for the hard-subset human analysis (A.17–A.18), not for the primary who–how claim. Add a model-level correlation (and optionally leave-one-out rank stability) between who and how on the full evaluation so the decoupling claim is a measured effect size, not only a pair of rank inversions.
minor comments (7)
  1. Table 1 and abstract claim “comprehensive” coverage; with 209 generation items this is better framed as a controlled diagnostic (as Limitations already does). Align abstract wording with Limitations to avoid overclaim.
  2. Figure 1(c) and Figure 3 are hard to read in grayscale; ensure distinct line styles/markers for the 12 models and define all radar axes (e.g., Robustness = 100−|Δ_cons|) in the caption.
  3. Inconsistent numbers: Table 5 lists OmniVinci overall Acc 15.15% while Table 2/3 list ~35.86%; VITA-1.5 Acc is 36.95 vs 36.97 in places. Reconcile perception tables and regenerate CIs if needed.
  4. A.11 defines large-gap as |s(a)−s(b)|≥20 while §3.4 uses ≥25 points for R_gap. Unify the threshold definition.
  5. Repeated wording artifacts (“consistent consistent,” “robust robust,” “systematically systematically”) appear in §3.3–3.4 and related work; a careful copy-edit pass is needed.
  6. MiniOmni2 is perception-only (*); state clearly in Table 2 caption that when/how means exclude it from cross-axis correlation analyses.
  7. Prompt cards (A.14) for when ask “Is it your turn to speak now?” while Algorithm 2 uses “Should Xi speak now?” with a designated speaker Xi. Clarify whether models are role-playing a fixed participant or answering about a named candidate, and keep prompts identical to the evaluation code.

Circularity Check

0 steps flagged

Empirical benchmark paper: who/when/how metrics and the decoupling claim are independently measured, not forced by definition or self-citation.

full rationale

SocialOmni is a benchmark-and-evaluation paper, not a first-principles derivation. Task I (who) uses top-1 accuracy and macro-F1 on 2,000 MCQs with fixed options; Task II (when) uses signed response offset Δτ and fixed timing windows (1, 2, 5)s; how uses multi-judge means on a discrete {25,50,75,100} scale. These quantities are defined independently of any model’s training objective and of each other, so rank inversions (e.g., Qwen3-Omni high who / low how; GPT-4o low who / strong how) are empirical observations, not tautologies. No parameter is fitted on a subset and then re-reported as a prediction of a closely related quantity. Self-citations in Related Work (video frame selection, RAG, quantization) are ordinary prior-work pointers and do not underwrite the decoupling claim or the benchmark construction. Limitations of the 209-item generation set and LLM judges affect validity/reliability, not circularity. No self-definitional loop, fitted-input-as-prediction, uniqueness import, or renaming of a known result as a forced derivation appears in the load-bearing chain.

Axiom & Free-Parameter Ledger

4 free parameters · 3 axioms · 2 invented entities

The central empirical claim rests on a small set of design choices that define what counts as correct social interactivity and how generation quality is scored. Timing windows, streaming stride, judge scale, and the curated generation subset size are free parameters chosen by the authors; the who–when–how triad and the reliability of LLM judges are domain assumptions. The benchmark itself is the main invented entity, with independent evidence only insofar as others can re-run it on the released data.

free parameters (4)
  • timing thresholds (θ1, θ2, θ3) = (1, 2, 5) s
    Define Interrupted / Perfect / Delayed / TooLate categories for the when-score; chosen by authors and justified only by appendix discussion, not derived from external theory.
  • streaming query stride δ = 1 s (with 0.5/2 s ablations)
    Controls temporal resolution of turn-entry decisions; selected as cost–precision trade-off.
  • LLM-judge discrete scale {25, 50, 75, 100}
    Coarse score set used for how; affects mean scores and large-gap rate.
  • generation subset size N_g = 209
    Kept small for annotation control; directly limits statistical power of when/how claims.
axioms (3)
  • domain assumption Social interactivity for OLMs is adequately captured by the joint who–when–how triad under multi-party AV dialogue.
    Stated as the operational definition in §1 and §3.3; other facets (prosody, gesture, multi-turn repair) are deferred.
  • domain assumption Three commercial/open LLM judges under fixed prompts are a sufficiently stable proxy for human judgments of interruption appropriateness.
    Used for all how scores (§3.4, A.11); only partial human calibration on a hard subset is provided.
  • domain assumption Clips with clear alternating turns and CC-BY-compatible multi-party dialogue are representative enough for diagnostic evaluation.
    Construction pipeline in §3.1–3.2; stricter selection for generation items is acknowledged.
invented entities (2)
  • SocialOmni benchmark (who/when/how tasks + AV-inconsistency split) no independent evidence
    purpose: Provide a unified interaction-oriented evaluation suite missing from prior OLM benchmarks.
    Defined entirely by this paper’s construction and metrics; independent evidence will come only from external re-use of the released data.
  • consistency gap Δ_cons = Acc_cons − Acc_incons no independent evidence
    purpose: Quantify reliance on visual–audio alignment for speaker identification.
    Paper-defined diagnostic; useful but not an external physical quantity.

pith-pipeline@v1.1.0-grok45 · 27436 in / 2962 out tokens · 34379 ms · 2026-07-13T23:26:49.236262+00:00 · methodology

0 comments
read the original abstract

Omni-modal large language models (OLMs) redefine human-machine interaction by natively integrating audio, vision, and text. However, existing OLM benchmarks remain anchored to static, accuracy-centric tasks, leaving a critical gap in assessing social interactivity, the fundamental capacity to navigate dynamic cues in natural dialogues. To this end, we propose SocialOmni, a comprehensive benchmark that operationalizes the evaluation of this conversational interactivity across three core dimensions: (i) speaker separation and identification (who is speaking), (ii) interruption timing control (when to interject), and (iii) natural interruption generation (how to phrase the interruption). SocialOmni features 2,000 perception samples and a quality-controlled diagnostic set of 209 interaction-generation instances with strict temporal and contextual constraints, complemented by controlled audio-visual inconsistency scenarios to test model robustness. We benchmarked 12 leading OLMs, which uncovers significant variance in their social-interaction capabilities across models. Furthermore, our analysis reveals a pronounced decoupling between a model's perceptual accuracy and its ability to generate contextually appropriate interruptions, indicating that understanding-centric metrics alone are insufficient to characterize conversational social competence. More encouragingly, these diagnostics from SocialOmni yield actionable signals for bridging the perception-interaction divide in future OLMs.

Figures

Figures reproduced from arXiv: 2603.16859 by Jiebo Luo, Jinfa Huang, Qingchuan Ma, Rongfang Luo, Rongrong Ji, Ruize Fang, Tianyu Xie, Wang Chen, Xiawu Zheng, Yan Yang, Yixuan Zou, Yuexiao Ma, Yuhui Zeng, Zhiqiang Lu.

Figure 1
Figure 1. Figure 1: Overview of SocialOmni. (a) Benchmark data distribution across 15 subcategories and four domains, with consistent/inconsistent stratification and perception/generation task splits. (b) Overview of the proposed evaluation tasks and metrics. (c) Performance comparison of 12 OLMs on both Task I and Task II. benchmarks [2, 19, 27, 28, 43, 46] emphasize turn-taking timing and interruption detection, yet predomi… view at source ↗
Figure 2
Figure 2. Figure 2: Illustration of the SocialOmni evaluation pipeline. Given a multi-modal conversation stream (Zone 1), SocialOmni constructs both audio-vision inconsistent and consistent consistent (Zone 2), then evaluates models on speaker perception (Task I) and turn-entry generation (Task II) with LLM-based judging (Zone 3). deliberately includes both consistent clips (Ncons = 1,725) and inconsistent clips (Nincons = 27… view at source ↗
Figure 3
Figure 3. Figure 3: Cross-axis capability profiles. Each polygon shows one model over normalized who–when–how dimensions. No single model dominates all axes, revealing distinct strengths and weaknesses. 8 [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Timing-phase decomposition for turn entry. Early/On-time/Late rates expose whether a model tends to interrupt prematurely or miss the optimal conversational window during dialogue. 4.3.3 Failure Cases Beyond critically aggregate metrics, we systematically inspect cases where the majority of evaluated models consistently fail, thereby identifying systemic bottlenecks rather than individual model weaknesses.… view at source ↗
Figure 5
Figure 5. Figure 5: Precision–recall operating points for when decisions. Iso-F1 guides highlight the fundamental trade-off between cautious and trigger-happy turn-entry strategies in dialogue systems. the visually salient face rather than maintaining speaker–identity binding across frames. This reflects a failure to reconcile “who was speaking before the cut” with “who is visible now,” a deficit in temporal cross-modal coher… view at source ↗
Figure 6
Figure 6. Figure 6: Selected-subset human feedback across who/when/how. The pale interval spans the full-benchmark mean to the best reported model; the dark connector links the same-subset model mean and selected-subset human score. A.17 Human Feedback on a Challenging Subset This subsection reports an additional human-feedback analysis without changing the claims in the main paper. We follow the same who/when/how axes as the… view at source ↗
Figure 7
Figure 7. Figure 7: Correlation estimates on the selected challenging subset. Each point shows the estimated correlation; horizontal segments show the corresponding 95% confidence interval. The dashed vertical line marks zero association. A.18 Human Feedback Discussion This subsection discusses how the human-feedback results should be read. One finding is the negative item-level association between human judgments and model e… view at source ↗
Figure 8
Figure 8. Figure 8: Who failure case. The visually dominant frame favors the wrong speaker; The correct answer requires cross-modal speaker binding. The case illustrates a saliency-driven attribution error [PITH_FULL_IMAGE:figures/full_fig_p023_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: When failure case. The waveform contains a pause-like gap, but the turn remains unfinished. The case illustrates premature turn triggering from shallow silence-gap cues [PITH_FULL_IMAGE:figures/full_fig_p023_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: How failure case. The dialogue establishes an interpersonal context, but the response remains generic rather than empathetic. The case illustrates context-response decoupling in generation. For how, [PITH_FULL_IMAGE:figures/full_fig_p023_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Omni-DeepSearch: A Benchmark for Audio-Driven Omni-Modal Deep Search

    cs.SD 2026-05 unverdicted novelty 8.0

    Omni-DeepSearch is a 640-sample benchmark for audio-driven omni-modal search where the best model reaches only 43.44% accuracy, exposing bottlenecks in audio inference, tool use, and cross-modal reasoning.

  2. GRASP: Learning to Ground Social Reasoning in Multi-Person Non-Verbal Interactions

    cs.CV 2026-05 unverdicted novelty 7.0

    GRASP is a large-scale dataset and benchmark for social reasoning grounded in gaze and gesture events in multi-person videos, with Social Grounding Reward (SGR) proposed to improve model performance on GRASP-Bench.

  3. OmniFocus: Query-Guided Modality-Balanced Token Compression for Omni-Modal Large Language Models

    cs.LG 2026-07 conditional novelty 5.5

    Query-guided, modality-symmetric token compression preserves audio-visual evidence better than audio-guided budgets at 25–35% retention on Qwen2.5-Omni.

  4. CogniRoute: Learning to Route Social Evidence in Omni-Modal Models

    cs.CV 2026-06 unverdicted novelty 5.0

    CogniRoute adds a cognitive schema and route-aware RL to an omni-modal MoE, reaching 59.38% accuracy on a new 118K-example social video QA benchmark and beating prior baselines by 15-27 points.

Reference graph

Works this paper leans on

62 extracted references · 1 canonical work pages · cited by 4 Pith papers

  1. [1]

    Flamingo: A visual language model for few-shot learning.arXiv preprint arXiv:2204.14198, 2022

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katie Millican, Malcolm Reynolds, et al. Flamingo: A visual language model for few-shot learning.arXiv preprint arXiv:2204.14198, 2022. URLhttps://arxiv.org/abs/2204.14198

    Pith/arXiv arXiv 2022

  2. [2]

    Talking turns: Bench- marking audio foundation models on turn-taking dynamics.arXiv preprint arXiv:2503.01174, 2025

    Siddhant Arora, Zhiyun Lu, Chung-Cheng Chiu, Ruoming Pang, and Shinji Watanabe. Talking turns: Bench- marking audio foundation models on turn-taking dynamics.arXiv preprint arXiv:2503.01174, 2025. URL https://arxiv.org/abs/2503.01174

    Pith/arXiv arXiv 2025

  3. [3]

    JointAVBench: A benchmark for joint audio-visual reasoning evaluation.arXiv preprint arXiv:2512.12772, 2025

    Jianghan Chao, Jianzhang Gao, Wenhui Tan, Yuchong Sun, Ruihua Song, and Liyun Ru. JointAVBench: A benchmark for joint audio-visual reasoning evaluation.arXiv preprint arXiv:2512.12772, 2025. URL https: //arxiv.org/abs/2512.12772

    Pith/arXiv arXiv 2025

  4. [4]

    Event-anchored frame selection for effective long-video understanding.arXiv preprint arXiv:2603.00983, 2026

    Wang Chen, Yongdong Luo, Yuhui Zeng, Luojun Lin, Tianyu Xie, Fei Chao, Rongrong Ji, and Xiawu Zheng. Event-anchored frame selection for effective long-video understanding.arXiv preprint arXiv:2603.00983, 2026

    arXiv 2026

  5. [5]

    Wavelet-based frame selection by detecting semantic boundary for long video understanding.arXiv preprint arXiv:2603.00512, 2026

    Wang Chen, Yuhui Zeng, Yongdong Luo, Tianyu Xie, Luojun Lin, Jiayi Ji, Yan Zhang, and Xiawu Zheng. Wavelet-based frame selection by detecting semantic boundary for long video understanding.arXiv preprint arXiv:2603.00512, 2026

    arXiv 2026

  6. [6]

    AMUSE: Audio-visual benchmark and alignment framework for agentic multi-speaker understanding.arXiv preprint arXiv:2512.16250, 2025

    Sanjoy Chowdhury, Karren Dai Yang, Xudong Liu, Fartash Faghri, Pavan Kumar Anasosalu Vasu, Oncel Tuzel, Dinesh Manocha, Chun-Liang Li, and Raviteja Vemulapalli. AMUSE: Audio-visual benchmark and alignment framework for agentic multi-speaker understanding.arXiv preprint arXiv:2512.16250, 2025. URL https://arxiv.org/abs/2512.16250

    arXiv 2025

  7. [7]

    Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities.arXiv preprint arXiv:2507.06261, 2025

    Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities.arXiv preprint arXiv:2507.06261, 2025. URLhttps://arxiv.org/abs/2507.06261

    Pith/arXiv arXiv 2025

  8. [8]

    Bootstrap methods: Another look at the jackknife.The Annals of Statistics, 7(1):1–26, 1979

    Bradley Efron. Bootstrap methods: Another look at the jackknife.The Annals of Statistics, 7(1):1–26, 1979. doi: 10.1214/aos/1176344552. URL https://projecteuclid.org/journals/annals-of-statistics/volume-7/iss ue-1/Bootstrap-Methods--Another-Look-at-the-Jackknife/10.1214/aos/1176344552.full

    doi:10.1214/aos/1176344552 1979

  9. [9]

    Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis

    Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 24108–24118,

  10. [10]

    URL https://openaccess.thecvf.com/content/CVPR2025/html/Fu_Video-MME_The_First-Ever_Comp rehensive_Evaluation_Benchmark_of_Multi-modal_LLMs_in_CVPR_2025_paper.html

  11. [11]

    Vita-1.5: Towards gpt-4o level real-time vision and speech interaction.arXiv preprint arXiv:2501.01957, 2025

    Chaoyou Fu, Haojia Lin, Xiong Wang, Yi-Fan Zhang, Yunhang Shen, Xiaoyu Liu, Yangze Li, Zuwei Long, Heting Gao, Ke Li, et al. Vita-1.5: Towards gpt-4o level real-time vision and speech interaction.arXiv preprint arXiv:2501.01957, 2025. URLhttps://arxiv.org/abs/2501.01957

    Pith/arXiv arXiv 2025

  12. [12]

    Funasr: A fundamental end-to-end speech recognition toolkit.arXiv preprint arXiv:2305.11013, 2023

    Zhifu Gao, Zerui Li, Jiaming Wang, Haoneng Luo, Xian Shi, Mengzhe Chen, Yabin Li, Lingyun Zuo, Zhihao Du, Zhangyu Xiao, and Shiliang Zhang. Funasr: A fundamental end-to-end speech recognition toolkit.arXiv preprint arXiv:2305.11013, 2023. URLhttps://arxiv.org/abs/2305.11013

    Pith/arXiv arXiv 2023

  13. [13]

    Gemini 3: Introducing the latest gemini ai model from google.https://blog.google/products-and-p latforms/products/gemini/gemini-3/, 2025

    Google. Gemini 3: Introducing the latest gemini ai model from google.https://blog.google/products-and-p latforms/products/gemini/gemini-3/, 2025. Google Blog, Nov 18, 2025. Accessed: 2026-03-01

    2025

  14. [14]

    Release notes | gemini api | google ai for developers

    Google AI for Developers. Release notes | gemini api | google ai for developers. https://ai.google.de v/gemini- api/docs/changelog , 2026. Documents launch/update records for gemini-3-pro-preview and gemini-3-flash-preview. Accessed: 2026-03-01

    2026

  15. [15]

    WorldSense: Evaluating real- world omnimodal understanding for multimodal llms.arXiv preprint arXiv:2502.04326, 2025

    Jack Hong, Shilin Yan, Jiayin Cai, Xiaolong Jiang, Yao Hu, and Weidi Xie. WorldSense: Evaluating real- world omnimodal understanding for multimodal llms.arXiv preprint arXiv:2502.04326, 2025. URL https: //arxiv.org/abs/2502.04326

    Pith/arXiv arXiv 2025

  16. [16]

    Determining layer-wise sparsity for large language models through a theoretical perspective.arXiv preprint arXiv:2502.14770, 2025

    Weizhong Huang, Yuxin Zhang, Xiawu Zheng, Fei Chao, and Rongrong Ji. Determining layer-wise sparsity for large language models through a theoretical perspective.arXiv preprint arXiv:2502.14770, 2025

    Pith/arXiv arXiv 2025

  17. [17]

    Discovering important 12 experts for mixture-of-experts models pruning through a theoretical perspective

    Weizhong Huang, Yuxin Zhang, Xiawu Zheng, Fei Chao, Rongrong Ji, and Liujuan Cao. Discovering important 12 experts for mixture-of-experts models pruning through a theoretical perspective. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025

    2025

  18. [18]

    Dynamic low-rank sparse adaptation for large language models.arXiv preprint arXiv:2502.14816, 2025

    Weizhong Huang, Yuxin Zhang, Xiawu Zheng, Yang Liu, Jing Lin, Yiwu Yao, and Rongrong Ji. Dynamic low-rank sparse adaptation for large language models.arXiv preprint arXiv:2502.14816, 2025

    Pith/arXiv arXiv 2025

  19. [19]

    GPT-4o System Card.arXiv preprint arXiv:2410.21276, 2024

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. GPT-4o System Card.arXiv preprint arXiv:2410.21276, 2024. URL https://arxiv.org/abs/2410.21276

    Pith/arXiv arXiv 2024

  20. [20]

    From specific-mllms to omni-mllms: a survey on mllms aligned with multi-modalities

    Shixin Jiang, Jiafeng Liang, Jiyuan Wang, Xuan Dong, Heng Chang, Weijiang Yu, Jinhua Du, Ming Liu, and Bing Qin. From specific-mllms to omni-mllms: a survey on mllms aligned with multi-modalities. InFindings of the Association for Computational Linguistics: ACL 2025, pages 8617–8652, 2025. doi: 10.18653/v1/2025.finding s-acl.453. URLhttps://aclanthology.o...

    work page doi:10.18653/v1/2025.finding 2025

  21. [21]

    SIV-Bench: A video benchmark for social interaction understanding and reasoning.arXiv preprint arXiv:2506.05425, 2025

    Fanqi Kong, Weiqin Zu, Xinyu Chen, Yaodong Yang, Song-Chun Zhu, and Xue Feng. SIV-Bench: A video benchmark for social interaction understanding and reasoning.arXiv preprint arXiv:2506.05425, 2025. URL https://arxiv.org/abs/2506.05425

    Pith/arXiv arXiv 2025

  22. [22]

    Seed-bench- 2: Benchmarking multimodal large language models.arXiv preprint arXiv:2311.17092, 2023

    Bohao Li, Yuying Ge, Yixiao Ge, Guangzhi Wang, Rui Wang, Ruimao Zhang, and Ying Shan. Seed-bench- 2: Benchmarking multimodal large language models.arXiv preprint arXiv:2311.17092, 2023. URL https: //arxiv.org/abs/2311.17092

    Pith/arXiv arXiv 2023

  23. [23]

    OmniVideoBench: Towards audio-visual understanding evaluation for omni mllms.arXiv preprint arXiv:2510.10689, 2025

    Caorui Li, Yu Chen, Yiyan Ji, Jin Xu, Zhenyu Cui, Shihao Li, Yuanxing Zhang, Jiafu Tang, Zhen Song, Dingling Zhang, Yinghui He, Haoxian Liu, Yuxuan Wang, Qiufeng Wang, Zhenhe Wu, Jiehui Luo, Zhiyu Pan, Weihao Xie, Chenchen Zhang, Zhaohui Wang, Jiayi Tian, Yanghai Wang, Zhe Cao, Minxin Dai, Kefeng Wang, Runzhe Wen, Ying Ma, Yaning Pan, Sungkyun Chang, Term...

    arXiv 2025

  24. [24]

    WOW-seg: A word-free open world segmentation model

    Danyang Li, Tianhao Wu, Bin Lin, Zhenyuan Chen, Yang Zhang, Yuxuan Li, Ming-Ming Cheng, and Xiang Li. WOW-seg: A word-free open world segmentation model. InThe Fourteenth International Conference on Learning Representations, 2026. URLhttps://openreview.net/forum?id=AyJPSnE1bq

    2026

  25. [25]

    Mvbench: A comprehensive multi-modal video understanding benchmark

    Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, et al. Mvbench: A comprehensive multi-modal video understanding benchmark. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22195–22206, 2024. URL https://openaccess.thecvf.com/content/CVPR2024/html/Li_MVBench_A_C...

    2024

  26. [26]

    Baichuan-Omni-1.5 Technical Report.arXiv preprint arXiv:2501.15368, 2025

    Yadong Li, Jun Liu, Tao Zhang, Song Chen, Tianpeng Li, Zehuan Li, Lijun Liu, Lingfeng Ming, Guosheng Dong, Da Pan, et al. Baichuan-Omni-1.5 Technical Report.arXiv preprint arXiv:2501.15368, 2025. URL https://arxiv.org/abs/2501.15368

    arXiv 2025

  27. [27]

    OmniBench: Towards the future of universal omni-language models.arXiv preprint arXiv:2409.15272, 2024

    Yizhi Li, Ge Zhang, Yi Ma, Ruibin Yuan, Kang Zhu, Hangyu Guo, Yiming Liang, Jiaheng Liu, Jian Yang, Siwei Wu, Xingwei Qu, Jinjie Shi, Xinyue Zhang, Zhen Yang, Xiangzhou Wang, Zhaoxiang Zhang, Zachary Liu, Emmanouil Benetos, Wenhao Huang, and Chenghua Lin. OmniBench: Towards the future of universal omni-language models.arXiv preprint arXiv:2409.15272, 2024...

    arXiv 2024

  28. [28]

    Liu, and Hung-yi Lee

    Guan-Ting Lin, Jiachen Lian, Tingle Li, Qirui Wang, Gopala Anumanchipalli, Alexander H. Liu, and Hung-yi Lee. Full-duplex-bench: A benchmark to evaluate full-duplex spoken dialogue models on turn-taking capabilities. arXiv preprint arXiv:2503.04721, 2025. URLhttps://arxiv.org/abs/2503.04721

    Pith/arXiv arXiv 2025

  29. [29]

    Wearvox: An egocentric multichannel voice assistant benchmark for wearables.arXiv preprint arXiv:2601.02391, 2025

    Zhaojiang Lin, Yong Xu, Kai Sun, Jing Zheng, Yin Huang, Surya Teja Appini, Krish Narang, Renjie Tao, Ishan Kapil Jain, Siddhant Arora, et al. Wearvox: An egocentric multichannel voice assistant benchmark for wearables.arXiv preprint arXiv:2601.02391, 2025. URLhttps://arxiv.org/abs/2601.02391

    arXiv 2025

  30. [30]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. arXiv preprint arXiv:2310.03744, 2023. URLhttps://arxiv.org/abs/2310.03744

    Pith/arXiv arXiv 2023

  31. [31]

    G-Eval: Nlg evaluation using gpt-4 with better human alignment.arXiv preprint arXiv:2303.16634, 2023

    Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. G-Eval: Nlg evaluation using gpt-4 with better human alignment.arXiv preprint arXiv:2303.16634, 2023. URLhttps://arxiv.org/ab s/2303.16634. 13

    Pith/arXiv arXiv 2023

  32. [32]

    MMBench: Is your multi-modal model an all-around player? European Conference on Computer Vision, 2023

    Yuanzhan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, Kai Chen, and Dahua Lin. MMBench: Is your multi-modal model an all-around player? European Conference on Computer Vision, 2023. URLhttps://arxiv.org/abs/2307.06281

    Pith/arXiv arXiv 2023

  33. [33]

    Video-rag: Visually-aligned retrieval-augmented long video comprehension.arXiv preprint arXiv:2411.13093, 2024

    Yongdong Luo, Xiawu Zheng, Guilin Li, Shukang Yin, Haojia Lin, Chaoyou Fu, Jinfa Huang, Jiayi Ji, Fei Chao, Jiebo Luo, et al. Video-rag: Visually-aligned retrieval-augmented long video comprehension.arXiv preprint arXiv:2411.13093, 2024

    arXiv 2024

  34. [34]

    Quota: Query-oriented token assignment via cot query decouple for long video comprehension.arXiv preprint arXiv:2503.08689, 2025

    Yongdong Luo, Wang Chen, Xiawu Zheng, Weizhong Huang, Shukang Yin, Haojia Lin, Chaoyou Fu, Jinfa Huang, Jiayi Ji, Jiebo Luo, et al. Quota: Query-oriented token assignment via cot query decouple for long video comprehension.arXiv preprint arXiv:2503.08689, 2025

    Pith/arXiv arXiv 2025

  35. [35]

    Ompq: Orthogonal mixed precision quantization

    Yuexiao Ma, Taisong Jin, Xiawu Zheng, Yan Wang, Huixia Li, Yongjian Wu, Guannan Jiang, Wei Zhang, and Rongrong Ji. Ompq: Orthogonal mixed precision quantization. InProceedings of the AAAI conference on artificial intelligence, volume 37, pages 9029–9037, 2023

    2023

  36. [36]

    Affinequant: Affine transformation quantization for large language models.arXiv preprint arXiv:2403.12544, 2024

    Yuexiao Ma, Huixia Li, Xiawu Zheng, Feng Ling, Xuefeng Xiao, Rui Wang, Shilei Wen, Fei Chao, and Rongrong Ji. Affinequant: Affine transformation quantization for large language models.arXiv preprint arXiv:2403.12544, 2024

    Pith/arXiv arXiv 2024

  37. [37]

    Outlier-aware slicing for post-training quantization in vision transformer

    Yuexiao Ma, Huixia Li, Xiawu Zheng, Feng Ling, Xuefeng Xiao, Rui Wang, Shilei Wen, Fei Chao, and Rongrong Ji. Outlier-aware slicing for post-training quantization in vision transformer. InForty-first International Conference on Machine Learning, 2024

    2024

  38. [38]

    Flow caching for autoregressive video generation.arXiv preprint arXiv:2602.10825, 2026

    Yuexiao Ma, Xuzhe Zheng, Jing Xu, Xiwei Xu, Feng Ling, Xiawu Zheng, Huafeng Kuang, Huixia Li, Xing Wang, Xuefeng Xiao, et al. Flow caching for autoregressive video generation.arXiv preprint arXiv:2602.10825, 2026

    arXiv 2026

  39. [39]

    Social genome: Grounded social reasoning abilities of multimodal models.Conference on Empirical Methods in Natural Language Processing, 2025

    Leena Mathur, Marian Qian, Paul Pu Liang, and Louis-philippe Morency. Social genome: Grounded social reasoning abilities of multimodal models.Conference on Empirical Methods in Natural Language Processing, 2025. URLhttps://arxiv.org/abs/2502.15109

    Pith/arXiv arXiv 2025

  40. [40]

    Omni-SafetyBench: A benchmark for safety evaluation of audio-visual large language models.arXiv preprint arXiv:2508.07173, 2025

    Leyi Pan, Zheyu Fu, Yunpeng Zhai, Shuchang Tao, Sheng Guan, Shiyu Huang, Lingzhe Zhang, Zhaoyang Liu, Bolin Ding, Felix Henry, Lijie Wen, and Aiwei Liu. Omni-SafetyBench: A benchmark for safety evaluation of audio-visual large language models.arXiv preprint arXiv:2508.07173, 2025. URLhttps://arxiv.org/abs/2508.07173

    arXiv 2025

  41. [41]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational Conference on Machine Learning, pages 8748–8763, 2021. URLhttps://proceedi ngs.mlr.press/v139/radford21a

    2021

  42. [42]

    Robust speech recognition via large-scale weak supervision.Proceedings of the 40th International Conference on Machine Learning (ICML), 202:28492–28518, 2023

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision.Proceedings of the 40th International Conference on Machine Learning (ICML), 202:28492–28518, 2023. URLhttps://proceedings.mlr.press/v202/radford23a.html

    2023

  43. [43]

    Schegloff, and Gail Jefferson

    Harvey Sacks, Emanuel A. Schegloff, and Gail Jefferson. A simplest systematics for the organization of turn-taking for conversation.Language, 50(4):696–735, 1974. doi: 10.2307/412243

    doi:10.2307/412243 1974

  44. [44]

    Multivox: A benchmark for evaluating voice assistants for multimodal interactions.arXiv preprint arXiv:2507.10859, 2025

    Ramaneswaran Selvakumar, Ashish Seth, Nishit Anand, Utkarsh Tyagi, Sonal Kumar, Sreyan Ghosh, and Dinesh Manocha. Multivox: A benchmark for evaluating voice assistants for multimodal interactions.arXiv preprint arXiv:2507.10859, 2025. URLhttps://arxiv.org/abs/2507.10859

    arXiv 2025

  45. [45]

    Turn-taking in conversational systems and human-robot interaction: A review.Computer Speech & Language, 67:101178, 2021

    Gabriel Skantze. Turn-taking in conversational systems and human-robot interaction: A review.Computer Speech & Language, 67:101178, 2021. doi: 10.1016/j.csl.2020.101178. URLhttps://doi.org/10.1016/j.csl.2020.101178

    doi:10.1016/j.csl.2020.101178 2021

  46. [46]

    Audiobench: A universal benchmark for audio large language models

    Bin Wang, Xunlong Zou, Geyu Lin, Shuo Sun, Zhuohan Liu, Wenyu Zhang, Zhengyuan Liu, AiTi Aw, and Nancy Chen. Audiobench: A universal benchmark for audio large language models. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pa...

    doi:10.18653/v1/2025.naacl-long.218 2025

  47. [47]

    Voiceassistant-eval: Benchmarking ai assistants across listening, speaking, and viewing.arXiv preprint arXiv:2509.22651, 2025

    Ke Wang, Houxing Ren, Zimu Lu, Mingjie Zhan, and Hongsheng Li. Voiceassistant-eval: Benchmarking ai assistants across listening, speaking, and viewing.arXiv preprint arXiv:2509.22651, 2025. URL https: //arxiv.org/abs/2509.22651. 14

    arXiv 2025

  48. [48]

    OmniMMI: A comprehensive multi-modal interaction benchmark in streaming video contexts.2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    Yuxuan Wang, Yueqian Wang, Borun Chen, Tong Wu, Dongyan Zhao, and Zilong Zheng. OmniMMI: A comprehensive multi-modal interaction benchmark in streaming video contexts.2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025. doi: 10.1109/CVPR52734.2025.01763. URL https://arxiv.org/abs/2503.22952

    doi:10.1109/cvpr52734.2025.01763 2025

  49. [49]

    Mini-Omni2: Towards open-source GPT-4o with vision, speech and duplex capabilities.arXiv preprint arXiv:2410.11190, 2024

    Zhifei Xie and Changqiao Wu. Mini-Omni2: Towards open-source GPT-4o with vision, speech and duplex capabilities.arXiv preprint arXiv:2410.11190, 2024. URLhttps://arxiv.org/abs/2410.11190

    Pith/arXiv arXiv 2024

  50. [50]

    Qwen2.5-Omni Technical Report.arXiv preprint arXiv:2503.20215, 2025

    Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, Bin Zhang, Xiong Wang, Yunfei Chu, and Junyang Lin. Qwen2.5-Omni Technical Report.arXiv preprint arXiv:2503.20215, 2025. URLhttps://arxiv.org/abs/2503.20215

    Pith/arXiv arXiv 2025

  51. [51]

    Qwen3-Omni Technical Report.arXiv preprint arXiv:2509.17765, 2025

    Jin Xu, Zhifang Guo, Hangrui Hu, Yunfei Chu, Xiong Wang, Jinzheng He, Yuxuan Wang, Xian Shi, Ting He, Xinfa Zhu, Yuanjun Lv, Yongqi Wang, Dake Guo, He Wang, Linhan Ma, Pei Zhang, Xinyu Zhang, Hongkun Hao, Zishan Guo, Baosong Yang, Bin Zhang, Ziyang Ma, Xipin Wei, Shuai Bai, Keqin Chen, Xuejing Liu, Peng Wang, Mingkun Yang, Dayiheng Liu, Xingzhang Ren, Bo ...

    Pith/arXiv arXiv 2025

  52. [52]

    OmniVinci: Enhancing architecture and data for omni-modal understanding llm

    Hanrong Ye, Chao-Han Huck Yang, Arushi Goel, Wei Huang, Ligeng Zhu, Yuanhang Su, Sean Lin, An-Chieh Cheng, Zhen Wan, Jinchuan Tian, et al. OmniVinci: Enhancing architecture and data for omni-modal understanding llm. arXiv preprint arXiv:2510.15870, 2025. URLhttps://arxiv.org/abs/2510.15870

    arXiv 2025

  53. [53]

    SALMONN-omni: A codec-free llm for full-duplex speech understanding and generation

    Wenyi Yu, Siyin Wang, Xiaoyu Yang, Xianzhao Chen, Xiaohai Tian, Jun Zhang, Guangzhi Sun, Lu Lu, Yuxuan Wang, and Chao Zhang. SALMONN-omni: A codec-free llm for full-duplex speech understanding and generation. arXiv preprint arXiv:2411.18138, 2024. URLhttps://arxiv.org/abs/2411.18138

    Pith/arXiv arXiv 2024

  54. [54]

    MMMU: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi.arXiv preprint arXiv:2311.16502, 2023

    Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. MMMU: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi.arXiv preprint arXiv:2311.16502, 2023. URLhttps://arxiv.org/abs/2311.16502

    Pith/arXiv arXiv 2023

  55. [55]

    Mixture of experts in large language models.arXiv preprint arXiv:2507.11181, 2025

    Danyang Zhang, Junhao Song, Ziqian Bi, Yingfang Yuan, Tianyang Wang, Joe Yeong, and Junfeng Hao. Mixture of experts in large language models.arXiv preprint arXiv:2507.11181, 2025. URLhttps://arxiv.org/abs/2507 .11181

    arXiv 2025

  56. [56]

    Unichange: Unifying change detection with multimodal large language model.arXiv preprint arXiv:2511.02607, 2025

    Xu Zhang, Danyang Li, Xiaohang Dong, Tianhao Wu, Hualong Yu, Jianye Wang, Qicheng Li, and Xiang Li. Unichange: Unifying change detection with multimodal large language model.arXiv preprint arXiv:2511.02607, 2025

    arXiv 2025

  57. [57]

    Crystal: Spontaneous emergence of visual latents in mllms.arXiv preprint arXiv:2602.20980, 2026

    Yang Zhang, Danyang Li, Yuxuan Li, Xin Zhang, Tianyu Xie, Mingming Cheng, and Xiang Li. Crystal: Spontaneous emergence of visual latents in mllms.arXiv preprint arXiv:2602.20980, 2026

    arXiv 2026

  58. [58]

    OmniEval: A benchmark for evaluating omni-modal models with visual, auditory, and textual inputs.arXiv preprint arXiv:2506.20960, 2025

    Yiman Zhang, Ziheng Luo, Qiangyu Yan, Wei He, Borui Jiang, Xinghao Chen, and Kai Han. OmniEval: A benchmark for evaluating omni-modal models with visual, auditory, and textual inputs.arXiv preprint arXiv:2506.20960, 2025. URLhttps://arxiv.org/abs/2506.20960

    Pith/arXiv arXiv 2025

  59. [59]

    Xing, Hao Zhang, Joseph E

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging llm-as-a-judge with mt-bench and chatbot arena.arXiv preprint arXiv:2306.05685, 2023. URLhttps://arxiv.org/abs/2306.05685

    Pith/arXiv arXiv 2023

  60. [60]

    An information theory-inspired strategy for automatic network pruning.arXiv preprint arXiv:2108.08532, 2021

    Xiawu Zheng, Yuexiao Ma, Teng Xi, Gang Zhang, Errui Ding, Yuchao Li, Jie Chen, Yonghong Tian, and Rongrong Ji. An information theory-inspired strategy for automatic network pruning.arXiv preprint arXiv:2108.08532, 2021

    Pith/arXiv arXiv 2021

  61. [61]

    Mlvu: Benchmarking multi-task long video understanding.2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13691–13701, 2025

    Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Shitao Xiao, Xi Yang, Yongping Xiong, Bo Zhang, Tiejun Huang, and Zheng Liu. Mlvu: Benchmarking multi-task long video understanding.2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13691–13701, 2025. URLhttps://openaccess.thecvf.com/cont ent/CVPR2025/html/Zhou_MLVU_Benchmarking_Multi...

    2025

  62. [62]

    That’s so true

    Ziwei Zhou, Rui Wang, and Zuxuan Wu. Daily-Omni: Towards audio-visual reasoning with temporal alignment across modalities.arXiv preprint arXiv:2505.17862, 2025. URLhttps://arxiv.org/abs/2505.17862. 15 Appendix Appendix Table of Contents Appendix A Additional Method Details for SocialOmni .................................17 A.1 Inter-Annotator Agreement .....

    arXiv 2025