{"id":"4ca51a99-daec-41e0-9b71-e9f455e68b18","arxiv_id":"2412.12453","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"MIntOOD mixes synthetic out-of-distribution features from multiple known intent classes and learns weighted multimodal fusion with binary, cosine-classifier, and contrastive losses, improving OOD AUROC by 2.5 to 8.4 points and ID accuracy by 0.5 to 1.7 points over baselines on three datasets.","lead":"A new system, MIntOOD, detects user intents in text-video-audio conversations and flags utterances that do not fit any known intent, improving out-of-distribution detection by several AUROC points over prior multimodal methods. It matters because conversational agents that cannot recognize out-of-scope inputs are brittle and can act on unintended user messages.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline OOD gains rest on same-domain OOD test sets; pseudo-OOD convex combinations inhabit the ID convex hull, so transfer to truly unseen OOD domains is unvalidated.","rationale":"The reader's conditional verdict and weakest assumption point to pseudo-OOD fidelity, which is also the most load-bearing issue here. The paper's central claim is that synthetic convex-combination OOD transfers to genuine unseen OOD, but the three evaluations are all same-domain: MIntRec's OOD is self-annotated from the same TV series, and MELD-DA/IEMOCAP-DA use the 'Others' class from the same corpus. This means the method is validated mainly for OOD that is close to the ID manifold, which is precisely the region where the pseudo-OOD generation is strongest. The MIntOOD(R) variant provides partial evidence that cross-dataset real OOD can work, but it is not tested on a held-out cross-domain OOD set for the main datasets, and the IEMOCAP-DA OOD test set is tiny. The duplicated MIntOOD(R) row in Table II is a separate reliability concern but does not by itself overturn the method; it reinforces the need for independent verification. A concrete cross-domain OOD evaluation, preferably using MIntRec2.0's out-of-scope split, would settle whether the 3-10% AUROC gain is real generalization or an artifact of the same-domain benchmark. Until then, the conditional verdict is appropriate.","tokens_in":24827,"tokens_out":6528,"duration_ms":59899,"concrete_test":"Download the released MIntOOD code, train on the MIntRec ID split, and evaluate OOD detection on the MIntRec2.0 out-of-scope test split (or an independently collected OOD set from a different TV series or domain). Compare AUROC, FPR95, and DER against the same baselines used in Table II. If the AUROC margin over the best baseline falls below the claimed 3-10% (or differs materially from the 80.54 AUROC on the same-series OOD), the central transfer claim fails. Independently re-run the IEMOCAP-DA MIntOOD(R) configuration to verify the duplicated Table II row.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that pseudo-OOD generated by Eq. 1-2 (convex combinations of ID features from at least two classes) transfers to real unseen OOD. The evaluation never actually tests transfer to a different domain. For MIntRec, the OOD test set is 450 utterances self-collected from the same TV series (Superstore) as the ID data; for MELD-DA and IEMOCAP-DA, OOD is the 'Others' dialogue-act label from the same corpus. All three OOD sets therefore sit near the ID manifold, exactly the region where convex-combination pseudo-OOD is most representative. At inference the method uses a Mahalanobis score (Eq. 14-15), which rewards separating the convex hull of ID class centroids from the centroids themselves; this separation is not the same as detecting OOD that lies outside the hull, such as a new recording environment or a semantically unrelated domain. MIntOOD(R) is trained on real OOD from another dataset, but it is not evaluated on a held-out cross-domain OOD set for MIntRec or MELD-DA, and the IEMOCAP-DA 'Others' test set contains only 70 samples, making the near-perfect AUROC (97.19) high-variance. A secondary data-integrity issue is that the IEMOCAP-DA MIntOOD(R) row in Table II shows ID metrics (WF1 71.86, WP 72.59, F1 68.29, P 69.75, R 68.43) identical to the MIntRec MIntOOD(R) row, which is implausible and reduces confidence in the table's accuracy.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes MIntOOD, a method for joint multimodal intent classification and out-of-distribution (OOD) detection. The model extracts text, video, and audio features with pretrained backbones (BERT, Swin Transformer, WavLM), fuses them via a learned weighted fusion network, and generates pseudo-OOD samples as Dirichlet-weighted convex combinations of ID features from at least two classes (Eqs. 1–2). Training combines coarse-grained binary ID/OOD classification, a cosine classifier for ID classes, and contrastive learning for instance-level separation. At inference, OOD detection is performed with the Mahalanobis distance to the nearest class centroid (Eqs. 14–15). Experiments on MIntRec, MELD-DA, and IEMOCAP-DA report state-of-the-art ID classification and AUROC gains of roughly 3–10 absolute points over existing multimodal fusion baselines.","tokens_in":25207,"tokens_out":6685,"duration_ms":56490,"significance":"If the reported results hold, MIntOOD is a useful contribution to multimodal intent understanding: it provides a practical way to train OOD detectors without labeled OOD data, introduces a dynamic weighting fusion that benefits both tasks, and releases baselines and a new OOD benchmark for MIntRec. The ablation study (Table III) consistently shows that each component contributes on most datasets and metrics, and the public code and data availability are commendable. However, the significance is tempered by the evaluation protocol's limited out-of-domain scope and the absence of statistical uncertainty quantification, both of which directly affect the abstract's claim of generalization to 'unseen OOD data in real-world scenarios.'","major_comments":[{"comment":"All three OOD test sets are drawn from the same distributional domain as the ID data: the MIntRec OOD utterances come from the same TV series (Superstore) as the ID data, and MELD-DA and IEMOCAP-DA treat the 'Others' dialogue-act label of the same corpus as OOD. Consequently, the reported AUROC improvements for the full MIntOOD (which trains only on pseudo-OOD generated by convex combinations of ID features) validate detection of near-manifold OOD, not the 'unseen OOD data in real-world scenarios' claimed in the abstract. A cross-domain experiment—for example, training the full MIntOOD on MIntRec ID data and testing on MELD-DA 'Others' or on an unrelated OOD collection—would directly substantiate the generalization claim. MIntOOD(R) does use real cross-domain OOD for training, but it does not test the pseudo-OOD generation itself in a cross-domain setting.","section":"Sections IV-A and V-A (Tables I and II)"},{"comment":"The MIntOOD(R) row for IEMOCAP-DA lists WF1=71.86, WP=72.59, F1=68.29, P=69.75, R=68.43, which are identical to the values in the MIntRec MIntOOD(R) row (only ACC differs). This is implausible for a different dataset and strongly suggests a copy-paste error. The authors should verify the reported averages and correct the table, since this error undermines confidence in the accuracy of the numerical results.","section":"Table II, IEMOCAP-DA block"},{"comment":"All experimental results are reported as averages over five random seeds, but no standard deviations, confidence intervals, or significance tests are provided. Many of the claimed improvements are only 1–3 absolute points in AUROC or ACC (e.g., MIntRec AUROC 80.54 vs. 75.85 for the best baseline), and without variance estimates the reader cannot determine whether the differences are statistically reliable. The authors should report per-seed variability and, where possible, paired significance tests across the five seeds.","section":"Section IV.D"},{"comment":"The IEMOCAP-DA OOD test set contains only 70 samples. The near-perfect AUROC of 97.19 for MIntOOD on this set is therefore high-variance, and the large improvements in DER and FPR95 (44.85 and 46.57 points, respectively) should be interpreted with caution. A 95% confidence interval or a discussion of the small OOD sample size would make the strength of this result more assessable.","section":"Section IV.A and Table I"}],"minor_comments":[{"comment":"The notation C({yj}kj=1) is undefined; please define it as the set of unique class labels among the selected samples.","section":"Section III-B, Eq. (2)"},{"comment":"The figure appears to plot AUROC values, but the y-axis label is missing; please add a clear axis label and, if applicable, error bars.","section":"Figure 3"},{"comment":"The sentence 'with improvements of about 1% to 3%, 2% to 4%, and 2% to 40%, respectively' is ambiguous because the per-dataset mapping is not stated; specify which range corresponds to which dataset.","section":"Section V-A, last paragraph"},{"comment":"The sentence fragment 'the performance is comparable or even improves. .' contains a double period and is incomplete; rephrase to 'the performance is comparable or even slightly improves on IEMOCAP-DA.'","section":"Section V-B, third paragraph"},{"comment":"Reference [43] (MIntRec2.0) is cited but not discussed in relation to the proposed OOD benchmark; clarify the relationship between MIntOOD and this closely related benchmark, including whether MIntRec2.0 contains an OOD split that could have been used for cross-domain evaluation.","section":"Related Work, Section II-A"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid incremental contribution, but the central OOD generalization claim is not yet supported by the evaluation protocol: the full method is only tested on same-domain OOD sets, while the cross-domain evidence comes only from the MIntOOD(R) variant that uses real external OOD for training. The duplicated row in Table II is a data-integrity concern; the editor may wish to request the per-seed raw results to verify the reported tables. An independent or cross-domain OOD benchmark would significantly strengthen the paper's claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nYou can read this paper as a solid engineering contribution to multimodal intent recognition plus OOD detection. The genuinely new pieces: it is the first multimodal OOD method for this task; the pseudo-OOD generator uses Dirichlet-sampled convex combinations of k ID features from at least two classes, which is a real variation on prior binary mixup; and the authors release a new MIntRec OOD benchmark along with code. The ablations in Table III mostly support the design: removing the cosine classifier hurts ID accuracy substantially, and removing the binary or contrastive losses degrades OOD metrics. That is evidence the method works as described.\n\nThe soft spots are real but not fatal, and one is fixable. The OOD test sets never leave the ID domain. MIntRec's OOD is 450 self-collected utterances from the same TV series; MELD-DA and IEMOCAP-DA use the 'Others' dialogue-act label from the same corpus. That means the pseudo-OOD convex combinations, which live in the convex hull of ID features, are actually a good proxy for the tested OOD. The 3-10% AUROC gains are credible for near-boundary OOD, but the abstract's generalization claim is broader than the evaluation. There is no cross-domain OOD test, so we don't know how the method behaves on a new recording environment or semantically unrelated domain. That should be stated and tested.\n\nTwo concrete issues. First, Table II has a duplicated row: the IEMOCAP-DA MIntOOD(R) row shows WF1 71.86, WP 72.59, F1 68.29, P 69.75, R 68.43, exactly matching the MIntRec MIntOOD(R) row, with only ACC differing. That is almost certainly a copy-paste error and needs correction. Second, the five-seed averages have no error bars or significance tests; for claims like 'significantly improves', that matters, especially on the small 70-sample IEMOCAP-DA OOD set. Also, MIntRec2.0, which the authors cite, is a closely related same-group benchmark and is never compared against; the paper should at least explain why.\n\nWho benefits: anyone working on multimodal intent understanding, conversational OOD detection, or multimodal fusion. It deserves a serious referee pass, but with major revision: fix the table, add variance reporting, run or clearly scope a cross-domain OOD evaluation, and address MIntRec2.0. If those are done, this becomes a useful contribution with a reusable benchmark.\n\nRecommendation: send to peer review, conditional on those revisions.","headline":"Credible method, near-boundary-only OOD evaluation; needs a table fix and a cross-domain test before the headline claim is convincing.","tokens_in":25725,"tokens_out":3012,"would_cite":true,"duration_ms":26943,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MIntOOD combines weighted multimodal fusion, Dirichlet-synthesized pseudo-OOD examples, a cosine classifier, and contrastive learning so one model can classify known conversational intents and flag out-of-distribution utterances.","keywords":["multimodal intent understanding","out-of-distribution detection","pseudo-OOD generation","weighted feature fusion","cosine classifier","contrastive learning","Dirichlet mixing","dialogue act classification"],"falsifier":"Build an OOD test set from a genuinely different source, such as utterances from a different conversational domain or language annotated by the same protocol, and compare MIntOOD's AUROC against baselines; if the margin disappears when the test OOD lies far from the convex combinations used in training, the pseudo-OOD proxy is the bottleneck. More directly, measure whether the model assigns high in-distribution scores to points on the inside of the ID convex hull; if it does, the binary head has learned to label hull-interior points as ID rather than semantic novelty.","tokens_in":24647,"feed_emoji":"🎙️","tokens_out":6887,"duration_ms":58678,"temperature":0.7,"pith_summary":"Multimodal intent understanding asks a system to read what a speaker means from text, video, and audio, and to know when an utterance belongs to none of the trained intents. This paper proposes MIntOOD as one method for both jobs: it fuses the three modalities with per-utterance learned weights, and trains on original in-distribution data plus pseudo-OOD features synthesized by mixing in-distribution features with Dirichlet-sampled coefficients. The paper's reported results are state-of-the-art in-distribution accuracy on MIntRec, MELD-DA, and IEMOCAP-DA, and out-of-distribution AUROC gains of about 3–10 points over the best baselines. If correct, the contribution is evidence that open-world robustness and closed-world accuracy are not in conflict for multimodal intent systems.","feed_headline":"Synthetic mixups lift OOD intent detection by 3–10 AUROC points","feed_subtitle":"Fusing text, video, and audio with learned weights, it also sets new benchmarks on in-distribution intent classification.","key_machinery":"The load-bearing object is the pseudo-OOD feature, built as $z_{\\mathrm{OOD}} = \\sum_j \\lambda_j z_{\\mathrm{ID},j}$ with $\\lambda$ sampled from a Dirichlet distribution with unit sum, drawn from $k=3$ ID features spanning at least two classes. It underpins all three objectives: a binary classifier gives coarse ID/OOD separation, a cosine classifier isolates directional information among intent classes, and a contrastive loss refines instance-level relations with dropout-changed duplicates as positive pairs. A weighted fusion network computes softmax-normalized modality weights per utterance and sums the encoded text, video, and audio representations; the Mahalanobis distance over per-class covariance then performs the final OOD scoring.","core_discovery":"The central claim is that representations trained at several granularities on both in-distribution and synthesized out-of-distribution examples can simultaneously separate known intents and flag unseen ones. Pseudo-OOD features are made per modality as convex combinations of $k=3$ in-distribution features drawn from at least two classes, with weights sampled from a Dirichlet distribution; these are treated as negative examples in a binary ID/OOD head, while a cosine classifier separates the known classes and a contrastive loss organizes instances so that same-class samples cluster and pseudo-OOD samples stand apart. At inference, the Mahalanobis distance to per-class centroids in the fused feature space scores how out-of-distribution an utterance is. The paper reports that the full recipe beats every baseline on nearly all ID and OOD metrics across the three datasets, with the largest absolute gains on OOD detection.","pith_inferences":["Beyond the paper: a direct extension is to vary the number of mixed samples $k$ and the Dirichlet concentration $\\alpha$ and measure AUROC on held-out OOD; the method's logic implies a sweet spot exists, with too-small mixtures indistinguishable from ID and too-large mixtures drifting beyond plausible OOD.","Beyond the paper: the paper observes that gains are largest for the fine-grained 20-class MIntRec data, suggesting that the richness of the intent taxonomy may be what makes pseudo-OOD training pay off; this could be checked by subsampling MIntRec to fewer classes.","Beyond the paper: because the pseudo-OOD operation happens in feature space before fusion, the same generation procedure could be dropped into other feature extractors or modality sets, though the paper does not demonstrate this transfer."],"forward_implications":["If the reported gains hold, a single multimodal model can serve both closed-world intent classification and open-world robustness, so dialogue systems need not degrade in accuracy to gain OOD awareness.","Because pseudo-OOD data is generated in feature space from ID examples, the approach sidesteps the cost of collecting real out-of-scope utterances, which the paper calls prohibitively expensive.","The ablation results indicate that the cosine classifier and Mahalanobis scoring are the main drivers of OOD performance, so future multimodal OOD detectors should not rely only on raw logits.","The pseudo-OOD generation strategy is effective across two dialogue-act datasets and a fine-grained intent dataset, indicating the recipe transfers beyond 20-class intent taxonomies."],"supporting_citations":[{"why":"Supplies the MIntRec benchmark, its 20 intent classes, and the feature-extraction and fusion baselines that define the task.","marker":"[1]"},{"why":"The current state-of-the-art intent method TCL-MAP, which the paper must beat and which is shown to generalize poorly to OOD.","marker":"[11]"},{"why":"Earlier self-supervised pseudo-OOD generation for text-only out-of-scope detection, the line the paper extends with Dirichlet multi-sample mixing.","marker":"[18]"},{"why":"Supplies the Mahalanobis-distance OOD scoring function used at inference and for all compared methods.","marker":"[54]"},{"why":"The cosine classifier with normalized weights, whose removal causes the largest ID accuracy drops in ablations.","marker":"[66]"},{"why":"The dropout-based positive augmentation used as the contrastive pair in the instance-level loss.","marker":"[67]"},{"why":"Provides the pre-trained BERT text encoder whose [CLS] representation anchors the fused multimodal space.","marker":"[62]"},{"why":"Provides the WavLM audio encoder used for the audio modality in the fusion.","marker":"[59]"},{"why":"Provides the Swin Transformer video backbone used to produce visual sequence embeddings.","marker":"[57]"}],"fun_headline_variants":["Pseudo-OOD mixups boost multimodal intent OOD detection by 3-10 AUROC","Fusing text, video, audio with synthetic OOD data improves intent detection","Learned modality weights and pseudo-OOD samples lift AUROC by 3-10 points","MIntOOD generates pseudo-OOD from ID data to lift OOD detection and intent accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"All of the OOD gains depend on the assumption that a convex mixture of a few in-distribution feature vectors, weighted by a Dirichlet draw, is close enough to a real out-of-scope utterance that training to reject it transfers to unseen inputs.","fun_headline_variants_meta":{"raw":{"variants":["Pseudo-OOD mixups boost multimodal intent OOD detection by 3-10 AUROC","Fusing text, video, audio with synthetic OOD data improves intent detection","Learned modality weights and pseudo-OOD samples lift AUROC by 3-10 points","MIntOOD generates pseudo-OOD from ID data to lift OOD detection and intent accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000736,"raw_usage":{"total_tokens":3313,"prompt_tokens":993,"completion_tokens":2320,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":609,"completion_tokens_details":{"reasoning_tokens":2226}},"tokens_in":609,"tokens_out":2320,"duration_ms":14849,"temperature":1.0,"reasoning_tokens":2226,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:04:32.301055+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build an OOD test set from a genuinely different source, such as utterances from a different conversational domain or language annotated by the same protocol, and compare MIntOOD's AUROC against baselines; if the margin disappears when the test OOD lies far from the convex combinations used in training, the pseudo-OOD proxy is the bottleneck. More directly, measure whether the model assigns high in-distribution scores to points on the inside of the ID convex hull; if it does, the binary head has learned to label hull-interior points as ID rather than semantic novelty.","supporting_citations":[{"cited_title":"Mintrec: A new dataset for multimodal intent recognition,","cited_arxiv_id":null,"evidence_quote":"Supplies the MIntRec benchmark, its 20 intent classes, and the feature-extraction and fusion baselines that define the task."},{"cited_title":"Token-level contrastive learning with modality-aware prompting for multimodal intent recognition,","cited_arxiv_id":null,"evidence_quote":"The current state-of-the-art intent method TCL-MAP, which the paper must beat and which is shown to generalize poorly to OOD."},{"cited_title":"Out-of-scope intent detection with self-supervision and discriminative training,","cited_arxiv_id":null,"evidence_quote":"Earlier self-supervised pseudo-OOD generation for text-only out-of-scope detection, the line the paper extends with Dirichlet multi-sample mixing."},{"cited_title":"A simple unified framework for detecting out-of-distribution samples and adversarial attacks,","cited_arxiv_id":null,"evidence_quote":"Supplies the Mahalanobis-distance OOD scoring function used at inference and for all compared methods."},{"cited_title":"Low-shot learning with imprinted weights,","cited_arxiv_id":null,"evidence_quote":"The cosine classifier with normalized weights, whose removal causes the largest ID accuracy drops in ablations."},{"cited_title":"Simcse: Simple contrastive learning of sentence embeddings,","cited_arxiv_id":null,"evidence_quote":"The dropout-based positive augmentation used as the contrastive pair in the instance-level loss."},{"cited_title":"Bert: Pre-training of deep bidirectional transformers for language understanding,","cited_arxiv_id":null,"evidence_quote":"Provides the pre-trained BERT text encoder whose [CLS] representation anchors the fused multimodal space."},{"cited_title":"Wavlm: Large-scale self-supervised pre- training for full stack speech processing,","cited_arxiv_id":null,"evidence_quote":"Provides the WavLM audio encoder used for the audio modality in the fusion."},{"cited_title":"Swin transformer: Hierarchical vision transformer using shifted windows,","cited_arxiv_id":null,"evidence_quote":"Provides the Swin Transformer video backbone used to produce visual sequence embeddings."}],"review_version":1}