{"id":"570627b9-4506-4e0a-bef0-ccccd64725c9","arxiv_id":"2412.01986","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"HybridMQA combines a 3D graph network with rendered 2D projections via cross-attention to assess colored mesh quality, outperforming prior full-reference methods on four public datasets.","lead":"The authors built a system, HybridMQA, that judges the visual quality of colored 3D meshes by looking at both the 3D shape and its 2D rendered appearance together. It combines 3D graph analysis with rendered 2D projections and uses cross-attention to model how geometry and texture distortions affect each other, and it reports higher agreement with human ratings than earlier methods on four public benchmark datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SOTA claim depends on an unfair baseline comparison: 3D-PSSIM, the closest competitor, was not re-run under HybridMQA's rendering/evaluation protocol despite Section 4.2 saying all projection-based methods were.","rationale":"I read the paper as a solid engineering contribution: the component ablations (Tables 3-5), cross-dataset generalization (Table 2), gMAD competition, and GradCAM analyses provide genuine evidence for the architecture's design. The central mathematical construction, GCN features rendered into projections aligned with colored renderings and combined by cross-attention, is coherent and internally consistent. The reader's weakest assumption concerns whether the fixed six-view Phong rendering captures the appearance humans use. That is a real risk, but the more immediately load-bearing threat to the paper's strongest claim (outperforming all methods) is the asymmetry in baseline evaluation: Section 4.2 promises a controlled comparison, yet Table 1's note says the closest competitor 3D-PSSIM was taken from its published numbers. Because 3D-PSSIM is the only baseline that comes close on the two largest datasets, the SOTA margin may be due to differences in rendering, patch count, splits, or fitting rather than to geometry-texture interaction modeling. This is not an accusation of bad faith; unreleased baseline code is common, and the authors disclosed the provenance. But it means the central empirical claim cannot be taken as verified under the paper's own fairness standard. My recommendation is therefore to keep the reader's CONDITIONAL verdict: the method is promising and well-ablated, but release of the implementation and a controlled re-run of 3D-PSSIM (and ideally GeodesicPSIM/Fu) are required before accepting the SOTA comparison as apples-to-apples.","tokens_in":15952,"tokens_out":5896,"duration_ms":53744,"concrete_test":"Re-run 3D-PSSIM under exactly the protocol used for HybridMQA: same source-disjoint 5-fold splits, same six perpendicular Phong renderings with matching lighting, same patch selection (>=10% non-background, Eq. 5), same 128x128 feature and 512x512 color resolutions, same logistic PLCC fitting, and same median-over-5-folds reporting. Also obtain a TSMD score for 3D-PSSIM on the same 39-mesh split. If 3D-PSSIM's SRCC/PLCC under this protocol come within about 0.01-0.02 of HybridMQA's values, or exceed them, the claimed SOTA superiority is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"HybridMQA's central claim is that it outperforms all model- and projection-based methods across all four datasets (Section 4.3). The main empirical support is Table 1. But the paper is internally inconsistent about protocol control. Section 4.2 says 'all projection-based methods are evaluated under the same rendering settings as HybridMQA, with no prior assumptions about object orientation.' The Table 1 note, however, states that GeodesicPSIM, Fu et al., and 3D-PSSIM scores 'are reported directly from their publications as their implementations are not publicly available.' 3D-PSSIM is a projection-based method and is the second-best baseline on Nehmé et al. and SJTU-TMQA, the two datasets where the claimed margins are largest (6.5% and 7.7% PLCC over 3D-PSSIM). Its published scores were obtained with its own rendering and evaluation choices (viewpoints, lighting, patch sampling, fitting), not with HybridMQA's six perpendicular Phong views, patch filtering (Eq. 5), 128/512 resolutions, source-disjoint 5-fold splits, and median-over-folds reporting. Therefore the reported advantage over 3D-PSSIM could be partly an artifact of protocol mismatch. In addition, 3D-PSSIM has no TSMD row, so the 'across all datasets' claim is not tested against the strongest projection baseline on TSMD. This makes the headline SOTA claim unverified, not necessarily false.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"HybridMQA proposes a full-reference colored mesh quality assessment method that combines a model-based 3D graph branch with a projection-based appearance branch. The 3D branch builds a feature graph from UV-aligned normal, vertex, and texture maps, processes it with a GCN, and renders the resulting vertex features into 2D projections aligned with six perpendicular colored renderings. Cross-attention modules then fuse the two projection streams, and the absolute difference of reference/distorted quality representations is regressed to a MOS. Experiments on Nehmé et al., SJTU-TMQA, TSMD, and CMDM report state-of-the-art SRCC/PLCC, with ablations on the cross-attention mechanism, model-branch components, augmentations, and viewpoint sampling. The paper also includes a cross-dataset generalization test and qualitative gMAD and GradCAM analyses.","tokens_in":16245,"tokens_out":2920,"duration_ms":28752,"significance":"If the results hold, HybridMQA is a useful step for colored mesh quality assessment: it explicitly couples a true 3D surface representation with rendered appearance, and the detailed ablations in Tables 3–5 and the supplementary material support the contribution of the proposed cross-attention design. The cross-dataset generalization evaluation in Table 2 is a valuable addition, as is the gMAD comparison. The public implementation promise, cross-validation without source-content overlap, and thorough reporting of training details are strengths. The central claim, however, is currently weakened by an internal inconsistency in how baseline methods were evaluated, which matters because the headline margins over 3D-PSSIM are computed against published scores rather than scores obtained under the paper's own protocol.","major_comments":[{"comment":"There is a direct contradiction between the protocol stated in Section 4.2 and the footnote under Table 1. Section 4.2 says 'all projection-based methods are evaluated under the same rendering settings as HybridMQA,' while Table 1 states that GeodesicPSIM, Fu et al., and 3D-PSSIM scores 'are reported directly from their publications as their implementations are not publicly available.' 3D-PSSIM is the second-best method on Nehmé et al. and SJTU-TMQA, and the claimed 6.5% and 7.7% PLCC gains are measured against it. Those published scores were obtained with 3D-PSSIM's own rendering, patch sampling, and fitting protocol, not with HybridMQA's six perpendicular views, Phong shading, 10% background patch filtering, 128/512 resolutions, source-disjoint 5-fold splits, and median-over-folds reporting. The advantage over 3D-PSSIM is therefore unverified under a common protocol, and the 'outperforms all ... across all datasets' claim is not supported by the evidence as presented. Please either re-run 3D-PSSIM (and the other cited methods with public implementations) under the same protocol, or explicitly rephrase the SOTA claim to acknowledge the protocol mismatch and report the comparison using published results as a separate row.","section":"§4.2 / Table 1"},{"comment":"The universality of the 'across all datasets' claim is limited by missing baseline entries that are not accounted for in the discussion. 3D-PSSIM has no TSMD row, GeodesicPSIM appears only on TSMD, and Fu et al. appears only on Nehmé et al. Consequently, on TSMD the strongest projection baseline is not compared, and on SJTU-TMQA the strongest model-based baselines are absent. This makes it possible that the 'outperforms all' conclusion is an artifact of which baselines happen to have publicly available numbers. Please report all baselines that can be obtained (or reimplemented) on all four datasets, and add a sentence explaining how missing entries affect the comparison.","section":"Table 1"},{"comment":"No variance or significance information is provided for the 5-fold results, despite the use of a median-over-folds protocol. On CMDM the margins over LPIPS are very small (SRCC 0.897 vs 0.865, PLCC 0.927 vs 0.925), and CMDM contains only five source meshes, so the difference may not be statistically meaningful. Please report per-fold scores or standard deviations, and if possible a paired significance test, for at least the main comparison in Table 1.","section":"§4.3 / Table 1"}],"minor_comments":[{"comment":"The statement that rendering uses 'directional or ambient light to match the conditions of subjective tests' is ambiguous; please specify which dataset used which lighting configuration, or state that the same configuration was used throughout.","section":"§3.3"},{"comment":"The 10% non-background patch threshold is introduced without justification or sensitivity analysis; a short ablation or a citation to a prior study would help.","section":"Eq. (5)"},{"comment":"The viewpoint-dropout ablation shows a large spread (SRCC 0.837–0.892 depending on number of viewpoints and batch size), so the choice of two viewpoints with batch size 8 is a meaningful hyperparameter; please describe this configuration clearly in the main text alongside the default training settings.","section":"Supplementary Table 7"},{"comment":"The distortion-type breakdown would be more informative with the number of samples per distortion type, since some distortion types may contain very few meshes and the SRCC/PLCC values could be noisy.","section":"Figure 5"},{"comment":"The reference list contains several formatting glitches, such as 'San Diega' in [15], 'V ol.2' in [46], and stray page numbers after some entries in the supplementary references; these should be cleaned up.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The core method is reasonable, and the ablation studies are fairly thorough, so I see no reason to doubt the internal consistency of the proposed architecture. The main risk is that the headline SOTA claim is built on a comparison where the strongest competitor, 3D-PSSIM, was not run under the paper's own protocol. Given that the paper explicitly promises code release and that 3D-PSSIM's implementation is described in its paper, I would encourage the editor to ask the authors to make the protocol re-run a condition of revision. Also, the 'first hybrid' and 'first geometry-texture interaction' framing may need to be softened given that 3D-PSSIM already combines projected geometry and texture information, even if not through cross-attention."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a real contribution to colored mesh quality assessment, and the architecture is genuinely new. But the headline 'outperforms all baselines' claim is not fully supported as written because Table 1 mixes your own evaluation protocol with published scores from 3D-PSSIM, the closest competitor on two of the four datasets.\n\nThe core idea is solid: use a GCN on the mesh graph to get 3D surface features, render those features into 2D projections pixel-aligned with standard color renderings, and then use cross-attention to let geometry and texture representations interact. That combination is new relative to the cited literature, and the design is well motivated by the geometry-texture interaction examples in Figure 1. The ablations are thorough: each model branch component matters, cross-attention beats addition, concatenation, multiplication, and self-attention alternatives, and the viewpoint dropout experiment is a nice practical touch. The cross-dataset generalization test (train on Nehmé and TSMD, test on SJTU-TMQA) is exactly the kind of evaluation this field needs, and the results are strong.\n\nThe main soft spot is the baseline comparison. Section 4.2 says all projection-based methods are evaluated under the same rendering settings as HybridMQA, but Table 1's note says GeodesicPSIM, Fu et al., and 3D-PSSIM scores are taken directly from their publications because implementations are not public. 3D-PSSIM is projection-based and is the second-best method on Nehmé and SJTU-TMQA, which are exactly the datasets where the claimed PLCC gains are largest (6.5% and 7.7%). Published scores come from those methods' own viewpoints, lighting, patch sampling, and fitting, so the reported advantage may be partly a protocol artifact. The paper does not give error bars or significance tests across the five folds, so we cannot tell how much of the margin is noise. This does not sink the method, but it does mean the 'state-of-the-art' claim is unverified rather than demonstrated. A re-run of 3D-PSSIM under the new protocol, or a clear justification of why published scores are comparable, would fix the issue.\n\nMinor soft spot: the 'first hybrid' and 'first geometry-texture interaction' phrasing would benefit from a broader related-work search, but this is easy to address in revision.\n\nWho should read this: anyone building or benchmarking mesh quality metrics, especially for textured and vertex-color meshes. It deserves a serious external review; the architecture and evaluation methodology are worth the referee time. My recommendation: send it to review, but make protocol comparability and variance reporting conditions for acceptance.","headline":"Solid hybrid MQA paper with a credible architecture and strong ablations, but the headline SOTA claim is undercut by mixing published baseline scores with your own evaluation protocol.","tokens_in":16807,"tokens_out":2551,"would_cite":true,"duration_ms":21764,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By fusing 3D graph features with aligned color projections, HybridMQA reports the most human-aligned quality scores for colored meshes tested.","keywords":["mesh quality assessment","full-reference","geometry-texture interaction","graph convolutional network","cross-attention","projection-based quality metric","colored 3D mesh","perceptual quality"],"falsifier":"Take a distorted mesh whose most visible geometry artifact appears only from a viewpoint outside the six fixed perpendicular cameras, while the six standard views look near-identical to the reference; if HybridMQA then scores the distorted mesh almost as high as the reference, the fixed-viewpoint representation has missed a human-visible distortion.","tokens_in":15727,"feed_emoji":"🧊","tokens_out":5044,"duration_ms":271409,"temperature":0.7,"pith_summary":"This paper tries to establish that colored mesh quality is best judged by combining both places where distortion lives: the 3D surface topology and the rendered 2D appearance. It introduces HybridMQA, a full-reference metric that extracts 3D surface features with a graph convolutional network, renders those features into the same six viewpoints as the mesh's color images, and uses cross-attention to let geometry and texture representations modify each other before scoring. The authors report that this hybrid outperforms eleven model-based and projection-based methods across four colored-mesh datasets, with the largest gains on distortions that change geometry or both geometry and texture. If HybridMQA is right, projection-only metrics miss geometry distortions visible only across viewpoints, while model-only metrics miss the final rendered appearance, and fusing the two closes both gaps.","feed_headline":"Hybrid mesh metric beats 11 baselines by fusing 3D and 2D views","feed_subtitle":"Rendering 3D features into color-image viewpoints lets cross-attention link shape and texture distortions.","key_machinery":"The load-bearing mechanism is a differentiable feature graph rendering that projects GCN-learned 3D surface features into 2D using the same six camera viewpoints and a Phong shader as the color renderings, giving pixel-to-pixel correspondence between geometry and appearance. A cross-attention module made of two transformer blocks, applied at five scales with alternating query and key-value roles, lets the two domains influence each other; the cross-attended scales are concatenated with the raw 3D feature embeddings to form the mesh quality representation. Patch selection discards patches with less than 10% non-background pixels, focusing the comparison on visible surface regions.","core_discovery":"The central claim is that HybridMQA, a full-reference framework, achieves the best reported SRCC and PLCC correlations with mean opinion scores on the Nehmé et al., SJTU-TMQA, TSMD, and CMDM datasets by integrating model-based and projection-based information. The model branch turns texture, normal, and vertex maps into per-vertex features, refines them via graph convolution over mesh connectivity, and renders those 3D surface features into six perpendicular projections that are pixel-aligned with color renderings. Cross-attention blocks at five scales alternate the query and key-value roles of the two modalities, so texture can steer where geometry distortions matter and geometry can expose texture artifacts that flat projections hide. The final quality representation concatenates the cross-attended scales with direct 3D feature embeddings, and the absolute difference between reference and distorted representations is regressed to a quality score. Ablations show that removing the 3D branch drops SRCC from 0.892 to 0.820, and replacing cross-attention with addition, concatenation, multiplication, or self-attention drops SRCC to at most 0.852 on the Nehmé et al. dataset.","pith_inferences":["A testable extension is to replace the six fixed perpendicular viewpoints with saliency-weighted or free-viewing viewpoint sampling; if performance rises further, the fixed-view assumption is the bottleneck, and if it stays flat, the six views already capture the perceptually relevant content.","The same feature-rendering alignment could transfer to no-reference mesh quality and to point-cloud quality by training a regressor directly on the quality representation, directions the paper lists as future work.","A sharper separation of interaction effects would come from a synthetic experiment that varies texture complexity while holding a fixed geometry distortion; if cross-attention truly encodes geometry-texture interactions, the predicted quality gap should change with texture complexity even though the geometry distortion is identical."],"forward_implications":["HybridMQA reports higher SRCC and PLCC than all eleven compared model-based and projection-based baselines on the four test datasets, including PLCC gains of 6.5% over 3D-PSSIM on Nehmé et al. and 7.7% on SJTU-TMQA.","On texture-only distortions HybridMQA is comparable to Graphics-LPIPS, while on geometry-only and mixed geometry-texture distortions it shows the largest improvements, supporting the claim that 3D understanding drives the gains.","The cross-attention mechanism is essential: replacing it with addition, weighted addition, concatenation, elementwise multiplication, or self-attention drops SRCC from 0.892 to at most 0.852 on Nehmé et al.","Generalization experiments trained on Nehmé et al. or TSMD and tested on SJTU-TMQA show HybridMQA outperforming LPIPS and Graphics-LPIPS and matching a 3D-PSSIM model trained directly on the test set.","Because the feature rendering is differentiable, GradCAM on the 3D feature projections localizes geometry artifacts on the mesh surface, which could guide perceptually optimized mesh compression or restoration."],"supporting_citations":[{"why":"Supplies the largest textured-mesh MQA dataset, the Graphics-LPIPS projection-based baseline, and the main evaluation and ablation test bed.","marker":"[29]"},{"why":"Provides the SJTU-TMQA dataset with mixed geometric and color distortions, used for evaluation, distortion-type breakdown, and generalization testing.","marker":"[7]"},{"why":"Provides the TSMD static color mesh dataset used for evaluation and as a training source in generalization experiments.","marker":"[49]"},{"why":"Provides the CMDM vertex-color mesh dataset and motivates the vertex-color rendering path where the base encoder is skipped.","marker":"[30]"},{"why":"Defines 3D-PSSIM, the strongest projection-based baseline that HybridMQA reports beating by 6.5% and 7.7% PLCC on two datasets.","marker":"[20]"},{"why":"Supplies the differentiable renderer that the feature graph rendering and colored projection pipeline customizes, enabling gradients to flow to the GCN and base encoder.","marker":"[32]"},{"why":"Supplies the graph convolutional update rule used to refine vertex features through message passing over mesh connectivity.","marker":"[28]"},{"why":"Supplies the transformer block architecture used in the cross-attention modules.","marker":"[42]"},{"why":"Provides the ImageNet-pretrained ResNet50 used as the image encoder for color projections.","marker":"[14]"},{"why":"Supplies the differentiable rank-loss approximation used with MAE to train the quality regressor.","marker":"[38]"}],"fun_headline_variants":["Hybrid MQA links 3D graphs and 2D renders via cross-attention","Cross-attention between 3D features and color views tops mesh QA","HybridMQA: fusing 3D graph features with 2D renders for best QA","New mesh QA model uses cross-attention to sync texture and geometry","Geometry-texture cross-attention lifts mesh quality scores"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that six fixed perpendicular viewpoints rendered with a simple Phong shader, with lighting chosen to match the subjective test setup and near-empty patches discarded, expose the same geometry-texture interactions that human viewers rely on when judging mesh quality.","fun_headline_variants_meta":{"raw":{"variants":["Hybrid MQA links 3D graphs and 2D renders via cross-attention","Cross-attention between 3D features and color views tops mesh QA","HybridMQA: fusing 3D graph features with 2D renders for best QA","New mesh QA model uses cross-attention to sync texture and geometry","Geometry-texture cross-attention lifts mesh quality scores"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000807,"raw_usage":{"total_tokens":3557,"prompt_tokens":974,"completion_tokens":2583,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":590,"completion_tokens_details":{"reasoning_tokens":2482}},"tokens_in":590,"tokens_out":2583,"duration_ms":18890,"temperature":1.0,"reasoning_tokens":2482,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T23:57:41.009283+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a distorted mesh whose most visible geometry artifact appears only from a viewpoint outside the six fixed perpendicular cameras, while the six standard views look near-identical to the reference; if HybridMQA then scores the distorted mesh almost as high as the reference, the fixed-viewpoint representation has missed a human-visible distortion.","supporting_citations":[{"cited_title":"Textured mesh quality assessment: Large-scale dataset and deep learning-based quality metric","cited_arxiv_id":null,"evidence_quote":"Supplies the largest textured-mesh MQA dataset, the Graphics-LPIPS projection-based baseline, and the main evaluation and ablation test bed."},{"cited_title":"Sjtu-tmqa: A quality assessment database for static mesh with texture map","cited_arxiv_id":null,"evidence_quote":"Provides the SJTU-TMQA dataset with mixed geometric and color distortions, used for evaluation, distortion-type breakdown, and generalization testing."},{"cited_title":"Tsmd: A database for static color mesh quality assessment study","cited_arxiv_id":null,"evidence_quote":"Provides the TSMD static color mesh dataset used for evaluation and as a training source in generalization experiments."},{"cited_title":"Visual quality of 3d meshes with diffuse colors in virtual reality: Subjective and objective evaluation","cited_arxiv_id":null,"evidence_quote":"Provides the CMDM vertex-color mesh dataset and motivates the vertex-color rendering path where the base encoder is skipped."},{"cited_title":"3d-pssim: Projective structural simi- larity for 3d mesh quality assessment robust to topological irregularities","cited_arxiv_id":null,"evidence_quote":"Defines 3D-PSSIM, the strongest projection-based baseline that HybridMQA reports beating by 6.5% and 7.7% PLCC on two datasets."},{"cited_title":"Hamilton, Jan Eric Lenssen, Gaurav Rattan, and Martin Grohe","cited_arxiv_id":null,"evidence_quote":"Supplies the graph convolutional update rule used to refine vertex features through message passing over mesh connectivity."},{"cited_title":"A deep learning based no-reference quality assessment model for ugc videos","cited_arxiv_id":null,"evidence_quote":"Supplies the differentiable rank-loss approximation used with MAE to train the quality regressor."}],"review_version":1}