{"id":"f252b053-9091-4e86-a223-3ed9332ee841","arxiv_id":"2506.22662","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A MeshCNN-style convolutional network adapted to 2D CFD airfoil meshes classifies airfoil thickness ranges with roughly 67% stable and 83% peak accuracy, but the small self-made dataset and missing artifacts limit the result.","lead":"The paper adapts MeshCNN, a convolutional neural network built for 3D shapes, to read 2D unstructured CFD meshes around airfoils and classify the airfoil's thickness range. It reports stable accuracy near 67% and peaks near 83% on a small self-generated dataset, suggesting mesh geometry alone carries useful aerodynamic shape information.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 67% stable accuracy may be inflated by file-level train/test splits that place fine and coarse meshes of the same airfoil in both sets; with ~50 meshes, the generalization claim is not yet established.","rationale":"The paper pursues a plausible and modest goal: test whether an edge-based CNN (MeshCNN) can classify 2D airfoil meshes by thickness interval from geometry alone. A 3-class task with 67% stable accuracy is above chance and, if real, would support the feasibility claim. I credit the authors for using unstructured meshes, for comparing dense versus sparse configurations, and for explicitly acknowledging in Section 5 that \"insufficient training data can limit the learning capability, resulting in reduced prediction accuracy and generalization.\" That acknowledged limitation is exactly where the central claim is least secure. The load-bearing evidence is Table 2, but the evaluation protocol is not described in enough detail to rule out the most mundane explanation: because each airfoil geometry is meshed at two densities (Table 1) and the split is only specified as a file ratio, dense and coarse versions of the same airfoil may straddle the train/test boundary. With roughly 50 files and a 4:1 split, the test set has ~10 examples, so stable accuracy 66.7% corresponds to just 6-7 correct classifications. The same configuration reaches peak 83.3% and stable 66.7%, while a nearly identical configuration in Table 2 gives stable 59.1%, a 7.6-point swing with no seed control. This is the signature of small-sample instability rather than a robust learned mapping. The proper test is an airfoil-grouped split: if it preserves roughly 67% accuracy, the concern is resolved; if it drops, the headline overclaim should be withdrawn or heavily qualified. The verdict therefore stays CONDITIONAL, with a sharper condition: release the dataset with airfoil identifiers and demonstrate grouped-split generalization.","tokens_in":11370,"tokens_out":5894,"duration_ms":79083,"concrete_test":"Obtain the actual dataset with a unique airfoil identifier per geometry. Reproduce the best configuration in Table 2 (4:1 split, lr=2e-4, filters [64,128,256,256], pooling [510,360,210,180]) under an airfoil-grouped split: all meshes of each airfoil must lie entirely in train or entirely in test, across 10 random grouped seeds. Report mean and standard deviation of stable and peak accuracy, plus per-class confusion. If the grouped-split stable accuracy falls to near chance (~33%) or shows >±15-point variance, the original 66.7% was inflated by geometric leakage; if it remains near 67% with tight variance, the generalization claim survives.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that AirCANS generalizes to CFD airfoil meshes rests on the Table 2 accuracies (best stable 66.7%, peak 83.3%). Section 3.2 says the dataset contains \"over 50 sets\" of airfoil meshes, and Table 1 shows each airfoil is meshed at two densities (fine and coarse). The train/test splits in Table 2 are reported only as file ratios (3:1, 4:1), so a random file-level split can assign the dense and coarse meshes of the same airfoil to different sides. A model can then succeed by recognizing near-duplicate geometry (identical boundary coordinates, similar edge-length ratios, shared topology) rather than by learning thickness as a general property. With roughly 50 files and three classes, the 4:1 test set is about 10 meshes; 66.7% stable accuracy is just 6-7 correct, and moving one test sample changes accuracy by 10 points. No repeated-seed statistics, no airfoil-grouped split, and no data release are provided, so this alternative explanation is not excluded. The instability across Tables 2-3 (stable 33-67%, peaks 50-83%) is consistent with small-sample variance and potential leakage.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents AirCANS, a framework based on MeshCNN for classifying 2D CFD unstructured airfoil meshes into three thickness ranges (1-10%, 11-20%, 21-30%). The authors generate over 50 self-made airfoil meshes at two densities (fine and coarse), adapt the MeshCNN edge-based convolution/pooling architecture to 2D, and report stable accuracies around 67% and peak accuracies around 83% for the best configuration. They also compare dense versus sparse meshes, claiming that dense meshes yield more stable performance. The central claim is that CNNs are fully adaptable and understandable for CFD airfoil mesh data structures, with potential applications in mesh refinement and acceleration.","tokens_in":11569,"tokens_out":7227,"duration_ms":75074,"significance":"If the results were robust, this paper would provide a useful proof-of-concept for applying geometric deep learning to unstructured CFD mesh data, potentially enabling fast preprocessing and classification without flow solving. The dataset-generation effort and the comparative dense/sparse design are positive aspects, as is the intended public release of a mesh dataset. However, the experimental evidence is thin: roughly 50 samples, no repeated-seed statistics or airfoil-grouped splits, and the mathematical description in Section 3.4 is internally inconsistent. The paper does not provide the promised dataset or code, so reproducibility cannot be checked. These issues substantially weaken the support for the abstract's strong claim.","major_comments":[{"comment":"Equations (1)-(13) are inconsistent with the described architecture and with the input specification in Section 3.1. Equations (1)-(4) define Sobel gradient features on a dense 2D pixel grid I ∈ R^{H×W×C}, whereas Section 3.1 defines the input as an edge-based tensor R^{B×5×E×1} with features [θ_i, α_i1, α_i2, r_i1, r_i2] computed from mesh edges. Equations (5)-(13) describe 5×5 and 2×2 image-style neighborhoods and a dynamic importance score, but the text states that convolution operates on each edge and its four neighboring edges in a mesh. Please provide a coherent mathematical description of the actual edge-based convolution and pooling operations, or clearly explain why these image-based equations are included.","section":"Section 3.4"},{"comment":"There are numerical contradictions between the text and the tables. Section 4.2 states that with a 3:1 train-test ratio the highest accuracy was about 83% and the stable accuracy about 67%, but Table 2 shows the 3:1 row with Stable Acc 59.091% and Highest Acc 81.818%; the 66.667% stable and 83.333% highest values correspond to the 4:1 row with pooling [510,360,210,180]. Figure 8 and its caption claim dense grids 'regularly achieving accuracy levels >80%' and sparse grids occasionally reaching '>99%', neither of which appears in Tables 2 and 3 (best stable are 66.7% and 50%, best peaks are 83.3%). Please correct these inconsistencies and define precisely what 'Stable ACC' and 'Highest ACC' measure (e.g., best test accuracy over all epochs, final-epoch test accuracy).","section":"Section 4.2 / Tables 2-3 / Figure 8"},{"comment":"The experimental protocol does not rule out data leakage between training and testing. The dataset contains over 50 airfoil meshes, with each airfoil meshed at both fine and coarse densities (Table 1). The train/test splits are reported only as file ratios (3:1, 4:1), with no statement that all meshes of a single airfoil are kept in one split. If fine and coarse versions of the same airfoil appear in both training and test sets, the model can achieve high accuracy by recognizing near-duplicate geometry rather than by learning thickness as a general property. Please perform a grouped split by airfoil identity, report the resulting accuracies, and, given the small test set (~10 samples), also provide per-class results and confidence intervals.","section":"Section 3.2 / Section 4.1"},{"comment":"Hyperparameters appear to have been selected by inspecting the same training curves that are reported as results. The text says that 'Based on the convergence trends, we selected the pooling parameters [510, 360, 210, 180] ... with a learning rate' directly from the loss trajectories shown in Figure 7. Without a separate validation set or repeated-seed trials, this procedure risks overfitting to the test set. Please report mean and standard deviation across multiple random seeds, and explicitly separate model selection (validation) from final evaluation (test), or justify why this is not needed for the small dataset.","section":"Section 4.1 / Figure 7"},{"comment":"The manuscript makes claims that are not supported by its content. The Introduction and Section 2 announce a publicly released comprehensive CFD mesh dataset, but no dataset URL, repository, or access details are given anywhere. The task performed is airfoil thickness classification, not 'mesh assessment' or 'mesh optimisation'; the title uses 'Mesh Optimisation-based' although no mesh optimization is conducted (only fine vs. coarse meshes are compared). The abstract's statement that 'CNNs are fully adaptable as well as understandable to CFD airfoil mesh data structures' is an overreach given the limited experiments. Please align the claims, title, and release statement with what the paper actually delivers.","section":"Section 2 / Section 6 / Title"}],"minor_comments":[{"comment":"There are many typographical and grammatical errors, e.g., 'exposion' in Table 1, 'airfilm' in Section 2, 'hypermeters' in Section 6, 'parse matrices' in Section 5, and 'we Our aim' in Section 3.5. A careful proofread is needed.","section":"Throughout"},{"comment":"The input tensor shape is given as R^{B×C_in×E×1} and later as R^{B×5×E×1}; please clarify the channel order and whether the extra dimension is a singleton, since this affects how the convolution is applied.","section":"Section 3.1"},{"comment":"The terms 'Stable Acc' and 'Highest Acc' are used without formal definitions. Please state whether 'Highest Acc' is the best test accuracy during training (which is an optimistically biased statistic) and how 'Stable' is computed (e.g., average over last N epochs).","section":"Section 4.2"},{"comment":"The paper frequently cites MeshCNN [24] and its fundamentals [26] but does not specify which version of the code or which training hyperparameters (e.g., optimizer, batch size, epoch count) were used. Please add these details for reproducibility.","section":"References"},{"comment":"The text says 'the pooling layer requires soft-edge attribution; otherwise, the lack of geometric information may not adequately represent key features' (Section 5), but this concept is not defined or elaborated anywhere. Please explain what soft-edge attribution means and how it affects the experiments.","section":"Section 3.4"}],"recommendation":"major_revision","confidential_remarks":"The paper has a plausible core idea, but the current manuscript has several load-bearing problems. The stress-test concern about file-level train/test leakage is valid and should be addressed with an airfoil-grouped split. The inconsistency between Section 3.4's image-based equations and the described mesh-based architecture is a serious presentation flaw that calls the technical understanding into question. Given the small dataset and no repeated-seed statistics, the reported accuracies are not convincing evidence for the abstract's strong claim. I would be willing to see a revised version that corrects these issues, but the current version is not acceptable for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a proof-of-concept that an off-the-shelf MeshCNN can classify 2D CFD airfoil meshes into three thickness bins. That's a legitimate thing to try, and the paper does a decent job of framing it. But the headline number—67% stable accuracy—isn't solid enough to bear the abstract's 'fully adaptable' claim, and the most likely reason is the train/test setup.\n\nWhat's actually new: the application of MeshCNN to airfoil mesh classification, plus a data loader and a comparison of dense vs sparse mesh representations. The mesh-density experiment is a sensible idea. The authors also say they'll release the dataset, which would be a real contribution if a link appears. The limitations section is honest about the small data size.\n\nThe problems: the dataset is roughly 50 meshes, with each airfoil meshed twice (fine and coarse). The train/test split is reported only as a file ratio, so a random split can place the fine and coarse version of the same airfoil on opposite sides. That's a leak: the model can recognize near-duplicate geometry rather than learn thickness as a general property. With a 4:1 split you have ~10 test meshes; 67% is 6 or 7 correct, so one sample moves the number by 10 points. No repeated-seed statistics, no error bars, and hyperparameters are chosen by looking at the same training curves. Section 3.4's equations look like standard image CNN math (Sobel filters, 2x2 pooling) and don't match the edge-collapse pipeline described in Section 3.3—that inconsistency needs fixing. The abstract also overclaims: classification of thickness bins is not 'accelerating the solution' or 'mesh refinement.'\n\nThe citation pattern is fine—MeshCNN and relevant CFD-ML work is cited. No invention of entities.\n\nVerdict: the core idea is plausible and worth a look, but the evidence as presented doesn't establish generalization. This deserves a serious referee to force a proper validation protocol, but it's not ready in this form. I'd send it out with major revisions expected, not desk reject.","headline":"Proof-of-concept for airfoil mesh classification with MeshCNN, but the accuracy claim is undercut by possible train/test leakage.","tokens_in":12170,"tokens_out":2524,"would_cite":false,"duration_ms":26677,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A CNN can classify airfoil thickness directly from CFD mesh geometry, without solving the flow equations.","keywords":["convolutional neural network","CFD mesh classification","airfoil thickness","unstructured mesh","edge-based convolution","mesh pooling","MeshCNN","deep learning"],"falsifier":"Generate meshes for a held-out set of airfoil shapes that were not used in training, for example NACA 4-digit series with thickness values not present in the training bands, and run the trained AirCANS model on them; if stable accuracy falls to the chance level of about 33% while training accuracy stays high, the mesh-edge features do not generalize and the central claim fails.","tokens_in":11100,"feed_emoji":"🛩️","tokens_out":3295,"duration_ms":33458,"temperature":0.7,"pith_summary":"This paper tries to show that a convolutional neural network can take a 2D unstructured CFD mesh around an airfoil and classify which thickness range the airfoil belongs to, using only local edge geometry rather than solving the flow equations. The authors build a framework called AirCANS, adapted from MeshCNN, that represents each mesh edge by angles and normalized length ratios and then applies convolution and edge-collapse pooling. On their own generated dataset of more than 50 airfoil meshes grouped into three thickness bands, the best configuration reaches about 83% peak accuracy and settles near 67% stable accuracy. The intended payoff is faster mesh assessment for CFD, with classification used to guide mesh refinement and cut simulation time.","feed_headline":"CNN sorts airfoil meshes by thickness without solving the flow","feed_subtitle":"AirCANS reads edge geometry from 2D CFD meshes and keeps ~67% stable accuracy on three thickness classes.","key_machinery":"The input tensor has shape (batch, 5, edges, 1); each edge is described by the dihedral angle between adjacent face normals (constant in 2D), the two opposite angles within its neighboring triangles, and the two normalized edge-length ratios inside those triangles. The backbone is four MResConv blocks that convolve each edge jointly with its four neighbors using symmetric functions, then MeshPool collapses low-importance edges based on an L2-norm score, progressively reducing edge counts from 3600 to 1200. This edge-based representation lets the network ignore raw vertex positions and operate purely on local geometry and topology.","core_discovery":"The central claim is that CNNs are fully adaptable and understandable to CFD airfoil mesh data structures. Concretely, AirCANS classifies an airfoil's thickness range from the unstructured triangular mesh alone, without the Navier-Stokes solve, and the experimental evidence is that with a 4:1 train-test split the model maintains roughly 67% stable accuracy with peaks near 83%, while denser meshes train more stably than sparse ones. The paper treats this as evidence that mesh geometry encodes enough aerodynamic-relevant information for neural assessment, so that mesh quality and thickness could be evaluated quickly before or during simulation.","pith_inferences":["The 67% stable accuracy on a 3-class problem is modest; a stronger test would be whether the edge features generalize to airfoil families not used in generating the meshes, since the current dataset is self-generated and small.","If verified on held-out geometry, the approach could be extended to predict continuous thickness values rather than coarse bands, or to regress other geometric parameters such as camber and chord position.","A label-shuffle control would isolate whether the network learns genuine geometric signals versus dataset artifacts such as mesh statistics correlated with thickness."],"forward_implications":["Mesh quality and airfoil thickness can be assessed directly from mesh files, avoiding an initial simulation pass.","The same edge-convolution pipeline could be extended back to 3D by re-enabling the dihedral angle feature, which is kept in the 5-channel format.","Dense meshes give more stable training than sparse ones, suggesting mesh resolution should be preserved when feeding data to such networks.","Classification output could be used to trigger mesh refinement or coarsening decisions, accelerating CFD workflows."],"supporting_citations":[{"why":"Supplies the edge-convolution and edge-collapse pooling architecture that AirCANS adapts from MeshCNN.","marker":"[24]"},{"why":"Provides the pooling parameters and the reconstructable representation that AirCANS inherits and modifies.","marker":"[26]"},{"why":"Justifies the use of unstructured meshes for complex geometries, the data setup on which the experiments depend.","marker":"[29]"}],"fun_headline_variants":["Airfoil thickness from mesh alone: CNN skips flow solve","CNN reads airfoil mesh, predicts thickness without CFD solve","Mesh-based airfoil classification: AirCANS sees thickness","Neural net assesses airfoil mesh geometry, not flow"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The classification target must be recoverable from the local edge geometry of the unstructured mesh; if thickness information is not robustly encoded there for unseen airfoils, the reported accuracy reflects memorization of the small self-generated dataset rather than a generalizable skill.","fun_headline_variants_meta":{"raw":{"variants":["Airfoil thickness from mesh alone: CNN skips flow solve","CNN reads airfoil mesh, predicts thickness without CFD solve","Mesh-based airfoil classification: AirCANS sees thickness","Neural net assesses airfoil mesh geometry, not flow"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000205,"raw_usage":{"total_tokens":1333,"prompt_tokens":825,"completion_tokens":508,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":441,"completion_tokens_details":{"reasoning_tokens":436}},"tokens_in":441,"tokens_out":508,"duration_ms":5367,"temperature":1.0,"reasoning_tokens":436,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T22:01:00.019277+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate meshes for a held-out set of airfoil shapes that were not used in training, for example NACA 4-digit series with thickness values not present in the training bands, and run the trained AirCANS model on them; if stable accuracy falls to the chance level of about 33% while training accuracy stays high, the mesh-edge features do not generalize and the central claim fails.","supporting_citations":[{"cited_title":"MeshCNN: A Network with an Edge","cited_arxiv_id":"1809.05910","evidence_quote":"Supplies the edge-convolution and edge-collapse pooling architecture that AirCANS adapts from MeshCNN."},{"cited_title":"MeshCNN Fundamentals: Geometric Learning through a Reconstructable Representation","cited_arxiv_id":"2105.13277","evidence_quote":"Provides the pooling parameters and the reconstructable representation that AirCANS inherits and modifies."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Justifies the use of unstructured meshes for complex geometries, the data setup on which the experiments depend."}],"review_version":1}