{"id":"9a2c35e8-8f88-4b51-974e-7107483e5908","arxiv_id":"2608.11447","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"A dual-head CNN predicts density, specific compression modulus, yield strength, and compression strength of lignin-containing polyurethane foams from SEM images with reported R2 values of 0.85 to 0.91, but the reported evaluation does not clearly isolate held-out test data.","lead":"Researchers trained a dual-head convolutional neural network to predict density and mechanical properties of lignin-based polyurethane foams directly from SEM images, reporting R2 between 0.85 and 0.91. The work is a candidate non-destructive screening tool for bio-based foam formulations, but the evaluation and data limitations currently cap confidence in the claimed accuracy.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 2 metrics appear to be computed on the entire dataset including training images, so the reported R2/MAPE may be fit statistics rather than held-out generalization, leaving the central non-destructive-prediction claim unsupported.","rationale":"The reader's weakest-assumption analysis identifies the same key issue: Section 4.1 says the entire dataset was assessed and Table 2 is labeled as averaged for all samples, so the reported R2/MAPE may include training data. That concern is load-bearing because the paper's fundamental value proposition is that a CNN can predict mechanical properties from SEM images without destructive testing—a claim that only holds if performance is demonstrated on unseen data. The current text does not provide such a demonstration, and Table 3's examples do not establish split membership. The paper has other issues (no code/data, unverified pretraining claim, no error bars), but the evaluation-protocol ambiguity is the most direct threat to the central claim. My stress-test does not change the reader's REJECT verdict; if the authors provide test-only metrics and they are strong, the paper could be reconsidered, but as written the evidence is insufficient.","tokens_in":14831,"tokens_out":1594,"duration_ms":17566,"concrete_test":"Request the exact train/validation/test sample indices (or the trained model plus data) and recompute R2, MAPE, and Pearson r on the held-out test subset only, exactly as defined in Section 3.2. If the current Table 2 was computed on all 107 images, the reported values will not necessarily reflect generalization; the test-only values are the ones that must be reported. Additionally, run a 5-fold cross-validation stratified by foam formulation (so no images from the same batch are shared across folds) and report mean and standard deviation of R2 and MAPE. If test-only R2 drops materially below the claimed 0.85-0.91 range, the central claim must be revised.","verdict_should_be":"REJECT","load_bearing_attack":"The paper's core claim is that a dual-head CNN generalizes from SEM images to density and three specific mechanical properties with R2 0.850-0.91, r>0.92, MAPE<9% (Abstract; Section 4.1; Table 2). That claim requires evaluation on samples not used for training. However, Section 4.1 states: 'The entire dataset was assessed using several quantitative performance metrics', and Table 2 is labeled 'averaged for all samples'. This phrasing strongly implies that the reported R2, MAPE, and Pearson r values include training samples. If so, the metrics quantify how well the model fits the data it was trained on, not how well it predicts unseen foam microstructures. The paper never reports test-only metrics, error bars, or repeated random-split statistics. Table 3 lists predictions for seven specific formulations, but does not state whether those formulations were in the training, validation, or test split, so it cannot rescue generalization. Additionally, the data partition is described only as 40/30/30 with matched means and standard deviations (Section 3.4); no per-specimen grouping is described, raising the possibility that images from the same foam batch appear in both training and test sets. Without held-out test metrics, the stated accuracy and the 'non-destructive framework' claim are not established. This is the single load-bearing weakness: every downstream claim about predictive capability, physical interpretability, and practical utility depends on the model performing well on unseen data.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes a dual-head convolutional neural network (CNN) that takes SEM images of lignin-containing rigid polyurethane foams as input and predicts four target quantities: density, specific compression modulus, specific yield strength, and specific compression strength. The authors report R2 values between 0.850 and 0.914, Pearson correlation coefficients above 0.92, and MAPE below 9.12% (Abstract; Section 4.1, Table 2). They further use Grad-CAM to argue that the network focuses on physically meaningful microstructural features such as cell walls and strut junctions, and they position the method as a non-destructive, interpretable framework for predicting foam mechanical properties (Section 4.2). The work is based on 107 SEM images with corresponding compression-test data from an earlier study.","tokens_in":15200,"tokens_out":4638,"duration_ms":42916,"significance":"If the reported predictive performance were established on held-out data, the paper would offer a useful non-destructive screening tool for lignin-based PU foam formulations and a reasonable demonstration of image-to-property learning for a small, heterogeneous materials dataset. The dual-head architecture motivated by target correlation is a sensible design choice, and the inclusion of Grad-CAM visualization is a positive step toward interpretability in materials machine learning. The paper also documents many training details (Section 3.4) that aid reproducibility. However, the central claim of generalization is currently unsupported because the key performance metrics appear to be computed over the entire dataset, including training samples, and because data-splitting and leakage issues are not addressed. The significance of the work therefore hinges on a re-evaluation that is not provided in the manuscript.","major_comments":[{"comment":"The reported R2, MAPE, and Pearson r are described as 'averaged for all samples' and the text states 'The entire dataset was assessed using several quantitative performance metrics.' Since Section 3.4 states the data were split into 40% training, 30% validation, and 30% testing, the metrics in Table 2 appear to include training and validation samples as well as test samples. If that is the case, the results quantify training fit, not predictive generalization, and the Abstract's claim of a network that 'predicts' material properties is not established. Please report metrics computed only on the held-out 30% test split, and additionally provide repeated random-split or cross-validated results with means and standard deviations.","section":"Section 4.1, Table 2, Eqs. (31)-(33)"},{"comment":"The data-partitioning description only says that the three subsets have matched means and standard deviations; it does not specify whether images from the same foam specimen or formulation are grouped together. If multiple SEM images come from the same foam panel and images from that panel appear in both training and test sets, the reported accuracy may be optimistically biased due to image-level leakage. The authors should describe how images relate to specimens and perform a specimen-level or formulation-level split (e.g., group by foam panel) before recomputing all metrics.","section":"Section 3.2 and Section 3.4"},{"comment":"The text states that Phase 1 'prevents degradation of pretrained representations' and refers to 'optimal utilization of pre-trained backbone features,' but nowhere does the manuscript specify what pretrained backbone is used, how it was pretrained, or on what data. The architecture in Table 1 describes layers initialized with He initialization, which suggests random initialization. If pretraining is used, the source, architecture, and pretraining dataset must be identified; if no pretraining is used, the claim should be removed. This is a load-bearing detail because the training protocol and its effect on the reported performance cannot be evaluated without it.","section":"Section 3.4.1"},{"comment":"The Grad-CAM analysis is used to conclude that 'the network focuses its predictions on physically meaningful microstructural regions' and that the model 'does not rely on superficial image statistics but rather captures true morphological determinants of mechanical behavior.' This reasoning is circular: the model's own saliency maps are used to confirm the model's physical meaningfulness without any external validation, such as comparison to known structure-property relationships on independent data, ablation of highlighted regions, or quantitative attribution metrics. The interpretability claim exceeds what Grad-CAM alone can support and should be substantially softened or supplemented with an external validation.","section":"Section 4.2"},{"comment":"No baseline models are compared, and no uncertainty quantification is provided. With only 107 images, a simple baseline (e.g., linear regression on handcrafted morphological features, a single-head CNN, or random forest on histogram features) is necessary to interpret whether the dual-head CNN's performance is actually strong. Additionally, the absence of confidence intervals or repeated-split statistics means the reader cannot assess the stability of the reported R2 and MAPE values. Please include baselines and repeated-evaluation statistics in the revised manuscript.","section":"Section 4.1 and Discussion"}],"minor_comments":[{"comment":"Equation numbering is inconsistent: equations are numbered 1-10, then jump to 21 onwards; the intermediate equations (11-20) are missing. Please renumber sequentially.","section":"Section 3.1"},{"comment":"The text says the three mechanical parameters 'demonstrate strong independence with r=0.61-0.93'; given the context and the correlation heat map, this should read 'strong correlation' or 'strong interdependence,' not 'independence.'","section":"Section 3.1"},{"comment":"MAPE as defined uses the raw yi denominator. For small-magnitude targets such as density (~0.05-0.07 g/cm3), MAPE can be sensitive to small absolute errors; reporting MAE and RMSE alongside MAPE would give a more complete picture.","section":"Section 4.1, Eq. (31)"},{"comment":"For the seven formulations listed, the manuscript does not state whether these images were part of the training, validation, or test split. This information is essential for interpreting the 'predicted vs. actual' comparison in that table.","section":"Table 3"},{"comment":"The formulation text contains an unresolved placeholder '(ref)' in the description of the foam ingredients; please provide the intended citation.","section":"Section 2"},{"comment":"There are minor typographical issues, e.g., 'state-of-art' should be 'state-of-the-art,' 'diver lignin foams' in the Conclusion, and 'weaklyy' in Section 3.1. Please proofread the manuscript.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The reader's report recommends rejection, and I agree that the current evaluation protocol is seriously flawed. However, the central issue is potentially fixable because the manuscript already describes a 30% test split; if the authors recompute all performance metrics on that held-out set, group images by specimen, and add baseline comparisons, the core claim could become testable. Given the current manuscript as written, the reported metrics are not evidence of generalization, and the editors may wish to require a rigorous re-analysis before any further consideration."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The punchline: the reported R2/MAPE are almost certainly fit statistics, not generalization numbers. Section 4.1 says 'the entire dataset was assessed' and Table 2 is 'averaged for all samples'. That means the central claim—that the CNN predicts unseen foam microstructures—is not established. This is a load-bearing weakness, and it is right there in the text.\n\nWhat the paper does well: the application is sensible and the dataset is real: 107 SEM images with compression data from lignin-modified PU foams. The dual-head architecture, with density separated from the three correlated mechanical properties, is a reasonable design choice. The augmentation is modest and physically plausible. Grad-CAM is used in a standard way, and attention at cell walls and struts is consistent with foam mechanics. The citation pattern is fine; the authors cite prior CNN work on porous media and their own classification paper, which is appropriate.\n\nSoft spots, in order of severity. First, no test-only metrics. That is not a minor omission. Every downstream statement about generalization, robustness, and practical screening depends on held-out predictions. Without them, an R2 of 0.85 to 0.91 on training data is not evidence of predictive power. Second, no repeated splits, confidence intervals, or baseline comparison. With 107 images, a trivial baseline like predicting training-set means could look worse or better; we aren't told. Third, the pretraining claim in Section 3.4.1 is unsupported: the paper refers to pretrained backbone features but never describes any pretraining. If it was trained from scratch, that section is misleading; if it was pretrained, the details are missing. Fourth, Table 3 lists predictions for seven formulations but never says whether those formulations were in train, validation, or test. With only 107 images across formulations, batch-level leakage is a real risk. Fifth, data and code are not available, which would be acceptable with a clean evaluation but here blocks verification.\n\nThe stress-test note is correct; I don't think it overreaches. This is not a paper about math or algorithms—it's an application paper, and the evaluation protocol is the whole ballgame. The material is worth publishing eventually, but only after proper held-out evaluation.\n\nWho it is for: materials scientists working on bio-based foams who want a fast, non-destructive screen. It could become citable if the evaluation is fixed. My recommendation: send it to peer review, but the referee should insist on test-only metrics, repeated splits, a baseline, and clarification of the pretraining claim. As it stands, it is not acceptable.","headline":"Reported R2/MAPE are almost certainly fit statistics on the full dataset, so the paper's central generalization claim is not established; the materials data and architecture are reasonable, but the evaluation must be redone.","tokens_in":15703,"tokens_out":3573,"would_cite":false,"duration_ms":31014,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A dual-head CNN predicts density and three mechanical properties of lignin-containing polyurethane foams from SEM images, with reported R² between 0.850 and 0.91.","keywords":["Polyurethane foam","lignin","mechanical property prediction","convolutional neural networks","sustainable materials","Grad-CAM visualization","SEM imaging","multi-output regression"],"falsifier":"Evaluate the trained network on SEM images of lignin-containing foam formulations whose images were held out of all training phases, and compare the resulting $R^2$ and MAPE with Table 2; if the held-out values fall well below the reported 0.850–0.91, the generalization claim is unsupported. Recomputing the metrics on the untouched 30% test subset alone, as the paper's split description allows, would provide this check directly.","tokens_in":14682,"feed_emoji":"🔬","tokens_out":12130,"duration_ms":98383,"temperature":0.7,"pith_summary":"This paper argues that the load-bearing mechanical properties of lignin-containing rigid polyurethane foams can be read directly from scanning electron microscopy images, without destructive testing. A dual-head convolutional neural network is trained on 107 SEM images to predict density, specific compression modulus, specific yield strength, and specific compression strength, reporting $R^2$ values between 0.850 and 0.91, Pearson correlations above 0.92, and mean absolute percentage errors below 9%. The two-head design reflects the measured correlation structure: density is nearly independent of the three mechanical properties, while those three are strongly coupled. Grad-CAM heatmaps show the network attends to cell walls, struts, and junctions rather than pore voids, which the authors take as evidence that it learned physically meaningful morphology-property relationships. If correct, the approach offers a fast, non-destructive route to screening bio-based foam formulations.","feed_headline":"A dual-head CNN predicts foam strength and density from SEM images","feed_subtitle":"Lignin-based polyurethane foams can be screened without destructive compression tests, with errors below 9 percent.","key_machinery":"The load-bearing mechanism is a dual-head convolutional neural network. A shared backbone — three convolutional stages with 3×3 kernels, batch normalization, LeakyReLU activation, 2×2 max pooling, global average pooling, and a shared fully connected layer — produces a compact latent embedding. One fully connected head outputs density, while a second head outputs the three strongly correlated mechanical properties. The split is justified by the measured correlation structure: density correlates only weakly with the mechanical properties ($r$ between 0.29 and 0.36), whereas the three mechanical properties correlate strongly with each other ($r$ between 0.61 and 0.93), so sharing features helps the mechanical head without forcing density into an irrelevant correlation. Training uses a weighted multi-output loss, an adaptive three-phase schedule, gradient clipping, and early stopping; Gradient-weighted Class Activation Mapping (Grad-CAM), a standard technique for highlighting the image regions that drive a convolutional prediction, is then applied to each head to produce the attribution heatmaps.","core_discovery":"The paper's central claim is that a custom dual-head convolutional neural network can predict four properties of lignin-containing rigid polyurethane foams — density, specific compression modulus, specific yield strength, and specific compression strength, each of the latter normalized by density in MPa/(g·cm⁻³) — directly from 40× SEM images of the cellular microstructure. On its dataset of 107 images covering several lignin-based polyols and loadings, the network achieves reported $R^2$ values of 0.8505 for density, 0.897 for specific compression modulus, 0.888 for specific yield strength, and 0.914 for specific compression strength, with Pearson correlations between 0.922 and 0.956 and MAPE between 5.57% and 9.13%. The claim is not only numerical: Grad-CAM visualizations show that the density head activates around thicker struts and reduced pore volume, while the mechanical-property heads activate across interconnected junction networks and aligned pore chains, matching the load-bearing skeleton expected to govern foam stiffness and strength. The authors present this as an interpretable, image-only surrogate for mechanical testing of bio-based foams.","pith_inferences":["A testable extension would hold out entire formulations or specimens during training and then measure whether the reported $R^2$ survives; the paper's description of evaluating the entire dataset leaves open whether some of the accuracy comes from images already seen during training.","Because the three mechanical properties are tightly correlated, a natural baseline is a single-output CNN predicting only compression strength; comparing the dual-head model against such a baseline would reveal how much the extra outputs and shared backbone actually contribute.","The Grad-CAM foci suggest a simpler structural measurement — for example, strut-junction density or cell-wall thickness extracted from the same SEM images — might already predict the mechanical targets; testing that feature-based regression would clarify what the convolutional features add beyond a hand-engineered descriptor."],"forward_implications":["One SEM image could replace a compression test for routine screening of foam formulations, removing specimen cutting, conditioning, and destruction from the workflow.","Because the model preserves the ranking of materials by stiffness and strength across diverse lignin polyols, it can serve as a fast comparative tool for selecting promising formulations before full mechanical characterization.","The two-head design shows that weakly correlated property groups can be predicted from a single shared image representation, a pattern that transfers to other multi-output microstructure-property tasks.","The Grad-CAM maps give a per-image sanity check that a prediction is driven by load-bearing morphology, so the approach can be applied to new compositions whose microstructure resembles the training set.","The small per-image errors, mostly between 1% and 5% on the formulations shown in detail, suggest the model is precise enough to flag formulation-level differences rather than only global trends."],"supporting_citations":[{"why":"Supplies the 107 SEM images and compression measurements of lignin-containing PU foams that form the training data and ground truth.","marker":"[20]"},{"why":"Prior machine-learning study on classifying lignin-based PU foams from imaging, establishing the structure-property context that this work extends to regression.","marker":"[17]"},{"why":"Precedent for using an adaptive residual CNN with Grad-CAM to predict material properties from SEM images, supporting the interpretability approach.","marker":"[15]"},{"why":"Defines Grad-CAM, the method used to localize the image regions driving each prediction.","marker":"[36]"},{"why":"Supplies the uncertainty-based loss weighting used to balance the multi-task training objectives.","marker":"[34]"},{"why":"AdamW, the optimizer used for adaptive weight updates with decoupled weight decay.","marker":"[28]"}],"fun_headline_variants":["CNN predicts four foam properties from SEM images","Deep learning reads foam microstructure for mechanical traits","Lignin foam strength predicted from microscope images","AI model links foam images to stiffness and yield","Interpretable CNN screens bio-based foams without testing"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the 107 images behave like independent samples and that the reported accuracy numbers come from images the network was not trained on, so they describe prediction of new foams rather than recall of images it had already seen.","fun_headline_variants_meta":{"raw":{"variants":["CNN predicts four foam properties from SEM images","Deep learning reads foam microstructure for mechanical traits","Lignin foam strength predicted from microscope images","AI model links foam images to stiffness and yield","Interpretable CNN screens bio-based foams without testing"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000515,"raw_usage":{"total_tokens":2548,"prompt_tokens":1042,"completion_tokens":1506,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":658,"completion_tokens_details":{"reasoning_tokens":1436}},"tokens_in":658,"tokens_out":1506,"duration_ms":32369,"temperature":1.0,"reasoning_tokens":1436,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:12:18.509978+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate the trained network on SEM images of lignin-containing foam formulations whose images were held out of all training phases, and compare the resulting $R^2$ and MAPE with Table 2; if the held-out values fall well below the reported 0.850–0.91, the generalization claim is unsupported. Recomputing the metrics on the untouched 30% test subset alone, as the paper's split description allows, would provide this check directly.","supporting_citations":[{"cited_title":"Preparation of Mechanically Robust Bio-Based Polyurethane Foams Using Depolymerized Native Lignin,","cited_arxiv_id":null,"evidence_quote":"Supplies the 107 SEM images and compression measurements of lignin-containing PU foams that form the training data and ground truth."},{"cited_title":"Neural network-optimized imaging for classifying lignin-based polyurethane foams: Linking molecular composition to cellular microstructure using advanced machine learning,","cited_arxiv_id":null,"evidence_quote":"Prior machine-learning study on classifying lignin-based PU foams from imaging, establishing the structure-property context that this work extends to regression."},{"cited_title":"Adaptive residual convolutional neural network for compressive strength prediction of energetic materials using SEM images,","cited_arxiv_id":null,"evidence_quote":"Precedent for using an adaptive residual CNN with Grad-CAM to predict material properties from SEM images, supporting the interpretability approach."},{"cited_title":"Grad-cam: Visual explanations from deep networks via gradient-based localization,","cited_arxiv_id":null,"evidence_quote":"Defines Grad-CAM, the method used to localize the image regions driving each prediction."}],"review_version":1}