{"id":"085488e7-bf6d-4e69-93c7-7d5ee63d93a3","arxiv_id":"2411.11826","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"Two minimal convolutional networks match big pretrained models on an easy fake-face dataset and train far faster, but they fail on a harder 140k face dataset.","lead":"The authors built two very small neural networks that can tell real from fake face images, and showed they train much faster than large pretrained models. On an easy dataset they match near-perfect accuracy, but on a harder 140k face dataset they fall far behind.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claim of computational efficiency is not supported by the evidence: it relies on wall-clock MATLAB training time, omits parameter/FLOP counts, and Table 7 contradicts the text's assertion that v1 is the fastest model.","rationale":"The reader's weakest assumption identified the same core issue: computational efficiency is judged by wall-clock time and layer count without parameter/FLOP metrics. My read confirms this and adds a concrete internal contradiction in the timing tables (v2 faster than v1 on 140k; AlexNet close to v1 on Hard) that further undermines the specific 'v1 fastest' claim. The accuracy concern on the 140k dataset is also present, but the efficiency claim is the most load-bearing for the paper's 'rapid/lightweight' selling point. A conditional verdict with requests for parameter/FLOP reporting and a controlled timing comparison is appropriate.","tokens_in":16708,"tokens_out":9600,"duration_ms":90817,"concrete_test":"Compute and report trainable parameter counts and inference FLOPs for all ten models using a standard tool (e.g., MATLAB's analyzeNetwork or a public FLOPs counter). If the LightFFDNet models do not have at least an order-of-magnitude fewer FLOPs than AlexNet, the wall-clock speed advantage is likely implementation-specific rather than intrinsic to the lightweight architecture.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The abstract and Section 5 claim that the proposed LightFFDNet models are 'computationally efficient' and that v1 is 'significantly faster than all other models.' This is based on (i) counting only convolutional and fully connected layers (Section 4.2) and (ii) wall-clock training times on a single laptop (Tables 5 and 7). Neither is a valid measure of computational complexity: layer count ignores filter numbers, input resolutions, and FLOPs, and wall-clock time conflates implementation details (e.g., MATLAB's pretrained-network machinery, GPU utilization) with model efficiency. The internal evidence is also inconsistent: in Table 7 on the 140k dataset, LightFFDNet v2 (84 s) is faster than v1 (94 s), yet Section 5 says v1 is 'significantly faster than all other models.' On the Hard dataset, AlexNet is only 1.18x slower than v1 (78 s vs 66 s), contradicting 'leaving all other models significantly behind.' Without parameter counts or FLOPs, the central speed advantage is unverified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes two small convolutional neural networks, LightFFDNet v1 (2 convolutional + 1 fully connected layer) and LightFFDNet v2 (5 + 1), for binary real-vs-fake face classification. The models are trained and evaluated on two public datasets: the Fake-Vs-Real-Faces (Hard) dataset and a 1288-image subset of the 140k Real and Fake Faces dataset. The authors compare their models against eight ImageNet-pretrained architectures (AlexNet, VGG-16, VGG-19, ResNet-50, ResNet-101, GoogleNet, MobileNet-V2, DarkNet-53) in terms of validation accuracy, test accuracy, F1/precision/recall, and wall-clock training time across 3, 5, and 10 epochs. The central claims are that the proposed models are lightweight, accurate, and computationally efficient, with v1 being the fastest model.","tokens_in":16940,"tokens_out":4661,"duration_ms":43312,"significance":"If the claims held, the paper would provide a simple, fast baseline for facial forgery detection that could be useful in resource-constrained settings. The experimental setup is transparent: publicly available datasets, a described hardware/software environment, and repeated trials for each configuration. The proposed models are genuinely small in layer count, and on the Hard dataset they reach test accuracies near 99.7-99.9%, competitive with the pretrained models. However, the evidence does not support the full strength of the claims: accuracy on the 140k subset is only about 70%, computational efficiency is measured only by wall-clock training time and layer count rather than parameter/FLOP counts, and the speed claims are internally inconsistent. The paper's contribution is better framed as a modest empirical comparison of small CNNs versus pretrained models on two Kaggle datasets, not as a demonstration of state-of-the-art accuracy or rigorously established computational efficiency.","major_comments":[{"comment":"The statement that LightFFDNet v1 is 'significantly faster than all other models' is contradicted by the paper's own tables. In Table 7 (140k dataset, 10 epochs), LightFFDNet v2 trains in 84 s while LightFFDNet v1 takes 94 s, so v1 is not the fastest model on that dataset. In Table 5 (Hard dataset, 10 epochs), AlexNet takes 78 s versus 66 s for v1, a factor of only 1.18, which does not support 'leaving all other models significantly behind.' The speed claims should be restated per model and per dataset, and the internal inconsistency between Section 5 and Tables 5/7 should be resolved.","section":"Section 5, Tables 5 and 7"},{"comment":"Computational efficiency is assessed only by wall-clock training time on one laptop and by counting only convolutional and fully connected layers. This is not a valid measure of model complexity: layer count ignores the number of filters, input resolution, and multiply-accumulate operations, and wall-clock time conflates implementation details (e.g., MATLAB's pretrained-model machinery, GPU utilization, data-loading overhead) with model efficiency. The central 'lightweight and computationally efficient' claim requires reporting parameter counts and FLOPs/MACs for all models, and ideally inference time per image, before the claim can be evaluated.","section":"Section 4.2 and Table 4"},{"comment":"The abstract claims that the proposed models 'detect forgeries of facial imagery accurately,' but Table 8 shows test accuracies of only 69.90% (v1) and 71.19% (v2) on the 140k dataset, while Section 5 concedes that 'all models, especially the sequential models, did not perform well' on that dataset. The accuracy claim must be qualified by dataset and by the fact that on the 140k subset the proposed models are substantially below DarkNet-53 (92.12%) and ResNet-50 (86.05%).","section":"Abstract, Table 8, Section 5"},{"comment":"The paper states that experiments were repeated three times and that 'the average of these trials was taken,' but later states that confusion matrices were 'calculated based on the trial that yielded the best results among three attempts over 10 epochs.' Additionally, the headline numbers in Tables 5-8 appear to be selected from a scan over 3, 5, and 10 epochs with no variance or confidence intervals reported. Because the number of layers was also tuned on the same validation sets, the reported best results are optimistic. Please report the mean and standard deviation over repetitions for each epoch count, and clarify whether the reported values are averages or best-of-three.","section":"Section 4.4"}],"minor_comments":[{"comment":"The F1 entry for DarkNet-53 is '0.99640', which has an extra trailing zero; the precision entry in Table 10 for DarkNet-53 is '0.99457', which appears to be a typo (likely 0.9457 or 0.9946).","section":"Table 9"},{"comment":"The sentence 'it fell short of only the VGG-19 architecture by a difference of 0.26%' is misleading because Table 6 shows that ResNet-50 and AlexNet also achieve 99.74% test accuracy, so v1 is tied with them rather than being uniquely second to VGG-19.","section":"Section 4.4"},{"comment":"Reference [36] is cited for VGG-16 and is a fruit-fly classification paper, not the original VGG architecture paper; VGG-16 should cite Simonyan and Zisserman (which appears as reference [48] for VGG-19).","section":"References"},{"comment":"The final paragraph on future work mentions applying the models to BRDFs, BSDFs, and BSSRDFs in computer graphics; this is unrelated to facial forgery detection and should be removed or moved to a separate context, as it currently reads as boilerplate.","section":"Section 5"},{"comment":"There are several grammatical and typographical errors, such as 'It's shown' in the abstract and 'addresses' for 'address' in the first sentence of Section 3; a careful language edit is needed.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a straightforward empirical comparison of two small CNNs against pretrained classifiers on two Kaggle datasets. The main issues are overclaiming in the abstract and conclusions, an internally inconsistent speed claim, and missing complexity metrics. These are fixable within the manuscript's scope: the authors can report parameter/FLOP counts, correct the speed statements, qualify the accuracy claims, and add variance information. I also note that the comparison is only against generic ImageNet classifiers, not against recent state-of-the-art forgery detection methods (e.g., FaceForensics++ based approaches), which limits the significance; the authors should temper their novelty claims accordingly. The self-citations and out-of-scope future-work paragraph suggest the manuscript may have been assembled from a template and needs careful revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing you should know: this is a straightforward empirical comparison of two small CNNs against eight pretrained models on two public fake-face datasets. The models are basic sequential stacks of 3x3 conv, batch norm, ReLU, max pool, and a fully connected layer, with the layer count chosen by scanning 2 through 6 layers. There is no new mechanism, loss, or architecture idea. But the measurements are real, the datasets are public, and the paper is honest enough to show the 140k results where the proposed models get 69.9% and 71.2% test accuracy versus DarkNet-53's 92.1%.\n\nThe paper does give credit where due: Tables 5-8 and the F1/precision/recall tables report all models fairly, and the conclusion explicitly admits the sequential models did not perform well on the 140k set. Doing three trials and averaging is a good habit, even without reporting variance.\n\nThe soft spots are mostly in the claims, not the numbers. The abstract says the models detect forgeries 'accurately, and computationally efficiently,' but on the 140k set they are 20 points below the best baseline. The efficiency argument rests on counting only conv+FC layers and on wall-clock training time on one laptop; there are no parameter counts or FLOPs, and layer count is a poor proxy for computational cost. The internal comparison is sloppy too: in Table 7, v2 is faster than v1 on the 140k set, yet the text says v1 is the fastest model. The best results come from a scan over 3, 5, and 10 epochs without variance, and the 140k subset is described only as 'a portion of it was taken.' These are fixable. The citation pattern is odd—the future work lists a bunch of the authors' own BRDF papers—but self-citation isn't the issue here.\n\nWho would get value from this? Someone doing a quick engineering comparison of small CNNs for binary face classification might use the tables. I wouldn't cite it. If it comes across your desk, treat it as borderline: reject-and-resubmit or major revision, not a desk reject, because the measurements are honest and the missing details are straightforward to supply. But as it stands, the central claim overreaches.","headline":"A modest, honest benchmark; the speed claim is under-supported and the accuracy claim only holds on the small dataset, but with heavy revision it could pass at a low-tier venue.","tokens_in":17499,"tokens_out":4099,"would_cite":false,"duration_ms":40226,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Two tiny CNNs match deep networks at fake-face detection.","keywords":["facial forgery detection","lightweight CNN","convolutional neural network","transfer learning","StyleGAN fake faces","binary image classification","real vs fake faces"],"falsifier":"Compute and publish the trainable parameter count, FLOPs, and per-image inference time for LightFFDNet v1/v2 and the eight pretrained models on identical hardware; if the small models are not dramatically smaller by these standard measures, or if their training-time advantage disappears in a non-MATLAB framework, the central efficiency claim would be refuted.","tokens_in":16483,"feed_emoji":"🕵️","tokens_out":5532,"duration_ms":47683,"temperature":0.7,"pith_summary":"This paper proposes two extremely shallow convolutional neural networks, one with three layers and one with six, for telling real and fake face images apart, and compares them against eight pretrained models. On the Fake-Vs-Real-Faces (Hard) dataset, the small models reach average test accuracies of 99.74 percent and 99.87 percent, matching or beating most large pretrained networks while training in about 66 to 88 seconds instead of hundreds or thousands. On the larger 140k Real and Fake Faces dataset, they are faster than every pretrained model but less accurate than the best deep models, reaching about 70 to 76 percent validation accuracy. The paper argues that for this binary forgery task, a minimal sequential CNN trained for at most ten epochs can be accurate enough while being dramatically cheaper to train.","feed_headline":"Two tiny CNNs match deep networks at fake-face detection","feed_subtitle":"A 3-layer and a 6-layer network hit ~99.7–99.9 percent accuracy on the Hard dataset while training 8–17 times faster.","key_machinery":"The load-bearing mechanism is the layer-count definition of lightness: the paper counts only convolutional and fully connected layers, giving the proposed models 3 and 6 layers, versus 8 layers for AlexNet and up to roughly 100 for ResNet-101. Combined with training for at most 10 epochs, this yields the reported training-time reductions of roughly 2x to 17x. The individual components are standard CNN building blocks; the distinctive move is the deliberate minimalism of the stack rather than any new operation.","core_discovery":"The paper's central claim is that facial forgery detection can be handled by a tiny sequential CNN without giving up accuracy. LightFFDNet v1, made of two convolutional layers and one fully connected layer, and LightFFDNet v2, made of five convolutional layers and one fully connected layer, both use 3x3 filters, batch normalization, ReLU activation, 2x2 max pooling, and a softmax output. Trained with Adam, a learning rate of 0.0001, and a batch size of 16 for 3, 5, or 10 epochs, they achieve near-perfect results on the Hard dataset: 99.74 percent and 99.87 percent average test accuracy, with F1, precision, and recall all at 1.0, while VGG-19 reaches 100 percent test accuracy and ResNet-50, DarkNet-53, and AlexNet trail slightly or match them. On the 140k dataset they reach 69.90 percent and 71.19 percent test accuracy, outperforming VGG-16, VGG-19, and AlexNet but falling behind ResNet-50, ResNet-101, MobileNet-V2, and DarkNet-53. The paper interprets these results as evidence that large pretrained architectures are unnecessary for two-class face forgery detection, and that the shallow models' speed advantage makes them a practical choice.","pith_inferences":["For the speed claim to hold as a statement about model efficiency, parameter counts and FLOPs should be reported; without them, wall-clock time on one laptop conflates model size with implementation and hardware details.","The Hard dataset contains only 1,288 StyleGAN2-generated fake images, so the near-perfect results may reflect dataset simplicity; a stronger test would evaluate the same models on harder or more diverse forgery benchmarks.","A direct measurement of per-image inference latency, not just training time, would clarify whether the speed advantage persists at deployment time, especially on CPU-only devices.","The shallow architecture is a sensible baseline for other binary image tasks, but its accuracy edge over deep models is unlikely to survive on tasks with high intra-class variation, where deeper features become necessary."],"forward_implications":["On the Hard dataset, face forgery detection does not require deep networks: a 3-layer CNN reaches 99.74 percent test accuracy and a 6-layer CNN reaches 99.87 percent.","Training time drops to tens of seconds on a laptop GPU, with reported speedups of 2x to 17x over the eight pretrained models.","The small models achieve perfect F1, precision, and recall on the Hard test set, so the accuracy advantage is not limited to a single metric.","The same architecture transfers to a second binary face dataset, where it is faster than all compared models, though less accurate than the strongest deep models.","Because the architecture has no face-specific components, the authors claim it can be applied to other two-class object recognition problems."],"supporting_citations":[{"why":"Supplies the Fake-Vs-Real-Faces (Hard) dataset, the primary benchmark where the proposed models reach about 99.7 to 99.9 percent test accuracy.","marker":"[10]"},{"why":"Supplies the 140k Real and Fake Faces dataset, the second benchmark used for comparing speed and accuracy.","marker":"[61]"},{"why":"Provides AlexNet, one of the eight pretrained baselines with the fewest layers.","marker":"[20]"},{"why":"Provides ResNet-50 and ResNet-101, the pretrained baselines that achieve the highest accuracies on the 140k dataset.","marker":"[16]"},{"why":"Provides VGG-19, the only pretrained model that beats LightFFDNet v1 on the Hard test set.","marker":"[48]"},{"why":"Provides MobileNet-V2, a lightweight pretrained baseline used for speed and accuracy comparison.","marker":"[17]"},{"why":"Provides DarkNet-53, the highest-accuracy model on the 140k dataset and a key speed comparison target.","marker":"[45]"},{"why":"Provides GoogleNet, another pretrained baseline against which the lightweight models are compared.","marker":"[52]"}],"fun_headline_variants":["Tiny CNNs match deep nets on fake-face detection","Lightweight CNN hits 99.87% fake-face accuracy","Lightweight nets rival heavy ones on fake-face tests","Minimalist CNNs catch fake faces with near-perfect accuracy","Tiny neural nets spot fakes fast and accurately"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central 'lightweight and fast' claim rests on wall-clock training time measured in one MATLAB environment on one laptop, with model size judged only by counting convolutional and fully connected layers; if another implementation or hardware shows the small models are not actually cheaper, the paper's main advantage collapses.","fun_headline_variants_meta":{"raw":{"variants":["Tiny CNNs match deep nets on fake-face detection","Lightweight CNN hits 99.87% fake-face accuracy","Lightweight nets rival heavy ones on fake-face tests","Minimalist CNNs catch fake faces with near-perfect accuracy","Tiny neural nets spot fakes fast and accurately"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001129,"raw_usage":{"total_tokens":4793,"prompt_tokens":1144,"completion_tokens":3649,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":760,"completion_tokens_details":{"reasoning_tokens":3567}},"tokens_in":760,"tokens_out":3649,"duration_ms":30755,"temperature":1.0,"reasoning_tokens":3567,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T18:05:37.926256+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute and publish the trainable parameter count, FLOPs, and per-image inference time for LightFFDNet v1/v2 and the eight pretrained models on identical hardware; if the small models are not dramatically smaller by these standard measures, or if their training-time advantage disappears in a non-MATLAB framework, the central efficiency claim would be refuted.","supporting_citations":[{"cited_title":"https: //www.kaggle.com/datasets/hamzaboulahia/ hardfakevsrealfaces (2023)","cited_arxiv_id":null,"evidence_quote":"Supplies the Fake-Vs-Real-Faces (Hard) dataset, the primary benchmark where the proposed models reach about 99.7 to 99.9 percent test accuracy."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the 140k Real and Fake Faces dataset, the second benchmark used for comparing speed and accuracy."},{"cited_title":"In: Advances in Neural Information Processing Systems, vol","cited_arxiv_id":null,"evidence_quote":"Provides AlexNet, one of the eight pretrained baselines with the fewest layers."},{"cited_title":"In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp","cited_arxiv_id":null,"evidence_quote":"Provides ResNet-50 and ResNet-101, the pretrained baselines that achieve the highest accuracies on the 140k dataset."},{"cited_title":"Computers & Graphics 32(2), 149–158 (2008)","cited_arxiv_id":null,"evidence_quote":"Provides DarkNet-53, the highest-accuracy model on the 140k dataset and a key speed comparison target."},{"cited_title":"In: Proceedings of the 26th LightFFDNets: Lightweight Convolutional Neural Networks for Rapid Facial Forgery Detection 13 Spring Conference on Computer Graphics, SCCG ’10, pp","cited_arxiv_id":null,"evidence_quote":"Provides GoogleNet, another pretrained baseline against which the lightweight models are compared."}],"review_version":1}