{"id":"ea441c5d-ecac-4948-87ad-0bf3e0ab097d","arxiv_id":"2412.09936","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"A LoRA-fine-tuned MiniGPT-v2 with USDA retrieval answers food and calorie questions from one image, but calorie accuracy is never measured.","lead":"This paper fine-tunes a vision-language model with LoRA on food images and adds a USDA nutrition lookup so a user can ask for calories from a single photo. It is a practical food-tracking application, but the paper never measures whether the calorie numbers are actually correct.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Fine-tuning is evaluated only with text-overlap metrics (ROUGE/BLEU/BERTScore); no comparison of predicted versus true calorie values is reported, so the central claim of accurate calorie estimation is unsupported.","rationale":"The reader's verdict identifies the same load-bearing weakness: text-overlap metrics are used as a proxy for calorie estimation accuracy, and no true calorie comparison is made. This is the central issue because the paper's stated purpose is calorie estimation from a single image. If the evaluation does not measure calorie correctness, the reported improvements in ROUGE and BLEU are irrelevant to the central claim. The dataset-size inconsistency in Section 3 is a secondary concern that further weakens trust in the evaluation setup, but the primary problem remains the missing calorie-level metric. The paper also provides no independent verification of its open-sourced code or data, and the RAG component is not evaluated in isolation. Given these gaps, the REJECT verdict is appropriate and no adjustment is needed. The concrete test proposed above would settle the concern by directly measuring calorie prediction error and isolating the RAG contribution.","tokens_in":7795,"tokens_out":1981,"duration_ms":26303,"concrete_test":"Construct a calorie-focused evaluation on the CalData test split: for each test image, take the recipe's ground-truth ingredient quantities and the USDA calorie conversion as the target calorie value; parse the model's calorie output from the generated text; report mean absolute error, mean absolute percentage error, and Pearson correlation between predicted and true calories, for both the baseline and fine-tuned models. Additionally, run the fine-tuned model with RAG disabled and compare calorie errors, to verify that the retrieval component contributes to any improvement. If the fine-tuned model's calorie error is not substantially lower than a trivial mean-prediction baseline, or if disabling RAG does not change the error, the paper's central accuracy claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim, stated in the abstract and Section 5.3, is that CaLoRAify provides accurate calorie estimation from a single monocular image and that fine-tuning improves accuracy. The only quantitative evidence, Table 1, reports ROUGE, BLEU, SacreBLEU, and BERTScore computed between generated text and reference text. These are surface-level text-overlap and embedding-similarity metrics; they do not measure whether the predicted calorie number matches the true calorie content. A model could reproduce the reference wording while outputting wrong calorie values, or it could output correct calories while paraphrasing poorly. The paper never reports mean absolute error, percentage error, correlation, or any calorie-level accuracy metric against ground-truth nutritional values. The RAG component in Section 4.2 is claimed to ground predictions in USDA data, but no experiment isolates or ablates RAG, and no retrieval accuracy or end-to-end calorie error attributable to RAG is reported. Section 3 also contains internal inconsistencies: it states 330K image-text pairs, then describes 5,801 unique samples with up to five images and five instruction sets, yielding roughly 145K pairs, while the stated split sums to 319,055 pairs and the relationship to 330K is unexplained. These inconsistencies undermine confidence in the dataset construction and in the reference texts used for the reported metrics. Because no calorie-level ground-truth evaluation exists, the central claim that fine-tuning improves calorie estimation accuracy is not supported by the evidence presented.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces CaLoRAify, a vision-language model framework for ingredient recognition and calorie estimation from a single monocular food image. The authors curate CalData, an image-text dataset built from Recipe1M+ with nutritional instructions and question paraphrases, and fine-tune MiniGPT-v2 using LoRA, with a RAG component that retrieves nutritional facts from a USDA database. The paper claims that the fine-tuned system improves accuracy over the baseline, citing text-overlap metrics in Table 1. The reported experiments, however, contain only ROUGE, BLEU, SacreBLEU, and BERTScore values, with an aggregate metric defined as a weighted average of ROUGE-L and BLEU. No calorie-level ground-truth evaluation is reported, and the dataset description contains internal numerical inconsistencies.","tokens_in":8117,"tokens_out":3882,"duration_ms":46430,"significance":"If validated, the system would address a practical need: calorie estimation from a single food image with conversational flexibility, supported by an openly released dataset and a RAG component grounded in USDA data. The authors are to be credited for curating a domain-specific dataset, for describing a clean LoRA fine-tuning recipe, and for committing to open-source code and data. However, the central claim of accurate calorie estimation is not supported by the evidence presented. The quantitative evaluation measures text overlap between generated and reference strings, not the correctness of predicted calorie values, and no ablation isolates the contributions of LoRA, RAG, or the dataset. The dataset size statements are mutually inconsistent, further weakening confidence in the evaluation. As presented, the paper does not establish its headline contribution.","major_comments":[{"comment":"The only quantitative evidence for the paper's central claim is Table 1, which reports ROUGE, BLEU, SacreBLEU, and BERTScore between generated text and reference text, plus an aggregate Lagg = λ_rouge L_rouge + λ_bleu L_bleu with unspecified weights. These are text-overlap and embedding-similarity metrics; they do not measure whether the predicted calorie number matches the true calorie content of the depicted dish. A model could reproduce reference wording while returning wrong calorie values, or return correct calories while paraphrasing poorly. The paper never reports mean absolute error, percentage error, correlation, or any calorie-level accuracy metric against ground-truth nutritional values. Consequently, both the abstract's claim of 'accurate calorie estimation' and Section 5.3's statement that fine-tuning 'improves the accuracy of the model' are unsupported by the reported data.","section":"§5.3, Table 1 and Eq. (1)"},{"comment":"The dataset size statements are internally inconsistent. The abstract and Section 3 state that CalData contains 330K image-text pairs, but Section 3 also reports 5,801 unique samples, an initial pool of 76,767 images, and a design with at most five images and five instruction sets per sample, which implies roughly 145K pairs. The stated split sizes sum to 191,433 + 63,811 + 63,811 = 319,055 pairs, which is neither 330K nor 145K. The sentence 'We split the dataset within each sample (recipe)' is also ambiguous and could imply that images from the same recipe appear in multiple splits, creating a leakage risk. Because CalData provides the reference texts used for all reported metrics, these unresolved numerical and procedural inconsistencies undermine the evaluation.","section":"§3, dataset construction"},{"comment":"The paper claims that the RAG module grounds calorie estimates in USDA nutritional data and mitigates hallucinations, but no experiment isolates or evaluates this component. There is no retrieval accuracy measure, no end-to-end comparison with and without RAG, and no ablation of the retrieved context's effect on the final output. As a result, the reported aggregate improvement cannot be attributed to RAG, LoRA, or the training data, and the specific contribution of retrieval to calorie accuracy is unverified.","section":"§4.2, RAG component"},{"comment":"The term 'accuracy' is used without definition, and the baseline is described only as 'before fine-tuning' while the intervention bundles dataset curation, LoRA adaptation, and RAG. The aggregate metric uses arbitrary weights λ_rouge and λ_bleu that are never specified, and no statistical significance, confidence intervals, or comparisons to prior calorie-estimation methods are provided. The qualitative results in Figure 3 are illustrative but lack ground truth and error analysis. The evaluation protocol therefore does not support the conclusion that the system is accurate or that the proposed method improves over a meaningful baseline.","section":"§5.3, evaluation protocol"}],"minor_comments":[{"comment":"There are several typos and formatting errors: 'receipt dataset' should be 'recipe dataset', 'recovered generation' should be 'Retrieval-Augmented Generation', and the heading 'Multi-Modal Large Language Models for Food.' is duplicated.","section":"Throughout"},{"comment":"The values of λ_rouge and λ_bleu in the aggregate metric are not reported; please specify them or justify why the weighted combination is meaningful.","section":"§5.3, Eq. (1)"},{"comment":"The input format template uses placeholders like [Task Identifier] and [/INST] without a concrete example; including one full training or inference example would clarify the method.","section":"§5.2"},{"comment":"Some citations for market claims and related results are to non-archival or difficult-to-access sources (e.g., the CalAI revenue claim and the LIMA-based motivation), and should be replaced or supplemented with verifiable references.","section":"References"},{"comment":"The qualitative examples in Figure 3 would be more informative if accompanied by the corresponding ground-truth nutritional values and a discussion of failure cases or error patterns.","section":"§5.4"}],"recommendation":"reject","confidential_remarks":"The manuscript's central claim is not supported by the reported evaluation, and the dataset description contains inconsistencies that would require substantial rework. A future version with calorie-level ground-truth evaluation, component ablations, and a corrected dataset description could be reconsidered, but in its current form the paper does not meet the bar for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the authors fine-tune MiniGPT-v2 with LoRA and RAG over USDA data for food-image QA and release everything. That open-source contribution is real. But the paper's core claim—better calorie estimation—is never actually measured. Table 1 reports ROUGE, BLEU, SacreBLEU, and BERTScore, which compare generated text to reference text. They do not compare predicted calories to true calories. A model that writes the right words while emitting wrong numbers scores the same as one that writes wrong words with right numbers. No MAE, no percentage error, no correlation. The aggregate \"metric\" is a weighted average of ROUGE-L and BLEU, with weights never reported, so the headline 8.16% is uninterpretable.\n\nThe dataset description also has arithmetic gaps. The abstract says 330K pairs; the stated split sums to 319,055; the construction text (5,801 samples, up to five images and five instruction sets) implies a number far from either. Since the reference texts are the ground truth for Table 1, this inconsistency weakens the only quantitative evidence.\n\nWhat the paper does well: curating CalData from Recipe1M+ with nutritional facts is genuinely useful, and the code and data are public. The qualitative examples look plausible. RAG is standard but applied sensibly. Citations are fine. The problem is that the central argument—that this pipeline \"improves accuracy\"—is unsupported as written.\n\nI'd send this back to the authors with a clear revision path: evaluate on a held-out split with ground-truth calorie values; report calorie-level error metrics; ablate RAG; reconcile dataset numbers. Without that, the paper is a well-intentioned engineering write-up, not a research contribution. A main-track referee would be wasting time on it. That said, the open-sourced assets and the public-health motivation make it worth a second look if revised. For a workshop or challenge paper it might be acceptable now; for a serious venue, desk reject is the right call.","headline":"A well-intentioned fine-tuning exercise with a useful open-source dataset, but the calorie-accuracy claim is unmeasured and the dataset numbers do not add up.","tokens_in":8628,"tokens_out":3698,"would_cite":false,"duration_ms":41723,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CaLoRAify: one food photo yields ingredient and calorie estimates","keywords":["Calorie estimation","Vision-language model","Low-rank adaptation","Retrieval-augmented generation","Food image recognition","Ingredient analysis","Image-text dataset","USDA nutrition database"],"falsifier":"Compute per-sample calorie error on a held-out set of food images with ground-truth kilocalorie values, for example mean absolute error and correlation between predicted and true calories, for both the baseline and fine-tuned models; if fine-tuning improves text-overlap metrics but not calorie error, the claimed accuracy improvement is not about calorie estimation.","tokens_in":7613,"feed_emoji":"🍽️","tokens_out":4916,"duration_ms":51842,"temperature":0.7,"pith_summary":"The paper aims to show that calorie estimation can be reduced to a single-image task by pairing vision-language training with retrieval. It introduces CaLoRAify, a MiniGPT-v2-based system fine-tuned with LoRA on CalData, a dataset of 330K image-text pairs built from Recipe1M+ and nutrition instructions, so that at inference a user supplies one food photo and receives ingredient quantities and calorie estimates. The authors argue this approach replaces traditional multi-step pipelines that depend on reference objects, depth maps, or multi-view cameras, and that retrieval-augmented generation grounded in the USDA database reduces hallucinated nutrition values. If the central claim holds, food-photo calorie tracking could become practical on ordinary mobile hardware without extra metadata.","feed_headline":"Single food photo yields ingredient and calorie estimates","feed_subtitle":"CaLoRAify fine-tunes a vision-language model on 330K image-text pairs and grounds final numbers in USDA nutrition data.","key_machinery":"The central object is CalData, a dataset of 330K image-text pairs created by sampling 5,801 recipes from Recipe1M+, pairing each with up to five images and five instruction sets, and adding nutrition facts. The carrying mechanism is the CaLoRAify inference chain: a frozen Vision Transformer encodes the image, the LLaMA-2 backbone under a [vqa] task identifier generates ingredient and quantity predictions, LoRA (rank 64, alpha 16) adapts the model efficiently, and a Sentence-BERT encoder plus a USDA vector database retrieves nutrition facts that are concatenated back into the generation context.","core_discovery":"The paper claims that a vision-language model fine-tuned on the CalData image-text pairs can accept a single monocular food image at inference and output ingredient names, quantities, and a calorie estimate. Ingredient recognition is used as an intermediate step, and a retrieval-augmented generation module then queries the USDA nutritional database before the model produces its final answer. The paper reports that fine-tuning improves text-overlap metrics over the unfine-tuned MiniGPT-4 baseline, with the reported aggregate metric rising 8.16%, and takes this as evidence of improved accuracy for calorie estimation.","pith_inferences":["Editorial extension: if the ingredient-recognition step misnames an ingredient, the retrieval step will faithfully return nutrition facts for the wrong food, so calorie error will inherit ingredient error; a direct end-to-end calorie-error evaluation would reveal this coupling.","Editorial extension: because ROUGE and BLEU reward phrase overlap with template reference answers, part of the reported gain may reflect the model learning answer formatting rather than better visual calorie perception; testing on paraphrased calorie answers would disentangle the two.","Editorial extension: real phone photos have varied lighting, angles, and occlusions, so the relative gains observed here could be re-tested on a naturally captured meal dataset to see whether they persist outside Recipe1M+ imagery.","Editorial extension: the two-stage design uses one retriever for nutrition facts, but a comparative test of different retrievers or different nutrition databases could show how much of the final calorie estimate depends on retrieval quality versus the vision-language model's ingredient prediction."],"forward_implications":["Users can photograph a dish with an ordinary phone and receive ingredient names, quantities, and a calorie estimate, with no reference object, depth map, or multi-view rig.","Because the RAG module retrieves nutrition facts from the USDA database using the predicted ingredients, the final calorie number is grounded in external data rather than generated purely from memory.","Fine-tuning with LoRA on 330K image-text pairs improves the model's text-overlap scores against the unfine-tuned baseline, including ROUGE-L, BLEU, and BERTScore.","The same visual-text training and retrieval pipeline can be adapted to other food-related vertical tasks, such as recipe generation or personalized dietary advice.","The system retains conversational interaction, so users can ask follow-up questions about the estimated meal rather than receiving only a static number."],"supporting_citations":[{"why":"Supplies the MiniGPT-v2 backbone and task-identifier architecture that CaLoRAify adapts for food VQA.","marker":"[7]"},{"why":"Provides the Recipe1M+ source recipes and images from which CalData's 330K pairs are derived.","marker":"[22]"},{"why":"Supplies the LoRA method used to fine-tune the LLM efficiently.","marker":"[12]"},{"why":"Supplies the retrieval-augmented generation paradigm for grounding outputs in external knowledge.","marker":"[17]"},{"why":"Provides the USDA nutritional database that the RAG module queries for calorie facts.","marker":"[5]"},{"why":"Supplies the Sentence-BERT encoder used to embed retrieval queries against the nutrition database.","marker":"[25]"},{"why":"Provides the rephrasing model used to diversify the question set during data augmentation.","marker":"[30]"},{"why":"Defines the MiniGPT-4 baseline model that the paper fine-tunes and compares against.","marker":"[32]"}],"fun_headline_variants":["One food photo estimates calories and ingredients","LoRA-tuned VLM counts calories from a single image","Calorie estimate from one photo via LoRA and RAG","Single image, ingredient breakdown, calorie total","Photo to calories: LoRA + RAG on 330K pairs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim depends on text-overlap metrics standing in for calorie accuracy, because the paper's only quantitative evidence is ROUGE, BLEU, and BERTScore against reference text and includes no comparison between predicted and true calorie values.","fun_headline_variants_meta":{"raw":{"variants":["One food photo estimates calories and ingredients","LoRA-tuned VLM counts calories from a single image","Calorie estimate from one photo via LoRA and RAG","Single image, ingredient breakdown, calorie total","Photo to calories: LoRA + RAG on 330K pairs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000198,"raw_usage":{"total_tokens":1341,"prompt_tokens":893,"completion_tokens":448,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":509,"completion_tokens_details":{"reasoning_tokens":369}},"tokens_in":509,"tokens_out":448,"duration_ms":4696,"temperature":1.0,"reasoning_tokens":369,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T16:32:37.405069+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute per-sample calorie error on a held-out set of food images with ground-truth kilocalorie values, for example mean absolute error and correlation between predicted and true calories, for both the baseline and fine-tuned models; if fine-tuning improves text-overlap metrics but not calorie error, the claimed accuracy improvement is not about calorie estimation.","supporting_citations":[{"cited_title":"https://fdc.nal.usda","cited_arxiv_id":null,"evidence_quote":"Provides the USDA nutritional database that the RAG module queries for calorie facts."},{"cited_title":"Sentence-bert: Sentence embeddings using siamese bert-networks","cited_arxiv_id":null,"evidence_quote":"Supplies the Sentence-BERT encoder used to embed retrieval queries against the nutrition database."},{"cited_title":"Harris, and Marcel Carlsson","cited_arxiv_id":null,"evidence_quote":"Provides the rephrasing model used to diversify the question set during data augmentation."}],"review_version":1}