{"id":"1a6754cb-eede-483b-9bff-de48143ef680","arxiv_id":"1909.05620","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A CNN regresses the four edges of a bounding box from loose pre-labels, lifting the fraction of edges within 1% tolerance on Cityscapes Faster-RCNN boxes from 25.1% to 37.7%.","lead":"DeepBbox is a deep network that tightens rough bounding boxes around objects in driving images, reducing the need for human annotators to redraw them. The paper reports that on Cityscapes it raises the share of automatically precise box edges from 25.1% to 37.7% within a 1% tolerance.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline improvement is per-edge, not per-box; no time or box-level metric is reported, so the claim that 50% more precise edges reduces manual annotation time is not established.","rationale":"The reader's weakest assumption about Gaussian synthetic patches is reasonable but, in my reading, not the most load-bearing point. The main Table III result is obtained on actual Faster-RCNN pre-labels, and the mismatch-robustness columns provide some evidence against distribution overfitting. The bigger gap is between the measured quantity (per-edge tolerance) and the claimed benefit (reduced manual annotation time). The paper itself defines the contribution as increasing \"percentages of per-label edges that are precise and do not need human effort\" and then in the abstract states this reduces manual annotation time. That leap is the load-bearing inference. I do not think the paper is wrong; I think the evidence is incomplete. A box-level reanalysis or small timing study would settle it. This does not change the CONDITIONAL verdict, but it identifies a sharper condition than the one the reader flagged.","tokens_in":7304,"tokens_out":6929,"duration_ms":70381,"concrete_test":"Recompute Table III at box level: on the Cityscapes test set, count the fraction of Faster-RCNN pre-label boxes whose all four edges are within 1% of the longest edge before and after DeepBbox, and apply a simple annotation cost model (e.g., a box with any edge out of tolerance costs c seconds to fix; otherwise 0). If the modeled time saving is far below the implied 50% edge improvement, or if the box-level no-correction rate increases by less than 10 relative points, the headline claim should be narrowed to per-edge accuracy and the annotation-time conclusion dropped. Ideally, run a small user study timing annotators on raw vs. corrected pre-labels.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central numerical result, 25.1% to 37.7% (Section IV-B, Table III), is a per-edge percentage, not a per-box percentage. The abstract and conclusion translate it into \"50% more bounding box edges do not have to go through human correction\" and claim this reduces manual annotation time. This inference is not supported: annotation correction is done at box level, and a box with any out-of-tolerance edge still requires a human pass. The paper reports no box-level fraction of fully precise boxes and no direct time measurement. For independent edges, all-four-edges-within-1% goes from roughly 0.25^4 ≈ 0.4% to 0.38^4 ≈ 2.0% of boxes; even a five-fold relative increase leaves about 98% of boxes needing correction, so the edge-level gain is not evidence of a comparable reduction in annotation effort. Unless the paper shows box-level no-correction rates or measures annotator time, the central practical claim is unverified.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DeepBbox, a convolutional network that takes an image patch cropped around a pre-labeled bounding box and regresses the four edges of a tight bounding box for the main object. The network is trained on Cityscapes by expanding ground-truth boxes and perturbing their edges with Gaussian noise that is fitted to Faster-RCNN edge-error statistics, and it is evaluated on real Faster-RCNN pre-labels on Cityscapes test data, on a transfer setting on BDD, and on a few tracker examples from Caltech. The headline result is that the percentage of bounding-box edges falling within 1% of the longest edge rises from 25.1% to 37.7% after DeepBbox correction, which the paper interprets as a 50% increase in edges that need no human correction and hence a reduction in annotation time.","tokens_in":7508,"tokens_out":3215,"duration_ms":31712,"significance":"If the reported improvement is robust, the method is a useful component for semi-automatic annotation pipelines: it is evaluated on real detector pre-labels rather than only simulated ones, it compares three feature extractors, and it reports a low-data fine-tuning experiment (Fig. 5) that supports early adoption during annotation. The paper also honestly states known limitations, including occlusion, and discusses alternative correction methods. However, the central practical claim that the edge-level gain translates into a meaningful reduction in manual annotation effort is not directly established, which limits the significance until that link is quantified.","major_comments":[{"comment":"The headline improvement is per-edge, but human annotation correction is performed at the box level: a box with any out-of-tolerance edge still requires a human pass. If the four edges were independent, the all-edges-within-1% box rate would rise from roughly 0.25^4 = 0.4% to roughly 0.38^4 = 2.0%, leaving about 98% of boxes needing correction. The paper reports no box-level no-correction rate and no measured annotation time, so the abstract and conclusion statement that 50% more edges 'do not have to go through human correction' and that this reduces manual annotation time is not supported as stated. Please report the fraction of boxes whose four edges are all within tolerance before and after correction, and, if possible, annotator time for a small user study.","section":"Section IV-B, Table III"},{"comment":"Training patches are synthesized by expanding ground-truth boxes and shifting edges with Gaussian noise whose variances are fitted to Faster-RCNN on Cityscapes train, and the main test set is real Faster-RCNN pre-labels from the same distribution. This is a favorable setting. The mismatch experiments only scale the same fitted Gaussian by plus or minus 30%, which does not test systematic bias, correlated edge errors, class-dependent errors, or a different detector/tracker distribution. The claim that DeepBbox 'can be applied to various video annotation pipelines' is therefore only weakly supported. Please report the actual error statistics of the corrected pre-labels (e.g., per-edge bias and correlation), and evaluate on at least one distinct source of pre-labels or on a synthetic distribution with correlated or biased edge shifts.","section":"Section III-B, Section IV-B"},{"comment":"All numeric results appear to come from a single training run, with no variance across initializations, no confidence intervals, and no significance tests. The differences among VGG16, MobileNet, and ResNet50 are within about two percentage points at the 1% tolerance, so the reported architecture ranking may not be robust. Please report means and standard deviations over at least three to five runs or use bootstrap resampling over test objects.","section":"Section IV-B, Tables II and III"}],"minor_comments":[{"comment":"The abstract says 'within 1% error' but the precise definition is that the absolute error is within 1% of the true bounding box's longest edge; Section IV-A also says vertical and horizontal edge errors are normalized by width and height respectively. Please make the normalization used in Table III explicit and consistent.","section":"Abstract and Section IV-A"},{"comment":"There are recurring typos, including 'pre-lables' instead of 'pre-labels' in several places and 'ground through bounding boxes' in Section IV-A. Please proofread the manuscript.","section":"Throughout"},{"comment":"The description of the BDD masks is confusing: the text says BDD contains fine-grained object segmentation masks but 'does not contain object instance segmentation.' Please clarify what the masks contain and how connected pedestrian pixels were converted to instance-level tight boxes without instance IDs.","section":"Section IV-A"},{"comment":"The fixed expansion ratio used to enlarge the ground-truth boxes is not given numerically. Please report the value used in the experiments.","section":"Section III-B, Fig. 3"},{"comment":"In Fig. 5, it is unclear whether the x-axis is the fraction of images or the fraction of bounding boxes used for fine-tuning. Please clarify the axis and the training protocol.","section":"Section IV-B, Fig. 5"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is an unpub- lished-looking arXiv preprint, and for a journal venue the novelty section should be strengthened in light of subsequent work on label refinement and semi-automatic annotation; as written, the per-edge evaluation does not yet justify the practical claims. No code or data release is mentioned, which limits reproducibility. These issues are secondary to the box-level and robustness concerns in the major comments."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: DeepBbox is a straightforward regressor that tightens loose boxes, and the 25.1% to 37.7% edge-precision improvement on Cityscapes Faster-RCNN pre-labels appears to be a genuine empirical result. The paper's application-level claim, that this reduces manual annotation time, is not established. The stress-test note is correct: that number is per edge, not per box, and the paper never reports the box-level fraction that still needs human correction. If independent, all-four-edges-within-1% goes from about 0.4% to 2.1% of boxes. Even a five-fold relative increase leaves ~98% of boxes requiring a human pass. So the abstract's 'reduce manual annotation time' is an inference, not a result.\n\nWhat is new: the synthetic-error augmentation—fitting Gaussians to the detector's edge errors and using them to perturb ground-truth boxes during training—is a sensible way to decouple DeepBbox from a specific detector. Showing that MobileNet and VGG16 reach similar correction quality, and that fine-tuning with half the Cityscapes set gives comparable results, is useful for annotation pipelines. The limitations section is honest about occluded objects and clearly describes alternatives.\n\nSoft spots, in proportion. The missing box-level metric is the biggest one. No time measurement, no annotator study, no baseline comparison against simpler methods (e.g., thresholding a segmentation mask, or iterative regression). The BDD experiment is tiny (652 boxes) and the transfer result without fine-tuning is marginal. There are no error bars or statistical tests anywhere. The citation for Faster R-CNN points to the Fast R-CNN paper. The matched-error setting is favorable, since training and test share the same detector statistics, but not circular: the paper also shows mismatched-error performance.\n\nBottom line: this is a plausible piece of engineering with a real but contained result. It is not a strong paper as written, but it deserves referee time rather than a desk reject, because the core idea is relevant to anyone building annotation tooling and the synthetic-error trick is worth examining. I would send it to review with a clear request for box-level no-correction rates, a time proxy, and at least one baseline.","headline":"The edge-level gain is real, but the paper's practical claim that this meaningfully cuts annotation time does not survive the per-edge/per-box gap.","tokens_in":8017,"tokens_out":2553,"would_cite":false,"duration_ms":24969,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a small network, DeepBbox, corrects rough detector boxes so precisely that 50% more bounding-box edges fall within 1% error and can skip human correction.","keywords":["bounding box refinement","ground truth annotation","autonomous driving","object detection pre-labels","coordinate regression","synthetic training patches","transfer learning","pedestrian detection"],"falsifier":"Measure the actual edge-error distribution of several object detectors and trackers across driving datasets and compare it with the independent Gaussian assumption; if a common detector shows biased or strongly correlated edge errors, retrain DeepBbox on the true distribution and check whether the percentage of edges within 1% error still beats the raw pre-labels. If that improvement disappears, the synthetic-error premise is what carried the result.","tokens_in":7113,"feed_emoji":"🚗","tokens_out":7067,"duration_ms":67254,"temperature":0.7,"pith_summary":"This paper tries to establish that a deliberately small neural network can take loose bounding boxes produced by an object detector and turn them into tight boxes, reducing the manual work of labeling driving footage. The headline number is that the fraction of bounding-box edges accurate to within 1 percent of the longest edge rises from 25.1% for raw Faster-RCNN pre-labels to 37.7% after DeepBbox correction, i.e., about 50% more edges would not need a human annotator to touch them. The motivation is cost: precise labels for datasets with millions of boxes cost tens of thousands of human hours, so any correction that survives review has direct practical value. The paper also claims the network needs only a small fine-tuning set of about 7.6 thousand boxes when moving to a new dataset.","feed_headline":"50% more bounding box edges need no human correction","feed_subtitle":"A small regression network tightens rough detector boxes, lifting precise edges from 25.1% to 37.7%.","key_machinery":"The engine of the method is a synthetic training procedure that turns ground-truth boxes into realistic loose patches. For each fully visible object, DeepBbox expands the true box, perturbs each edge independently with a Gaussian whose variances are fitted to Faster-RCNN's observed error ratios ($\\mathcal{N}(0, 0.0064)$ for vertical edges and $\\mathcal{N}(0, 0.0196)$ for horizontal edges), crops the patch, scales it to 256x256 while preserving aspect ratio and padding with zeros, and feeds it to a convolutional feature extractor followed by three fully connected layers that regress the two opposite corners. Huber loss replaces pure L1 to keep optimization smooth. Because the perturbation distribution, not any particular detector, defines the task, the same trained model can be applied to different detectors and trackers.","core_discovery":"In the paper's account, the central discovery is that a comparatively simple regression network, trained on synthetically perturbed crops, can make loose bounding boxes tight enough that 50% more edges meet a 1% precision tolerance than the raw pre-labels do. On the Cityscapes pedestrian test set, the share of edges within 1% of the longest edge rises from 25.1% for Faster-RCNN pre-labels to 37.7% with DeepBbox using a MobileNet feature extractor, while mean absolute edge error relative to the longest edge falls from 4.25% to 3.34%; a VGG16 variant gives 35.5% and a similar error drop. The paper frames this as an annotation-cost result: more pre-labels would pass review untouched, and the network needs only about 7.6 thousand boxes to be fine-tuned to a new dataset such as BDD, where it still improves on raw Faster-RCNN pre-labels.","pith_inferences":["If the per-edge Gaussian error model were replaced by a distribution that also captures correlations among edges, such as systematic detector bias shared by vertical edges, the same training procedure might handle detector families the current setup does not test; the paper does not investigate correlated errors.","The 1%-edge metric is an edge-level statistic, so a direct claim about saved annotator hours would require an annotator study that converts edge-level pass rates into per-box review time; this paper does not include such a study.","The same regression target could be redefined for rotated boxes or polygon outlines by changing the four-coordinate output to other corner or vertex coordinates, though the paper's experiments stop at axis-aligned boxes on pedestrians."],"forward_implications":["At the reported 1% tolerance, roughly half again as many bounding-box edges survive without human correction, so the per-box review workload drops for the same precision standard.","Because about 7.6 thousand annotated boxes suffice to fine-tune the network to a new dataset, DeepBbox can be inserted early in an annotation pipeline rather than after the dataset is already labeled.","The method's relative insensitivity to a 30% mismatch in the assumed error size suggests one model can serve multiple detectors or trackers whose error scales differ.","Making key-frame and intermediate-frame pre-labels tighter before human review means fewer boxes need to be re-drawn, which translates directly into saved annotation time."],"supporting_citations":[{"why":"Supplies the Cityscapes fine-grained segmentation-derived boxes used as ground truth for training and testing.","marker":"[1]"},{"why":"Faster-RCNN is the pre-label generator whose edge error statistics are measured and corrected in the main scenario.","marker":"[13]"},{"why":"BDD provides the transfer dataset and the reported pre-labeling annotation-time reduction that motivates the method.","marker":"[3]"},{"why":"Caltech Pedestrian Detection dataset is used for the illustrative tracker pre-label correction examples.","marker":"[14]"}],"fun_headline_variants":["DeepBbox tightens loose boxes, cutting manual annotation time","50% more precise edges with DeepBbox, less human fixes","DeepBbox: Turn loose labels into precise boxes fast","Annotation made easier: DeepBbox corrects 50% more edges","Squeeze more precision out of loose boxes with DeepBbox"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that synthetic training patches, made by expanding true boxes and shifting each edge by random amounts chosen to mimic Faster-RCNN's typical error sizes, resemble the real loose boxes DeepBbox will be asked to fix.","fun_headline_variants_meta":{"raw":{"variants":["DeepBbox tightens loose boxes, cutting manual annotation time","50% more precise edges with DeepBbox, less human fixes","DeepBbox: Turn loose labels into precise boxes fast","Annotation made easier: DeepBbox corrects 50% more edges","Squeeze more precision out of loose boxes with DeepBbox"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000451,"raw_usage":{"total_tokens":2240,"prompt_tokens":880,"completion_tokens":1360,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":496,"completion_tokens_details":{"reasoning_tokens":1271}},"tokens_in":496,"tokens_out":1360,"duration_ms":9805,"temperature":1.0,"reasoning_tokens":1271,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T10:14:12.450234+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the actual edge-error distribution of several object detectors and trackers across driving datasets and compare it with the independent Gaussian assumption; if a common detector shows biased or strongly correlated edge errors, retrain DeepBbox on the true distribution and check whether the percentage of edges within 1% error still beats the raw pre-labels. If that improvement disappears, the synthetic-error premise is what carried the result.","supporting_citations":[{"cited_title":"Pedestrian detection: A benchmark","cited_arxiv_id":null,"evidence_quote":"Caltech Pedestrian Detection dataset is used for the illustrative tracker pre-label correction examples."}],"review_version":1}