{"id":"36d7583e-9f1c-4cd0-b244-de85f12a7ea3","arxiv_id":"2506.14271","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Leader360V provides a 10,000+ video, 198-class, densely annotated 360-degree video dataset with an LLM-assisted automatic annotation pipeline, and shows fine-tuning on it improves 360 video segmentation and tracking models.","lead":"This paper presents Leader360V, a collection of more than 10,000 real-world 360-degree videos with dense instance segmentation and tracking annotations, built by combining existing 360 video data, online videos, and new recordings. A smart generalist might read it because large panoramic datasets are a missing ingredient for training perception models for robots, autonomous vehicles, and AR/VR systems.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Evaluation split in Sec. 4.1 lets 66% of test clips share source videos with training clips, so the reported fine-tuning gains may reflect scene memorization rather than generalization; the central empirical claim is not yet supported.","rationale":"The reader's weakest assumption correctly identifies the Section 4.1 evaluation split as the main load-bearing concern: 66% of validation and test clips are clipped from original train-set videos, so the fine-tuning gains on the test set may be inflated by source-video-level leakage. The paper's central contribution is not merely the dataset's existence but its demonstrated value for training models, and that demonstration currently rests on a split that does not establish generalization to new 360 scenes. This is a fixable issue rather than a fundamental invalidation, so a conditional disposition remains appropriate. I also note supporting concerns: the dataset and code are not yet released, and the data-source counts in Tables 1 and 10 are internally inconsistent (e.g., 120 vs. 232 videos for the same source). These reinforce the need for release with corrected statistics and a leakage-free benchmark before the headline claims can be fully verified. No additional independent concern is needed: if the video-disjoint re-evaluation preserves the gains, the core claim would be substantially supported; if not, the empirical contribution would need to be reframed.","tokens_in":21609,"tokens_out":2525,"duration_ms":24338,"concrete_test":"Release the dataset with source-video IDs and recompute the main experiments (Tables 4, 5, 6, 7) using a video-disjoint split: no two clips in different splits may originate from the same source recording, and no clip may be a temporal crop overlapping a training clip. Report J&F and Sdual/Pdual on the test set under this split, along with overlap statistics for the current split. If the fine-tuning gains persist under the disjoint split (e.g., PSCFormer still gains more than 25 J&F over its untrained baseline), the generalization claim is supported; if gains shrink substantially, the reported numbers should be revised and the claim softened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract's claim that Leader360V 'significantly enhances model performance' is supported primarily by Tables 4-7, which show large gains after fine-tuning on the Leader360V training subset. Section 4.1 (Evaluation Subset) states that for the 125-video validation and 125-video test sets, '66% of the clips are clipped from the original train set videos as val and test sets.' If a test clip is truncated from the same source recording as a training clip, the model has already seen the same scene, static background, lighting, and often the same object instances during training. The reported +36.3 J&F (PSCFormer, Table 4), +19.5 (SwinB-AOT-L, Table 4), and +12.6 Sdual (SimTrack, Table 5) could then be inflated by memorizing source-video-specific content rather than by learning a generalizable 360 video representation. The paper neither reports source-video IDs nor evaluates a video-disjoint split, so the central empirical claim is not currently established. A secondary issue is the internal source-count inconsistency (Table 1 lists 360VOT with 120 videos, while Table 10 reports selecting 232 videos from 360VOTS at 80%), but the leakage issue alone is sufficient to condition the verdict.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Leader360V, presented as the first large-scale (10K+) real-world 360 video dataset with instance segmentation and tracking annotations covering 198 object categories, built from a combination of existing 360 video datasets, online collections, and self-captured footage. To reduce annotation cost, the authors propose A3360V, a three-phase automatic annotation pipeline that combines pre-trained 2D segmentors, SAM2, and LLM-based checkers to produce and refine masks. Experiments on a 500-video sample and domain-transfer settings report substantial improvements on 360 VOS and VOT benchmarks after fine-tuning on Leader360V. The paper also includes user studies and ablations to support the pipeline's effectiveness.","tokens_in":21881,"tokens_out":7285,"duration_ms":62213,"significance":"If the dataset and pipeline claims are substantiated, Leader360V would be a valuable community resource, addressing the scarcity of large-scale annotated 360 video for segmentation and tracking, and the A3360V pipeline could lower annotation costs. The paper's strengths include a modular pipeline design, a broad 198-category taxonomy, explicit failure-case discussion in Appendix C.7, and a user study with multiple skill levels. However, the current evaluation protocol introduces potential video-level leakage between training and test clips, and internal dataset statistics are inconsistent, so the headline performance gains and dataset composition claims are not yet established.","major_comments":[{"comment":"The split protocol states that 66% of validation and test clips are clipped from the original train set videos, meaning the same source recordings appear in both training and evaluation. This creates a video-level leakage that can inflate the reported fine-tuning gains (e.g., +36.3 J&F for PSCFormer in Table 4, +12.6 Sdual for SimTrack in Table 5) by allowing models to memorize scene-specific backgrounds and object instances. The paper does not report source-video IDs or a video-disjoint split, so the central claim that Leader360V significantly enhances model performance is not currently supported.","section":"Sec. 4.1 (Evaluation Subset)"},{"comment":"Appendix B.1 states that the 360VOTS dataset comprises 120 high-resolution video sequences, yet Table 10 reports selecting 232 videos from 360VOTS at an 80% selection rate. This is arithmetically impossible and indicates a factual error in the dataset composition statistics. The related row labeled 'Self-Collected 360VOTS' is also ambiguous. The authors should correct these entries and provide a source-by-source reconciliation that sums to the claimed 10,180 videos.","section":"Table 10 and Sec. B.1"},{"comment":"The ablation study evaluates Phase I and Phase II outputs against 'the final annotations' as ground truth, but those final annotations are themselves the output of the same pipeline after human revision in Phase III. This is a self-consistency check rather than an external validation of annotation quality. The claim that A3360V produces high-quality annotations would be strengthened by a comparison with a fully manual annotation set or by reporting inter-annotator agreement between Phase III outputs and independent human annotations.","section":"Sec. 4.3 / Table 8"},{"comment":"No dataset release artifacts or project page are provided in the paper. For a dataset contribution of this type, a download link, benchmark server, and code/data documentation are essential for the stated claims to be verifiable and usable by the community. The authors should clearly state data availability and provide these artifacts.","section":"General (Dataset Release)"}],"minor_comments":[{"comment":"In the abstract, 'appications' should be 'applications', and the phrase 'the first large-scale (10K+), labeled real-world 360 video datasets' mixes singular and plural; it should be 'dataset'.","section":"Abstract"},{"comment":"Table 1 uses the citation [10] for '360VOT', but the tracking benchmark 360VOT is correctly cited as [13] in the introduction; the table also seems to conflate 360VOT and 360VOTS.","section":"Table 1"},{"comment":"The heading of Table 4 and Table 5 says 'Qualitative comparison', but the content is quantitative; the heading should say 'Quantitative comparison'.","section":"Tables 4 and 5"},{"comment":"The evaluation subset description is ambiguous: it is unclear whether the 500-video sample includes the 250 training videos plus 125 val plus 125 test, and the relationship between 'clips' and 'videos' should be clarified; the paper should report the exact overlap in source video IDs between the training and test splits.","section":"Sec. 4.1"},{"comment":"In Table 10, the 'Source*' column includes a row 'Self-Collected 360VOTS' which appears to be a typographical error; this row should likely be 'Self-Collected' only, and the table's 'Task' and 'Pct' fields for open-source and self-collected rows are vague.","section":"Table 10"},{"comment":"Tables 6 and 7 have a confusing header structure for the 'Training Dataset' columns; please clarify what 'Self-Collected' means in those tables (likely the Leader360V fine-tuning data) and ensure the column labels are self-explanatory.","section":"Tables 6 and 7"}],"recommendation":"major_revision","confidential_remarks":"The evaluation leakage in Section 4.1 is the most serious issue and should be the primary focus of the revision; the source-count inconsistency in Table 10 also needs correction. If the authors can provide a video-disjoint evaluation split and reconcile the dataset numbers, the paper could become acceptable. The lack of any release artifacts is also a concern for a dataset paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Leader360V could be a genuinely useful resource. Ten thousand-plus real-world 360 videos with dense instance masks and tracks, 198 classes, assembled from existing datasets plus self-collected footage – that is a real step up from PanoVOS and 360VOT. The A3360V pipeline is also a reasonable engineering contribution: coordinating multiple segmentors with LLM-based label checking and SAM2 propagation, and the appendix documents failure cases honestly. If the dataset ships with clean splits and corrected statistics, it deserves a place in the 360-video toolbox.\n\nThe problem is that the paper currently does not support its central empirical claim. Section 4.1 states that 66% of the validation and test clips are clipped from the original train set videos. Unless the source recordings are disjoint between splits, the fine-tuning gains in Tables 4–7 (up to +36.3 J&F) could reflect memorization of the same scenes rather than generalization to new 360 content. The paper gives no source-video IDs and no video-disjoint evaluation, so the performance claim is not established. This is a load-bearing flaw, not a nitpick.\n\nThere is also an internal inconsistency: Table 10 reports selecting 232 videos from 360VOTS, while Appendix B.1 says 360VOTS has 120 sequences. That is a nearly two-fold discrepancy. One of the two tables is wrong. And the pipeline validation in Table 8 uses the final manual-refined annotations as ground truth – but those annotations are themselves the output of the same pipeline plus human edits. That is a self-consistency check, not an external validation. The user study provides some independent signal, but it is limited and does not fix the circularity.\n\nWho is this for? People who want to train or evaluate 360-video segmentation and tracking models. The resource is important enough that it should go to peer review, not be desk-rejected. But the reviewers should require a video-disjoint test split, corrected source counts, and a clear release plan before the headline claims are accepted. If those are fixed, this becomes a solid dataset paper; as written, it is a promising prototype with two fixable but serious flaws.","headline":"An important 360 video dataset in the making, but the current eval split leaks source videos between train and test, and the source counts are inconsistent; the headline performance claim isn't yet supported.","tokens_in":22443,"tokens_out":1960,"would_cite":false,"duration_ms":19990,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Leader360V claims to be the first 10K+ labeled real-world 360-degree video dataset for segmentation and tracking.","keywords":["360 video dataset","video object segmentation","multi-object tracking","panoramic understanding","automatic annotation","large language models","SAM2","instance segmentation"],"falsifier":"Train a segmentation or tracking model on the Leader360V training clips and evaluate it on a holdout of videos whose source videos never appear in training; if the J&F or Sdual gains shrink to the level of the untrained baselines, the dataset's claimed value as a generalization resource would be refuted.","tokens_in":21442,"feed_emoji":"🎥","tokens_out":3769,"duration_ms":33392,"temperature":0.7,"pith_summary":"The paper introduces Leader360V, a claimed first large-scale real-world 360 video dataset with over 10,180 videos and 198 object categories, annotated for both instance segmentation and tracking. To build it affordably, the authors propose A3360V, an automatic annotation pipeline that combines pre-trained 2D segmentors with large language models, followed by manual refinement. The central claim is that fine-tuning existing models on Leader360V significantly improves their performance on 360 video segmentation and tracking. A sympathetic reader would care because it addresses a bottleneck: the lack of large, labeled panoramic video data for foundation-model-era training.","feed_headline":"10,180 annotated 360 videos promise better segmentation and tracking","feed_subtitle":"The paper reports fine-tuning on Leader360V lifts panoramic segmentation J&F by up to 36 points and tracking by 12.6 points.","key_machinery":"The central object is the A3360V annotation pipeline, whose load-bearing component is the Semantic- and Distortion-aware Refinement (SDR) module. SDR merges patch-wise mask proposals from multiple 2D segmentors, uses an LLM-based semantic label checker to unify labels, and feeds the resulting masks as prompts into SAM2, with a mask-prompt shifting step that selects the most consistent distortion-aware mask. The pipeline then propagates annotations through frames, using a coverage-rate threshold to trigger a Motion-Continuity Refinement module that resolves ERP border discontinuities, missing masks, and new objects before a final LLM-guided human revision pass.","core_discovery":"On its own terms, the paper establishes Leader360V as the first 10K+ labeled real-world 360 video dataset for instance segmentation and tracking, spanning 198 object types across indoor, urban, natural, and dynamic outdoor scenes. It further claims that its three-phase A3360V annotation pipeline—initial annotation with a Semantic- and Distortion-aware Refinement module, auto-refinement with a Motion-Continuity Refinement module, and LLM-assisted manual revision—produces high-quality annotations at a fraction of manual cost. Experiments on segmentation and tracking benchmarks report that models fine-tuned on Leader360V gain, for example, +36.3 J&F for PSCFormer and +12.6 Sdual for SimTrack over their untrained counterparts.","pith_inferences":["If the leakage concern is real, the headline gains may overstate the benefit; a video-disjoint evaluation could still rank the dataset fairly but with smaller numbers.","The pipeline's reliance on SAM2 and GPT-4o means its quality ceiling may track improvements in those foundation models, so the dataset could become richer automatically as base models improve.","Because the taxonomy merges rare classes from COCO, ADE20K, and Cityscapes, models trained on Leader360V may inherit coarse-grained labels that limit fine-grained recognition; this trade-off is worth testing explicitly.","The 360-degree nature of the data makes it a natural testbed for evaluating whether models learn true spherical consistency rather than merely ERP-specific artifacts."],"forward_implications":["Fine-tuning on Leader360V becomes a practical recipe for adapting 2D video segmentation models to 360 input, closing part of the domain gap from distortion and border discontinuity.","The dataset provides a unified mask-and-box annotation resource that can support training of 360-specific instance segmentation and multi-object tracking models end to end.","The A3360V pipeline offers a reusable template for scaling annotation of other 360 video collections without prohibitive human effort.","Performance comparisons on Leader360V can serve as a new benchmark for panoramic VOS and VOT, complementing the smaller 360VOTS and PanoVOS sets."],"supporting_citations":[{"why":"Provides the 360VOTS benchmark, which supplies prior dataset content, evaluation protocol, and baseline numbers for 360 visual object tracking and segmentation.","marker":"[10]"},{"why":"Provides the PanoVOS dataset and the PSCFormer baseline, which serves as the 360 VOS comparison point and a source of selected videos.","marker":"[5]"},{"why":"Supplies SAM2, the video foundation model used as the mask-prompt backbone for propagation and refinement in the A3360V pipeline.","marker":"[12]"},{"why":"Supplies CropFormer, the entity segmentation model that generates class-agnostic instance mask proposals in the SDR module.","marker":"[15]"},{"why":"Supplies OneFormer, the panoptic segmentor whose class-aware predictions feed the semantic label checker in the pipeline.","marker":"[16]"},{"why":"Supplies GPT-4o, the large language model used for semantic label checking, blank area classification, object retrieval, and annotation feedback.","marker":"[40]"},{"why":"Supplies YouTubeVOS as the planar-domain training source in the domain-transfer experiments that motivate the need for panoramic training data.","marker":"[51]"},{"why":"Supplies TrackingNet as the planar-domain training source in the tracking domain-transfer experiments that quantify the 360-domain gap.","marker":"[55]"}],"fun_headline_variants":["10,180 real-world 360 videos lift segmentation by 36 points","Leader360V: 10K+ 360 videos, +36 segmentation, +12 tracking","First large-scale real 360 dataset boosts tracking by 12.6 points","360 video dataset of 10,180 clips sharpens segmentation and tracking","10,180 panoramic videos: 36-point segmentation boost, 12.6-point tracking gain"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported performance gains rest on the assumption that the validation and test clips, 66% of which are cut from the training set's own source videos, actually test generalization rather than memory of the same scenes.","fun_headline_variants_meta":{"raw":{"variants":["10,180 real-world 360 videos lift segmentation by 36 points","Leader360V: 10K+ 360 videos, +36 segmentation, +12 tracking","First large-scale real 360 dataset boosts tracking by 12.6 points","360 video dataset of 10,180 clips sharpens segmentation and tracking","10,180 panoramic videos: 36-point segmentation boost, 12.6-point tracking gain"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001362,"raw_usage":{"total_tokens":5567,"prompt_tokens":1027,"completion_tokens":4540,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":643,"completion_tokens_details":{"reasoning_tokens":4433}},"tokens_in":643,"tokens_out":4540,"duration_ms":35842,"temperature":1.0,"reasoning_tokens":4433,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T00:17:31.573148+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a segmentation or tracking model on the Leader360V training clips and evaluate it on a holdout of videos whose source videos never appear in training; if the J&F or Sdual gains shrink to the level of the untrained baselines, the dataset's claimed value as a generalization resource would be refuted.","supporting_citations":[{"cited_title":"Panovos: Bridging non-panoramic and panoramic views with transformer for video segmentation","cited_arxiv_id":null,"evidence_quote":"Provides the PanoVOS dataset and the PSCFormer baseline, which serves as the 360 VOS comparison point and a source of selected videos."},{"cited_title":"High quality entity segmentation","cited_arxiv_id":null,"evidence_quote":"Supplies CropFormer, the entity segmentation model that generates class-agnostic instance mask proposals in the SDR module."},{"cited_title":"Trackingnet: A large-scale dataset and benchmark for object tracking in the wild","cited_arxiv_id":null,"evidence_quote":"Supplies TrackingNet as the planar-domain training source in the tracking domain-transfer experiments that quantify the 360-domain gap."}],"review_version":1}