{"id":"532a1e89-c5ef-4b29-93ce-ed93497167f6","arxiv_id":"2509.00053","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"Traj-MLLM claims a training-free MLLM framework outperforms task-specific trajectory models on four tasks, but the input may leak the target for travel time and destination prediction.","lead":"Traj-MLLM feeds human GPS trajectories to commercial multimodal AI models as map images and text descriptions, then uses the model's reasoning instead of training to estimate travel times, detect anomalies, predict destinations, and identify transport modes. The paper reports large gains over specialized baselines, but the evaluation may accidentally give the AI the answer in the input.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"TTE and MP labels are present in the multimodal input: no masking is reported, so claimed gains may be answer extraction rather than generalization.","rationale":"The reader's weakest assumption is the same load-bearing concern: the multimodal input for TTE contains start/end timestamps and duration, and the MP input contains the destination in the rendered full trajectory, with no masking described. I agree with that assessment. The paper's own ablation (Table 5) shows that removing images or reordering tokens hurts performance, but it does not include a masking control, so it cannot distinguish genuine reasoning from direct label extraction. The abstract's 'without requiring any training data' is also softened by the prompt-optimization procedure in Section 4.3, which uses labeled seed trajectories, and by per-dataset threshold selection in Appendix A.4; however, the masking issue alone is sufficient to invalidate the headline TTE and MP results. The proposed framework is novel and the direction is worth exploring, but as written the central empirical claim is not supported. Since the reader already reached REJECT and my analysis confirms that verdict, no change is needed.","tokens_in":22383,"tokens_out":4048,"duration_ms":44868,"concrete_test":"Inspect the released code (anonymous.4open.science/r/Traj-MLLM/) for the TTE and MP input builders to check whether masking exists. Then run a control experiment: for TTE, remove Duration and End Time from all text tokens while keeping the map images; for MP, crop or blur the terminal segment and destination icon from the global and local map images while keeping the rest of the trajectory. Rerun the same o4-mini prompts on Xian, Porto, and Chengdu and recompute MAE and ACC@1. If TTE MAE rises to near the best baseline (e.g., above roughly 180 seconds) or MP ACC@1 drops toward chance, the reported gains are label leakage; if performance is largely unchanged after masking, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that a frozen MLLM performs four trajectory tasks across cities without training. The load-bearing condition is that the reported improvements measure reasoning/generalization rather than reading the label out of the input. For TTE, Section 4.1.2 and Appendix A.2 construct each sub-trajectory text token from Start Time, End Time, and Duration in seconds; the TTE target is total travel time, so the answer is literally one field of the text token. For MP, Section 4.1.2 renders the full trajectory with start/end icons and Section 4.2.1 builds a global view of the whole trajectory; the destination is thus visible as the final map endpoint and is also the final segment in the text sequence. The paper nowhere states that timestamps or durations are masked for TTE, or that the destination is cropped/omitted for MP. If no masking is applied, the reported 48.05% TTE improvement and 15.52% MP improvement are consistent with simple extraction or arithmetic on the provided input, not with a general trajectory-mining ability. This is an internal validity flaw, not a disagreement with community consensus, and it directly affects the paper's strongest empirical claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Traj-MLLM, a training-free framework that converts raw GPS trajectories into interleaved image-text sequences through map-anchored semantic segmentation and multiview modeling, and then uses a frozen multimodal large language model with task prompts optimized on a few seed trajectories to perform travel time estimation, anomaly detection, mobility prediction, and transportation mode identification. Experiments on Xian, Chengdu, Porto, and Geolife report large improvements over state-of-the-art baselines, and the authors release their code and a dataset of MLLM responses.","tokens_in":22590,"tokens_out":10963,"duration_ms":115291,"significance":"If the reported results were valid, Traj-MLLM would be a significant demonstration that a single frozen MLLM can generalize across cities and trajectory-mining tasks without parameter updates. The paper contributes a modular tokenization pipeline and publicly releases both code and a dataset of MLLM responses, which are valuable assets for the community. However, the current evaluation does not establish the central claim: the TTE and MP targets appear to be present in the multimodal inputs unless masking is applied, and no masking is described anywhere in the manuscript. In addition, the headline improvements are selected from the most favorable per-metric, per-dataset cells. The significance of the paper therefore depends on a repaired evaluation protocol.","major_comments":[{"comment":"The TTE target is directly available in the text tokens. Equation (3) and Appendix A.2 specify that each sub-trajectory text token contains Start Time, End Time, and Duration in seconds. Since the TTE task is defined as predicting total travel time (Appendix A.3.2), the MLLM can obtain the answer by subtracting the first start time from the last end time or by summing the segment durations. The manuscript never states that timestamps or durations are masked for TTE, so the reported 48.05% improvement may reflect label extraction or arithmetic on the input rather than trajectory reasoning. The authors must either mask these fields and re-run the experiments, or demonstrate that performance is unaffected when they are removed.","section":"Section 4.1.2 and Appendix A.2"},{"comment":"The mobility prediction target is visible in the input. Mobility prediction is defined as predicting the destination region, but the global spatial view renders the entire trajectory and the interleaved text sequence includes the final sub-trajectory, whose endpoint is the destination. Section 4.1.2 additionally states that start and end points of each segment are marked with unique icons on the rendered map. Unless the destination is cropped from the images or the final segment is withheld from the text, the MLLM can read the answer directly from the global view or the last text token. No such masking is described, so the claimed 15.52% ACC@1 improvement does not currently constitute evidence of destination prediction.","section":"Sections 4.1.2, 4.2.1, and Appendix A.3.2"},{"comment":"The headline percentages are cherry-picked from individual cells. The abstract's 48.05% for TTE is the RMSE improvement on Xian in Table 1; 15.52% for MP is the ACC@1 improvement on Porto; 51.52% for AD is the PR-AUC improvement for switch anomalies with μ=0.3 on Xian in Table 2; and 1.83% for TMI is the accuracy improvement on Geolife in Table 3. Many cells show far smaller gains, e.g., AD high-detour improvements of 0.20%, 0.72%, and 0.93%. The 'Improvement' rows are also computed against the best baseline per metric rather than a single consistent SOTA method. The authors should report all metrics across all datasets, state a primary metric per task in advance, and provide averages or ranges instead of selecting the most favorable numbers for the abstract.","section":"Abstract and Tables 1-3"},{"comment":"The claim that Traj-MLLM works 'without requiring any training data' is contradicted by the prompt optimization procedure. This section describes selecting labeled seed trajectories, using the true label ŷ to generate feedback from disagreement samples, and iteratively refining the prompt until it achieves satisfactory performance on the seed set. This is a supervised adaptation step that uses labeled data, even if it does not update model parameters. The authors must state how seed trajectories are selected, whether they are disjoint from the evaluation set, and should rephrase the claim to something like 'no model parameter training or fine-tuning' if that is what is intended.","section":"Section 4.3.2"}],"minor_comments":[{"comment":"The tasks section lists Accuracy, Precision, and PR-AUC for anomaly detection, but Table 2 reports only PR-AUC; please clarify which metrics are used and where the other metrics are reported.","section":"Appendix A.3.2"},{"comment":"The contextual views are defined as C = {C_1, ..., C_K} in the first paragraph and then as C = {C_1, ..., C_Z} in Algorithm 1; please use a single index bound consistently.","section":"Section 4.2"},{"comment":"No confidence intervals, standard deviations, or repeated runs are reported for the MLLM outputs. Given the stochasticity of MLLM decoding, the small 1.83% TMI improvement may be within run-to-run noise; please report variance across multiple inference runs or state the decoding temperature and sampling protocol.","section":"Tables 1-3"},{"comment":"The paper states that 'over 80,000 trajectories' are selected, but does not specify how many trajectories are used for each task and dataset; please provide a per-task, per-dataset breakdown.","section":"Section 5.1"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The two things you should know: the representation idea is genuinely new, and the headline TTE/MP numbers are probably label leakage, not generalization. Traj-MLLM's map-anchored segmentation plus multiview interleaved image-text sequence is a sensible, well-engineered way to present a trajectory to an MLLM. The problem is that the input contains the answer for two of the four tasks. Each sub-trajectory text token includes Start Time, End Time, and Duration in seconds (Appendix A.2), so the TTE target—total travel time—is just the sum of the durations. For MP, the global map view renders the full trajectory with start/end icons, so the destination is visible in the input. The paper never states that timestamps, durations, or destinations are masked or cropped. Without that, the reported 48% TTE and 15.5% MP improvements are consistent with the model doing arithmetic or reading the map, not with a general trajectory-mining ability.\n\nWhat is actually good: the framework itself is a real contribution. The semantic segmentation, the multiple context views (POI, road network), and the prompt optimization loop are all clearly described and plausible. For AD and TMI, where the label is an anomaly status or transport mode rather than a visible field or endpoint, the results are more believable, though AD uses synthetically injected anomalies and the improvements are uneven across settings.\n\nWhere it's soft: the paper overstates the 'without any training data' claim—prompt optimization uses a few labeled seeds, and per-city theta thresholds are tuned on each dataset. That's a weaker issue than the leakage, but it should be stated more carefully. The main issue is the missing masking. This is not a disagreement about community consensus; it's an internal validity problem.\n\nBottom line: this could be an important paper if the experiments are redone with the target information withheld. As written, the central empirical claim is unsupported. I would send it to peer review—the direction is worth referee time and the flaw is concrete and fixable—but I would not accept the current version. If the authors can show the performance survives masking, it's a strong paper.","headline":"Clever representation, but TTE and MP results likely reflect label leakage (durations in text, destination in map), not generalization.","tokens_in":23149,"tokens_out":4086,"would_cite":true,"duration_ms":39342,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A frozen multimodal LLM, fed map-cropped images and structured text, can answer four trajectory-mining tasks with no training, the paper argues.","keywords":["trajectory data mining","multimodal large language models","training-free inference","map-anchored tokenization","travel time estimation","mobility prediction","anomaly detection","transportation mode identification"],"falsifier":"Re-run travel time estimation and mobility prediction with the label-bearing pieces removed: drop the end timestamp and duration from every text block and crop the destination region out of the global map image, then compare the reported metrics. If accuracy collapses toward baseline chance, the original numbers came from reading the label in the input. A complementary check is to feed the text statistics alone, without any image, to the same MLLM and see how much of the reported travel-time and mobility improvement survives.","tokens_in":22129,"feed_emoji":"🗺️","tokens_out":4416,"duration_ms":41394,"temperature":0.7,"pith_summary":"The paper claims that a single frozen multimodal large language model can perform four standard trajectory mining tasks—travel time estimation, mobility prediction, anomaly detection, and transportation mode identification—across cities with no training or fine-tuning. To make this work, the authors convert each GPS trajectory into an interleaved sequence of map images and structured text descriptions, then hand a task prompt to the MLLM. They report large gains over task-specific and foundation models, for example cutting travel-time RMSE by over 48% on one city and more than doubling PR-AUC on a low-magnitude detour anomaly setting. A prompt-optimization loop using a few labeled seed trajectories produces data-invariant prompts that transfer across regions. The central question—whether MLLMs can reform trajectory mining—is answered affirmatively on these four tasks and four cities.","feed_headline":"Frozen MLLMs answer trajectory tasks without training","feed_subtitle":"Paper reports 48% travel-time error cut and 51% anomaly-detection jump using a frozen MLLM, no training data.","key_machinery":"The central mechanism is map-anchored tokenization combined with multiview trajectory modeling. Segmentation minimizes a cost function built from speed consistency, road-type changes, and segment length regularization; each resulting sub-trajectory becomes both a map-cropped image (with context elements filtered by a distance threshold) and a text block of summary statistics. Spatial views at global and local scales plus contextual views (POI, road network) are arranged in an ordered interleaved image-text sequence, letting the MLLM's sequential reasoning capture temporal dependencies without a trainable temporal encoder. Task adaptation rides on a prompt template of role, task description, domain knowledge, and output format, which is automatically refined over a handful of seed trajectories until it performs satisfactorily, then reused across cities.","core_discovery":"Traj-MLLM claims to be the first general, training-free framework that uses multimodal LLMs for trajectory data mining. Raw GPS trajectories are segmented into semantically coherent sub-trajectories via a map-anchored dynamic program, rendered as map images with layered POI and road network context, and paired with structured statistics such as start and end times, duration, distance, and speeds. The interleaved image-text sequence preserves spatial, temporal, and contextual information, while a few-shot prompt optimization yields task prompts that transfer across regions. The claimed outcome is that the frozen MLLM outperforms state-of-the-art baselines by 48.05% on travel time estimation, 15.52% on mobility prediction, 51.52% on anomaly detection, and 1.83% on transportation mode identification, with ablation studies showing that the visual modality, semantic segmentation, interleaved order, POI context, and road network context all contribute to these results.","pith_inferences":["If the reported numbers hold with properly masked inputs, the real contribution would be a test-time-compute substitute for model training: any trajectory task becomes a prompt, and performance would track the underlying MLLM's reasoning power, which the paper's own backbone comparison already hints at.","The prompt-optimization loop is de facto few-shot learning: the seed trajectories and their labels leak task information into the prompt, so the 'training-free' claim reduces to 'no gradient updates,' which matters for deployment but is not the same as zero-shot learning.","A natural testable extension is to probe how the distance threshold should scale with urban density; the paper's sensitivity analysis shows that too small a threshold can hide connecting roads and too large a threshold adds clutter, suggesting a dataset-dependent optimum that may not transfer to cities with sparse POI coverage."],"forward_implications":["A single frozen MLLM could replace task-specific and region-specific trained models for four trajectory tasks, requiring no training data for new cities.","Because the trajectory-to-sequence mapping is task-independent, new trajectory tasks can be added by writing a new task prompt and reusing the same multimodal representation.","Cross-city transfer is claimed to hold without retraining: the same optimized prompts work on Xi'an, Chengdu, Porto, and the GeoLife Beijing trajectories.","The released multimodal response dataset could serve as training data for building future MLLMs specialized for trajectory analysis.","The framework is extensible to additional contextual views, such as traffic lights, as long as they can be rendered as map layers and filtered by relevance."],"supporting_citations":[{"why":"Supplies the GeoLife GPS trajectory dataset used for transportation mode identification and for cross-user generalization.","marker":"[44]"},{"why":"Provides the multi-scale anomaly detection baseline (MST-OATD) and the detour/switch evaluation protocol the paper follows.","marker":"[29]"},{"why":"Supplies the synthetic anomaly construction method (disturbance-based detour and switch injection) used to create AD ground truth.","marker":"[23]"},{"why":"The strongest multimodal baseline for travel time estimation, which Traj-MLLM claims to beat by over 29% MAE on Xi'an.","marker":"[31]"},{"why":"The strongest LLM-based baseline for TTE and mobility prediction, which Traj-MLLM claims to beat by over 34.9% on MAPE.","marker":"[47]"},{"why":"A trajectory foundation model baseline (START) for TTE and mobility prediction, used to establish the comparison set.","marker":"[10]"},{"why":"Supports the premise that LLMs struggle with numerical and spatio-temporal reasoning, motivating the multimodal design.","marker":"[17]"}],"fun_headline_variants":["Frozen MLLM beats SOTA on four trajectory tasks","Training-free MLLM masters trajectory mining across regions","Traj-MLLM: one frozen multimodal model for all trajectory tasks","Multimodal LLM outperforms trajectory baselines with zero training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The inputs given to the language model do not already contain the answer to the task: in particular, the global map image shows the full trajectory including the destination, and the text blocks state start and end timestamps and duration, so the reported travel-time and mobility numbers could be read off or arithmetically derived from the input rather than inferred from spatial semantics.","fun_headline_variants_meta":{"raw":{"variants":["Frozen MLLM beats SOTA on four trajectory tasks","Training-free MLLM masters trajectory mining across regions","Traj-MLLM: one frozen multimodal model for all trajectory tasks","Multimodal LLM outperforms trajectory baselines with zero training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000555,"raw_usage":{"total_tokens":2685,"prompt_tokens":1030,"completion_tokens":1655,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":646,"completion_tokens_details":{"reasoning_tokens":1583}},"tokens_in":646,"tokens_out":1655,"duration_ms":12823,"temperature":1.0,"reasoning_tokens":1583,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:01:18.840378+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run travel time estimation and mobility prediction with the label-bearing pieces removed: drop the end timestamp and duration from every text block and crop the destination region out of the global map image, then compare the reported metrics. If accuracy collapses toward baseline chance, the original numbers came from reading the label in the input. A complementary check is to feed the text statistics alone, without any image, to the same MLLM and see how much of the reported travel-time and mobility improvement survives.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the GeoLife GPS trajectory dataset used for transportation mode identification and for cross-user generalization."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the multi-scale anomaly detection baseline (MST-OATD) and the detour/switch evaluation protocol the paper follows."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the synthetic anomaly construction method (disturbance-based detour and switch injection) used to create AD ground truth."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The strongest multimodal baseline for travel time estimation, which Traj-MLLM claims to beat by over 29% MAE on Xi'an."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supports the premise that LLMs struggle with numerical and spatio-temporal reasoning, motivating the multimodal design."}],"review_version":1}