{"id":"b0f3edd2-24be-4929-821f-073039544fb9","arxiv_id":"2412.07591","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"CoinCLIP fuses CLIP image and text features with community comments and likes to classify Pump.fun memecoins as viable or non-viable, using Raydium listing as the label.","lead":"This paper introduces CoinVibe, a dataset of 6,231 Pump.fun memecoins with text, logos, and community comments, plus CoinCLIP, a CLIP-based classifier that predicts which meme coins are viable. The claimed value is a data-driven filter for the flood of low-quality token launches, but the label and features raise leakage concerns.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Community features are collected after the Raydium-listing event that defines the label, so CoinCLIP's 92.1% AUC may encode post-success engagement rather than pre-listing viability; the paper describes no temporal cutoff and performs no leakage test.","rationale":"The paper's contribution is a dataset and a framework for memecoin viability; the central claim is that CoinCLIP identifies viable coins. For that claim, the temporal ordering of features and labels is decisive. The label is defined by Raydium listing (Section 3.3), and the community features used in Section 4 are scraped with no pre-listing cutoff. A model with access to post-listing likes and comments can trivially separate listed from unlisted coins; the 4.8-point AUC gap between CoinCLIP and CLIP's image-only result coincides with the addition of these leaked features. The reader's verdict anticipated this by identifying the label-proxy and leakage assumption; my analysis agrees and makes it concrete. I do not question the honest effort of releasing a dataset or the novelty of applying CLIP to memecoins; the issue is that the evaluation cannot support the predictive claim because the information set is contaminated. A feasible test exists because comment timestamps are part of the dataset and listing times are on-chain. Until such a test is run, the reported numbers cannot be interpreted as evidence of viability prediction. Therefore the reader's REJECT verdict stands, with no adjustment needed.","tokens_in":6184,"tokens_out":5043,"duration_ms":45285,"concrete_test":"Use the comment timestamps already collected in CoinVibe to reconstruct, for every viable coin, the first Raydium listing time (from Dune/Raydium logs). Re-train and re-evaluate CoinCLIP with community features truncated to comments posted strictly before that listing time (or before the $69k market-cap threshold if listing time is unavailable); keep the same train/validation/test split. If accuracy/AUC drops materially—for example toward the CLIP image-only result of 79.1/88.3 or toward chance—the original 84.7/92.1 result is leakage-driven. As a secondary check, rerun the 'without community data' condition on the same truncated features to quantify how much of the reported gain comes from post-listing engagement.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim—CoinCLIP reaches 84.72% accuracy, 92.07% AUC, and 83.74% F1 on CoinVibe (Table 1, Section 5.2)—is only meaningful if the model predicts viability from information available before the success outcome. Section 3.3 labels a coin viable iff it was listed on Raydium. Section 3.1 scrapes community comments, timestamps, and likes, but gives no statement that these were collected before the listing event; the natural reading is that all data were scraped once, after coin creation and after any Raydium listing. Section 4 then uses these same comments, timestamps, and likes as features, aggregating them by like-weighted sum. For a viable coin, post-listing engagement is a consequence of listing; including it in X while labeling Y by listing creates a direct causal leak. The model can learn 'this coin has attracted comments after the 69k threshold,' which is not predictive viability. In addition, the Section 3.2 filtering step drops coins with missing descriptions or comments, removing the least-engaged non-viable coins and further biasing the comparison. The ablation in Table 2 cannot rescue this because its caption references a 'hate detection task' and its accuracy values are incompatible with Table 1, indicating the ablation is not on the same task. Consequently, the reported superiority over baselines does not establish the paper's central claim of assessing memecoin viability before success.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces CoinVibe, a multimodal dataset of 6,231 Pump.fun memecoins (text descriptions, logos, user comments, timestamps, likes), labels coins as viable if they were listed on Raydium, and proposes CoinCLIP, a frozen-CLIP architecture with per-modality projection layers, feature adapters with residual connections, and community-data integration. On an 80/10/10 split of CoinVibe, CoinCLIP is reported to outperform unimodal and multimodal baselines, reaching 84.72% accuracy, 92.07% AUROC, and 83.74% macro-F1 (Table 1). An ablation study is presented in Table 2, and the code and dataset are made publicly available.","tokens_in":6629,"tokens_out":2812,"duration_ms":28328,"significance":"If the reported result measured true predictive viability, the paper would provide a useful benchmark and a reasonable strong baseline for a practically relevant Web3 problem. The stated strengths are the public release of code and data, a three-seed evaluation protocol with mean and standard deviation, and a broad comparison against unimodal and multimodal baselines. However, the central claim of assessing viability before success is not supported as reported, because the community features are collected without any demonstrated temporal cutoff relative to the Raydium-listing label. The evaluation therefore cannot distinguish predictive signal from leakage of post-listing engagement, and the ablation study is described as being on a different task, further weakening the empirical case.","major_comments":[{"comment":"The main load-bearing issue is temporal leakage between features and label. Section 3.3 defines viability by whether the coin was listed on Raydium, and Section 2 states that a coin must reach a $69,000 market capitalization to become eligible for that listing. Section 3.1 describes scraping comments, timestamps, and likes, but it does not state that these were collected before the listing event; the natural reading is that all community data were scraped after coin creation, which for viable coins is after the Raydium listing. Section 4 then uses these same comments, timestamps, and likes as features, aggregating comment representations by a like-weighted sum. Because post-listing engagement is a consequence of listing, the model can learn to detect 'this coin already attracted attention after crossing the listing threshold,' which is not predictive viability. The reported 92.07% AUROC in Table 1 therefore does not establish the paper's claim of assessing memecoin viability before success. The authors must provide a temporal split or otherwise demonstrate that community features were observed before the label was determined; without that, the central empirical claim is not supported.","section":"§3.1, §3.3, §4, Table 1"},{"comment":"The data-cleansing step excludes tokens with missing descriptions or comments. This systematically removes non-viable coins that attracted no community engagement, and it changes the composition of the dataset relative to the population of all Pump.fun launches. The paper reports that 44.27% of the cleaned dataset is viable, but no comparison with the overall listing rate is provided. This filtering can inflate apparent separability because the remaining classes are conditioned on having at least some content and community activity. The paper should report the number of tokens removed at each cleansing step and show that the results are robust to this exclusion.","section":"§3.2, §3.3"},{"comment":"The ablation study does not support the conclusion drawn from it. Table 2's caption states that the ablation was conducted on the 'hate detection task from the CoinVibe dataset,' while the main experiments and the paper's stated contribution concern memecoin viability classification. The accuracy values in Table 2 (71.23–76.44) are also incompatible with the viability accuracies in Table 1 (70.17–84.72 for the corresponding configurations). The progressive integration of projection layers, feature adapters, and community data is therefore not shown to contribute to the viability result. The paper should either report the ablation on the same task and split as Table 1 or explicitly state that Table 2 concerns a separate task and discuss why that evidence bears on CoinCLIP's viability performance.","section":"Table 2, §5.3"}],"minor_comments":[{"comment":"The conclusion mentions 'semantic-aware initialization' as a contributing component, but this component is not introduced in Section 4 and does not appear in the ablation table. The manuscript should either describe and evaluate it or remove the reference.","section":"§6"},{"comment":"The text says that corrupted or missing images are 'replaced with zero vectors,' but a zero vector is not a standard valid input to the CLIP image encoder. Please clarify whether the replacement happens before encoding, whether the encoder sees a zero image, or whether a zero embedding is used instead.","section":"§3.2"},{"comment":"The term 'disentangle' is used for the linear projection layers, but there is no training objective or evaluation that enforces or measures disentanglement. Consider using a more neutral description such as 'separate linear projections for each modality.'","section":"§4"},{"comment":"The abstract frames the goal as filtering out 'low-quality or bot-driven projects,' but the operational label is Raydium listing, which is a market-capitalization threshold and not a direct measure of quality or bot activity. The gap between the stated motivation and the actual label should be acknowledged explicitly.","section":"Abstract and §3.3"}],"recommendation":"reject","confidential_remarks":"The paper's core empirical claim cannot be assessed from the reported experiments because community features are collected without a stated temporal cutoff relative to the Raydium-listing label. This is not a matter of tuning or presentation; it requires re-collecting data with a pre-listing cutoff or otherwise establishing that no information following the label was used. If the authors can obtain such data and rerun the evaluation, a resubmission could be appropriate. I would also flag that Table 2's ablation is on a different task, which further weakens the empirical support for the framework's components."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the CoinVibe dataset is the real contribution here; the CoinCLIP framework is a modest CLIP-Adapter variant, and the evaluation has a temporal leakage problem that makes the headline accuracy numbers untrustworthy as evidence of predictive viability.\n\nWhat's actually new and good: 6,231 Pump.fun memecoins with text, logos, comments, timestamps, likes, labeled by Raydium listing. That's a new public dataset, and it's non-trivial to collect. They run multiple baselines, report mean/std over three seeds, and publicly release code and data. They also honestly state that CLIP-Adapter is conceptually similar, which is more than many papers do.\n\nThe soft spots are real. The biggest issue is temporal leakage. The label is whether the coin was listed on Raydium. The community features—comments, timestamps, likes—are scraped without any stated cutoff before the listing event. Listing drives attention and comments, so the model can learn 'this coin is attracting engagement after hitting the 69k threshold,' which is not the same as predicting viability from pre-listing information. The paper never frames this as a prediction task with a temporal boundary, but the abstract and intro clearly imply filtering before success. As it stands, the 92% AUC probably overstates what the model would do on genuinely new coins.\n\nA second issue: they drop entries with missing descriptions or comments. Non-viable coins are more likely to have no comments, so this filtering biases the sample and inflates apparent accuracies. That's a design choice, but it should be discussed.\n\nThird, the ablation in Table 2 is on a 'hate detection task' from CoinVibe, which is confusing—CoinVibe is a memecoin viability dataset. The accuracy values (71-76%) are incompatible with Table 1 (81-84%), suggesting it's a different task entirely. That needs clarification or correction.\n\nOn method novelty: separate per-modality adapters plus community-data concatenation is a small step from CLIP-Adapter. Not a fatal flaw, but it is what it is.\n\nWho's this for? Researchers working on blockchain or crypto analytics, and anyone who wants an example of how easy it is to leak the label into the features in a retrospective dataset. A serious referee could help the authors fix the leakage by re-annotating with features as of a fixed time before the Raydium threshold, or re-framing the task as post-hoc classification.\n\nRecommendation: send it to review. The dataset is worth publishing with the evaluation fixed; the current central claim doesn't stand, but it's fixable.","headline":"Useful new memecoin dataset, but the evaluation leaks the label into the features and the 92% AUC does not measure predictive viability as claimed.","tokens_in":6999,"tokens_out":2702,"would_cite":false,"duration_ms":22967,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that memecoin viability can be predicted at 84.7% accuracy by combining a frozen CLIP model with community comment data, and tests this on a new 6,231-token dataset.","keywords":["memecoin","Web3","multimodal classification","CLIP","community data","Pump.fun","Raydium listing","viability prediction"],"falsifier":"Take the same CoinVibe features but split the dataset by creation date rather than randomly, or relabel viability by whether a token still has liquidity and a non-zero price 90 days after launch; if CoinCLIP's accuracy drops substantially, then its current performance depends on leakage from post-listing engagement or on a label proxy that does not measure long-term survival. A simpler check is to collect tokens that were scraped as non-viable but listed on Raydium days later and see whether the model flags them as viable.","tokens_in":5971,"feed_emoji":"🪙","tokens_out":4241,"duration_ms":36887,"temperature":0.7,"pith_summary":"The paper tries to establish that the viability of a newly launched memecoin can be predicted from how it presents itself and how its community reacts, before any substantial trading history exists. It introduces CoinVibe, a dataset of 6,231 Pump.fun tokens with logos, textual descriptions, user comments, timestamps, and likes, and labels each token viable if it was later listed on the Raydium exchange. On top of this it builds CoinCLIP, which keeps a frozen CLIP model, adds per-modality projection layers and residual feature adapters, and injects comment data weighted by likes. On a held-out test split CoinCLIP reaches 84.72% accuracy, 92.07% AUC, and 83.74% F1, outperforming CLIP itself, CLIP-Adapter, and unimodal text or image classifiers. The point of caring is that most Pump.fun tokens are bot-generated or short-lived, and a cheap multimodal screen could filter them before investors commit.","feed_headline":"Multimodal AI spots viable memecoins with 84.7% accuracy","feed_subtitle":"CoinCLIP fuses logos, text, and community comments to separate lasting tokens from pump-and-dumps.","key_machinery":"The mechanism is a frozen CLIP vision-language backbone plus three lightweight add-ons. Linear projection layers map each modality's CLIP embedding into a task-aligned space; Feature Adapters with residual connections let the model adjust to memecoin data without drifting from pre-trained features; and a community module encodes user comments through the CLIP text encoder, embeds normalized timestamps and likes, and aggregates comments by a like-weighted sum. Image and text representations are merged by Hadamard product, the community vector is concatenated, and a small MLP makes the final viable/non-viable decision. CLIP is the named central object: a contrastively pre-trained model that puts images and text in one embedding space.","core_discovery":"On the paper's own terms, the central discovery is that community interaction data—comments, timestamps, and likes—carries signal for memecoin viability that neither image nor text alone contains, and that this signal can be fused into a frozen CLIP backbone with lightweight adapters rather than expensive fine-tuning. The authors claim that separating image and text streams with independent projection layers and feature adapters, then combining them through a Hadamard product and concatenating an aggregated community representation, yields the best classifier for the CoinVibe task. They support this with a full benchmark: CoinCLIP beats CLIP by 3.4 accuracy points, CLIP-Adapter by 2.5, and the best unimodal model (CLIP image-only) by 5.6, with similar margins in AUC and F1. The ablation study attributes the gains to the projection layers, the dual feature adapters, and the community data integration in turn.","pith_inferences":["If the Raydium-listing label is replaced by a longer-horizon outcome such as 90-day survival or trading volume persistence, the reported 84.7% accuracy is likely to decrease; that gap would measure how much of the model's signal is about reaching the listing threshold rather than about lasting value.","Because the article's features include comment likes and timestamps gathered at scrape time, a temporal split (train on earlier coins, test on later coins) is the natural stress test; the paper's random split may overstate generalizability.","The same architecture should transfer to other bonding-curve launchpads on other blockchains, and its community module could be extended to social-platform signals such as X/Twitter engagement without changing the fusion design.","At 83.7% macro F1, the false-negative rate for viable coins is still high enough that a practical deployment should treat the model as a filter rather than a validator."],"forward_implications":["On coins like those in CoinVibe, multimodal screening reaches about 84.7% accuracy, which beats unimodal methods and plain CLIP by several points.","Separate adapters per modality outperform a single shared adapter, suggesting images and text need independent adjustment for this domain.","Community engagement features (comment volume, timing, likes) add measurable signal beyond the token's own image and description.","Image-only classification using CLIP's visual encoder already outperforms text-only methods, so visual branding matters more than narrative text for memecoin survival.","The frozen-backbone design means the method can be applied without expensive fine-tuning of large vision-language models."],"supporting_citations":[{"why":"Supplies the frozen CLIP encoders whose shared embedding space CoinCLIP builds on.","marker":"[7]"},{"why":"Provides the CLIP-Adapter baseline that CoinCLIP extends with separate per-modality adapters.","marker":"[4]"},{"why":"Defines the ViT-L/14 architecture used as image encoder and image-only baseline.","marker":"[3]"},{"why":"Provides the BERT text baseline used for unimodal comparison.","marker":"[2]"}],"fun_headline_variants":["CoinCLIP merges logos, text, and comments to rate memecoin viability","Community likes and comments sharpen memecoin viability predictions","Frozen CLIP plus lightweight adapters yields best memecoin classifier","Multimodal memecoin filter gains 5.6% accuracy over image-only CLIP"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a token being listed on Raydium is a faithful label for 'viable,' and that the scraped comments, timestamps, and likes reflect community interest before the listing rather than being inflated by the listing event itself.","fun_headline_variants_meta":{"raw":{"variants":["CoinCLIP merges logos, text, and comments to rate memecoin viability","Community likes and comments sharpen memecoin viability predictions","Frozen CLIP plus lightweight adapters yields best memecoin classifier","Multimodal memecoin filter gains 5.6% accuracy over image-only CLIP"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000417,"raw_usage":{"total_tokens":2171,"prompt_tokens":988,"completion_tokens":1183,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":604,"completion_tokens_details":{"reasoning_tokens":1100}},"tokens_in":604,"tokens_out":1183,"duration_ms":11508,"temperature":1.0,"reasoning_tokens":1100,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:40:31.583973+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the same CoinVibe features but split the dataset by creation date rather than randomly, or relabel viability by whether a token still has liquidity and a non-zero price 90 days after launch; if CoinCLIP's accuracy drops substantially, then its current performance depends on leakage from post-listing engagement or on a label proxy that does not measure long-term survival. A simpler check is to collect tokens that were scraped as non-viable but listed on Raydium days later and see whether the model flags them as viable.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the CLIP-Adapter baseline that CoinCLIP extends with separate per-modality adapters."}],"review_version":1}