{"id":"4784fd4e-fa49-471a-9259-d3151a4fc352","arxiv_id":"2508.02247","paper_version":2,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Introduces ByteGen, a tokenizer-free byte-level model for limit order book events, trained on 34 million CME Bitcoin futures messages, reproducing heavy-tailed returns and bursty event timing.","lead":"ByteGen is a new machine learning model that predicts market data streams byte by byte, avoiding the usual step of converting numbers into tokens. It is designed for realistic simulation of financial markets, which could improve how trading strategies are tested.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Packed 32-byte format may incorporate significant hand-designed structure, undermining the 'raw byte space' and 'no feature engineering' claim.","rationale":"The reader's weakest assumption focused on H-Net's dynamic chunking discovering message structure from bytes. My concern is upstream of that: even if dynamic chunking works, the input bytes are produced by a custom 32-byte packing scheme. The abstract's claim to 'complete elimination of feature engineering' is load-bearing for the paper's novelty, but the packing step is a form of feature engineering unless it is a trivial, information-preserving byte serialization of the original event fields. Since the abstract does not describe the packing details or provide round-trip guarantees, this remains an unverified assumption. The reader's verdict is already UNVERDICTED because the full text is unavailable; my concern does not move that verdict, but it identifies a specific validation step that would be needed before the central claim can be accepted. I therefore leave the verdict unchanged.","tokens_in":788,"tokens_out":3657,"duration_ms":47626,"concrete_test":"Obtain the preprocessing code from the authors, reimplement the 32-byte packing, and run a round-trip test on the full CME Bitcoin futures dataset: encode every event to bytes and decode back to the original fields. If any event fails to round-trip exactly, or if the byte layout varies with message type in ways that require manual offsets, then the 'without information loss' and 'no feature engineering' claims are weakened. Additionally, retrain or probe ByteGen with a deliberately altered but still lossless packing (e.g., reversed field order or different byte widths); if the model's market quality metrics change materially, the results depend on the specific engineered packing, not on raw byte learning.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract claims ByteGen eliminates feature engineering and tokenization by operating on 'raw byte streams,' yet it also introduces a 'compact and efficient 32-byte packed binary format to represent market messages without information loss.' This packed format is itself a pre-processing choice: field ordering, fixed widths, and byte layout must be manually selected to encode timestamps, prices, sizes, side codes, and message types. If the layout is derived from the CME protocol rather than emerging from raw bytes, the model is not learning from the most fundamental representation but from a hand-crafted binary schema. The 'without information loss' assertion is also not self-evident for a fixed-width format; it requires verifying that every message in the 34M-event training set maps injectively to 32 bytes, with no ambiguity, overflow, or loss of precision. H-Net's dynamic chunking can only discover structure within the given byte layout; it cannot recover information discarded or reordered by the packing. Therefore, the central novelty depends on the packing being a truly neutral, lossless byte encoding, and the abstract does not provide evidence for that.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript introduces ByteGen, a tokenizer-free generative model for limit order book events that predicts the next byte of a 32-byte packed binary encoding of CME Bitcoin futures market messages. The authors claim that this approach eliminates feature engineering and tokenization, that the packed format is lossless, and that a hybrid Mamba-Transformer architecture with dynamic chunking (H-Net) discovers the inherent structure of LOB data. They report training on over 34 million events and claim that the model reproduces stylized facts such as heavy-tailed returns and bursty event timing, while remaining competitive on standard market quality metrics. The abstract presents the framework's contributions and qualitative results but does not include numerical evidence or methodological details.","tokens_in":1008,"tokens_out":1745,"duration_ms":21866,"significance":"If the central claims hold, ByteGen would be a meaningful step toward end-to-end generative modeling of high-frequency financial data, potentially avoiding the discretization biases introduced by tokenization. The idea of autoregressive next-byte prediction over a compact binary representation is plausible and aligns with recent successes of byte-level models in other domains. The paper also promises a concrete advance in applying structured-state-space and attention hybrids to orderbook data. However, the significance cannot be fully assessed from the abstract alone because no quantitative results, baselines, or statistical analyses are provided. The claimed elimination of feature engineering is a strong novelty statement that requires careful defense given the introduced packed format.","major_comments":[{"comment":"The abstract claims 'complete elimination of feature engineering and tokenization' while simultaneously introducing a 'compact and efficient 32-byte packed binary format.' This packing requires manual design decisions about field ordering, fixed byte widths, integer versus floating-point representation, and handling of timestamps and prices. Unless it is shown that the packing is a direct, unmodified byte-level representation of the native exchange protocol, the claim of operating on 'raw byte streams' is overstated. The authors should clarify whether the 32-byte format is derived from the existing FIX/CME binary protocol or is a custom encoding that already constitutes feature engineering.","section":"Abstract"},{"comment":"The assertion that the packed format represents market messages 'without information loss' is not self-evident and is load-bearing for the paper's premise. A fixed-width format can lose information through truncation, overflow, or ambiguous ordering if the field ranges exceed the allocated bytes. The authors should state whether injectivity of the 32-byte mapping has been verified across all message types and value ranges in the 34M-event training set, and should specify how edge cases (e.g., extreme prices, timestamps at the boundary, nanosecond precision) are handled.","section":"Abstract"},{"comment":"The abstract reports 'competitive performance on standard market quality metrics' and 'successfully reproduces key stylized facts' without any numerical results, baselines, or confidence intervals. As an abstract-only submission, this leaves the reader unable to evaluate the magnitude of the claimed success or whether the performance is statistically distinguishable from simpler baselines. The authors should include at least representative quantitative measures (e.g., Wasserstein distances for return distributions, event-rate autocorrelations, or comparison against tokenized baselines) in the abstract or in a results summary.","section":"Abstract"}],"minor_comments":[{"comment":"The term 'raw byte streams' is used ambiguously: it could mean the native exchange byte encoding, a normalized byte sequence, or the custom 32-byte format. Please define the exact input representation.","section":"Abstract"},{"comment":"The paper is described as 'the first end-to-end, byte-level framework for LOB modeling,' but the abstract does not cite prior work on byte-level financial sequence models or on tokenization-free methods for LOB generation. A brief comparison or citation would help position the novelty.","section":"Abstract"},{"comment":"The abbreviation H-Net is introduced without explanation of the architecture's provenance; if it refers to an existing model, a citation is needed.","section":"Abstract"},{"comment":"The phrase 'without the biases of tokenization' is a causal claim that needs a defined metric (e.g., which specific biases are measured and how they are quantified).","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"This review is based only on the abstract because the full text was not available. The central conceptual issue—whether the 32-byte packed format undermines the 'raw byte space' claim—is genuine and must be addressed. The lack of numerical evidence in the abstract is understandable for a paper with a separate results section, but the claims are currently unfalsifiable on the submitted material. I would recommend the editor seek the full manuscript before making a final decision, but the abstract as written needs substantial revision to state the methodology and results more precisely."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Hi,\n\nQuick take on arXiv:2508.02247. The core idea is genuinely new: treat limit order book messages as a byte stream and train an autoregressive next-byte predictor with a Mamba-Transformer hybrid using dynamic chunking. If the empirical claims hold, this could bypass the discretization artifacts that tokenization introduces for high-precision financial data. That is a real contribution, and the scale (34M events from CME Bitcoin futures) suggests they actually built the thing.\n\nWhat the paper does well, from the abstract, is frame a concrete problem and propose a specific architecture plus a packed binary representation. The ambition to reproduce stylized facts like heavy tails and bursty timing is the right target. I can't verify numbers, but the direction is sound.\n\nThe soft spot is the one the stress-test note flags, and I think it is the paper's weakest point. The 32-byte packed format is not \"the most fundamental representation\" and it is not the same as learning from raw bytes. Somebody had to decide field order, fixed widths, and how to encode timestamps, prices, and sides. That is feature engineering, just dressed in byte clothes. The \"without information loss\" claim is also not self-evident for fixed-width packing; it needs a proof or at least an explicit description of how every possible message maps injectively. The model's dynamic chunking can only work with what the format leaves in. So the paper's headline novelty depends on the packing being neutral and lossless, and the abstract gives no evidence for that. That's a legitimate concern, not a nitpick.\n\nAlso, the abstract claims competitive performance on market quality metrics but gives no numbers. That is normal for an abstract, but it does mean we are buying the conclusion on faith.\n\nAll that said, this is not a desk-reject paper. The idea is important enough and the authors clearly did the work. I would send it to a referee who will push on the packing format and ask for a head-to-head comparison against tokenized baselines on the same data. If the full paper shows the packing is genuinely lossless and the byte-level model beats a properly tokenized baseline, that is a solid contribution.\n\nRecommendation: it deserves a serious referee; I would engage with it, but the \"no feature engineering\" language should be walked back or defended.","headline":"Byte-level LOB modeling is a genuinely fresh idea, but the 'raw byte space' claim is undercut by the hand-designed 32-byte packed format; the paper deserves a referee who will push on that point.","tokens_in":1421,"tokens_out":2106,"would_cite":false,"duration_ms":23481,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ByteGen shows that high-frequency limit order book dynamics can be modeled directly as raw byte streams, with no tokenization.","keywords":["limit order book","generative modeling","byte-level modeling","tokenization-free","next-byte prediction","H-Net","Mamba-Transformer","high-frequency finance"],"falsifier":"Train ByteGen and an otherwise matched tokenized baseline on the same Bitcoin futures data and compare realized bid-ask spread, depth, price distribution, and return-tail statistics across several seeds; if the tokenized baseline matches or beats ByteGen on these metrics, the claim that byte-level representation avoids tokenization bias is unsupported.","tokens_in":648,"feed_emoji":"📈","tokens_out":7105,"duration_ms":80158,"temperature":0.7,"pith_summary":"ByteGen claims that high-frequency limit order book dynamics can be generated directly from raw byte streams, with no tokenization, no binning, and no hand-designed features. The paper turns each orderbook event into a lossless 32-byte packed binary record and trains an autoregressive model to predict the next byte on millions of Bitcoin futures events. The generated data reproduces price distributions, heavy-tailed returns, and bursty event timing consistent with real markets. The authors report competitive scores on standard market quality metrics, which they read as evidence that byte-level learning avoids the distortions of tokenization. If that holds, a major source of artificial structure in financial generative models disappears.","feed_headline":"Byte-level model generates realistic orderbooks without tokenizers","feed_subtitle":"Packing each orderbook event into 32 bytes, the model learns market dynamics as next-byte prediction.","key_machinery":"The load-bearing object is ByteGen's 32-byte packed binary message format, a fixed-length, lossless serialization of each limit order book event that becomes the unit of next-byte prediction. It is paired with H-Net, a hybrid Mamba-Transformer architecture whose dynamic chunking mechanism learns where meaningful boundaries lie in byte space instead of relying on predefined tokens. Together they replace feature engineering and tokenization with a single objective: predict the next byte.","core_discovery":"The paper's central claim is that limit order book modeling does not need a token or feature layer: a generative model operating on the raw byte representation of market events can learn market structure on its own. ByteGen serializes each LOB message into a fixed 32-byte packed format that preserves all numerical precision, then frames the task as autoregressive next-byte prediction. The H-Net backbone, a hybrid Mamba-Transformer architecture, uses dynamic chunking to group raw bytes into meaningful units without predefined rules. Trained on millions of CME Bitcoin futures events, ByteGen produces realistic price distributions, heavy-tailed returns, and bursty event timing, and the authors say it performs competitively on standard market quality metrics. The intended lesson is that byte space is a flexible, unbiased substrate for financial generative modeling.","pith_inferences":["Going beyond the paper, the 32-byte packed-byte recipe should transfer to other high-frequency instruments and venues whose messages fit a fixed-width format; the paper trains only on Bitcoin futures and does not test this.","The authors do not isolate whether byte-level training or the H-Net backbone drives the result; a matched-size tokenized baseline would separate those factors.","If dynamic chunking works broadly, tokenization may be the main source of artificial discreteness in existing orderbook simulators, an interpretation beyond the reported experiments."],"forward_implications":["Market simulators can be trained directly from packed exchange feeds, removing per-dataset feature engineering and field-specific tokenizers.","Evaluation of generated market quality—price distribution, tail weight, event burstiness—no longer depends on how the input was discretized.","The same training objective, next-byte prediction, applies to any instrument whose messages fit the fixed-width packed format without redesign.","Lossless byte encoding means no model input precision is sacrificed before training begins."],"supporting_citations":[],"fun_headline_variants":["ByteGen models orderbooks in raw bytes, no tokenizers","Next-byte prediction yields realistic orderbook events","Tokenizer-free byte model reproduces market stylized facts","Orderbook generation directly from 32-byte events","Byte-level AI learns market dynamics without feature engineering"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The model's dynamic chunking genuinely recovers orderbook message structure from raw bytes without any predefined boundaries; if it does not, the claimed advantage over tokenization evaporates.","fun_headline_variants_meta":{"raw":{"variants":["ByteGen models orderbooks in raw bytes, no tokenizers","Next-byte prediction yields realistic orderbook events","Tokenizer-free byte model reproduces market stylized facts","Orderbook generation directly from 32-byte events","Byte-level AI learns market dynamics without feature engineering"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000558,"raw_usage":{"total_tokens":2686,"prompt_tokens":1011,"completion_tokens":1675,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":627,"completion_tokens_details":{"reasoning_tokens":1599}},"tokens_in":627,"tokens_out":1675,"duration_ms":13573,"temperature":1.0,"reasoning_tokens":1599,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T05:02:29.111204+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train ByteGen and an otherwise matched tokenized baseline on the same Bitcoin futures data and compare realized bid-ask spread, depth, price distribution, and return-tail statistics across several seeds; if the tokenized baseline matches or beats ByteGen on these metrics, the claim that byte-level representation avoids tokenization bias is unsupported.","supporting_citations":[],"review_version":1}