{"id":"2ac52a5b-7ac0-41b0-9868-26f1fb440230","arxiv_id":"2508.16589","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A 4-action reinforcement learning market maker trained with Hawkes order arrivals at low volatility continues to provide two-sided quotes over 92% of the time and holds stable Sharpe ratios when tested at 100x higher volatility in simulation.","lead":"This paper trains market-making agents with adversarial reinforcement learning and a Hawkes process order-flow model, then stress-tests them by raising volatility from 2 to 200. The main finding is that a 4-action agent (bilateral, one-sided, or no quoting) trained in low volatility stays stable and quotes two-sided over 92% of the time in the high-volatility regime. The result is a simulation outcome, not a live-market validation.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"At vol=200 the price-setting sub-policy (Always Quoting, trained at vol=2) can only use offsets in [0,3] while per-step price std is ~14; the 4-action agent's >92% quoting ratio may be a non-trading artifact, not adaptation.","rationale":"The reader's conditional verdict is appropriate, but we sharpen the load-bearing risk. The paper's own numbers: in Table 5 All, the 4A v2/v200 has 92.02% bilateral quoting, which is likely the source of the abstract's 'at least 92%'. But Table 5 Fix (eta=1.0, Fixed adversary) appears to show 45.45% no-quote and only 46.96% bilateral for the same agent, which would contradict the abstract; at minimum, the paper never specifies whether 'at least 92%' is an aggregate, a minimum, or a selected row. That ambiguity, together with the missing fill data, makes the central claim fragile. The key mechanism we propose—offset saturation—is directly testable and is a superset of the reader's concern: even if the Always Quoting policy does not 'fail' in a way that collapses Sharpe, the quoting ratio is not evidence of market-making activity unless the quotes are executable. The paper also admits the Hawkes process underperforms Poisson (§5.2), undercutting the broad claim that Hawkes 'significantly enhances effectiveness.' These issues justify keeping the conditional verdict; additional instrumentation and artifact release are needed.","tokens_in":15917,"tokens_out":17907,"duration_ms":162664,"concrete_test":"Instrument the vol=200 evaluation of 4-Action MM (Train @ vol=2, Test @ vol=200) to log, per episode: the bid/ask offsets chosen by the Always Quoting sub-policy, the number of trades, and the proportion of time both quotes are within one tick of the mid-price. Also run the Always Quoting MM (Train & Test @ vol=2) directly at vol=200 and report its Sharpe and expected fills. If average offsets are at or near the 3.0 upper bound and fills per 1000-step episode are close to zero, the adaptation result is an artifact. As a further check, retrain the Always Quoting sub-policy at vol=200 (or extend the offset range) and re-run the 4-action experiment; if the high quoting ratio disappears or Sharpe changes materially, the original claim is not robust.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on the 4-action MM (Train @ vol=2, Test @ vol=200) using the vol=2-trained Always Quoting MM to set prices. Under §3.1.1, with sigma=200 and dt=0.005, the per-step price standard deviation is 200*sqrt(0.005) ≈ 14.14. But the Always Quoting action space (§3.2.2) is bounded to offsets in [0,3], i.e., less than a quarter of one per-step standard deviation. A policy trained at sigma=2 (per-step sd ≈ 0.14) cannot widen spreads proportionally; it saturates at 3. The Hawkes execution intensity (§3.1.2) is lambda = Hawkes_intensity * exp(-k*delta); with k ≈ 1.5 and delta=3, the intensity is ~1% of baseline, so fills per episode become very rare. In that regime the 4-action policy's choice to 'quote' is nearly costless, and the reported stable Sharpe and >90% quoting ratio may simply reflect the agent rarely trading, not effective adaptation. The paper reports no fill counts, average offsets, or spread widths, so this artifact cannot be ruled out. If the sub-policy is not actually providing executable two-sided liquidity at vol=200, the central claim fails.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper extends the authors' prior ARL market-making framework ([30]) by replacing Poisson execution arrivals with a Hawkes process and by testing a four-action market maker (no quote, bilateral, ask-only, bid-only) under volatility levels 2 and 200. The headline claim is that a 4-action MM trained at volatility 2 and tested at volatility 200 adapts effectively, preserving Sharpe-type performance and providing two-sided quotes at least 92% of the time. The evaluation is reported in seven tables across risk-coefficient configurations, with several adversarial training regimes.","tokens_in":16192,"tokens_out":11506,"duration_ms":120522,"significance":"If established, the result would be a useful robustness finding for RL-based market making: a discrete quoting policy trained in a calm market could remain profitable and liquid in a much more volatile market. The paper also responsibly reports that the Hawkes-process agents perform worse in absolute terms than the earlier Poisson-process results, which is a useful caveat. The tables provide a large amount of comparative information. However, several load-bearing issues prevent the stated contribution from being accepted as written: the 92% quoting claim is contradicted by multiple entries in the authors' own tables, there is an internal inconsistency about the test volatility of one agent, and the economic meaning of the quoting decision at volatility 200 is not established without fill and spread diagnostics.","major_comments":[{"comment":"The abstract and Section 1 claim that the 4-action MM trained at vol=2 and tested at vol=200 provides two-sided quotes at least 92% of the time. This is contradicted by the tables. Bilateral quote ratios below 92% include: Table 1, row A: 77.27%; Table 2, Fix: 69.74%; Table 5, Fix: 46.96%; Table 6, Fix: 31.44%; Table 7, Fix: 48.77% and All: 74.64%. In these rows the agent instead chooses no-quote or unilateral quoting. The central claim must be restricted, or the heterogeneity must be explained and the abstract revised.","section":"Abstract; Tables 1–7, column '4-Action MM (Train @ vol=2, Test @ vol=200)'"},{"comment":"Section 5.1 states that the 4-Action MM (Train & Test @ vol=200) is 'trained in an environment with Volatility=200 ... but ultimately tested in an environment with Volatility=2.' The table headers label this column '4-Action MM (Train @ vol=200, Test @ vol=200)', and Section 5.2 discusses it as a high-volatility-tested agent. This is a direct inconsistency. The test environment determines whether the reported conservatism (e.g., bid-only 21.80% in Table 1 All row, or unilateral quoting in Table 7) supports the paper's conclusion about high-volatility training. The text and labels must be reconciled.","section":"Section 5.1 vs Table headers and Section 5.2"},{"comment":"The 4-action MM's 'quote' decision delegates price setting to the Always Quoting MM trained at vol=2. The sub-policy's offsets are bounded in [0,3] (Section 3.2.2), while at vol=200 with dt=0.005 the per-step price standard deviation is about 14.14 (Section 3.1.1). The paper does not report average offsets, fill counts, or spread widths at test time, so the reader cannot tell whether the observed bilateral quote ratios represent executable two-sided liquidity or nearly inert quotes. I do not press the strongest version of this concern: terminal wealth and inventory in Tables 1–7 are nonzero, so some trading does occur. But without these diagnostics, the 'effective adaptation' claim is not fully supported.","section":"Section 5.2, '2-Action MM ... & 4-Action MM ...'; Sections 3.1.1 and 3.2.2"},{"comment":"The 'stable performance' claim rests on point estimates. Sharpe ratios are E/σ computed from the evaluation runs, but no confidence intervals, standard errors, or significance tests are reported. For example, Table 1 Fix gives Sharpe 0.6810 for 4-Action MM (Train & Test @ vol=2) and 0.6837 for 4-Action MM (Train @ vol=2, Test @ vol=200); many differences are of this size and will be within sampling noise. Please provide uncertainty quantification, at least for the key train@2/test@200 versus train&test@2 comparison.","section":"Section 5.1; Tables 1–7"}],"minor_comments":[{"comment":"The phrase 'evaluated 100 times, each with 1000 episodes' is ambiguous: does this mean 100 independent evaluations of 1000 episodes each, or 100 episodes in 1000 runs? Please clarify the total number of evaluation trajectories and how the reported means and standard deviations are aggregated.","section":"Section 5.1"},{"comment":"The quantity 'last_match_result' is used in the Hawkes intensity update but is not formally defined. State that it is the indicator of a trade in the previous time step, and specify whether the market maker observes its own fills or all market trades.","section":"Section 3.1.2, Eq. (1)"},{"comment":"Terminal Wealth is described as 'mean ± variance' but the tables report standard deviations (e.g., 3.2613 when the mean is 2.1945). Please use 'mean ± standard deviation' for clarity.","section":"Section 5.1"},{"comment":"The Hawkes parameters (mean_reversion_speed=60, baseline_arrival_rate=10, jump_size=40, dt=0.005) are taken from the mbt_gym defaults without sensitivity analysis or calibration. At least one robustness check would strengthen the claim that the results are not tied to this particular parameter choice.","section":"Section 3.1.2 / Section 4"},{"comment":"The description of the figures is terse. In particular, Figure 2 is said to contain 'six curves' and color-coded Sharpe ratios, but the caption and text do not identify which curve corresponds to which agent/adversary configuration. Please add a legend or a more complete caption.","section":"Figures 2–7"}],"recommendation":"major_revision","confidential_remarks":"The paper is an eight-page conference-style manuscript whose headline claim is stronger than the data support. The internal inconsistency about the test volatility of the 4-Action MM (Train & Test @ vol=200) is the kind of issue that must be fixed before any publication decision. The self-citations to the authors' prior work [30] are extensive but not disqualifying; the real novelty is the Hawkes and volatility extension, and the paper should be judged on that. I would like to see a revised version that corrects the 92% claim, reconciles the test-volatility descriptions, and adds fill/spread diagnostics."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Ziyi, quick take on the ARL market-making paper.\n\nWhat's new: they take the 4-action quote/don't-quote framework from their earlier work, drop in a Hawkes execution model from mbt_gym, and then train at vol=2 and test at vol=200. The specific result—that a 4-action agent trained at low vol still quotes bilaterally >92% of the time at high vol with Sharpe ratios close to the baseline—isn't in prior work. That's a legitimate new observation, and the paper is honest that the Hawkes process underperforms the Poisson baseline, which I read as a point in their favor.\n\nSoft spots, in proportion. The biggest issue is the numbers don't obviously add up. Price dynamics are a random walk with σ=200 and dt=0.005, so per-step price std is about 14 and terminal price std over 1000 steps should be hundreds. They report terminal wealth std around 3, with terminal inventory mean ~1.4. That combination doesn't make sense unless the terminal inventory is always near zero (it isn't) or the volatility isn't actually being applied the way the equations say. Something is off. If the 'high volatility' environment is not actually more volatile in the P&L sense, the central adaptation claim collapses to 'the agent behaves similarly in a slightly different environment.' The paper gives no code, no data, and no diagnostic stats—no fill counts, no average quote offsets, no actual price path variances—so I can't tell whether this is a typo in the text or a bug in the simulator. The stress-test note about quote offsets [0,3] being a quarter of the per-step std is plausible but not directly supported by the execution formula; execution intensity depends on offset and Hawkes intensity, not on σ. The deeper point stands, though: the price-setting sub-policy was trained at vol=2 and cannot widen offsets beyond 3, so it's reasonable to ask whether the 4-action agent is really providing executable two-sided quotes at vol=200 or just going through the motion of quoting that rarely results in trades.\n\nAlso, the comparison between low- and high-vol training is confounded: both use the vol=2-trained adversary and vol=2-trained Always Quoting sub-policy, so any differences could come from those fixed pieces rather than the agent's own adaptation. The paper's own conclusion hints at this for the Train & Test @ vol=200 case.\n\nWho's this for? People working on RL market-making in stylized simulators. It's a conference paper with a clean idea but a potentially fatal implementation ambiguity. I'd send it to peer review because the claim is falsifiable and the check is easy: ask the authors for code, a sanity check on the volatility implementation, and fill rates at vol=200. If the volatility is working, the paper is a modest but useful robustness study; if not, it's a simulation artifact. Either way, it deserves referee time to sort that out.","headline":"The 4-action adaptation claim is a real empirical finding, but the reported wealth variance at vol=200 looks too small for the modeled random walk, which makes me worry the volatility change isn't actually taking effect as described.","tokens_in":16756,"tokens_out":4484,"would_cite":false,"duration_ms":53255,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A market-making agent that chooses when to quote, rather than recomputing prices, keeps its edge when volatility jumps 100-fold.","keywords":["market making","adversarial reinforcement learning","Hawkes processes","limit order book","high-frequency trading","volatility robustness","flexible quoting","stochastic optimal control"],"falsifier":"Measure realized fill rates, time-to-fill, and fill-conditional P&L at $\\sigma=200$ for the four-action policy trained at $\\sigma=2$, with the same offset bounds $[0,3]$. If fills are rare or systematically one-sided, the stable terminal wealth is an artifact of non-execution; if fills occur at reasonable rates and capture spread, the adaptation claim is supported.","tokens_in":15711,"feed_emoji":"📈","tokens_out":11115,"duration_ms":110420,"temperature":0.7,"pith_summary":"This paper claims that a market-making agent with a discrete choice — quote both sides, quote one side, or stop quoting — can absorb a hundredfold increase in volatility without being retrained. Trained at volatility 2 under an adversary, with executions arriving through a self-exciting Hawkes process, the four-action agent tested at volatility 200 keeps its Sharpe ratio and terminal wealth close to the low-volatility baseline and provides two-sided quotes at least 92% of the time. The load-bearing design is separation: the high-level policy decides when to quote, while a frozen price-setting policy trained at volatility 2 chooses the actual bid/ask offsets. The authors interpret this as evidence that flexible quoting plus realistic order flow yields robustness to regime change. The paper also acknowledges that replacing Poisson arrivals with Hawkes arrivals lowers absolute performance, so the gain is in robustness rather than raw profitability.","feed_headline":"Market maker trained in calm markets survives 100x volatility jump","feed_subtitle":"Flexible quoting keeps two-sided quotes at or above 92% after a 100-fold volatility shock.","key_machinery":"The core mechanism is a two-level decision structure. A high-level discrete policy chooses one of four actions: no quote, bilateral quotes, ask-only, or bid-only; whenever it chooses to quote, the actual bid and ask offsets come from a frozen Always Quoting agent trained at volatility 2. Execution intensity is $$\\$lambda_n^{{\\pm}}$ = \\text{Hawkes\\_intensity}\\,$e^{{-k_n^{\\pm}}$\\$delta_n^{{\\pm}}$},$$ where $\\delta_n^{\\pm}$ are the bid/ask offsets and $k_n^{\\pm}$ is the volume-decay parameter; the intensity itself mean-reverts toward a baseline and jumps upward on each fill, which is the self-exciting feature that replaces Poisson arrivals. The adversary controls drift, baseline arrival rate, and volume de","core_discovery":"The central claim is that a four-action market maker trained with adversarial reinforcement learning under low volatility generalizes to a high-volatility regime without retraining. The agent chooses among four discrete quoting modes — no quote, bilateral, ask-only, bid-only — and, when quoting, uses offsets from a frozen Always Quoting agent trained at volatility 2. Execution follows a self-exciting Hawkes intensity that mean-reverts and jumps on fills, replacing the Poisson arrivals of earlier work. Across seven risk-coefficient tables and five adversary types, the policy trained at $\\sigma=2$ and tested at $\\sigma=200$ keeps terminal wealth and Sharpe close to the low-volatility baseline","pith_inferences":["The paper does not report fill rates or spread capture at $\\sigma=200$; computing those would separate genuine adaptation from quotes that are posted but rarely marketable.","A testable extension: retrain only the price-setting policy at $\\sigma=200$, keep the four-action supervisor frozen, and check whether bilateral quoting and Sharpe improve; this would isolate whether the discrete policy or the offset policy carries the robustness.","Because quote offsets are bounded in $[0,3]$ while price increments scale with $\\sigma$, scaling offset bounds with volatility could turn the reported stability into higher profitability; this is an editorial extrapolation, not a claim in the paper.","The paper acknowledges that Hawkes arrivals lower mean wealth and Sharpe relative to Poisson; a Poisson-baseline comparison at $\\sigma=200$ would show whether the volatility-transfer result is an effect of the arrival process or of the flexible action space alone."],"forward_implications":["A four-action quoting space generalizes across a 100-fold volatility change: the agent trained at $\\sigma=2$ and tested at $\\sigma=200$ keeps Sharpe ratio and terminal wealth close to baseline across all seven risk-coefficient tables.","The same agent meets exchange-style quoting obligations under stress: bilateral quotes appear at least 92% of the time, above the 90% continuous-quoting requirement the paper cites for regulated market makers.","The mismatch appears in price setting, not the action space: the agent trained and tested at $\\sigma=200$ with a frozen $\\sigma=2$ price policy quotes conservatively and has lower Sharpe.","Hawkes arrivals change quoting incentives: because a fill raises the intensity of further fills, agents quote more aggressively after trades, which the paper argues supports liquidity in low-liquidity states."],"supporting_citations":[{"why":"supplies the price dynamics, bid/ask offsets, and inventory accounting the simulation is built on.","marker":"[3]"},{"why":"supplies the soft actor-critic algorithm used to train the adversary and the Always Quoting price-setting agent.","marker":"[16]"},{"why":"supplies the self-exciting Hawkes arrival model (mean reversion, baseline, and jump on fill) used for execution dynamics.","marker":"[20]"},{"why":"establishes the adversarial reinforcement learning training method for robust market making that this paper extends.","marker":"[27]"},{"why":"introduces the flexible multi-action quoting problem and the 2- and 4-action agents on which this paper builds.","marker":"[30]"}],"fun_headline_variants":["Calm-trained market maker sails through 100x volatility storm","Four-action market maker shrugs off 100-fold volatility shock","Flexible quoting keeps market maker stable after 100x volatility jump","Trained at volatility 2, market maker holds up at volatility 200"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The price-setting sub-policy trained at volatility 2 must still produce sensible bid/ask prices when the price process moves at volatility 200; if its offsets become stale or nonmarketable, the four-action agent's stable numbers could simply reflect that its quotes rarely trade.","fun_headline_variants_meta":{"raw":{"variants":["Calm-trained market maker sails through 100x volatility storm","Four-action market maker shrugs off 100-fold volatility shock","Flexible quoting keeps market maker stable after 100x volatility jump","Trained at volatility 2, market maker holds up at volatility 200"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000245,"raw_usage":{"total_tokens":1334,"prompt_tokens":668,"completion_tokens":666,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":412,"completion_tokens_details":{"reasoning_tokens":591}},"tokens_in":412,"tokens_out":666,"duration_ms":6190,"temperature":1.0,"reasoning_tokens":591,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T23:05:02.189233+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure realized fill rates, time-to-fill, and fill-conditional P&L at $\\sigma=200$ for the four-action policy trained at $\\sigma=2$, with the same offset bounds $[0,3]$. If fills are rare or systematically one-sided, the stable terminal wealth is an artifact of non-execution; if fills occur at reasonable rates and capture spread, the adaptation claim is supported.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the price dynamics, bid/ask offsets, and inventory accounting the simulation is built on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the self-exciting Hawkes arrival model (mean reversion, baseline, and jump on fill) used for execution dynamics."},{"cited_title":"Robust Market Making via Adversarial Reinforcement Learning","cited_arxiv_id":"2003.01820","evidence_quote":"establishes the adversarial reinforcement learning training method for robust market making that this paper extends."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"introduces the flexible multi-action quoting problem and the 2- and 4-action agents on which this paper builds."}],"review_version":1}