{"id":"e9aea4cb-c59d-449e-95b8-134589bbeae6","arxiv_id":"2507.19788","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"MORL/D, a decomposition-based multi-objective RL method, yields the most balanced Pareto-front approximations across three supply chain network complexities when compared with weighted-sum PPO and NSGA-II.","lead":"The paper builds a generalized multi-objective supply chain model with profit, greenhouse gas emissions, and service-level fairness goals, and compares three algorithms on simple, moderate, and complex networks. It reports that a decomposition-based multi-objective reinforcement learning method, MORL/D, gives the most balanced trade-off between solution quality, diversity, and density.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The internal 'true PF' used for AHD is the non-dominated union of the algorithms' own outputs, making optimality comparisons circular; hypervolume reference points are hand-picked and untested, so the 'balance' claim is not yet established.","rationale":"I read the paper as proposing a generalized MOMDP supply-chain model plus a three-way algorithmic comparison, with the central claim that MORL/D gives the most balanced Pareto-front approximation. The model, the Messiah simulator, and the multi-seed experiments across three network sizes are genuine assets. The weakest point is the evaluation reference. Section 4.3 constructs the 'true PF' by merging the very solution sets being compared and selecting non-dominated points via Eq. (15). Any diversity-oriented algorithm that reaches a previously unpopulated region inserts its own points into the reference, so its IGD/AHD to that reference is partly self-comparison. This directly undermines AHD, the only metric the paper uses to correct for different numbers of solutions per front. The hypervolume reference points are also hand-picked with no sensitivity analysis, and the paper itself notes that profit's larger scale can dominate hypervolume contributions. Since the 'balance' claim combines optimality (hypervolume/AHD) with diversity and density (sparsity), the unsupported optimality leg makes the central claim conditional rather than demonstrated. I agree with the reader's CONDITIONAL verdict; the natural remedy is an independent reference set and reference-point sensitivity analysis, plus ideally release of the simulator. I also noticed an internal contradiction in Section 5.2.3/Figure 11 about which method has the lowest demand loss; it is secondary to the PF-balance claim but reinforces the need for careful revision.","tokens_in":29551,"tokens_out":10735,"duration_ms":134528,"concrete_test":"Recompute the AHD comparison in Section 5.2.1 using an independent reference PF rather than the union of the three algorithms' own outputs. For the simple and moderate SC instances, build the reference by running a high-budget MOEA/D or NSGA-II variant with a population of at least 1000 and 10x the generations reported in Table 7, or by discretized exhaustive search on a shortened horizon; then apply Eq. (32) to all three methods. If MORL/D no longer has the best AHD, or if the PPO/NSGA-II ordering changes, the circular-reference concern is confirmed and the 'balance' claim lacks support.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline claim that MORL/D provides the most balanced Pareto-front approximation rests on comparing optimality, diversity, and density. The optimality leg is measured by hypervolume and by AHD (Eq. 32), and AHD is computed against a 'true PF' that Section 4.3 defines as the non-dominated subset of the union of all three algorithms' own solution sets, selected via Eq. (15). This reference is not independent of the algorithms being compared: MORL/D is specifically designed to produce diverse points across the front, so any region it newly populates becomes part of the reference against which AHD is measured. An algorithm that contributes extreme or diverse points to the union automatically has zero generational distance to its own points in the reference, while competitors that do not cover that region are penalized in the inverted generational distance term. The balance conclusion therefore contains a circular component for any diversity-sensitive indicator. Hypervolume is also evaluated against hand-chosen reference points (Section 5.1.1: {0, -2e5, -100}, {0, -4e5, -200}, {0, -1e6, -500}), described as 'slightly lower than the minimum possible points,' but no sensitivity analysis is reported. The paper itself notes that the profit scale is much larger than the other objectives, so hypervolume can be dominated by a single objective. If an independent reference PF or different reference points change the optimality ranking, the claimed balance between optimality, diversity, and density is not supported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper develops a generalized multi-objective, multi-echelon supply chain (SC) model cast as a multi-objective Markov decision process (MOMDP), together with a customisable SC simulator called Messiah. The authors compare three solution approaches: MORL/D (decomposition-based multi-objective reinforcement learning) with and without shared experience buffer and Pareto simulated annealing, a modified single-objective PPO with weighted-sum scalarisation, and NSGA-II. Experiments are run on three SC network configurations of increasing complexity, with objectives of profit maximisation, GHG emission minimisation, and service-level inequality minimisation. The central claim is that MORL/D provides the most balanced Pareto-front approximation in terms of optimality, diversity, and density, and that a shared experience buffer improves its performance. The abstract further claims up to 75% higher hypervolume than the MOEA-based method and roughly eleven times denser solutions than the weighted-sum PPO in complex settings.","tokens_in":29844,"tokens_out":9542,"duration_ms":108823,"significance":"If the comparative evaluation were fully sound, the paper would make a useful contribution by demonstrating a decomposition-based MORL algorithm with a shared experience buffer on a reasonably realistic, multi-echelon, multi-objective SC problem with three sustainability pillars. The model formulation is explicit and the Messiah simulator is potentially a reusable environment. However, the evidence for the central 'most balanced' claim is weakened by several methodological issues: the AHD reference front is constructed from the algorithms' own outputs, hypervolume reference points are hand-chosen without sensitivity analysis, solution-set cardinalities differ substantially across methods, and the quantitative claims in the abstract are not backed by any table in the text. The paper ships no code, data, or full parameter tables, so the experiments cannot be reproduced. These issues are fixable within the scope of a revision, but the current empirical support is not robust enough for the headline conclusions.","major_comments":[{"comment":"The 'true PF' used for AHD is defined by 'merging all resultant solution sets across all algorithm runs and then selecting the best PF approximation set using Equation (15)'. This reference set is not independent of the compared algorithms. MORL/D is explicitly designed to produce diverse solutions, so any region that it newly populates is added to the reference and then penalises other algorithms in the inverted generational distance term. Consequently, the AHD results in Figure 7 and the 'most thoughtful balance' conclusion in Section 5.2.1 are partly circular. Please compute AHD against an independent reference front (for example, from a much larger multi-start run, a dedicated global solver, or a leave-one-algorithm-out cross-validation), and report GD and IGD separately.","section":"Section 4.3, Eq. (32)"},{"comment":"The hypervolume reference points are chosen by hand as 'slightly lower than the minimum possible points', with values {0, -2e5, -100}, {0, -4e5, -200}, and {0, -1e6, -500}. No sensitivity analysis is reported, even though the text itself notes that the profit scale is much larger than the other objectives and can dominate hypervolume. The ranking between PPO and MORL/D could change under alternative reference points or after objective normalisation. Please provide a sensitivity analysis over reference points, report hypervolume with normalised objectives, and include numeric values with standard errors. The abstract's '75% higher hypervolume' claim needs a concrete table reference.","section":"Section 5.1.1"},{"comment":"The reward penalty is defined as rho_t = sum_j min(I_tj, 0)*M. When inventory is negative, min(I_tj,0) is negative, so rho_t is non-positive. The reward vector is then defined as {Profit_t, -E_t, -F_t} * rho_t, which multiplies all rewards by a non-positive scalar. This flips the intended sign of every objective: profit becomes negative, while -E_t and -F_t become positive, so maximising the total return would maximise emissions and inequality rather than minimise them. This is inconsistent with the optimisation model's constraint (10) and with the stated maximisation convention. Please correct the reward definition (likely an additive penalty or a positive scaling with a sign-preserving structure) and verify that the RL training signal is not corrupted.","section":"Section 3.3.4, Eq. (24)"},{"comment":"The compared methods produce solution sets of very different sizes: PPO yields 21 points per set, MORL/D yields 30 to 70 points, and NSGA-II yields 50 to 130 points. Hypervolume and sparsity are sensitive to the number of points in the front, so the density comparisons in Figure 6 and the abstract's 'approximately eleven times denser' statement are not controlled for cardinality. AHD is introduced to mitigate this, but its reference set is the problematic circular construct noted above. Please equalise the number of points per front (e.g., by subsampling to a common cardinality) or use indicators that are unbiased with respect to cardinality, and report how the results change with front size.","section":"Section 5.2.2 and Table 8"},{"comment":"The experimental results are reported only as figures; the precise quantitative claims in the abstract ('up to 75% higher hypervolume' and 'approximately eleven times denser') do not appear in the main text or in any table. In addition, Section 5.2.2 states that the authors 'select the optimal PF approximation sets for each approach based on their hypervolume analysis' before visual comparison, which can bias the comparison even if the intention is only illustrative. Please add a table with means and standard deviations across all runs for hypervolume, sparsity, EUM, and AHD for each scenario and algorithm, state the number of independent runs, and replace the best-run selection with an analysis of the full distribution of results.","section":"Section 5.2 and results"},{"comment":"The paper refers to a supplementary material with 'the values of the detailed parameters' but the arXiv submission does not include it, and the Messiah simulator is described extensively but not released. No code, data, or seeds are provided. Given the number of hyperparameters and the in-house simulator, the experiments are not reproducible. Please provide the simulator code or a link, the complete parameter tables, and all seeds, or clearly state the conditions under which the implementation is available.","section":"Section 5.1.1 and Section 4.2.3"}],"minor_comments":[{"comment":"The summation limits in Eq. (11) are written as 'sum_{t=(t-L)}^T', where the index t also appears in the lower limit; this is not well-formed. Please clarify the intended time window for the service-level calculation. A similar notational issue appears in Eq. (28).","section":"Section 3.1, Eq. (11)"},{"comment":"The text says 21 pre-defined weight vectors 'whose sum values range from 0 to 1'; this should read 'whose components sum to 1', since the weights define a convex combination. The current phrasing is ambiguous.","section":"Section 4.2.1"},{"comment":"The hypervolume indicator is described as 'monotonic'; the standard technical property is that it is strictly Pareto-compliant. The weaker word can be misleading in a comparative study.","section":"Section 4.3"},{"comment":"The statement that the authors 'select the optimal PF approximation sets for each approach based on their hypervolume analysis' is not a statistically robust way to compare algorithms. Even if it is meant only for illustration, the selection criterion should be reported and the full set of runs should be shown in the supplementary figures.","section":"Section 5.2.2"},{"comment":"The computational time comparison mixes runs with different numbers of seeds and different ways of constructing a final front (e.g., PPO runs 21 weighted policies per seed, while MORL/D and NSGA-II run once per seed). A fairer comparison would report the total compute budget needed to produce a final front of a fixed size.","section":"Table 8 and Section 5.1.2"},{"comment":"Several references are incomplete: the Gymnasium citation in Section 4.2.3 lacks a year and venue, and the Reymond et al. (2022) citation lacks page numbers or a DOI. Please check that all references are complete and consistent.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is an extension of the authors' prior EMO paper (Rachman et al., 2025); the modelling novelty is incremental, but the generalised network representation and the MORL/SC benchmarking may still be publishable if the evaluation is substantially strengthened. The absence of code and data, plus the internal-reference and reward-typo issues, currently prevent verification of the central claims. I would not recommend rejection if the authors can address the evaluation protocol and reproducibility concerns, but the revision effort is substantial."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take. The paper's real contribution is the model: a clean, general multi-echelon MOMDP formulation that covers economic, environmental, and social objectives with flexible echelons and non-stationary demand. That generalization from their earlier case study is genuinely useful for anyone working on RL for supply chains, and the Messiah simulator description is detailed enough to reconstruct the environment even if the code isn't public. The three-way comparison across simple/moderate/complex networks is a sensible experimental design, and the operational analysis (production stability, inventory, demand loss) adds value beyond raw Pareto-front metrics.\n\nThe soft spot is the evaluation, and it's a real one. The 'true PF' used for AHD is defined as the non-dominated union of the three algorithms' own outputs (Section 4.3). That makes the optimality comparison partly circular: if MORL/D populates a new region, that region becomes part of the reference, and competitors get penalized for not being there. This is a standard practical workaround, but it undermines the strength of the 'most balanced' claim. Hypervolume reference points are hand-picked with no sensitivity analysis, and the paper itself admits the profit scale dominates the other objectives, so the hypervolume numbers may reflect profit more than a genuinely balanced front. Solution counts also differ across methods (21 for PPO, 30–70 for MORL/D, 50–130 for NSGA-II), which directly affects density and sparsity comparisons. Without released code or an independent reference, these issues are addressable but not yet addressed.\n\nThe paper is not sloppy; it does acknowledge the scale imbalance, and the visualizations and operational behavior analysis do support the qualitative conclusion that MORL/D produces more spread and denser solutions. The EUM instability of PPO is visibly real. So I'd call the central ranking plausible but unproven by the current evidence.\n\nThis paper deserves a serious referee. The model and the benchmark are useful to the MORL-for-supply-chain community, and the evaluation flaw is fixable. I'd send it to review with a request for the authors to release the simulator or a reimplementation, add reference-point sensitivity analysis, and explicitly justify or replace the internal 'true PF' construction. If they do that, the paper becomes a solid contribution rather than a conditional one.","headline":"A genuinely useful generalized MOMDP formulation for multi-echelon supply chains, but the algorithm ranking is built on a self-referential benchmark; the 'MORL/D is most balanced' claim is not yet proven.","tokens_in":30388,"tokens_out":3318,"would_cite":false,"duration_ms":37462,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["90C29","90C40","90B06"],"pacs":[],"model":"deepseek-v4-flash","headline":"A decomposition-based multi-objective reinforcement learning method with a shared experience buffer is the most balanced approach the authors test for approximating the Pareto front in non-stationary, multi-echelon supply chains balancing…","keywords":["multi-objective reinforcement learning","supply chain optimisation","multi-echelon network","Markov decision process","Pareto front approximation","shared experience buffer","sustainability"],"falsifier":"Run the simple network to exact Pareto optimality by exhaustive enumeration over its eight action dimensions, then recompute hypervolume, sparsity, and averaged Hausdorff distance using a fixed, independently generated reference front; if MORL/D's hypervolume no longer ranks as the best balanced trade-off or its density advantage shrinks below the reported eleven-fold, the central balance claim fails.","tokens_in":1740,"feed_emoji":"📦","tokens_out":1773,"duration_ms":84189,"temperature":0.7,"pith_summary":"This paper tries to establish that a decomposition-based multi-objective reinforcement learning algorithm, MORL/D, is the most balanced method for approximating the Pareto front in non-stationary, multi-echelon supply chain problems that trade off profit, greenhouse-gas emissions, and service-level inequality. It builds a generalised multi-objective Markov decision process formulation of the supply chain, provides a configurable simulator, and benchmarks MORL/D against weighted-sum PPO, a single-objective RL algorithm adapted to multiple objectives, and NSGA-II, a multi-objective evolutionary algorithm. In complex networks the paper reports up to 75% higher hypervolume than NSGA-II and roughly eleven times denser solution sets than weighted-sum PPO, while also producing more stable production and inventory levels and lower demand loss. A sympathetic reader would care because, if correct, the result gives practitioners a concrete MORL configuration that jointly covers optimality, diversity, and density in a class of problems where social objectives are rarely included.","feed_headline":"Decomposition-based RL is the most balanced supply-chain optimizer","feed_subtitle":"In the toughest network it reports 75% higher hypervolume than NSGA-II and 11 times denser results than weighted-sum PPO.","key_machinery":"The load-bearing object is the multi-objective Markov decision process formulation of the supply chain: the state includes inventory, outstanding orders, cumulative emissions, and average service-level inequality; actions are per-period manufacturing and delivery quantities; and rewards form a three-dimensional vector of profit, negative emissions, and negative service-level inequality, with an inventory-penalty term. MORL/D is the central mechanism: it decomposes the three objectives into scalarised subproblems, maintains a population of policies with an external Pareto archive, uses the multi-objective soft actor-critic algorithm as the underlying continuous-action policy, optionally adapts weights via Pareto simulated annealing, and transfers experience between neighbouring subproblems through a shared buffer, producing a dense, diverse approximation of the Pareto front in a single run.","core_discovery":"On the paper's own terms, the central discovery is that MORL/D generalises an earlier case-specific study to arbitrary network topologies and, with a shared experience buffer that lets subproblem policies exchange experience, offers the best tested balance of optimality, diversity, and density. The paper reports that MORL/D's hypervolume and expected utility metric rise steadily during training while sparsity falls, that the shared buffer improves hypervolume, EUM, and averaged Hausdorff distance in all three networks, and that Pareto simulated annealing weight adaptation helps in the more complex settings. It also finds that the apparent hypervolume winner, weighted-sum PPO, is actually the sparsest and becomes unstable in EUM as complexity grows, and that NSGA-II concentrates solutions in narrow, lower-quality regions. The conclusion is that decomposition-based MORL with experience sharing is the recommended approach for this generalised class of sustainable, non-stationary supply chain problems.","pith_inferences":["Editorial inference: the paper's 'true Pareto front' is assembled from the algorithms' own solution sets, so an independent reference constructed by exhaustive enumeration on the simple network or by a much larger external archive would be a stricter test of the reported hypervolume and density gaps.","Editorial inference: the eleven-fold density advantage partly follows from MORL/D returning 30 to 70 points while weighted-sum PPO returns a fixed 21; normalising density by archive size or reporting density per unit hypervolume would give a fairer operational comparison.","Editorial inference: because profit is far larger in scale than the other two objectives, hypervolume and density comparisons are sensitive to the chosen reference points and objective normalisation; the balance claim would be most credible if replicated with several reference points and normalisation schemes.","Editorial inference: a testable extension would run the same MOMDP environment under other MORL algorithms or longer, non-stationary demand regimes to see whether MORL/D's balanced-front advantage persists beyond the three instantiations in the paper."],"forward_implications":["The same MOMDP formulation can be reused across simple, moderate, and complex supply chain networks without rebuilding the model, because its sets and transition functions generalise over echelons, facilities, and markets.","The shared experience buffer is a recommended enhancement: the paper reports it improves hypervolume, EUM, and averaged Hausdorff distance while lowering sparsity across all tested network complexities.","Hypervolume alone is not a reliable robustness signal in this setting, because weighted-sum PPO scores highest on hypervolume while being the sparsest and most unstable on expected utility.","NSGA-II's concentration in narrow feasible regions, especially in the 5,900-decision-variable complex network, suggests that MOEA-based baselines need larger populations or different search operators before they can compete in this class of problems.","MORL/D's reduced sensitivity to demand fluctuations yields more stable production and inventory levels and lower demand loss, although it may sacrifice some precision in lean-inventory management."],"supporting_citations":[{"why":"Supplies the MORL/D algorithm, the paper's primary approach, which decomposes multi-objective problems into scalarised subproblems.","marker":"Felten et al., 2023b"},{"why":"Provides the PPO algorithm that the paper modifies with a weighted sum to create its single-objective-RL baseline.","marker":"Schulman et al., 2017"},{"why":"Defines NSGA-II, the MOEA baseline using non-dominated ranking and crowding distance.","marker":"Deb et al., 2002"},{"why":"Supplies the Das-Dennis method used to generate the 21 predefined weight vectors for weighted-sum PPO.","marker":"Das and Dennis, 1998"},{"why":"Defines the hypervolume performance measure used to compare Pareto front approximations.","marker":"Zitzler and Thiele, 1998"},{"why":"Defines the sparsity metric used to assess density and robustness of solution sets.","marker":"Xu et al., 2020"},{"why":"Defines the averaged Hausdorff distance used to compare fronts with different numbers of solutions.","marker":"Schutze et al., 2012"},{"why":"Supplies the MOMDP taxonomy and the Pareto-front definition used in the model formulation.","marker":"Hayes et al., 2022"},{"why":"Supplies MOSAC, the multi-objective soft actor-critic policy used inside MORL/D.","marker":"Chen et al., 2020"},{"why":"Provides Pareto simulated annealing, the weight adaptation variant tested with MORL/D.","marker":"Czyz˙zak and Jaszkiewicz, 1998"}],"fun_headline_variants":["Decomposition-based RL bests MOEA and weighted-sum RL in supply chains","MORL/D with experience sharing wins on optimality, diversity, density","Shared buffer gives RL the edge in multi-objective supply chain optimization","RL finds denser, more balanced Pareto fronts than MOEA and PPO variants","Decomposition RL beats NSGA-II and PPO in complex supply chain networks"],"cache_read_input_tokens":32512,"weakest_assumption_plain":"The comparison assumes that the 'true Pareto front' formed by merging all algorithms' own solution sets is an unbiased reference, and if that reference is skewed by how many points each algorithm returns or by the hand-chosen hypervolume reference points, the headline claims about hypervolume and density do not establish that MORL/D is more balanced.","fun_headline_variants_meta":{"raw":{"variants":["Decomposition-based RL bests MOEA and weighted-sum RL in supply chains","MORL/D with experience sharing wins on optimality, diversity, density","Shared buffer gives RL the edge in multi-objective supply chain optimization","RL finds denser, more balanced Pareto fronts than MOEA and PPO variants","Decomposition RL beats NSGA-II and PPO in complex supply chain networks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000285,"raw_usage":{"total_tokens":1679,"prompt_tokens":945,"completion_tokens":734,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":561,"completion_tokens_details":{"reasoning_tokens":634}},"tokens_in":561,"tokens_out":734,"duration_ms":7675,"temperature":1.0,"reasoning_tokens":634,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T14:00:17.732114+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the simple network to exact Pareto optimality by exhaustive enumeration over its eight action dimensions, then recompute hypervolume, sparsity, and averaged Hausdorff distance using a fixed, independently generated reference front; if MORL/D's hypervolume no longer ranks as the best balanced trade-off or its density advantage shrinks below the reported eleven-fold, the central balance claim fails.","supporting_citations":[{"cited_title":", author Esquivel, X","cited_arxiv_id":null,"evidence_quote":"Defines the averaged Hausdorff distance used to compare fronts with different numbers of solutions."},{"cited_title":", author Jaszkiewicz, A","cited_arxiv_id":null,"evidence_quote":"Provides Pareto simulated annealing, the weight adaptation variant tested with MORL/D."}],"review_version":1}