{"id":"d4b71edb-104b-484f-9b54-5744c241f798","arxiv_id":"2607.04066","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"low","formal_verification":"none","parameter_count":3,"one_line_summary":"An open-source MATLAB GUI package generates Bayesian D-optimal discrete choice designs via simulated annealing, supporting interactions, opt-outs, order effects, and direct Qualtrics export.","lead":"DCEDesignSA is a free MATLAB toolbox with a point-and-click interface that builds Bayesian D-optimal discrete choice experiment designs using simulated annealing and exports them ready for Qualtrics. It lowers the barrier for researchers in marketing, health, and transport who need efficient preference surveys without writing optimization code.","discovery_kind":"extension","skeptic_critique":{"model":"grok-4.5","headline":"No significant objection identified","rationale":"The reader correctly identifies that the statistical-efficiency advantage of SA is not re-validated inside this manuscript and rests on prior papers. That premise is the weakest link for any comparative claim, yet it is not load-bearing for the paper’s actual contribution—an accessible, feature-complete software tool. The manuscript supplies equations, algorithm, architecture, illustrative run, and public MIT code; these suffice for an ACCEPT software paper. No stronger technical concern (e.g., incorrect information-matrix formula, non-reproducible integration scheme, or broken export) is present. Therefore the reader’s ACCEPT / HIGH verdict stands without adjustment.","tokens_in":7482,"tokens_out":450,"duration_ms":4100,"concrete_test":"Clone the public GitHub repository (v1.0.0), launch the App Designer GUI, and regenerate the laundry-detergent example (4 attributes, main effects + Material×Brand interaction, 26 sets, 2 alternatives + opt-out, partial profile with 1 fixed attribute, order effects on, 600 s time limit). Confirm that a design matrix, Bayesian D-criterion, infinite-error rate, and Qualtrics .txt export are produced without runtime error.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper is a software contribution whose central claim is the existence and usability of an open-source MATLAB GUI that packages a previously published SA algorithm for Bayesian D-optimal DCE designs, with added support for interactions, partial profiles, no-choice, order effects, and Qualtrics export. That claim is internally consistent: architecture (Fig. 1), MNL information matrix and Bayesian criterion (Eqs. 1–4), SA procedure (Algorithm 1), GUI workflow, and repository metadata are all specified. The efficiency-superiority statements relative to CE/SWAP/MF are explicitly delegated to earlier peer-reviewed work [11,13,14] rather than re-demonstrated here; for a software paper this is standard and does not undermine the artifact claim. No derivation gap, missing dependency, or internal contradiction appears that would falsify the strongest claim.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"DCEDesignSA is presented as an open-source MATLAB toolbox (v1.0.0, MIT) with a GUI for constructing Bayesian D-optimal discrete choice experiment designs under the multinomial logit model. The package implements simulated annealing (Algorithm 1) to maximise the Bayesian D-criterion (Eq. 4) obtained by integrating the MNL Fisher information matrix (Eq. 3) over a user-specified prior via spherical-radial quadrature. Supported features include main effects and interactions, partial-profile designs, no-choice alternatives, presentation-order effects, and direct export to Qualtrics-compatible .txt and .csv. The manuscript supplies architecture (Fig. 1), a four-panel GUI workflow, a feature comparison with JMP, Ngene, idefix and choiceDes (Table 2), practical termination guidelines, and a laundry-detergent bottle walkthrough with reported D_B = 8.2926 and zero infinite-error rate.","tokens_in":7657,"tokens_out":1153,"duration_ms":21312,"significance":"If the software works as described, the contribution is practically useful: it lowers the barrier to Bayesian D-optimal DCE design for non-programmers, packages SA (an alternative to the CE/SWAP/MF algorithms used elsewhere), and closes the design-to-survey gap via Qualtrics export. Support for interactions, partial profiles, opt-out, and balanced profile order in one open GUI is a genuine gap relative to Table 2. Strengths include a public GitHub repository, explicit MNL/Bayesian formulae, a fully stated SA skeleton with reheating, and a concrete end-to-end example. The efficiency-superiority argument is delegated to prior peer-reviewed work rather than re-proved here, which is acceptable for a software paper provided the artifact itself is the primary claim.","major_comments":[{"comment":"Algorithm 1 (Section 3.2.2) invokes an unspecified “Exploration Rule” that “adapts to different design contexts, including full profile, partial profile, and balanced profile order design.” The candidate-move set is load-bearing for reproducibility of the method description: without stating which coordinates or profiles are swapped/exchanged under each design mode (and how balance constraints are enforced for order effects), readers cannot assess neighbourhood structure or verify that the GUI implements the SA procedure claimed in [11,13,14]. Please define the Exploration Rule formally (or in a short appendix) for each supported design type.","section":null},{"comment":"Section 2 and the Impact section assert that SA yields superior statistical efficiency to CE designs from existing packages, citing only the authors’ prior papers [11,13,14]. For a software contribution this is not fatal, but the claim is motivational and currently unillustrated in this manuscript. A minimal head-to-head in the laundry-detergent example (e.g., Bayesian D-criterion and infinite-error rate under SA vs a CE/MF baseline for the same prior, S, J, and model) would let readers see the advantage without consulting three external papers and would confirm that the GUI implementation preserves the published gains.","section":null}],"minor_comments":[{"comment":"Section 3.2.2: the three termination criteria (adaptive / cycle / time) are recommended by design complexity, but “adaptive” is never defined operationally (what triggers stop? relative improvement? plateau length?). A one-sentence definition would match the clarity of the cycle and time options.","section":null},{"comment":"Table 2: the comparison is restricted to D-optimal MNL designs, which is stated, but a footnote clarifying that Ngene and JMP support additional models/criteria (outside scope) would prevent over-reading the “No” cells as absolute feature absences.","section":null},{"comment":"Metadata C5 lists “MATLAB R2025”; if the package also runs on earlier releases, stating the minimum tested version would help users. If R2025-only features of App Designer are required, say so explicitly.","section":null},{"comment":"Section 4.1: the prior is described as “zero prior mean vector and an identity prior covariance matrix” plus an ASC mean of 1 for the opt-out. Confirm in the text whether the ASC is included in the identity covariance block or fixed, since that affects the dimension of β and of M(X,β).","section":null},{"comment":"Presentation: several places in the supplied text show concatenated words (e.g., abstract opening, “DCEDesignSAisafreelyavailable”); if these appear in the PDF, re-export with proper spacing. Also standardise “D-optimal” hyphenation and “opt-out / no-choice” terminology throughout.","section":null},{"comment":"Figure 5 caption mentions yellow highlighting for varying attributes; ensure the figure is colour-accessible or add a non-colour cue for print readers.","section":null}],"recommendation":"minor_revision","confidential_remarks":"Fit is appropriate for a software/stat.CO venue (SoftwareX-style structure). The efficiency narrative is self-citation heavy but the prior papers appear peer-reviewed; I would not treat that as a novelty or integrity issue. Main risk is incomplete method description of the Exploration Rule rather than any internal contradiction. I did not independently run the GitHub code."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"This is a clean software contribution, not a methods breakthrough. What is new is the integrated open-source MATLAB App Designer GUI that packages their previously published simulated-annealing search for Bayesian D-optimal discrete choice designs, adds balanced presentation-order designs, and ships a direct Qualtrics .txt export. That combination is missing from JMP, Ngene, idefix, and choiceDes, and it is useful for non-programmers who still want interactions, partial profiles, and opt-outs.\n\nThe technical core is standard and correctly written. Equations 1–4 give the usual MNL utility, choice probabilities, Fisher information, and Bayesian D-criterion; they use Gotwalt-style spherical-radial quadrature; Algorithm 1 is a transparent SA loop with temperature schedule, acceptance probability, and reheating. Architecture (Fig. 1), the four-panel GUI, and the laundry-detergent walkthrough are consistent. Code is public under MIT with version and platform metadata. Table 2 is a fair feature comparison.\n\nThe soft spot is exactly the one the reader flagged, and it is modest for this genre: superiority of SA over coordinate exchange is asserted via three self-citations [11,13,14] without fresh head-to-head numbers inside this manuscript. That is normal for a software paper that re-uses a published optimizer, but readers who care about efficiency will still need those earlier papers. Free parameters (temperature schedule, prior, termination rule) are user-exposed and documented; nothing is hidden. No derivation gaps or internal contradictions.\n\nWho it is for: applied DCE researchers who want a GUI-to-Qualtrics path and are willing to sit in MATLAB. It will not move theory or open new measurement regimes. It deserves a serious software-track referee rather than a desk reject. I would engage with the tool if I needed that workflow; I would not re-cite the efficiency claim without checking the prior papers.","headline":"Solid software paper: open MATLAB GUI packaging prior SA work for Bayesian D-optimal DCEs, with Qualtrics export and order-balance support; efficiency claims rest on earlier papers, not new runs.","tokens_in":8285,"tokens_out":500,"would_cite":false,"duration_ms":4726,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"An open MATLAB GUI uses simulated annealing to generate Bayesian D-optimal discrete choice designs and export them ready for Qualtrics.","keywords":["Discrete Choice Experiment","Simulated Annealing","Multinomial logit model","Bayesian Optimal Design","MATLAB","D-optimal design","partial profile"],"falsifier":"Take a fixed set of design problems (with and without interactions or partial profiles), generate designs with DCEDesignSA under a stated time or cycle budget and with a coordinate-exchange package under the same prior, then compare the realised Bayesian D-errors and infinite-error rates.","tokens_in":8328,"feed_emoji":"📊","tokens_out":539,"duration_ms":14572,"temperature":0.7,"pith_summary":"This paper introduces DCEDesignSA, a free MATLAB toolbox with a graphical interface that builds Bayesian D-optimal designs for discrete choice experiments without requiring users to write code. It maximises the expected information about preference parameters by searching the design space with simulated annealing under a user-specified prior. The tool supports main effects plus interactions, partial-profile designs, no-choice (opt-out) alternatives, and presentation-order effects, then exports the finished design as Qualtrics-ready files. The authors position the package as filling a practical gap between commercial GUI tools and code-only open packages, giving non-programmers access to statistically efficient designs that account for prior uncertainty. A sympathetic reader cares because better experimental designs under realistic uncertainty improve the precision of preference estimates that inform marketing, health, and transport decisions.","feed_headline":"Free MATLAB tool builds efficient choice surveys with annealing","feed_subtitle":"Open GUI maximises Bayesian D-optimality and exports designs straight to Qualtrics","key_machinery":"The Bayesian D-optimality criterion—the expected log-determinant of the multinomial-logit Fisher information matrix under a user prior—maximised by a simulated-annealing search whose exploration rule adapts to full-profile, partial-profile, and order-balanced designs.","core_discovery":"DCEDesignSA is an open-source MATLAB graphical toolbox that employs simulated annealing to construct Bayesian D-optimal designs for multinomial-logit discrete choice experiments. It lets users define attributes, model terms (including interactions), priors, and design settings through a four-panel interface, then returns designs that support full or partial profiles, opt-out alternatives, and balanced presentation order, with direct export to Qualtrics.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["MATLAB GUI builds Bayesian D-optimal DCEs with simulated annealing","Free toolbox uses annealing to craft efficient discrete choice designs","Open MATLAB package optimises choice experiments and exports to Qualtrics","Simulated annealing finds D-optimal designs for multinomial logit surveys","Interactive tool generates Bayesian optimal DCEs supporting interactions"],"cache_read_input_tokens":128,"weakest_assumption_plain":"The claim that simulated annealing yields superior statistical efficiency rests on earlier benchmark studies rather than new head-to-head runs performed inside this software paper.","fun_headline_variants_meta":{"raw":{"variants":["MATLAB GUI builds Bayesian D-optimal DCEs with simulated annealing","Free toolbox uses annealing to craft efficient discrete choice designs","Open MATLAB package optimises choice experiments and exports to Qualtrics","Simulated annealing finds D-optimal designs for multinomial logit surveys","Interactive tool generates Bayesian optimal DCEs supporting interactions"]},"model":"grok-4.5","effort":"low","cost_usd":0.003292,"raw_usage":{"total_tokens":1038,"prompt_tokens":638,"num_sources_used":0,"completion_tokens":85,"cost_in_usd_ticks":32920000,"prompt_tokens_details":{"text_tokens":638,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":315,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":638,"tokens_out":85,"duration_ms":3360,"temperature":1.0,"reasoning_tokens":315,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-11T21:54:42.819162+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Take a fixed set of design problems (with and without interactions or partial profiles), generate designs with DCEDesignSA under a stated time or cycle budget and with a coordinate-exchange package under the same prior, then compare the realised Bayesian D-errors and infinite-error rates.","supporting_citations":[],"review_version":1}