Pith. sign in

REVIEW 4 major objections 5 minor 33 references

Learning Quantitative Automata Modulo Theories

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that deterministic quantitative automata—machines that assign numbers to strings by summation, discounted summation, product, or classification—can be learned from preference and equivalence queries alone, with the…

desk verdict A plausible new problem and a clever algorithm, but the advertised guarantees don't hold—the infinite enumeration in Line 13 is a real blocker. read the letter →

arxiv 2411.10601 v1 pith:KZH6EKFB submitted 2024-11-15 cs.FL cs.LGcs.LO

Eric Hsiung , Swarat Chaudhuri , Joydeep Biswas This is my paper · ORCID
classification cs.FLcs.LGcs.LO MSC 68Q4568Q32
keywords activeautomatalearningquantitativepreferencequeriesequivalencesymbolicobservationtableconstraintsolvingMaxSMTiterativedeepening
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes learning deterministic quantitative automata—machines that assign numbers to strings by summing, discount-summing, multiplying, or classifying labels along a run—without ever asking for concrete input-output values. Instead, the learner asks only comparison questions and equivalence questions, translates the answers into constraints over table variables, and deduces the automaton using a background theory such as the theory of rationals. The claim is that the resulting algorithm, QUINTIC, exhaustively searches the space of automata, is guaranteed to return the minimal equivalent automaton, and is guaranteed to terminate. This matters because explicit numeric labels are often expensive or unnatural to obtain, whereas ranking and preference data are cheap and common in robotics, reinforcement learning, and other settings.

What carries the argument

The load-bearing object is the symbolic observation table, a two-dimensional array of variables with a constraint set, together with the requirement that a table be unified, closed, and consistent before it yields an automaton hypothesis. Equally central are the inference rules that convert pairs of preference-query answers into relations between table variables for each valuation function, encoded as decision trees, and the MaxSMT objective that maximizes the number of conjectured variable equalities while satisfying the constraints, guiding the search toward fewest states. Iterative deepening depth-first search, realized by budgeting symbolic-fill calls, is the mechanism that turns a potentially infinite conjecture–expansion chain into a complete, minimality-preserving search, and equivalence-query feedback supplies the constraints that invalidate wrong conjectures.

What would settle it

With weak feedback only, take a one-state summation target with an unknown rational label $c$ and answer every query consistently with $V(s)=c\cdot |s|$; because infinitely many rational $c$ satisfy any finite set of such constraints, watch whether the exhaustive hypothesis-testing loop terminates, and if it does, record what bound on $c$ it used.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that active learning of deterministic quantitative automata can be driven entirely by constraints derived from preferences rather than by membership observations. QUINTIC maintains a symbolic observation table whose entries are variables, uses inference rules tied to the recursive structure of the valuation function to decide which variables must be equal, and conjectures the remaining equalities as part of a maximum-satisfiability problem. Because variable equivalence can be genuinely ambiguous, greedy unification is impossible; the algorithm therefore backtracks over chains of conjectures and table expansions, with iterative deepening bounding the chain length. The authors argue that this yields a complete search over automata and a minimal result, and they demonstrate the method on summation, discounted summation, product, and classification automata over the rationals.

Load-bearing premise

To terminate, the algorithm must exhaustively try every concrete automaton that fits the current table and constraints, but with rational labels there are infinitely many such automata, so the proof depends on an unstated way to make that infinite list finite.

Editorial extensions

If this is right

  • For any target representable by a deterministic quantitative automaton under summation, discounted summation, product, or classification semantics, a learner with only preference and equivalence queries can in principle return an equivalent automaton.
  • The returned automaton is guaranteed to be minimal in the number of states.
  • The algorithm terminates despite an infinite hypothesis space because iterative deepening bounds the length of conjecture–expansion chains.
  • In the reported experiments QUINTIC learns all 25 ground-truth automata, with discounted summation and product being the hardest valuations and disabling iterative deepening causing non-termination for product targets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same recipe could be instantiated for other recursively defined valuation functions—average reward, min/max accumulation, or min-plus—whenever inference rules analogous to the paper's decision trees can be derived; the paper demonstrates only the four listed valuations.
  • A practical deployment with human preference data would infer reward machines from rankings without numeric rewards, provided the human's comparisons are consistent with a single target valuation; noisy preferences are outside the paper's model.
  • One could test how the completeness guarantee changes when preference queries are answered approximately or adversarially, since the proof assumes a teacher whose answers are perfectly consistent with the target.
  • The symbolic-table plus constraint-solving design suggests a modular library of valuation theories in which swapping the recursive definition and inference rules changes only the theory component of the learner.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces QUINTIC, an active learning algorithm for deterministic quantitative automata from preference and equivalence queries. The learner maintains a symbolic observation table, infers variable equivalences from preference constraints using valuation-specific inference rules, encodes equivalence conjectures as a MaxSMT problem, and uses iterative deepening over a bounded number of SymbolicFill calls to search for a hypothesis. The authors claim that the search is complete, that the output is minimal, and that the algorithm terminates, and they report experiments on 25 hand-constructed target automata under summation, discounted summation, product, and classification valuation functions.

Significance. The problem is well motivated, and the idea of combining symbolic observation tables with SMT/MaxSMT solving for learning quantitative automata from constraints is interesting and potentially useful. The paper also provides a nontrivial implementation and empirical ablations, which gives some support for the practical behavior of the algorithm. However, the central theoretical guarantees are not established: Theorem 1's completeness proof does not connect the bounded chain search to the target automaton, Theorem 2's minimality proof assumes the very object it must construct, and the exhaustive concrete-hypothesis loop in Algorithm 1 requires enumerating an infinite set for the rational output alphabets used in Table 1. The empirical evaluation is limited to hand-constructed targets, and the main baseline is the authors' prior work Remap, so the evidence for the claimed completeness and minimality is not convincing.

major comments (4)
  1. [§4, Algorithm 1 Line 13, Table 1] The claimed termination guarantee does not follow because Line 13 requires exhaustively testing all concrete hypotheses h under a symbolic hypothesis H and constraints C, but for summation, discounted summation, and product, Table 1 sets the output alphabet to the rationals or positive rationals, which is infinite. Even after preference constraints and weak feedback constraints are collected, the constraint set C typically admits infinitely many rational assignments, e.g., a constraint v1 < v2 has infinitely many rational solutions, and weak feedback only adds disequalities. Therefore the exhaustive loop in Line 13 cannot be executed in finite time, so the algorithm's termination guarantee is not established. This is compounded by Section 3, which defines a deterministic quantitative automaton as having a finite output alphabet, making the problem itself ill-posed under the paper's own definitions.
  2. [Theorem 1, §5.3] The proof of Theorem 1 asserts that iterative deepening depth-first search over SymbolicFill budgets is complete, but it does not show that the target automaton is represented by a chain of length at most M for some finite M, nor that the alternating SymbolicFill/Expansion process actually enumerates all relevant hypotheses. In particular, no argument is given that the space of output-label assignments is finite or enumerable in the claimed sense; as noted above, Line 13's exhaustive enumeration is over an infinite set for the rational-valued valuations. Thus the statement that QUINTIC performs a complete search over automata space is not established.
  3. [Theorem 2, §5.3] The minimality proof assumes the existence of a set Ω of infinitely many tables that are 'isomorphic by construction to the target' and then asserts that QUINTIC finds the shortest chain to an element of Ω. No construction or proof of the existence of Ω is given, and the claim that reaching any element of Ω yields a minimal automaton is asserted rather than demonstrated. The construction in Equation 1 only collapses duplicate rows; it does not by itself prove that the resulting automaton has the minimum number of states among all automata equivalent to the target. The proof therefore essentially restates the conclusion it is supposed to establish.
  4. [§5.1] The inference rules for determining variable equivalences are central to the algorithm, since they drive the closedness and consistency checks, but the paper states that 'inference rules can be derived' and presents a decision tree without proving that the rules are sound and complete for the stated preference-query models. The paper also explicitly acknowledges that variable equivalence cannot always be determined from preference queries alone, yet the correctness of the table-unification procedure depends on exactly which equivalences are inferred and which are merely conjectured. A formal statement and proof of the decision-tree inference rules are needed before the completeness and termination arguments can be assessed.
minor comments (5)
  1. [§6.1] The description of the variants contains an apparent typo: 'W-CC-VE (W feedback, CC disabled)' should presumably read 'W-CC-VE (W feedback, CC enabled)', since the preceding sentence states that CC-VE denotes the variant with the closedness-and-consistency objective enabled.
  2. [Table 1] The table header appears corrupted as 'Valuation, Symbol Output Alphabet'; the intended distinction between the symbolic output alphabet and the concrete output alphabet should be stated clearly, especially because the finiteness of the output alphabet is part of the automaton definition in Section 3.
  3. [Figure 3] The decision-tree diagram is hard to read: the labels 'Ris<', 'Ris>', 'Ris=', and 'Ris?' are abbreviated without a legend, and the relationship between the two preference-query pairs and the inferred relation R is not visually explicit. A concrete example of each region would improve clarity.
  4. [§6.6] The phrase 'learning underΠ fails to terminate' has a formatting issue ('underΠ'), and the ablation discussion would benefit from stating whether the non-termination is a divergence in the search chain or a failure of the underlying SMT loop to find a model.
  5. [§6.1] The paper does not state whether the implementation and experimental scripts are publicly available, which would be useful for reproducing the reported results and for comparing against future baselines.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular step in the core derivation: QUINTIC is an explicit search with independent SMT-based validation, and the main reviewer-visible risk is an infinite-enumeration termination gap, not circularity.

full rationale

I walked the derivation chain and found no case where a claimed prediction or guarantee is equivalent, by the paper's own equations or by a load-bearing self-citation, to its inputs. The completeness claim (Theorem 1, Section 5.3) is argued from iterative deepening depth-first search, a standard external completeness argument rather than a restatement of the algorithm's output. The minimalism claim (Theorem 2, Section 5.3) is under-supported: the proof asserts that reaching any table 'isomorphic by construction to the target' yields minimality because 'duplicate rows are collapsed into a single state,' without a lower-bound argument against all smaller automata; that is an omitted proof or correctness gap, not a circular reduction. The most serious reviewer-visible issue is Algorithm 1, line 13, which says 'Exhaustively test all concrete hypotheses h under symbolic hypothesis H and constraints C using EquivQ(h) ... until an h is found to be correct or there exist no more valid h under H and the updated constraints C.' Since Table 1 sets the output alphabet to the rationals (or positive rationals) for summation, discounted summation, and product, the set of concrete hypotheses can be infinite and this loop need not terminate; again, this is an implementability and termination gap rather than circular reasoning. The only self-reference is Remap [16], the authors' earlier algorithm, used as a baseline and algorithmic ancestor in Sections 2, 5, and 6.1. QUINTIC's own table construction, inference rules, and SMT encoding are described in the paper, and the correctness argument does not reduce to the Remap citation; no uniqueness theorem is imported from the authors' prior work to forbid alternatives. Under the rubric, this is a minor non-load-bearing self-citation, so I assign score 2 with no circular steps.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central claim rests on unproven search-completeness and finite-enumeration premises. No fitted numerical parameters are present, but several domain assumptions about teacher consistency, knowledge of Val, and the finiteness of the hypothesis space are relied upon without independent evidence.

assumptions (5)
  • domain assumption Teacher preferences are consistent with a total ordering induced by the target valuation function V.
    Section 4, Equation (7): prefQ responses are defined as P(s1,s2|V) based on comparisons of V(s1) and V(s2); the learner assumes this consistency, but nothing in the algorithm verifies it.
  • domain assumption The learner knows the algebraic form of Val and a formal theory T that contains V(s) R Val(s) for R in {=, !=}.
    Section 4 and Algorithm 1 assumptions: the algorithm requires a supplied recursive definition of Val and an SMT theory, e.g., theory of rationals. This is a strong prior knowledge assumption.
  • ad hoc to paper The decision-tree inference rules in Section 5.1 are sound and complete for determining variable equivalence relations.
    The paper derives the rules informally from rewriting Equation (9) in terms of Val, but provides no proof of soundness or completeness; incorrect rules would break closedness/consistency detection.
  • ad hoc to paper The set of concrete hypotheses consistent with a symbolic hypothesis and constraints C is finitely enumerable.
    Algorithm 1 line 13 requires exhaustive testing of all such hypotheses; with rational-valued outputs (Table 1), this set is generally infinite, so the premise is unexplained.
  • ad hoc to paper Iterative deepening over SymbolicFill chain length M eventually visits a chain leading to the minimal automaton.
    Theorem 1 asserts completeness but does not connect chain length to automaton size or show that the minimal automaton is reachable within finite M.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning Quantitative Automata Modulo Theories." pith.science (2026). https://pith.science/paper/KZH6EKFB

@misc{pith2026241110601,
  author       = {Pith},
  title        = {Pith review of: Learning Quantitative Automata Modulo Theories},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KZH6EKFB}},
  note         = {Machine review of arXiv:2411.10601}
}
read the original abstract

Quantitative automata are useful representations for numerous applications, including modeling probability distributions over sequences to Markov chains and reward machines. Actively learning such automata typically occurs using explicitly gathered input-output examples under adaptations of the L-star algorithm. However, obtaining explicit input-output pairs can be expensive, and there exist scenarios, including preference-based learning or learning from rankings, where providing constraints is a less exerting and a more natural way to concisely describe desired properties. Consequently, we propose the problem of learning deterministic quantitative automata from sets of constraints over the valuations of input sequences. We present QUINTIC, an active learning algorithm, wherein the learner infers a valid automaton through deductive reasoning, by applying a theory to a set of currently available constraints and an assumed preference model and quantitative automaton class. QUINTIC performs a complete search over the space of automata, and is guaranteed to be minimal and correctly terminate. Our evaluations utilize theory of rationals in order to learn summation, discounted summation, product, and classification quantitative automata, and indicate QUINTIC is effective at learning these types of automata.

Figures

Figures reproduced from arXiv: 2411.10601 by the authors.

Figure 1
Figure 1. Upper left: closure operation on symbolic observation table. Upper right: consistency opera￾tion on symbolic observation table. Bottom Left: high-level Remap algorithm, which is an algorithm based on L∗ . Bottom Right: Details for Remap SymbolicFill and MakeHypothesis procedures. The L∗ Algorithm. L∗ [1] is an active learning algorithm for learning automata, wherein the learner L must output an automaton H equivalen… view at source ↗
Figure 2
Figure 2. Example constraints obtained under Quintic for preference queries (left) and equivalence queries (right) under non-discounted summation Val and V functions [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. depicts the decision tree regions, which are points in {−1, 0, 1} × {−1, 0, 1} space. Note that the relation R in T(s · σ)RT(s ′ · σ ′ ) cannot always be determined, and therefore, unification cannot be greedily applied to replace all members of each equivalence class with their representatives. However, in order to make a hypothesis, the learner must ensure closedness and consistency, which entails determining whet… view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Results for 7 targets (columns) out of 25. Full results in Appendix. Each row is corresponds to a metric. Legend: S (strong feedback), W (weak feedback), VE (single MaxSMT objective Equa￾tion 12), CC-VE (joint MaxSMT objectives closedness and consistency objective, and…
Figure 5
Figure 5. Figure 5: Counterexample Expansion Ablation, measuring number of MaxSMT objective solves, under non-discounted summation valuation, with IDS enabled. This measures the number of (E, O) pairs considered. A.3 Full Results We provide the full range of plots for environments tested …
Figure 6
Figure 6. Figure 6: Number of Preference Queries, under CEX Expansion and IDS [PITH_FULL_IMAGE:figures/full_fig_p023_6.png]
Figure 7
Figure 7. Figure 7: Number of Equivalence Queries, under CEX Expansion and IDS [PITH_FULL_IMAGE:figures/full_fig_p024_7.png]
Figure 8
Figure 8. Figure 8: Number of Inequalities Gathered, under CEX Expansion and IDS [PITH_FULL_IMAGE:figures/full_fig_p025_8.png]
Figure 9
Figure 9. Figure 9: Size of the MaxSMT Objective, under CEX Expansion and IDS [PITH_FULL_IMAGE:figures/full_fig_p026_9.png]
Figure 10
Figure 10. Figure 10: Number of Variable Equivalence Conjectures, under CEX Expansion and IDS [PITH_FULL_IMAGE:figures/full_fig_p027_10.png]
Figure 11
Figure 11. Figure 11: Number of Variables, under CEX Expansion and IDS [PITH_FULL_IMAGE:figures/full_fig_p028_11.png]
Figure 12
Figure 12. Figure 12: Total Solver Time in Milliseconds, under CEX Expansion and IDS [PITH_FULL_IMAGE:figures/full_fig_p029_12.png]
Figure 13
Figure 13. Figure 13: Total Number of Backtracks, under CEX Expansion and IDS [PITH_FULL_IMAGE:figures/full_fig_p030_13.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 30 canonical work pages

  1. [1]

    Angluin, D.: Learning regular sets from queries and counterexamples. Inf. Comput. 75(2), 87–106 (1987), https://doi.org/10.1016/0890-5401(87)90052-6, URL https:// doi.org/10.1016/0890-5401(87)90052-6

    doi:10.1016/0890-5401(87)90052-6 1987

  2. [2]

    In: International Conference on Computer Aided Verification (2018)

    Argyros, G., D’antoni, L.: The learnability of symbolic automata. In: International Conference on Computer Aided Verification (2018)

    work page 2018

  3. [3]

    In: Conference on Algebraic Infor- matics (2015)

    Balle, B., Mohri, M.: Learning weighted automata. In: Conference on Algebraic Infor- matics (2015)

    work page 2015

  4. [4]

    Bergadano, F., Varricchio, S.: Learning behaviors of automata from multiplicity and equivalence queries. SIAM J. Comput.25, 1268–1280 (1994)

    work page 1994

  5. [5]

    ACM Transactions on Human-Robot Interaction13(2), 1–27 (2024)

    Biyik, E., Anari, N., Sadigh, D.: Batch active learning of reward functions from human preferences. ACM Transactions on Human-Robot Interaction13(2), 1–27 (2024)

    work page 2024

  6. [6]

    weighted automata

    Boker, U.: Quantitative vs. weighted automata. In: Reachability Problems: 15th In- ternational Conference, RP 2021, Liverpool, UK, October 25–27, 2021, Proceed- ings, p. 3–18, Springer-Verlag, Berlin, Heidelberg (2021), ISBN 978-3-030-89715- 4, https://doi.org/10.1007/978-3-030-89716-1_1, URL https://doi.org/10.1007/ 978-3-030-89716-1_1

    work page doi:10.1007/978-3-030-89716-1_1 2021

  7. [7]

    Boker, U.: Discounted-sum automata with real-valued discount factors. In: Pro- ceedings of the 39th Annual ACM/IEEE Symposium on Logic in Computer Sci- ence, LICS ’24, Association for Computing Machinery, New York, NY, USA (2024), ISBN 9798400706608, https://doi.org/10.1145/3661814.3662090, URL https://doi. org/10.1145/3661814.3662090

    arXiv 2024

  8. [8]

    Chalupa, M., Henzinger, T.A., Mazzocchi, N., Saraç, N.E.: Quak: Quantitative au- tomata kit (2024)

    work page 2024

Show all 33 references
  1. [9]

    Advances in neural information processing systems 30 (2017)

    Christiano, P.F., Leike, J., Brown, T., Martic, M., Legg, S., Amodei, D.: Deep rein- forcement learning from human preferences. Advances in neural information processing systems 30 (2017)

    2017

  2. [10]

    Preference Learning41(41-55), 48 (2009)

    De Gemmis, M., Iaquinta, L., Lops, P., Musto, C., Narducci, F., Semeraro, G., et al.: Preference learning in recommender systems. Preference Learning41(41-55), 48 (2009)

    2009

  3. [11]

    In: Proceedings of the Theory and Practice of Software, 14th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, p

    De Moura, L., Bjørner, N.: Z3: An efficient smt solver. In: Proceedings of the Theory and Practice of Software, 14th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, p. 337–340, TACAS’08/ETAPS’08, Springer- Verlag, Berlin, Heidelbe...

    2008

  4. [12]

    In: Kumar, A., Thiébaux, S., Varakantham, P., Yeoh, W

    Dohmen, T., Topper, N., Atia, G.K., Beckus, A., Trivedi, A., Velasquez, A.: Inferring probabilistic reward machines from non-markovian reward signals for reinforcement learning. In: Kumar, A., Thiébaux, S., Varakantham, P., Yeoh, W. (eds.) Proceedings of the Thirty-Second Inte...

    2022

  5. [13]

    In: International Conference on Tools and Algorithms for Construction and Analysis of Systems (2017)

    Drews, S., D’antoni, L.: Learning symbolic automata. In: International Conference on Tools and Algorithms for Construction and Analysis of Systems (2017)

    2017

  6. [14]

    In: Bouajjani, A., Holík, L., Wu, Z

    Fisman, D., Saadon, S.: Learning and characterizing fully-ordered lattice automata. In: Bouajjani, A., Holík, L., Wu, Z. (eds.) Automated Technology for Verification and Learning Quantitative Automata Modulo Theories 19 Analysis, pp. 266–282, Springer International Publishing,...

    2022

  7. [15]

    Gaon, M., Brafman, R.I.: Reinforcement learning with non-markovian rewards. In: The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty- Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Edu...

    2020

  8. [16]

    Hsiung, E., Biswas, J., Chaudhuri, S.: Learning reward machines through preference queries over sequences (2023), URLhttps://arxiv.org/abs/2308.09301

    2023 arXiv

  9. [17]

    In: Dy, J., Krause, A

    Icarte, R.T., Klassen, T., Valenzano, R., McIlraith, S.: Using reward machines for high- level task specification and decomposition in reinforcement learning. In: Dy, J., Krause, A. (eds.) Proceedings of the 35th International Conference on Machine Learning, Pro- ceedingsofMac...

  10. [18]

    Maccarini, M., Pura, F., Piga, D., Roveda, L., Mantovani, L., Braghin, F.: Preference-based optimization of a human-robot collaborative con- troller. IFAC-PapersOnLine 55(38), 7–12 (2022), ISSN 2405-8963, https://doi.org/https://doi.org/10.1016/j.ifacol.2023.01.126, URL https:...

    2022 doi

  11. [19]

    In: 2022 IEEE International Conference on Development and Learning (ICDL), pp

    Maroto-Gomez, M., Villarroya, S.M., Malfaz, M., Castro-Gonzalez, A., Castillo, J.C., Salichs, M.A.: A preference learning system for the autonomous selection and person- alization of entertainment activities during human-robot interaction. In: 2022 IEEE International Conferenc...

    2022

  12. [20]

    Narcomey, A., Tsoi, N., Desai, R., Vázquez, M.: Learning human preferences over robot behavior as soft planning constraints (2024), URL https://arxiv.org/abs/ 2403.19795

    2024 arXiv

  13. [21]

    In: Robotics: Science and Systems (2017)

    Sadigh, D., Dragan, A.D., Sastry, S., Seshia, S.A.: Active preference-based learning of reward functions. In: Robotics: Science and Systems (2017)

    2017

  14. [22]

    Shah, A., Vazquez-Chanlatte, M., Junges, S., Seshia, S.A.: Learning formal specifica- tions from membership and preference queries (2023)

    2023

  15. [23]

    Advances in Neural Information Processing Systems33, 3008–3021 (2020)

    Stiennon, N., Ouyang, L., Wu, J., Ziegler, D., Lowe, R., Voss, C., Radford, A., Amodei, D., Christiano, P.F.: Learning to summarize with human feedback. Advances in Neural Information Processing Systems33, 3008–3021 (2020)

    2020

  16. [24]

    In: International Symposium on Formal Methods, pp

    Tappler, M., Aichernig, B.K., Bacci, G., Eichlseder, M., Larsen, K.G.: L*-based learning of Markov decision processes. In: International Symposium on Formal Methods, pp. 651–669, Springer (2019)

    2019

  17. [25]

    Tzeng, W.G.: Learning probabilistic automata and markov chains via queries. Mach. Learn. 8(2), 151–166 (Mar 1992), ISSN 0885-6125, https://doi.org/10.1023/A:1022616503659, URL https://doi.org/10.1023/A: 1022616503659

    1992 doi

  18. [26]

    In: Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché- Buc, F., Fox, E., Garnett, R

    Weiss, G., Goldberg, Y., Yahav, E.: Learning deterministic weighted automata with queries and counterexamples. In: Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché- Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems, 20 E. Hsiung et al....

    2019

  19. [27]

    Journal of Machine Learning Research18(136), 1–46 (2017)

    Wirth, C., Akrour, R., Neumann, G., Fürnkranz, J.: A survey of preference-based re- inforcement learning methods. Journal of Machine Learning Research18(136), 1–46 (2017)

    2017

  20. [28]

    In: Proceedings of the AAAI conference on artificial intelligence, vol

    Wirth, C., Fürnkranz, J., Neumann, G.: Model-free preference-based reinforcement learning. In: Proceedings of the AAAI conference on artificial intelligence, vol. 30 (2016)

    2016

  21. [29]

    Xu, Z., Wu, B., Ojha, A., Neider, D., Topcu, U.: Active finite reward automaton in- ference and reinforcement learning using queries and counterexamples. In: Machine Learning and Knowledge Extraction: 5th IFIP TC 5, TC 12, WG 8.4, WG 8.9, WG 12.9 International Cross-Domain Con...

    2021 doi

  22. [30]

    In: Proceedings of the 29th ACM SIGKDD Confer- ence on Knowledge Discovery and Data Mining, p

    Xue, W., Cai, Q., Xue, Z., Sun, S., Liu, S., Zheng, D., Jiang, P., Gai, K., An, B.: Prefrec: Recommender systems with human preferences for reinforcing long-term user engagement. In: Proceedings of the 29th ACM SIGKDD Confer- ence on Knowledge Discovery and Data Mining, p. 287...

    2023

  23. [31]

    Closed and consistent tables are ranked highest

  24. [32]

    Closed, but not consistent table are ranked second

  25. [33]

    Within the 3 regimes, tables are ranked by how many states they contain

    Tables which are not closed are ranked the lowest. Within the 3 regimes, tables are ranked by how many states they contain. Tables with fewer states and ranked higher than tables with more states. The components of objective are constructed from 3 parts: (a) an expression whic...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.