REVIEW 3 major objections 5 minor 87 references
Beyond the Proxy: Trajectory-Distilled Guidance for Offline GFlowNet Training
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read TD-GFN trains offline GFlowNets without a proxy: distilled edge rewards guide DAG pruning and backward sampling, gradient updates use only ground-truth terminal rewards, and the paper reports faster convergence with higher-quality samples…
desk verdict A useful offline training recipe whose central claim—that IRL edge rewards can guide the policy without contaminating it—is plausible but not proven; the mechanism needs a sharper argument or a targeted experiment. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the transition-level edge reward $R_E(s,s')$ on the environment DAG, learned by adversarial imitation from a rebalanced offline dataset and converted from the discriminator output by the logit transform $R_E(s,s') = \log D_\phi(s,s') - \log(1 - D_\phi(s,s'))$ so that negative values are allowed. The machinery has two uses. First, the reward-guided pruning rule removes an edge when $R_E(s,s') < \mathrm{mean}(D_{R_E}) - K \cdot \mathrm{std}(D_{R_E})$, producing a smaller action space. Second, the backward policy $P_B(s_t \mid s_{t+1}) \propto \exp\{R_E(s_t, s_{t+1})\}$ over the pruned graph samples training trajectories backward from high-reward terminals. The theoretical bridge is the established equivalence between GFlowNet training and entropy-regularized reinforcement learning, which justifies applying IRL to edge-level quantities; the practical effect is that the policy receives dense guidance without any learned reward entering its loss.
What would settle it
Take a small DAG with two routes to a high-reward terminal, one common and one rare in the offline dataset, make the behavior policy's preferences oppose the reward landscape so that inverse reinforcement learning assigns higher edge rewards to the low-reward route, and check whether TD-GFN's pruning removes the rare high-reward edges; if the final policy's reward distribution is materially worse than pruning with ground-truth edge importance, the claim that inaccurate edge rewards cannot propagate into training is falsified.
Extended reading notes
Core claim
On its own terms, the paper discovers that the unequal importance of edges in a GFlowNet's environment DAG can be extracted as an edge-reward function from offline trajectories using maximum-causal-entropy inverse reinforcement learning on a reward-rebalanced dataset. These edge rewards are then used only indirectly: edges with rewards below a threshold are pruned from the DAG, and training trajectories are generated backward from reward-proportional terminal states using a softmax over edge rewards. The paper's key assertion is that because the rewards never enter the GFlowNet loss, the policy gradients depend only on recorded ground-truth terminal rewards, insulating the policy from proxy-style error propagation while still providing dense structural guidance. Empirically, the paper claims this yields the fastest convergence and best sample quality among the compared offline-GFlowNet and offline-RL baselines on all three benchmarks.
Load-bearing premise
The load-bearing premise is that the transition-level scores learned from the offline data are accurate enough that pruning and backward sampling do not systematically throw away paths to high-reward ends; if a wrong score removes or de-emphasizes the very paths that lead to the best outcomes, the final policy is still trained on ground-truth rewards but only over a biased subset of trajectories.
Editorial extensions
If this is right
- If TD-GFN's claim holds, offline GFlowNet training no longer requires a proxy reward model, removing the main source of out-of-distribution reward-query errors.
- The learned edge rewards generalize to transitions not present in the dataset, enabling discovery of high-reward modes beyond the support of the offline trajectories.
- On the molecule-design benchmark, TD-GFN converges in roughly $2.75\times10^4$ training trajectories, about twenty times fewer than Oracle-GFN's $4.41\times10^5$, with comparable or better top-$k$ rewards.
- The framework is agnostic to the downstream training objective: flow matching and trajectory balance both work after trajectory collection, and an offline RL loss can be substituted as well.
Reading between the lines
- A formal guarantee for the paper's core robustness claim would need an error bound showing that estimated edge rewards do not remove high-reward edges; without such a bound, the claimed insulation from error propagation rests on the empirical margins.
- The same edge-reward guidance could be run online, pruning the DAG dynamically from the agent's own experience; the paper lists this as a future direction, so it is an extension, not a paper claim.
- A soft alternative to hard pruning, such as reweighting policy logits by the learned edge rewards, would isolate whether the benefit comes from shrinking the action space or from the dense guidance itself; the ablation suggests the hard pruning specifically is doing important work.
- The paper itself notes in its discussion that TD-GFN relies on trajectories carrying meaningful structure, since uniform reverse-generated trajectories degrade performance; this delimits the method to datasets whose edge sequences contain information beyond terminal rewards.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Trajectory-Distilled GFlowNet (TD-GFN), a proxy-free method for training GFlowNets from offline trajectory datasets. TD-GFN first uses maximum causal entropy IRL (in a GAIL-style adversarial loop) on a reward-rebalanced dataset to learn dense edge-level rewards. These learned rewards are then applied indirectly: they are used to prune low-utility transitions from the environment DAG (Eq. 6) and to define a prioritized backward sampling distribution over the pruned graph (Eq. 7). The policy is trained with a GFlowNet objective (FM or TB) using only the ground-truth terminal rewards from the dataset. Experiments on Hypergrid, AMP design, and molecule design show that TD-GFN converges faster and achieves higher sample quality than existing proxy-free and proxy-based baselines, including COFlowNet, Dataset-GFN, GAIL, BC, CQL, IQL, Proxy-GFN, and Oracle-GFN. The paper claims that because the final gradient updates rely only on ground-truth terminal rewards, the learned edge rewards do not cause error propagation.
Significance. If the central claim is substantiated, TD-GFN would be a practically valuable contribution to offline GFlowNet training, as it avoids the need for a learned terminal-reward proxy while still providing dense structural guidance. The paper includes extensive experiments across three tasks, ablations of the main components (Appendix D.6), and a sensitivity analysis of the pruning threshold (Appendix D.4). The empirical results are generally convincing: TD-GFN consistently outperforms a wide range of baselines in both convergence speed and final sample quality. However, the load-bearing theoretical claim that indirect use of edge rewards 'prevents error propagation' is not supported by the analysis or experiments, and this is the main weakness of the manuscript.
major comments (3)
- [Section 3.2, Eq. (6)-(7), Algorithm 1] The assertion that indirect use of edge rewards 'insulates the policy' from IRL error is not justified. Although the final gradient updates use only ground-truth terminal rewards, the set of trajectories over which those updates are computed is determined by the learned edge rewards: pruning (Eq. 6) removes low-scoring edges from the DAG, and backward sampling (Eq. 7) weights trajectories by softmax of edge rewards. An inaccurate edge-reward estimate can therefore remove or de-emphasize edges that lead to high-reward terminals, biasing the training distribution before any ground-truth gradient is applied. The paper provides no bound or experiment that isolates this failure mode; the ablations in Appendix D.6 show the components matter, but they do not demonstrate robustness to misestimated edge rewards. This is a load-bearing issue because the central claim of the paper is that error propagation is prevented.
- [Section 3.1, Eq. (4)-(5)] The rebalancing scheme P(τ) ∝ R(s_T) is introduced as an ad-hoc approximation of expert behavior, and the edge rewards are learned from the same offline dataset that is later used for policy training. The paper claims strong generalization of edge rewards to unobserved transitions, but the only evidence is the qualitative visualization in Appendix D.3 and favorable downstream performance. Because pruning and backward sampling are applied to the full DAG (including edges never seen in the dataset), the method's success hinges on the accuracy of the learned edge rewards on unobserved regions. This is neither theoretically analyzed nor systematically tested. A concrete experiment that corrupts or ablates a subset of learned edge rewards would help establish whether the indirect guidance mechanism is as robust as claimed.
- [Table 1 and Section 4.3] The comparison between TD-GFN and the offline RL baselines CQL and IQL is not apples-to-apples: the table reports only the best-performing seed for CQL and IQL, while all other methods report mean ± standard deviation over three seeds. This can artificially inflate the performance gap. Additionally, the '20 times fewer trajectories' claim relative to Oracle-GFN relies on a convergence metric that is not precisely defined in the main text; the reader should be told how convergence is measured and whether the comparison accounts for the fact that Oracle-GFN uses a pre-trained oracle model (trained on 300k molecules) as its reward source.
minor comments (5)
- [Section 4.1 and Appendix D.7] Equation (8) (the FM objective) is referenced in Section 4.1 but is only defined in Appendix D.7. It should be introduced in the main text or the cross-reference made more prominent, since the main experiments are run with this objective.
- [Section 4.2] The description of the AMP dataset is slightly unclear: the text says 'we train a Proxy-GFN and compare its performance', but the dataset description mentions two datasets (one for training a reward proxy and another for an oracle). Clarify how many datasets are used and which trajectories are provided to TD-GFN.
- [Section 4.3 and Related Work (Appendix B)] The paper does not include RO-GFlowNets (Wang et al., 2023) in the experimental comparison, despite it being the earliest proxy-free GFlowNet method and being mentioned in the related work. Since the paper claims state-of-the-art performance, a comparison or a clear explanation for its omission is needed.
- [Section 4.3 and Figure 1] The 'Convergence' metric in Table 1 and Figure 1 is not defined. State explicitly what constitutes 'performance convergence' (e.g., reaching a fixed number of modes or a threshold on top-k reward) so that the reader can interpret the reported speedup.
- [Appendix D.4] The sensitivity analysis of the pruning threshold K uses a tuning criterion ('highest ratio of retained recorded transitions to retained randomly selected transitions') that is not described in the main text. Briefly explain this criterion when first mentioning K in Section 3.2 or Appendix D.4.
Circularity Check
No significant circularity: final GFlowNet updates use ground-truth terminal rewards; learned edge rewards enter only as data shaping.
full rationale
I traced the derivation chain. Edge rewards are extracted by GAIL on a rebalanced dataset (Eqs. 4-5), then used to prune the DAG (Eq. 6) and to define backward sampling (Eq. 7). Phase 3 samples terminals proportionally to recorded rewards and optimizes the policy with the FM or TB objective using those ground-truth terminal rewards (Eqs. 8-9). There is no equation-level reduction: learned edge rewards do not appear in the final loss, and the target distribution is not defined in terms of them. Inaccurate edge rewards could bias the support of training trajectories, but that is an unverified robustness assumption, not a self-referential derivation. The paper's own ablations and discussion acknowledge sensitivity to trajectory quality. Self-citations appear only in related work and metric references and are not load-bearing. No specific circular step can be quoted.
Assumptions & free parameters
free parameters (1)
- Pruning threshold K =
Hypergrid: 7.0; Molecule Design: 1.0; Biosequence Design: 2.0
assumptions (7)
- standard math GFlowNet training is equivalent to entropy-regularized RL with gamma=1, lambda=1 and the modified reward in Eq (1).
- standard math Maximum causal entropy IRL (Eq 3) and the GAIL objective (Eq 4) recover a reward that explains the expert behavior, and the logit transform in Eq (5) yields an unbiased edge reward.
- ad hoc to paper Reweighted trajectories P(tau) proportional to R(s_T) approximate an expert policy's visitation distribution.
- domain assumption Edge rewards learned on observed transitions generalize meaningfully to unobserved transitions.
- ad hoc to paper Edges with RE below mean(D_RE)-K*std(D_RE) are low-utility and can be pruned without losing important high-reward paths.
- domain assumption The backward policy PB(s_t | s_{t+1}) proportional to exp{RE(s_t, s_{t+1})} is consistent with reward shaping and with the GFlowNet objective.
- domain assumption The environment has deterministic dynamics and a DAG structure as in standard GFlowNet settings.
Cite this review
Pith. "Pith review of Beyond the Proxy: Trajectory-Distilled Guidance for Offline GFlowNet Training." pith.science (2026). https://pith.science/paper/RDF2K5EL
@misc{pith2026250520110,
author = {Pith},
title = {Pith review of: Beyond the Proxy: Trajectory-Distilled Guidance for Offline GFlowNet Training},
year = {2026},
howpublished = {\url{https://pith.science/paper/RDF2K5EL}},
note = {Machine review of arXiv:2505.20110}
}
read the original abstract
Generative Flow Networks (GFlowNets) excel at sampling diverse, high-reward objects. In many practical applications where active reward queries are infeasible, these models must be trained using static offline datasets. Prevailing training methods typically rely on a proxy model to provide reward feedback for online sampled trajectories. However, constructing a reliable proxy is often challenging due to data scarcity or high evaluation costs. While existing proxy-free approaches attempt to address this, they often impose coarse constraints that limit the model's ability to explore effectively. To overcome these limitations, we propose Trajectory-Distilled GFlowNet (TD-GFN), a novel proxy-free training framework. TD-GFN utilizes inverse reinforcement learning (IRL) to extract dense, transition-level edge rewards from offline trajectories, providing rich structural guidance for efficient exploration. Crucially, to ensure robustness, these rewards guide the policy indirectly through DAG pruning and prioritized backward sampling. This design ensures that gradient updates rely exclusively on ground-truth terminal rewards from the dataset, thereby preventing error propagation. Empirical results demonstrate that TD-GFN significantly outperforms a broad range of existing baselines in both convergence speed and sample quality, establishing a more robust and efficient paradigm for offline GFlowNet training.
Figures
Figures from the paper (15 more)
Reference graph
Works this paper leans on
-
[1]
Investigating Generalization Behaviours of Generative Flow Networks
Lazar Atanackovic and Emmanuel Bengio. Investigating generalization behaviours of generative flow networks. arXiv preprint arXiv:2402.05309, 2024
work page Pith review arXiv 2024
-
[2]
Why is tanimoto index an appropriate choice for fingerprint-based similarity calculations? Journal of cheminformatics, 7: 0 1--13, 2015
D \'a vid Bajusz, Anita R \'a cz, and K \'a roly H \'e berger. Why is tanimoto index an appropriate choice for fingerprint-based similarity calculations? Journal of cheminformatics, 7: 0 1--13, 2015
2015
-
[3]
Flow network based generative models for non-iterative diverse candidate generation
Emmanuel Bengio, Moksh Jain, Maksym Korablyov, Doina Precup, and Yoshua Bengio. Flow network based generative models for non-iterative diverse candidate generation. Advances in Neural Information Processing Systems, 34: 0 27381--27394, 2021
2021
-
[4]
Gflownet foundations
Yoshua Bengio, Salem Lahlou, Tristan Deleu, Edward J Hu, Mo Tiwari, and Emmanuel Bengio. Gflownet foundations. Journal of Machine Learning Research, 24 0 (210): 0 1--55, 2023
2023
-
[5]
A theory of non-acyclic generative flow networks
Leo Brunswic, Yinchuan Li, Yushun Xu, Yijun Feng, Shangling Jui, and Lizhuang Ma. A theory of non-acyclic generative flow networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp.\ 11124--11131, 2024
2024
-
[6]
Leo Maxime Brunswic, Mateo Clemente, Rui Heng Yang, Adam Sigal, Amir Rasouli, and Yinchuan Li. Ergodic generative flows. ArXiv, abs/2505.03561, 2025. URL https://api.semanticscholar.org/CorpusID:278339228
arXiv 2025
-
[7]
Jaakkola
Andrew Campbell, Jason Yim, Regina Barzilay, Tom Rainforth, and T. Jaakkola. Generative flows on discrete state-spaces: Enabling multimodal flows with applications to protein co-design. ArXiv, 2024. URL https://api.semanticscholar.org/CorpusID:267523194
2024
-
[8]
Offline goal-conditioned reinforcement learning for safety-critical tasks with recovery policy
Chenyang Cao, Zichen Yan, Renhao Lu, Junbo Tan, and Xueqian Wang. Offline goal-conditioned reinforcement learning for safety-critical tasks with recovery policy. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pp.\ 2838--2844. IEEE, 2024
2024
Show all 87 references
-
[9]
Kostas, Emma Jordan, and Philip S
Yash Chandak, Georgios Theocharous, James E. Kostas, Emma Jordan, and Philip S. Thomas. Learning action representations for reinforcement learning. In International Conference on Machine Learning, 2019. URL https://api.semanticscholar.org/CorpusID:59553460
2019
-
[10]
On the opportunities and challenges of offline reinforcement learning for recommender systems
Xiaocong Chen, Siyu Wang, Julian McAuley, Dietmar Jannach, and Lina Yao. On the opportunities and challenges of offline reinforcement learning for recommender systems. ACM Transactions on Information Systems, 42 0 (6): 0 1--26, 2024
2024
-
[11]
Order-preserving GF lownets
Yihang Chen and Lukas Mauch. Order-preserving GF lownets. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=VXDPXuq4oG
2024
-
[12]
Training verifiers to solve math word problems
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021
-
[13]
Towards guaranteed safe ai: A framework for ensuring robust and reliable ai systems
David Dalrymple, Joar Skalse, Yoshua Bengio, Stuart Russell, Max Tegmark, Sanjit Seshia, Steve Omohundro, Christian Szegedy, Ben Goldhaber, Nora Ammann, et al. Towards guaranteed safe ai: A framework for ensuring robust and reliable ai systems. arXiv preprint arXiv:2405.06624, 2024
2024 arXiv
-
[14]
Bayesian structure learning with generative flow networks
Tristan Deleu, Ant \'o nio G \'o is, Chris Emezue, Mansi Rankawat, Simon Lacoste-Julien, Stefan Bauer, and Yoshua Bengio. Bayesian structure learning with generative flow networks. In Uncertainty in Artificial Intelligence, pp.\ 518--528. PMLR, 2022
2022
-
[15]
Decomposition of uncertainty in bayesian deep learning for efficient and risk-sensitive learning
Stefan Depeweg, Jose-Miguel Hernandez-Lobato, Finale Doshi-Velez, and Steffen Udluft. Decomposition of uncertainty in bayesian deep learning for efficient and risk-sensitive learning. In International conference on machine learning, pp.\ 1184--1193. PMLR, 2018
2018
-
[16]
Learning robust rewards with adverserial inverse reinforcement learning
Justin Fu, Katie Luo, and Sergey Levine. Learning robust rewards with adverserial inverse reinforcement learning. In International Conference on Learning Representations, 2018. URL https://openreview.net/forum?id=rkHywl-A-
2018
-
[17]
Tucker, and Sergey Levine
Justin Fu, Aviral Kumar, Ofir Nachum, G. Tucker, and Sergey Levine. D4rl: Datasets for deep data-driven reinforcement learning. ArXiv, abs/2004.07219, 2020. URL https://api.semanticscholar.org/CorpusID:215827910
2004 arXiv
-
[18]
Iq-learn: Inverse soft-q learning for imitation
Divyansh Garg, Shuvam Chakraborty, Chris Cundy, Jiaming Song, and Stefano Ermon. Iq-learn: Inverse soft-q learning for imitation. Advances in Neural Information Processing Systems, 34: 0 4028--4039, 2021
2021
-
[19]
Generative flow networks assisted biological sequence editing
Pouya M Ghari, Alex Tseng, G \"o kcen Eraslan, Romain Lopez, Tommaso Biancalani, Gabriele Scalia, and Ehsan Hajiramezanali. Generative flow networks assisted biological sequence editing. In NeurIPS 2023 Generative AI and Biology (GenBio) Workshop, 2023
2023
-
[20]
Neural message passing for quantum chemistry
Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International conference on machine learning, pp.\ 1263--1272. PMLR, 2017
2017
-
[21]
Optimizing backward policies in gflownets via trajectory likelihood maximization
Timofei Gritsaev, Nikita Morozov, Sergey Samsonov, and Daniil Tiapkin. Optimizing backward policies in gflownets via trajectory likelihood maximization. arXiv preprint arXiv:2410.15474, 2024
2024 arXiv
-
[22]
Hoffman, Ofir Nachum, G
Caglar Gulcehre, Ziyun Wang, Alexander Novikov, Tom Le Paine, Sergio Gomez Colmenarejo, Konrad Zolna, Rishabh Agarwal, Josh Merel, Daniel Jaymin Mankowitz, Cosmin Paduraru, Gabriel Dulac-Arnold, Jerry Zheng Li, Mohammad Norouzi, Matthew W. Hoffman, Ofir Nachum, G. Tucker, Nico...
2006 arXiv
-
[23]
Generative adversarial imitation learning
Jonathan Ho and Stefano Ermon. Generative adversarial imitation learning. Advances in neural information processing systems, 29, 2016
2016
-
[24]
Harnessing mixed offline reinforcement learning datasets via trajectory weighting
Zhang-Wei Hong, Pulkit Agrawal, R \'e mi Tachet des Combes, and Romain Laroche. Harnessing mixed offline reinforcement learning datasets via trajectory weighting. arXiv preprint arXiv:2306.13085, 2023
2023 arXiv
-
[25]
Amortizing intractable inference in large language models
Edward J Hu, Moksh Jain, Eric Elmoznino, Younesse Kaddar, Guillaume Lajoie, Yoshua Bengio, and Nikolay Malkin. Amortizing intractable inference in large language models. In The Twelfth International Conference on Learning Representations, 2024 a . URL https://openreview.net/fo...
2024
-
[26]
Beyond squared error: Exploring loss design for enhanced training of generative flow networks
Rui Hu, Yifan Zhang, Zhuoran Li, and Longbo Huang. Beyond squared error: Exploring loss design for enhanced training of generative flow networks. arXiv preprint arXiv:2410.02596, 2024 b
2024 arXiv
-
[27]
Biological sequence design with gflownets
Moksh Jain, Emmanuel Bengio, Alex Hernandez-Garcia, Jarrid Rector-Brooks, Bonaventure FP Dossou, Chanakya Ajit Ekbote, Jie Fu, Tianyu Zhang, Michael Kilgour, Dinghuai Zhang, et al. Biological sequence design with gflownets. In International Conference on Machine Learning, pp.\...
2022
-
[28]
Gflownets for ai-driven scientific discovery
Moksh Jain, Tristan Deleu, Jason Hartford, Cheng-Hao Liu, Alex Hernandez-Garcia, and Yoshua Bengio. Gflownets for ai-driven scientific discovery. Digital Discovery, 2 0 (3): 0 557--577, 2023 a
2023
-
[29]
Multi-objective gflownets
Moksh Jain, Sharath Chandra Raparthy, Alex Hern \'a ndez-Garc a, Jarrid Rector-Brooks, Yoshua Bengio, Santiago Miret, and Emmanuel Bengio. Multi-objective gflownets. In International conference on machine learning, pp.\ 14631--14653. PMLR, 2023 b
2023
-
[30]
Learning energy decompositions for partial inference of gflownets
Hyosoon Jang, Minsu Kim, and Sungsoo Ahn. Learning energy decompositions for partial inference of gflownets. arXiv preprint arXiv:2310.03301, 2023
2023 arXiv
-
[31]
Pessimistic backward policy for gflownets
Hyosoon Jang, Yunhui Jang, Minsu Kim, Jinkyoo Park, and Sungsoo Ahn. Pessimistic backward policy for gflownets. arXiv preprint arXiv:2405.16012, 2024
2024 arXiv
-
[32]
Junction tree variational autoencoder for molecular graph generation
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation. In International conference on machine learning, pp.\ 2323--2332. PMLR, 2018
2018
-
[33]
Droid: A large-scale in-the-wild robot manipulation dataset
Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty Ellis, et al. Droid: A large-scale in-the-wild robot manipulation dataset. arXiv preprint arXiv:2403.1...
2024 arXiv
-
[34]
Ant colony sampling with gflownets for combinatorial optimization
Minsu Kim, Sanghyeok Choi, Jiwoo Son, Hyeonah Kim, Jinkyoo Park, and Yoshua Bengio. Ant colony sampling with gflownets for combinatorial optimization. CoRR, abs/2403.07041, 2024 a . URL https://doi.org/10.48550/arXiv.2403.07041
-
[35]
Adaptive teachers for amortized samplers
Minsu Kim, Sanghyeok Choi, Taeyoung Yun, Emmanuel Bengio, Leo Feng, Jarrid Rector-Brooks, Sungsoo Ahn, Jinkyoo Park, Nikolay Malkin, and Yoshua Bengio. Adaptive teachers for amortized samplers. arXiv preprint arXiv:2410.01432, 2024 b
-
[36]
Local search gflownets
Minsu Kim, Taeyoung Yun, Emmanuel Bengio, Dinghuai Zhang, Yoshua Bengio, Sungsoo Ahn, and Jinkyoo Park. Local search gflownets. URL https://arxiv. org/abs/2310, 2710, 2024 c
2024
-
[37]
Discriminator-actor-critic: Addressing sample inefficiency and reward bias in adversarial imitation learning
Ilya Kostrikov, Kumar Krishna Agrawal, Debidatta Dwibedi, Sergey Levine, and Jonathan Tompson. Discriminator-actor-critic: Addressing sample inefficiency and reward bias in adversarial imitation learning. arXiv preprint arXiv:1809.02925, 2018
2018 arXiv
-
[38]
Imitation learning via off-policy distribution matching
Ilya Kostrikov, Ofir Nachum, and Jonathan Tompson. Imitation learning via off-policy distribution matching. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=Hyg-JC4FDr
2020
-
[39]
Offline reinforcement learning with implicit q-learning
Ilya Kostrikov, Ashvin Nair, and Sergey Levine. Offline reinforcement learning with implicit q-learning. arXiv preprint arXiv:2110.06169, 2021
2021 arXiv
-
[40]
Conservative q-learning for offline reinforcement learning
Aviral Kumar, Aurick Zhou, George Tucker, and Sergey Levine. Conservative q-learning for offline reinforcement learning. Advances in neural information processing systems, 33: 0 1179--1191, 2020
2020
-
[41]
A theory of continuous generative flow networks
Salem Lahlou, Tristan Deleu, Pablo Lemos, Dinghuai Zhang, Alexandra Volokhova, Alex Hern \'a ndez-Garc a, L \'e na N \'e hale Ezzine, Yoshua Bengio, and Nikolay Malkin. A theory of continuous generative flow networks. In International Conference on Machine Learning, pp.\ 18269...
2023
-
[42]
Qgfn: Controllable greediness with action values
Elaine Lau, Stephen Lu, Ling Pan, Doina Precup, and Emmanuel Bengio. Qgfn: Controllable greediness with action values. Advances in neural information processing systems, 37: 0 81645--81676, 2024
2024
-
[43]
Energy-based imitation learning
Minghuan Liu, Tairan He, Minkai Xu, and Weinan Zhang. Energy-based imitation learning. ArXiv, abs/2004.09395, 2020. URL https://api.semanticscholar.org/CorpusID:215827862
2004 arXiv
-
[44]
Generative flow network for listwise recommendation
Shuchang Liu, Qingpeng Cai, Zhankui He, Bowen Sun, Julian McAuley, Dong Zheng, Peng Jiang, and Kun Gai. Generative flow network for listwise recommendation. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.\ 1524--1534, 2023
2023
-
[45]
Xiao, Weiyang Liu, Yoshua Bengio, and Dinghuai Zhang
Zhen Liu, Tim Z. Xiao, Weiyang Liu, Yoshua Bengio, and Dinghuai Zhang. Efficient diversity-preserving diffusion alignment via gradient-informed GF lownets. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=Aye5wL6TCn
2025
-
[46]
Transferable reward learning by dynamics-agnostic discriminator ensemble
Fan-Ming Luo, Xingchen Cao, Rong-Jun Qin, and Yang Yu. Transferable reward learning by dynamics-agnostic discriminator ensemble. arXiv preprint arXiv:2206.00238, 2022
2022 arXiv
-
[47]
Reward-consistent dynamics models are strongly generalizable for offline reinforcement learning
Fan-Ming Luo, Tian Xu, Xingchen Cao, and Yang Yu. Reward-consistent dynamics models are strongly generalizable for offline reinforcement learning. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=GSBHKiw19c
2024
-
[48]
Learning gflownets from partial episodes for improved convergence and stability
Kanika Madan, Jarrid Rector-Brooks, Maksym Korablyov, Emmanuel Bengio, Moksh Jain, Andrei Cristian Nica, Tom Bosc, Yoshua Bengio, and Nikolay Malkin. Learning gflownets from partial episodes for improved convergence and stability. In International Conference on Machine Learnin...
2023
-
[49]
Towards improving exploration through sibling augmented gflownets
Kanika Madan, Alex Lamb, Emmanuel Bengio, Glen Berseth, and Yoshua Bengio. Towards improving exploration through sibling augmented gflownets. In The Thirteenth International Conference on Learning Representations, 2025
2025
-
[50]
Trajectory balance: Improved credit assignment in gflownets
Nikolay Malkin, Moksh Jain, Emmanuel Bengio, Chen Sun, and Yoshua Bengio. Trajectory balance: Improved credit assignment in gflownets. Advances in Neural Information Processing Systems, 35: 0 5955--5967, 2022
2022
-
[51]
Gflownets and variational inference
Nikolay Malkin, Salem Lahlou, Tristan Deleu, Xu Ji, Edward Hu, Katie Everett, Dinghuai Zhang, and Yoshua Bengio. Gflownets and variational inference. URL https://arxiv. org/abs/2210.00580, 2023
2023 arXiv
-
[52]
Maximum entropy gflownets with soft q-learning
Sobhan Mohammadpour, Emmanuel Bengio, Emma Frejinger, and Pierre-Luc Bacon. Maximum entropy gflownets with soft q-learning. In International Conference on Artificial Intelligence and Statistics, pp.\ 2593--2601. PMLR, 2024
2024
-
[53]
Revisiting non-acyclic gflownets in discrete environments
Nikita Morozov, Ian Maksimov, Daniil Tiapkin, and Sergey Samsonov. Revisiting non-acyclic gflownets in discrete environments. arXiv preprint arXiv:2502.07735, 2025
2025 arXiv
-
[54]
Algorithms for inverse reinforcement learning
Andrew Y Ng, Stuart Russell, et al. Algorithms for inverse reinforcement learning. In Icml, volume 1, pp.\ 2, 2000
2000
-
[55]
Training language models to follow instructions with human feedback
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35: 0 2...
2022
-
[56]
Generative augmented flow networks
Ling Pan, Dinghuai Zhang, Aaron Courville, Longbo Huang, and Yoshua Bengio. Generative augmented flow networks. arXiv preprint arXiv:2210.03308, 2022
2022 arXiv
-
[57]
Better training of gflownets with local credit and incomplete trajectories
Ling Pan, Nikolay Malkin, Dinghuai Zhang, and Yoshua Bengio. Better training of gflownets with local credit and incomplete trajectories. In International Conference on Machine Learning, pp.\ 26878--26890. PMLR, 2023 a
2023
-
[58]
Stochastic generative flow networks
Ling Pan, Dinghuai Zhang, Moksh Jain, Longbo Huang, and Yoshua Bengio. Stochastic generative flow networks. In Uncertainty in Artificial Intelligence, pp.\ 1628--1638. PMLR, 2023 b
2023
-
[59]
Dbaasp v3: database of antimicrobial/cytotoxic activity and structure of peptides as a resource for development of new therapeutics
Malak Pirtskhalava, Anthony A Amstrong, Maia Grigolava, Mindia Chubinidze, Evgenia Alimbarashvili, Boris Vishnepolsky, Andrei Gabrielian, Alex Rosenthal, Darrell E Hurt, and Michael Tartakovsky. Dbaasp v3: database of antimicrobial/cytotoxic activity and structure of peptides ...
2021
-
[60]
Goal-conditioned gflownets for controllable multi-objective molecular design
Julien Roy, Pierre-Luc Bacon, Christopher Pal, and Emmanuel Bengio. Goal-conditioned gflownets for controllable multi-objective molecular design. arXiv preprint arXiv:2306.04620, 2023
2023 arXiv
-
[61]
Improved off-policy training of diffusion samplers
Marcin Sendera, Minsu Kim, Sarthak Mittal, Pablo Lemos, Luca Scimeca, Jarrid Rector-Brooks, Alexandre Adam, Yoshua Bengio, and Nikolay Malkin. Improved off-policy training of diffusion samplers. In Neural Information Processing Systems, 2024. URL https://api.semanticscholar.or...
2024
-
[62]
Towards understanding and improving gflownet training
Max W Shen, Emmanuel Bengio, Ehsan Hajiramezanali, Andreas Loukas, Kyunghyun Cho, and Tommaso Biancalani. Towards understanding and improving gflownet training. In International conference on machine learning, pp.\ 30956--30975. PMLR, 2023
2023
-
[63]
On divergence measures for training GF lownets
Tiago Silva, Eliezer de Souza da Silva, and Diego Mesquita. On divergence measures for training GF lownets. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net/forum?id=N5H4z0Pzvn
2024
-
[64]
When do GF lownets learn the right distribution? In The Thirteenth International Conference on Learning Representations, 2025
Tiago Silva, Rodrigo Barreto Alves, Eliezer de Souza da Silva, Amauri H Souza, Vikas Garg, Samuel Kaski, and Diego Mesquita. When do GF lownets learn the right distribution? In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.ne...
2025
-
[65]
Generative flow networks as entropy-regularized rl
Daniil Tiapkin, Nikita Morozov, Alexey Naumov, and Dmitry P Vetrov. Generative flow networks as entropy-regularized rl. In International Conference on Artificial Intelligence and Statistics, pp.\ 4213--4221. PMLR, 2024
2024
-
[66]
Dynameomics: a comprehensive database of protein dynamics
Marc W van der Kamp, R Dustin Schaeffer, Amanda L Jonsson, Alexander D Scouras, Andrew M Simms, Rudesh D Toofanny, Noah C Benson, Peter C Anderson, Eric D Merkley, Steven Rysavy, et al. Dynameomics: a comprehensive database of protein dynamics. Structure, 18 0 (4): 0 423--435, 2010
2010
-
[67]
Amortizing intractable inference in diffusion models for vision, language, and control
Siddarth Venkatraman, Moksh Jain, Luca Scimeca, Minsu Kim, Marcin Sendera, Mohsin Hasan, Luke Rowe, Sarthak Mittal, Pablo Lemos, Emmanuel Bengio, Alexandre Adam, Jarrid Rector-Brooks, Yoshua Bengio, Glen Berseth, and Nikolay Malkin. Amortizing intractable inference in diffusio...
2024
-
[68]
Regularized offline GF lownets, 2023
Haozhi Wang, Yinchuan Li, yunfeng shao, and Jianye HAO. Regularized offline GF lownets, 2023. URL https://openreview.net/forum?id=kbhUUAMZmQT
2023
-
[69]
Advdiffuser: Generating adversarial safety-critical driving scenarios via guided diffusion
Yuting Xie, Xianda Guo, Cong Wang, Kunhua Liu, and Long Chen. Advdiffuser: Generating adversarial safety-critical driving scenarios via guided diffusion. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp.\ 9983--9989. IEEE, 2024
2024
-
[70]
Error bounds of imitating policies and environments for reinforcement learning
Tian Xu, Ziniu Li, and Yang Yu. Error bounds of imitating policies and environments for reinforcement learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44: 0 6968--6980, 2020. URL https://api.semanticscholar.org/CorpusID:225040357
2020
-
[71]
Flow of reasoning: Training llms for divergent problem solving with minimal examples
Fangxu Yu, Lai Jiang, Haoqiang Kang, Shibo Hao, and Lianhui Qin. Flow of reasoning: Training llms for divergent problem solving with minimal examples. arXiv preprint arXiv:2406.05673, 2024
2024 arXiv
-
[72]
Learn what not to learn: Action elimination with deep reinforcement learning
Tom Zahavy, Matan Haroush, Nadav Merlis, Daniel J Mankowitz, and Shie Mannor. Learn what not to learn: Action elimination with deep reinforcement learning. Advances in neural information processing systems, 31, 2018
2018
-
[73]
Generative flow networks for discrete probabilistic modeling
Dinghuai Zhang, Nikolay Malkin, Zhen Liu, Alexandra Volokhova, Aaron Courville, and Yoshua Bengio. Generative flow networks for discrete probabilistic modeling. In International Conference on Machine Learning, pp.\ 26412--26428. PMLR, 2022
2022
-
[74]
Let the flows tell: Solving graph combinatorial problems with GF lownets
Dinghuai Zhang, Hanjun Dai, Nikolay Malkin, Aaron Courville, Yoshua Bengio, and Ling Pan. Let the flows tell: Solving graph combinatorial problems with GF lownets. In Thirty-seventh Conference on Neural Information Processing Systems, 2023 a . URL https://openreview.net/forum?...
2023
-
[75]
Distributional gflownets with quantile flows
Dinghuai Zhang, Ling Pan, Ricky TQ Chen, Aaron Courville, and Yoshua Bengio. Distributional gflownets with quantile flows. arXiv preprint arXiv:2302.05793, 2023 b
2023 arXiv
-
[76]
Adversarial generative flow network for solving vehicle routing problems
Ni Zhang, Jingfeng Yang, Zhiguang Cao, and Xu Chi. Adversarial generative flow network for solving vehicle routing problems. In The Thirteenth International Conference on Learning Representations, 2025 a . URL https://openreview.net/forum?id=tBom4xOW1H
2025
-
[77]
COF lownet: Conservative constraints on flows enable high-quality candidate generation
Yudong Zhang, Xuan Yu, Xu Wang, Zhaoyang Sun, Chen Zhang, Pengkun Wang, and Yang Wang. COF lownet: Conservative constraints on flows enable high-quality candidate generation. In The Thirteenth International Conference on Learning Representations, 2025 b . URL https://openrevie...
2025
-
[78]
Automatic curriculum learning through value disagreement
Yunzhi Zhang, Pieter Abbeel, and Lerrel Pinto. Automatic curriculum learning through value disagreement. Advances in Neural Information Processing Systems, 33: 0 7648--7659, 2020
2020
-
[79]
Sample-efficient multi-objective molecular optimization with gflownets
Yiheng Zhu, Jialu Wu, Chaowen Hu, Jiahuan Yan, Tingjun Hou, Jian Wu, et al. Sample-efficient multi-objective molecular optimization with gflownets. Advances in Neural Information Processing Systems, 36: 0 79667--79684, 2023
2023
-
[80]
Maximum entropy inverse reinforcement learning
Brian D Ziebart, Andrew L Maas, J Andrew Bagnell, Anind K Dey, et al. Maximum entropy inverse reinforcement learning. In Aaai, volume 8, pp.\ 1433--1438. Chicago, IL, USA, 2008
2008
-
[81]
Ziebart, J
Brian D. Ziebart, J. Andrew Bagnell, and Anind K. Dey. Modeling interaction via the principle of maximum causal entropy. In Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML'10, pp.\ 1255–1262, Madison, WI, USA, 2010. Omnipr...
2010
-
[82]
Fine-tuning language models from human preferences
Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593, 2019
1909 arXiv
-
[83]
A variational perspective on generative flow networks
Heiko Zimmermann, Fredrik Lindsten, Jan-Willem van de Meent, and Christian A Naesseth. A variational perspective on generative flow networks. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https://openreview.net/forum?id=AZ4GobeSLq
2023
-
[84]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
-
[85]
@esa (Ref
\@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...
-
[86]
\@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...
-
[87]
?+dp ǝA/8n6aN-Fo 7F<B :qRwl3d 9K 1
@open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...
2023
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.