Generative Question Refinement with Deep Reinforcement Learning in Retrieval-based QA System

Chenwei Zhang; Philip S. Yu; Xiaohui Yan; Ye Liu; Yi Chang

REVIEW 4 major objections 5 minor 43 references

Generative Question Refinement with Deep Reinforcement Learning in Retrieval-based QA System

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single generative model can repair typos, word-order errors, and noisy background in user questions, and the repair improves answer retrieval.

desk verdict A solid, incremental combination of existing ideas for question refinement, with credible but not airtight retrieval gains; the answer-reward proxy is the main unvalidated link. read the letter →

arxiv 1908.05604 v3 pith:AVH2M5SY submitted 2019-08-13 cs.IR cs.AI

Ye Liu , Chenwei Zhang , Xiaohui Yan , Yi Chang , Philip S. Yu This is my paper · ORCID

classification cs.IRcs.AI

keywords questionrefinementsequence-to-sequencereinforcementlearningproximalpolicyoptimizationBERTembeddingsanswerretrievalqueryreformulationanswering

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the quality of user questions, not just retrieval algorithms, is a bottleneck for retrieval-based QA systems, and that a single sequence-to-sequence model can repair the common failure modes: typos, word-order errors, and irrelevant background phrases. It proposes QREFINE, trained in two stages: supervised pretraining followed by proximal policy optimization with a reward combining a word-level language-model score and a sentence-level answer-correlation score. The central claim is that this generates more readable questions with fewer mistakes than the user's original question, and that retrieval of the correct answer, measured by Hits@K, improves substantially on both an English non-factoid QA dataset and a Chinese customer-service log. If true, question refinement becomes a practical preprocessing layer for existing QA systems without domain-specific rules.

What carries the argument

The load-bearing object is the QREFINE policy: a Seq2Seq LSTM encoder-decoder with attention whose input representation stacks three grain levels — word embeddings, character-level BiLSTM embeddings for handling typos, and BERT contextual embeddings — and whose training objective is a PPO-clipped reinforcement learning objective over a composite reward. The composite reward is the sum of a word-level wording reward (a pretrained LM probability plus a BERT contextual probability) and a question-level answer-correlation reward computed by a fixed QA-LSTM similarity model; because the answer-correlation reward is added at the final token and then propagated back through a discounted accumulated return, the policy can be trained to value retrievability even before the sentence is finished. This machinery is what lets one model handle wrong words, wrong word order, and noisy background as a single task.

What would settle it

Run the reward model on two sets of rewrites from the same policy — those that receive high answer-correlation rewards and those that receive low ones — and measure Hits@K of each set against the gold answers; if high-reward rewrites do not retrieve the gold answer more often than low-reward rewrites, the reward is being gamed rather than measuring retrievability. A second check is to have human annotators judge whether the refined question preserves the user's intent: a rewrite that fixes the wording but changes the meaning should be counted as a failure.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that ill-formed questions in real QA logs can be rewritten into well-formed ones by a unified generative model, and that the rewriting should be optimized explicitly for answer retrievability rather than for surface similarity alone. The model encodes each word by concatenating context-free embeddings, character-level BiLSTM embeddings, and BERT contextual embeddings, which lets it correct misspellings and understand noisy word order. Decoding is a policy that is pretrained by maximum likelihood and then refined with PPO, where each generated word receives an immediate wording reward and the completed question receives an answer-correlation reward from a fixed QA-LSTM trained with a hinge loss. The paper reports that this combination outperforms paraphrase, query-reformulation, and reading-comprehension baselines on BLEU, ROUGE, and METEOR, and raises Hits@K for answer retrieval on two datasets.

Load-bearing premise

The whole pipeline rests on the assumption that the QA-LSTM answer-correlation reward is a faithful proxy for whether a rewritten question will actually retrieve the right answer; if the policy learns to inflate that score without improving real retrieval, the reported Hits@K improvements would not reflect genuine question refinement.

Editorial extensions

If this is right

A retrieval-based QA system can prepend QREFINE as a rule-free preprocessing layer, so ill-formed user questions are repaired before retrieval.
Because the answer-correlation reward is trained on paired question-answer data, the approach transfers to any domain where such pairs exist, including non-English customer-service logs.
Combining character-level, context-free, and contextual embeddings improves robustness to input typos, so the model can handle out-of-vocabulary misspellings without an explicit spelling corrector.
Using PPO instead of REINFORCE yields more stable training and earlier convergence, making reinforcement learning more practical for sequence generation in this task.
The reported results imply that readability metrics and retrieval metrics move together: fixing word order and noise improves both generation-quality scores and Hits@K.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same reward-shaping recipe — word-level fluency reward plus task-level correlation reward — could transfer to other text normalization tasks, such as grammar correction or query rewriting for product search, provided a comparable task-level success signal exists.
The dependence on paired ill-formed and well-formed questions is the main practical bottleneck; an inverse-RL or unsupervised approach over a pool of well-formed questions, which the paper names as future work, would broaden applicability substantially.
A direct extension would be to test whether the policy games the QA-LSTM reward by measuring retrieval with a held-out retriever different from the one used in evaluation; if the Hits@K gains persist, the refinement effect is likely genuine rather than reward-specific.
Because the reward model itself is an answer-selection network, the refined questions could plausibly improve answer selection as well as retrieval, not just the top-K retrieval stage.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's the short version: QREFINE is a reasonable, well-motivated combination of existing ideas for question refinement, and the paper mostly does what it says. The new part is the specific combination—multi-grain embeddings (word, char, BERT) plus PPO with two rewards for this task—and that is not in prior work. The empirical work is more thorough than average: two datasets (one real), several baselines, ablations, case studies, and learning curves. The generation quality numbers are consistently better than baselines, and the retrieval Hits@K numbers mostly point in the same direction. So the paper earns a serious referee.

Now the soft spots. The main one is the answer-correlation reward in Eq. 6. It is a learned QA-LSTM trained on the same paired data, and it is never validated against the Hits@K metric used for the headline claim. The paper does not report the correlation between that reward and Hits@K, does not check whether high-reward generations actually retrieve gold answers, and evaluates only one lexical retriever. That means the contribution of the RL component to the retrieval gains is not fully pinned down. It is not fatal—the test-set Hits@K results are the final evidence, and they are positive—but without significance tests or error bars we cannot rule out chance on some of the smaller gaps, for instance the CSU Hits@1 difference between QREFINE and QR-word. A second issue: the wording reward includes the probability from the pretrained Seq2Seq decoder, which is also the policy initialization, so part of the reward is self-referential. That is a defensible reward shape but deserves a sentence of justification. Third, the synthetic corruption process for Yahoo is described too vaguely to reproduce, and the CSU data is proprietary, so the empirical story rests on one public dataset. These are real but not disqualifying.

Who is this for? Anyone working on query reformulation, question rewriting, or RL for text generation, especially in retrieval-based QA. The paper is a useful data point, and the ablation study is informative. I would cite it for the combination of char/BERT embeddings with PPO in a reformulation setting.

On peer review: send it out. It is not a breakthrough, but it is a competent piece of applied research with a clear task, reasonable experiments, and an honest enough presentation. The authors should be pushed to add error bars, validate the reward proxy, and clarify the synthetic data. All of that is fixable.

Would I bring it to reading group? Maybe—good for a discussion of reward design in text generation, less so for novel methodology.

Referee Report

4 major / 5 minor

Summary. The paper proposes QREFINE, a unified Seq2Seq model for refining ill-formed user questions in retrieval-based QA. The model enriches question representations with character, context-free, and BERT embeddings, and trains the generator with reinforcement learning using two reward components: a word-level wording reward from a language model and BERT, and a question-level answer-correlation reward from a QA-LSTM model. Experiments on a synthetic Yahoo dataset and a real Chinese customer-service dataset compare QREFINE-PPO against Seq2Seq, PARA-NMT, AQA, TOQR, and ablations, reporting gains in BLEU/ROUGE/METEOR for generation quality and in PyLucene Hits@K for downstream answer retrieval. The paper concludes that QREFINE generates more readable questions and significantly improves retrieval accuracy.

Significance. If the empirical claims hold, QREFINE addresses a practically important problem with a clean, unified architecture, and the combination of multi-grain embeddings with PPO-based sequence-level rewards is a reasonable contribution. The paper includes two datasets, one of which is a real industrial customer-service log, and the case studies illustrate clear qualitative improvements. However, the central retrieval claim currently rests on an unvalidated reward proxy and on point estimates without statistical support, so the significance of the reported gains is not yet established. The paper also provides a code link, which supports reproducibility if the code is actually released.

major comments (4)

[§3.3, Eq. (6) and §4.4, Table 6] The answer-correlation reward rac is trained on the same paired data and used as the RL reward, but the headline retrieval claim is measured with PyLucene Hits@K; no evidence is provided that rac correlates with Hits@K or that optimizing rac improves retrievability. The gap between QREFINE and its ablation QR-word (Yahoo Hits@5 23.95 vs 20.79; CSU Hits@1 22.10 vs 21.23) is exactly what is attributed to this reward, so the contribution of the RL component to the headline result is unverified. Please report the correlation between rac and Hits@K on a held-out set, or evaluate the policy with a retrieval-based reward, or at minimum show that generated questions receiving high rac actually retrieve the gold answer.
[§3.3, Eq. (5) and Algorithm 1] The wording reward rw includes plm(yt+1|kt), the probability from the 'pre-trained Seq2Seq' decoder, and Algorithm 1 initializes the policy pθ with the same supervised pre-trained policy. Unless the reward LM is frozen and kept separate from the policy parameters, the policy can increase its reward by making its own decoder more confident rather than by improving question quality. Please clarify whether the reward LM is frozen and detached, and if it is not, add an ablation or analysis that rules out this self-referential reward inflation.
[Abstract and §4.4, Tables 4 and 6] The abstract and conclusion claim that the refined questions 'significantly improve' answer retrieval, but the paper reports only point estimates with no error bars, no multiple random seeds, and no significance tests. The differences between QREFINE and the closest baseline in Table 6 are often a few percentage points, and without variance information it is impossible to judge whether these differences are meaningful. Please report means and standard deviations over at least three random seeds and run a paired significance test (e.g., bootstrap or paired t-test) for the Hits@K comparisons.
[§4.1] The construction of the synthetic Yahoo dataset is underspecified: 'randomly change the character of the words or change the order of the character of words' and 'randomly execute those three operations' do not state the corruption rate, the number of corrupted tokens per question, the size of the sampled background phrases, or how the three operations are mixed. This makes the synthetic benchmark hard to reproduce and the reported gains difficult to interpret. Please give the exact generation parameters and, ideally, release the generated triples or a generation script.

minor comments (5)

[§3.4.2, Eq. (14)] The text introduces the probability ratio as β_t, but the formula uses r_t(θ); please make the notation consistent.
[§3.3, Eq. (5) and Eq. (4)] The notation plm(yt+1|kt) is inconsistent with the earlier plm(ym|y1:m−1, x); please define kt and clarify that the probability is conditioned on the generated prefix and the source question.
[Table 3] In the 'Noisy Background' column the metrics are listed as Blue-1, Meteor, Rouge, while the other columns list Blue-1, Rouge, Meteor; please keep the metric order consistent across all columns.
[§3.4.1 and §5.2] TRPO and PPO are described as 'off-policy' methods, but they are more standardly classified as on-policy methods that use importance sampling from an older policy; please correct the terminology.
[Algorithm 1] The input list includes 'rating data R', but R is not used anywhere in the algorithm; please remove it or explain its role.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: final evaluations are external to the training rewards.

full rationale

The derivation chain is self-contained and the central claims are not reduced to their inputs. The RL objective in Eqs. 7-8 combines a wording reward (Eq. 5) and an answer-correlation reward (Eq. 6). The wording reward uses BERT probability plus the decoder probability plm of the pre-trained Seq2Seq module, and Algorithm 1 initializes the policy p_theta with that same supervised pre-trained policy p_theta'; this makes part of the training signal self-referential, effectively a KL-like regularizer toward the initialization, but it is not a fitted parameter renamed as the target result. The answer-correlation reward is a fixed QA-LSTM trained with a hinge loss on paired data; although this reward is a learned proxy, the headline retrieval results are measured with PyLucene Hits@K, an external lexical retriever, and generation quality with BLEU/ROUGE/METEOR against held-out references. No equation equates the predicted output with a fitted input, no load-bearing self-citation chain is used, and no uniqueness claim is imported from the authors' prior work. The lack of direct validation or significance testing of the QA-LSTM reward as a proxy for Hits@K is an empirical-validity concern, not circularity.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on paired supervision and on learned reward models. The free parameters are hyperparameters tuned on a development set; the axioms are standard ML and RL background plus domain assumptions about data representativeness and reward validity. No new physical or conceptual entities are introduced.

free parameters (5)

c1 = tuned over {0.1, 1, 10}
Weight balancing word-level wording reward and answer-correlation reward in Eq. 7, chosen on the development set.
gamma/lambda = tuned over [0, 1)
Discount factor for accumulated rewards in Eq. 8; notation inconsistently uses gamma and lambda.
epsilon (PPO clip) = tuned over {0.1, 0.2, 0.3}
Clipping threshold in the PPO objective, Eq. 14.
c2 = tuned over {0.1, 1}
Entropy regularization weight in Eq. 16.
margin epsilon in answer correlation loss = not specified
Margin in the hinge loss of Eq. 6; the value is not reported.

assumptions (4)

standard math Policy gradient theorem and PPO clipped surrogate objective are valid optimization methods.
Used in Section 3.4 to derive the update rule; standard background for RL-based sequence generation.
domain assumption The paired triples (ill-formed, well-formed, answer) are correctly labeled and representative of real QA interactions.
The entire supervised pretraining and reward-module training rely on these pairs, described in Section 4.1.
domain assumption The QA-LSTM reward model trained on ground-truth pairs provides a valid and transferable reward signal for RL.
Section 3.3 uses this reward to optimize the generator; if the reward is misaligned, the RL gains may not transfer.
domain assumption BERT and character embeddings improve representation of typo-laden questions.
This is a design assumption motivating the encoder; it is partially validated by the ablation study in Figure 2.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Generative Question Refinement with Deep Reinforcement Learning in Retrieval-based QA System." pith.science (2026). https://pith.science/paper/AVH2M5SY

@misc{pith2026190805604,
  author       = {Pith},
  title        = {Pith review of: Generative Question Refinement with Deep Reinforcement Learning in Retrieval-based QA System},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AVH2M5SY}},
  note         = {Machine review of arXiv:1908.05604}
}

read the original abstract

In real-world question-answering (QA) systems, ill-formed questions, such as wrong words, ill word order, and noisy expressions, are common and may prevent the QA systems from understanding and answering them accurately. In order to eliminate the effect of ill-formed questions, we approach the question refinement task and propose a unified model, QREFINE, to refine the ill-formed questions to well-formed question. The basic idea is to learn a Seq2Seq model to generate a new question from the original one. To improve the quality and retrieval performance of the generated questions, we make two major improvements: 1) To better encode the semantics of ill-formed questions, we enrich the representation of questions with character embedding and the recent proposed contextual word embedding such as BERT, besides the traditional context-free word embeddings; 2) To make it capable to generate desired questions, we train the model with deep reinforcement learning techniques that considers an appropriate wording of the generation as an immediate reward and the correlation between generated question and answer as time-delayed long-term rewards. Experimental results on real-world datasets show that the proposed QREFINE method can generate refined questions with more readability but fewer mistakes than the original questions provided by users. Moreover, the refined questions also significantly improve the accuracy of answer retrieval.

Figures

Figures reproduced from arXiv: 1908.05604 by the authors.

**Figure 1.** The architecture of the proposed model Qrefine. ○1 The encoder of the agent module reads the ill-formed question and the decoder generates a well-formed question, one word/phrase at a time. ○2 The well-formed question being generated so far is sent to a pre-trained reward module, which calculates a word-level wording reward from word-level LM and BERT Reward and a question-level answer correlation reward from QA sim… view at source ↗

**Figure 3.** The learning curve analysis on Yahoo and CSU [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

**Figure 2.** The ablation study on Yahoo and CSU dataset [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 17 canonical work pages

[1]

Dzmitry Bahdanau, Philemon Brakel, Kelvin Xu, Anirudh Goyal, Ryan Lowe, Joelle Pineau, Aaron Courville, and Yoshua Bengio. 2016. An actor-critic algorithm for sequence prediction. arXiv preprint arXiv:1607.07086 (2016)

arXiv 2016
[2]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural ma- chine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014)

arXiv 2014
[3]

Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization. 65–72

work page 2005
[4]

Andrew G Barto and Richard S Sutton. 1998. Reinforcement learning. Neural systems for control (1998), 7–29

work page 1998
[5]

Denny Britz, Anna Goldie, Minh-Thang Luong, and Quoc Le. 2017. Massive explo- ration of neural machine translation architectures.arXiv preprint arXiv:1703.03906 (2017)

arXiv 2017
[6]

Christian Buck, Jannis Bulian, Massimiliano Ciaramita, Wojciech Gajewski, An- drea Gesmundo, Neil Houlsby, and Wei Wang. 2018. Ask the Right Questions: Active Question Reformulation with Reinforcement Learning. ICLR (2018)

work page 2018
[7]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding.arXiv preprint arXiv:1810.04805 (2018)

arXiv 2018
[8]

Li Dong, Jonathan Mallinson, Siva Reddy, and Mirella Lapata. 2017. Learning to paraphrase for question answering. arXiv preprint arXiv:1708.06022 (2017)

arXiv 2017

Show all 43 references

[9]

Manaal Faruqui and Dipanjan Das. 2018. Identifying Well-formed Natural Lan- guage Questions. arXiv preprint arXiv:1808.09419 (2018)

2018 arXiv
[10]

Minwei Feng, Bing Xiang, Michael R Glass, Lidan Wang, and Bowen Zhou
[11]

Alex Graves. 2013. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850 (2013)

2013 arXiv
[12]

Evan Greensmith, Peter L Bartlett, and Jonathan Baxter. 2004. Variance reduction techniques for gradient estimates in reinforcement learning. Journal of Machine Learning Research 5, Nov (2004), 1471–1530

2004
[13]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory.Neural computation 9, 8 (1997), 1735–1780

1997
[14]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic opti- mization. arXiv preprint arXiv:1412.6980 (2014)

2014 arXiv
[15]

Zichao Li, Xin Jiang, Lifeng Shang, and Hang Li. 2017. Paraphrase generation with deep reinforcement learning. arXiv preprint arXiv:1711.00279 (2017)

2017 arXiv
[16]

Chen Liang, Jonathan Berant, Quoc Le, Kenneth D Forbus, and Ni Lao. 2016. Neural symbolic machines: Learning semantic parsers on freebase with weak supervision. arXiv preprint arXiv:1611.00020 (2016)

2016 arXiv
[17]

Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. Text Summarization Branches Out (2004)

2004
[18]

Minh-Thang Luong, Hieu Pham, and Christopher D Manning. 2015. Effec- tive approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025 (2015)

2015 arXiv
[19]

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems . 3111–3119

2013
[20]

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2013. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013)

2013 arXiv
[21]

Ramesh Nallapati, Bowen Zhou, Caglar Gulcehre, Bing Xiang, et al. 2016. Ab- stractive text summarization using sequence-to-sequence rnns and beyond.arXiv preprint arXiv:1602.06023 (2016)

2016 arXiv
[22]

Rodrigo Nogueira, Jannis Bulian, and Massimiliano Ciaramita. 2018. Learn- ing to Coordinate Multiple Reinforcement Learning Agents for Diverse Query Reformulation. arXiv preprint arXiv:1809.10658 (2018)

2018 arXiv
[23]

Rodrigo Nogueira and Kyunghyun Cho. 2017. Task-Oriented Query Reformula- tion with Reinforcement Learning. In EMNLP. 574–583

2017
[24]

Boyuan Pan, Hao Li, Zhou Zhao, Bin Cao, Deng Cai, and Xiaofei He. 2017. MEMEN: multi-layer embedding with memory networks for machine compre- hension. arXiv preprint arXiv:1707.09098 (2017)

2017 arXiv
[25]

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In ACL. ACL, 311–318

2002
[26]

Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In EMNLP. 1532–1543

2014
[27]

Marc’Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. 2016. Sequence level training with recurrent neural networks. ICLR (2016)

2016
[28]

John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz
[29]

John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov
[30]

In International Conference on Machine Learning

Trust region policy optimization. In International Conference on Machine Learning. 1889–1897
[31]

Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In NIPS. 3104–3112

2014
[32]

Richard S Sutton, David A McAllester, Satinder P Singh, and Yishay Mansour. 2000. Policy gradient methods for reinforcement learning with function approximation. In NIPS. 1057–1063

2000
[33]

Minjoon Seo, Sewon Min, Ali Farhadi, and Hannaneh Hajishirzi. 2016. Query- reduction networks for question answering. arXiv preprint arXiv:1606.04582 (2016)

2016 arXiv
[34]

Yi-Lin Tuan, Jinzhi Zhang, Yujia Li, and Hung-yi Lee. 2018. Proximal Policy Optimization and its Dynamic Version for Sequence Generation. arXiv preprint arXiv:1808.07982 (2018)

2018 arXiv
[35]

Xin Wang, Wenhu Chen, Yuan-Fang Wang, and William Yang Wang. 2018. No metrics are perfect: Adversarial reward learning for visual storytelling. arXiv preprint arXiv:1804.09160 (2018)

2018 arXiv
[36]

Ming Tan, Cicero dos Santos, Bing Xiang, and Bowen Zhou. 2015. LSTM- based deep learning models for non-factoid answer selection. arXiv preprint arXiv:1511.04108 (2015)

2015 arXiv
[37]

Ziang Xie, Anand Avati, Naveen Arivazhagan, Dan Jurafsky, and Andrew Y Ng
[38]

Zheng Yuan and Ted Briscoe. 2016. Grammatical error correction using neural machine translation. In Proceedings of the 2016 Conference of the North Ameri- can Chapter of the Association for Computational Linguistics: Human Language Technologies. 380–386

2016
[39]

Ronald J Williams and Jing Peng. 1991. Function optimization using connectionist reinforcement learning algorithms. Connection Science 3, 3 (1991), 241–268

1991
[43]

Xingxing Zhang and Mirella Lapata. 2017. Sentence Simplification with Deep Reinforcement Learning. In EMNLP. 584–594

2017
[2015]

In Automatic Speech Recognition and Understanding (ASRU), 2015 IEEE Workshop on

Applying deep learning to answer selection: A study and an open task. In Automatic Speech Recognition and Understanding (ASRU), 2015 IEEE Workshop on . IEEE, 813–820

2015
[2016]

arXiv preprint arXiv:1603.09727 (2016)

Neural language correction with character-based attention. arXiv preprint arXiv:1603.09727 (2016)

2016 arXiv
[2017]

arXiv preprint arXiv:1707.06347 (2017)

Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)

2017 arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Dzmitry Bahdanau, Philemon Brakel, Kelvin Xu, Anirudh Goyal, Ryan Lowe, Joelle Pineau, Aaron Courville, and Yoshua Bengio. 2016. An actor-critic algorithm for sequence prediction. arXiv preprint arXiv:1607.07086 (2016)

arXiv 2016

[2] [2]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural ma- chine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014)

arXiv 2014

[3] [3]

Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization. 65–72

work page 2005

[4] [4]

Andrew G Barto and Richard S Sutton. 1998. Reinforcement learning. Neural systems for control (1998), 7–29

work page 1998

[5] [5]

Denny Britz, Anna Goldie, Minh-Thang Luong, and Quoc Le. 2017. Massive explo- ration of neural machine translation architectures.arXiv preprint arXiv:1703.03906 (2017)

arXiv 2017

[6] [6]

Christian Buck, Jannis Bulian, Massimiliano Ciaramita, Wojciech Gajewski, An- drea Gesmundo, Neil Houlsby, and Wei Wang. 2018. Ask the Right Questions: Active Question Reformulation with Reinforcement Learning. ICLR (2018)

work page 2018

[7] [7]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding.arXiv preprint arXiv:1810.04805 (2018)

arXiv 2018

[8] [8]

Li Dong, Jonathan Mallinson, Siva Reddy, and Mirella Lapata. 2017. Learning to paraphrase for question answering. arXiv preprint arXiv:1708.06022 (2017)

arXiv 2017