Hybrid Data-Model Parallel Training for Sequence-to-Sequence Recurrent Neural Network Machine Translation

Eiichiro Sumita; Junya Ono; Masao Utiyama

REVIEW 3 major objections 5 minor 1 cited by

Hybrid Data-Model Parallel Training for Sequence-to-Sequence Recurrent Neural Network Machine Translation

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Training attention-based Seq2Seq RNN machine translation on four GPUs can run 4.13–4.20 times faster than on one GPU without lowering BLEU.

desk verdict Useful and honest engineering result, but the headline speedup conflates parallelization with removing input-feeding and a larger mini-batch; needs a same-model 1-GPU baseline before the 4.13–4.20x claim stands. read the letter →

arxiv 1909.00562 v2 pith:RUFAJQFO submitted 2019-09-02 cs.DC cs.CLcs.LGcs.NE

Junya Ono , Masao Utiyama , Eiichiro Sumita This is my paper · ORCID

classification cs.DCcs.CLcs.LGcs.NE

keywords sequence-to-sequencerecurrentneuralnetworkmachinetranslationdataparallelismmodelhybridattentionGPUtraining

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Training neural machine translation with attention is slow, often taking days on large corpora. This paper claims a way to make it much faster on a single machine with four GPUs: run the recurrent encoder and decoder in model-parallel fashion across GPUs, and run the attention-and-softmax part in data-parallel fashion, alternating the two modes within one training step. The reported result is a 4.13 to 4.20 times speed-up over one GPU on standard English–German benchmarks, with BLEU scores at least as high as the comparison baseline. The move that makes this schedule possible is removing input-feeding from the decoder so that, during training, all target words are known in advance and decoder hidden states can be computed for all time steps before attention is applied. If the claim holds, it gives practitioners a concrete way to scale RNN translation training beyond what pure data or pure model parallelism delivers.

What carries the argument

The load-bearing object is a two-phase training schedule made possible by teacher forcing. Removing input-feeding makes the decoder's LSTM hidden states independent of the attention context, so after one forward pass all encoder hidden states $S$ and decoder hidden states $H$ are known for every position. Attention and prediction then factorize as $\alpha = \mathrm{Softmax}(H^T W_\alpha S)$, $C = \alpha \cdot S$, $H_c = \tanh(W_c[H; C])$, and $P = \mathrm{Softmax}(F_c(H_c))$. The first phase computes $S$ and $H$ with model parallelism across GPUs; the second phase partitions the attention-score, context-vector, and softmax matrix computations over the mini-batch across all GPUs and accumulates the attention-softmax parameter gradients at a root GPU. The parameter distribution matters: the encoder-decoder holds the bulk of the roughly 138 million parameters and benefits from model parallelism, while the attention-softmax part is small enough that data parallelism on it avoids repeated synchronization of the large LSTM parameters.

What would settle it

Train the same model with the hybrid schedule but restore input-feeding, so the decoder is sequential at each step; or train HybridNMT with the 4-GPU mini-batch size capped to the 1-GPU size rather than tripled or quadrupled. If the speed-up collapses toward the model-parallel-only value, or if BLEU on a long-sentence test set drops noticeably versus the input-feeding baseline, the claimed speed-up and accuracy-neutrality would not hold in that regime.

Watch

Extended reading notes

Core claim

The discovery is that the attention mechanism, normally the piece that ties decoder steps together, can be moved into a separate data-parallel stage once input-feeding is removed. With teacher forcing, all decoder hidden states are available after one model-parallel pass, so attention scores, context vectors, and output softmax can be computed in bulk and split across GPUs by mini-batch. The paper reports scaling factors of 4.13 on WMT14 en–de and 4.20 on WMT17 en–de, measured as source tokens per second relative to one GPU, with test BLEU scores of 22.71 and 26.91, respectively. The same network without the hybrid schedule but with input-feeding trains at scaling factors of 3.43–3.57, while pure model parallelism reaches 2.32–2.51, so the authors attribute the speed-up to the hybrid alternation combined with the architectural simplification.

Load-bearing premise

The load-bearing premise is that training can supply all target words before attention is computed, so removing input-feeding leaves decoder hidden states independent of attention context; if decoder states had to incorporate attention at each step, the model-parallel decoder and the later data-parallel attention phase could not run in this order.

Editorial extensions

If this is right

Four GPUs can train an attention Seq2Seq RNN translation model more than four times as fast as one GPU, with BLEU at least as high as the input-feeding baseline on WMT14 and WMT17 English–German.
Model parallelism alone beats data parallelism for this architecture, with roughly 2.3–2.5x versus 1.6–1.7x scaling, because the encoder-decoder parameters dominate and avoid synchronization.
Removing input-feeding does not hurt translation accuracy in the tested settings and speeds convergence, since HybridNMT reached lower perplexity in less wall-clock time than the baselines.
The hybrid scheme, being component-wise, should generalize to other encoder-decoder models that have an encoder, a decoder, and a softmax layer, such as Transformer, though the paper does not implement that extension.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An implication left implicit is that the reported more-than-4x scaling with four GPUs may partly reflect the larger mini-batch made possible by spreading activations; the tokens-per-second measure includes this batch-size effect, so a fixed-total-batch comparison would be a sharper test of parallelism alone.
The teacher-forcing trick that removes decoder sequentiality is essentially the same assumption used by non-autoregressive and parallel-decoding models, so the hybrid schedule could be combined with those approaches or adapted to any setting where target prefixes are known in advance.
Because attention-softmax becomes the data-parallel stage, the method's benefit should grow as the attention context length or vocabulary size increases; a testable extension is measuring how the speed-up scales with longer source sentences or larger output vocabularies.
Beyond BLEU, removing input-feeding may change translation behavior on long or low-resource text; the paper evaluates only BLEU, so a targeted quality analysis on such segments would clarify whether accuracy-neutrality holds more broadly.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper is a clean, useful systems result: hybrid data-model parallelism for Seq2Seq RNN MT, with the encoder-decoder part model-parallel and the attention-softmax part data-parallel. The idea is straightforward, the implementation is described clearly, and the measurements on WMT14 and WMT17 are direct and reproducible in principle. BLEU scores are preserved or slightly improved, which is honest evidence that the architecture change did not hurt quality. The specific split is new as far as I know, and the authors are upfront that it combines known ingredients from Wu et al. (2016) and Wang et al. (2018). They don't compare against Wang et al., but they acknowledge it as future work; that is a minor gap.

The main soft spot is exactly what the stress-test note says. The abstract claims a 4.13–4.20x speedup versus 1 GPU, but the 1-GPU row in Table 3 is the original input-feeding model with 142M parameters and mini-batch size 64. The HybridNMT row removes input-feeding, has 138M parameters, and uses mini-batch 224. So the reported scaling factor conflates three changes: parallelization, architecture simplification, and larger batch size. A faster single-GPU decoder without input-feeding would inflate the ratio even if the 4-GPU schedule contributed nothing. The HybridNMTIF row helps separate input-feeding from parallelism, but it is itself a 4-GPU run, so it does not supply the missing 1-GPU HybridNMT baseline. The BLEU comparisons do not close this gap; they speak to quality, not to the denominator of the speedup. The paper's own note that mini-batch sizes were about four times larger when using 4 GPUs confirms part of the speedup is a batch-size effect. So the headline claim should be rephrased as a combined model-plus-parallel speedup, or the missing baseline should be supplied.

Minor issues: single runs without error bars limit statistical strength, and the comparison with an automatic hybrid parallelism method is left as future work. None of these are load-bearing for the core idea—the hybrid schedule is plausible and the paper's own teacher-forcing assumption is stated explicitly. But the headline number needs to be re-measured.

The paper is for people doing parallel training for RNN sequence-to-sequence models, or anyone interested in how to measure parallel speedups without confounds. It's a legitimate engineering contribution in a subfield that has largely moved to Transformers, though the hybrid idea may transfer. I'd send it to peer review with a request for the missing same-model baseline and some error bars. Conditional accept is the right call.

Referee Report

3 major / 5 minor

Summary. The paper proposes a hybrid data-model parallel training method for sequence-to-sequence RNN machine translation. The encoder-decoder layers are split across GPUs in a model-parallel fashion, while the attention-softmax computation is data-parallel. To make the decoder pipeline parallelizable, the paper removes input-feeding, exploiting the fact that during training all target words are known in advance (teacher forcing). Experiments on WMT14 and WMT17 English-German report training throughput, convergence speed, and BLEU scores, with a headline claim of a 4.13-4.20x speedup from 1 GPU to 4 GPUs without loss of translation accuracy. The proposed schedule itself is coherent: Section 3.2 explicitly justifies it by the teacher-forcing assumption, so the stress-test concern about decoder dependence on attention context does not land as a correctness issue for training.

Significance. If the claims are properly benchmarked, the paper makes a useful systems contribution: it gives a concrete division of labor between model parallelism and data parallelism for RNN encoder-decoder models, and it identifies input-feeding as the main obstacle to decoder-side parallelism. The method is clearly described with equations, and the work includes comparisons against pure data and pure model parallelism on standard public benchmarks. The main value is the scheduling idea and the observation that removing input-feeding enables it; the paper does not rely on fitted parameters or circular reasoning, and the reported BLEU results are broadly in line with published Seq2Seq RNN systems.

major comments (3)

[Abstract; Table 3] The headline 4.13-4.20x speedup is computed as HybridNMT at a 224-item mini-batch divided by the 1-GPU baseline with input-feeding at a 64-item mini-batch. This ratio conflates three effects: the removal of input-feeding (a model change), the larger mini-batch enabled by the parallel deployment, and the GPU parallelism itself. Note that HybridNMTIF already achieves 3.43-3.57x, so the incremental contribution of the hybrid parallelism over HybridNMTIF is only about 1.20x. To support the abstract's '4 GPUs compared with 1 GPU' wording, the authors should report a 1-GPU run of HybridNMT under the same mini-batch size, or rephrase the claim as a combined model-plus-parallel speedup.
[Section 4.3, Table 3] The comparison among data parallelism (256 mini-batch), model parallelism (224), HybridNMTIF (224), and HybridNMT (224) is useful, but the statement that scaling factors above 4 demonstrate the effectiveness of the hybrid method is not supported. Because the 1-GPU reference uses a 64-item mini-batch, a single GPU processing a 224-item batch would likely show higher throughput regardless of parallelization, so the >4 factors are not evidence of superlinear parallel efficiency. A controlled run that fixes the total batch size across configurations is needed to separate batch-size gains from parallelization gains.
[Section 4.5, Tables 4 and 5] The claim of 'without affecting machine translation accuracy' is not fully established by the presented BLEU comparisons. Table 5 compares HybridNMT against published systems with different preprocessing, model sizes, and decoding regimes, and Table 4 compares it with an OpenNMT-lua baseline under different length-normalization settings. Please include a matched comparison using the same data, tokenization, vocabulary, optimizer schedule, and decoding protocol between HybridNMT and the input-feeding baseline, and report BLEU for the same number of updates or epochs rather than only the best-perplexity runs.

minor comments (5)

[Section 3.2, Eq. (1)] The notation in Eq. (1) mixes 'a1' and 'α i' in the same sequence; please use a consistent symbol for the attention coefficients.
[Section 3.2] The text says 'Fc denotes a liner function'; this should be 'linear function'.
[Section 4.3] The sentence 'mini-batch sizes were about 4 times when using 4 GPUs compared with those obtained when using 1 GPU' conflates total batch size with per-GPU batch size; please clarify which quantity is meant.
[Figure 4] The convergence-speed figure should state whether all methods were trained for identical wall-clock budgets, identical numbers of batches, or identical numbers of epochs, and how many updates correspond to each plotted point.
[Section 3.2, Figure 3] The claim that one GPU 'stores the hidden states of all steps' suggests a possible memory bottleneck; please report GPU memory utilization or discuss how this storage scales with sequence length and batch size.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the speedup and BLEU claims are direct experimental measurements on public benchmarks, not derivations from fitted inputs or self-citations.

full rationale

The paper's central claims are empirical: training throughput in source tokens per second and BLEU scores on WMT14/WMT17. The proposed HybridNMT architecture is defined concretely in Section 3.2, and its motivation (removing input-feeding to enable parallel decoder computation) is presented as a design choice whose accuracy impact is then measured, not assumed. The speedup factors in Table 3 are ratios of measured throughputs, and the BLEU comparisons in Tables 4 and 5 are independent accuracy evaluations against published systems. No parameter is fitted to a target result and then renamed as a prediction; no uniqueness theorem or ansatz is imported from the authors' own prior work to force the conclusion. The footnote stating that the manuscript is identical to the formal publication is purely bibliographic and not load-bearing. The skeptic's observation that the 1-GPU baseline includes input-feeding and uses a different mini-batch size is a benchmarking-fairness concern, not a circularity concern: the reported ratio remains an empirical measurement and does not reduce to an identity or to a fitted parameter. Overall, the derivation chain is self-contained with respect to the evidence presented.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The central result is a measured speedup, so there are no free parameters fitted to the outcome. The design relies on two domain assumptions: teacher forcing during training and the accuracy-neutrality of removing input-feeding. No new physical or mathematical entities are introduced.

assumptions (2)

domain assumption During training, all target words are known beforehand, so decoder hidden states for all time steps can be computed without waiting for attention context.
Section 3.2 states this explicitly and it is what makes the decoder side model-parallel and the data-parallel attention phase possible.
domain assumption Removing input-feeding from the decoder does not reduce translation accuracy enough to offset the parallel speedup.
The paper tests this in Section 4.5, but it remains a modeling assumption that the simplified architecture is acceptable; the BLEU results support it.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Hybrid Data-Model Parallel Training for Sequence-to-Sequence Recurrent Neural Network Machine Translation." pith.science (2026). https://pith.science/paper/RUFAJQFO

@misc{pith2026190900562,
  author       = {Pith},
  title        = {Pith review of: Hybrid Data-Model Parallel Training for Sequence-to-Sequence Recurrent Neural Network Machine Translation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RUFAJQFO}},
  note         = {Machine review of arXiv:1909.00562}
}

read the original abstract

Reduction of training time is an important issue in many tasks like patent translation involving neural networks. Data parallelism and model parallelism are two common approaches for reducing training time using multiple graphics processing units (GPUs) on one machine. In this paper, we propose a hybrid data-model parallel approach for sequence-to-sequence (Seq2Seq) recurrent neural network (RNN) machine translation. We apply a model parallel approach to the RNN encoder-decoder part of the Seq2Seq model and a data parallel approach to the attention-softmax part of the model. We achieved a speed-up of 4.13 to 4.20 times when using 4 GPUs compared with the training speed when using 1 GPU without affecting machine translation accuracy as measured in terms of BLEU scores.

Figures

Figures reproduced from arXiv: 1909.00562 by the authors.

**Figure 3.** Proposed model for hybrid parallelism. start performing their assigned computations until all nodes related to the previous target words finish their computation. This is due to the input-feeding approach employed. For instance, the target word embedding of y2 needs to be concatenated with the attentional hidden state of y1 before being input into the first LSTM layer. 3.2 Proposed model for hybrid parallelism Herei… view at source ↗

**Figure 4.** Convergence speed for different methods. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 4 citations worldwide. Full citation record

Distributed Deep Learning using Stochastic Gradient Staleness
cs.LG 2025-09 reject novelty 5.0 of 10

A hybrid data- and model-parallel training scheme using stale gradients and consensus averaging, claimed to converge to critical points and speed up ResNet-20 training on CIFAR-10.

Reference graph

Works this paper leans on

23 extracted references · 17 canonical work pages · cited by 1 Pith paper

[1]

Murray, Benoit Steiner, Paul Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng

Mart\' i n Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek G. Murray, Benoit Steiner, Paul Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2016. TensorFlow: A system...

work page 2016
[2]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate . In Proceedings of the International Conference on Learning Representations (ICLR)

work page 2015
[3]

Ond r ej Bojar, Christian Buck, Christian Federmann, Barry Haddow, Philipp Koehn, Johannes Leveling, Christof Monz, Pavel Pecina, Matt Post, Herve Saint-Amand, Radu Soricut, Lucia Specia, and Ale s Tamchyna. 2014. Findings of the 2014 Workshop on Statistical Machine Translation . In Proceedings of the Ninth Workshop on Statistical Machine Translation (WMT)

work page 2014
[4]

Ond r ej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Shujian Huang, Matthias Huck, Philipp Koehn, Qun Liu, Varvara Logacheva, Christof Monz, Matteo Negri, Matt Post, Raphael Rubino, Lucia Specia, and Marco Turchi. 2017. Findings of the 2017 conference on machine translation (WMT17) . In Proceedings of the Second Conference o...

work page 2017
[5]

Denny Britz, Anna Goldie, Minh-Thang Luong, and Quoc Le. 2017. Massive exploration of neural machine translation architectures . In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP)

work page 2017
[6]

Tianqi Chen, Mu Li, Yutian Li, Min Lin, Naiyan Wang, Minjie Wang, Tianjun Xiao, Bing Xu, Chiyuan Zhang, and Zheng Zhang. 2015. MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems . In arXiv:1512.01274

arXiv 2015
[7]

Junyoung Chung, Kyunghyun Cho, and Yoshua Bengio. 2016. A Character-Level Decoder without Explicit Segmentation for Neural Machine Translation . In arXiv:1603.06147

arXiv 2016
[8]

S\' e bastien Jean, Kyunghyun Cho, Roland Memisevic, and Yoshua Bengio. 2015. On Using Very Large Target Vocabulary for Neural Machine Translation . In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (ACL)

work page 2015

Show all 23 references

[9]

Marcin Junczys-Dowmunt, Roman Grundkiewicz, Tomasz Dwojak, Hieu Hoang, Kenneth Heafield, Tom Neckermann, Frank Seide, Ulrich Germann, Alham Fikri Aji, Nikolay Bogoychev, Andr\' e F. T. Martins, and Alexandra Birch. 2018. Marian: Fast Neural Machine Translation in C++ . In arXi...

2018 arXiv
[10]

Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. 2017. On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima . In arXiv:1609.04836

2017 arXiv
[11]

Kingma and Jimmy Ba

Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization . In Proceedings of the International Conference on Learning Representations (ICLR)

2015
[12]

Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander M. Rush. 2017. OpenNMT: Open-Source Toolkit for Neural Machine Translation . In Proceedings of the 55th Association for Computational Linguistics (ACL), System Demonstrations

2017
[13]

Alex Krizhevsky. 2014. One weird trick for parallelizing convolutional neural networks . In arXiv:1404.5997

2014 arXiv
[14]

Minh-Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective Approaches to Attention-based Neural Machine Translation . In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP)

2015
[15]

Myle Ott, Sergey Edunov, David Grangier, and Michael Auli. 2018. Scaling Neural Machine Translation . In Proceedings of the Third Conference on Machine Translation (WMT): Research Papers

2018
[16]

Rico Sennrich, Alexandra Birch, Anna Currey, Ulrich Germann, Barry Haddow, Kenneth Heafield, Antonio Valerio Miceli Barone, and Philip Williams. 2017. The University of Edinburgh's Neural MT Systems for WMT17 . In Proceedings of the Second Conference on Machine Translation (WM...

2017
[17]

Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016 a . Improving Neural Machine Translation Models with Monolingual Data . In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL)

2016
[18]

Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016 b . Neural Machine Translation of Rare Words with Subword Units . In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL)

2016
[19]

Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A Simple Way to Prevent Neural Networks from Overfitting . Journal of Machine Learning Research, 15

2014
[20]

Gomez, ukasz Kaiser, and Illia Polosukhin

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need . In Advances in Neural Information Processing Systems (NIPS)

2017
[21]

Minjie Wang, Chienchin Huang, and Jinyang Li. 2018. Unifying Data, Model and Hybrid Parallelism in Deep Learning via Tensor Tiling . In arXiv:1805.04170

2018 arXiv
[22]

Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, ukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Ste...

2016 arXiv
[23]

Jie Zhou, Ying Cao, Xuguang Wang, Peng Li, and Wei Xu. 2016. Deep Recurrent Models with Fast-Forward Connections for Neural Machine Translation . Transactions of the Association for Computational Linguistics (TACL), 4

2016

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Murray, Benoit Steiner, Paul Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng

Mart\' i n Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek G. Murray, Benoit Steiner, Paul Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2016. TensorFlow: A system...

work page 2016

[2] [2]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate . In Proceedings of the International Conference on Learning Representations (ICLR)

work page 2015

[3] [3]

Ond r ej Bojar, Christian Buck, Christian Federmann, Barry Haddow, Philipp Koehn, Johannes Leveling, Christof Monz, Pavel Pecina, Matt Post, Herve Saint-Amand, Radu Soricut, Lucia Specia, and Ale s Tamchyna. 2014. Findings of the 2014 Workshop on Statistical Machine Translation . In Proceedings of the Ninth Workshop on Statistical Machine Translation (WMT)

work page 2014

[4] [4]

Ond r ej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Shujian Huang, Matthias Huck, Philipp Koehn, Qun Liu, Varvara Logacheva, Christof Monz, Matteo Negri, Matt Post, Raphael Rubino, Lucia Specia, and Marco Turchi. 2017. Findings of the 2017 conference on machine translation (WMT17) . In Proceedings of the Second Conference o...

work page 2017

[5] [5]

Denny Britz, Anna Goldie, Minh-Thang Luong, and Quoc Le. 2017. Massive exploration of neural machine translation architectures . In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP)

work page 2017

[6] [6]

Tianqi Chen, Mu Li, Yutian Li, Min Lin, Naiyan Wang, Minjie Wang, Tianjun Xiao, Bing Xu, Chiyuan Zhang, and Zheng Zhang. 2015. MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems . In arXiv:1512.01274

arXiv 2015

[7] [7]

Junyoung Chung, Kyunghyun Cho, and Yoshua Bengio. 2016. A Character-Level Decoder without Explicit Segmentation for Neural Machine Translation . In arXiv:1603.06147

arXiv 2016

[8] [8]

S\' e bastien Jean, Kyunghyun Cho, Roland Memisevic, and Yoshua Bengio. 2015. On Using Very Large Target Vocabulary for Neural Machine Translation . In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (ACL)

work page 2015