Pith. sign in

REVIEW 4 cited by

Monotonic Chunkwise Attention

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1712.05382 v2 pith:EQANFU2T submitted 2017-12-14 cs.CL stat.ML

Chung-Cheng Chiu , Colin Raffel This is my paper · ORCID
classification cs.CLstat.ML
keywords attentionmonotonicsoftappliedchunkwisedecodingmochamodel
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Sequence-to-sequence models with soft attention have been successfully applied to a wide variety of problems, but their decoding process incurs a quadratic time and space cost and is inapplicable to real-time sequence transduction. To address these issues, we propose Monotonic Chunkwise Attention (MoChA), which adaptively splits the input sequence into small chunks over which soft attention is computed. We show that models utilizing MoChA can be trained efficiently with standard backpropagation while allowing online and linear-time decoding at test time. When applied to online speech recognition, we obtain state-of-the-art results and match the performance of a model using an offline soft attention mechanism. In document summarization experiments where we do not expect monotonic alignments, we show significantly improved performance compared to a baseline monotonic attention-based model.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. PairAlign: A Framework for Sequence Tokenization via Self-Alignment with Applications to Audio Tokenization

    cs.LG 2026-05 unverdicted novelty 7.0 of 10

    PairAlign learns compact variable-length token sequences for audio via self-alignment on paired content-preserving views, achieving 55% fewer archive tokens than VQ while preserving edit-distance retrieval at 12.71 tokens/s.

  2. SimulS2ST-Omni: Data-Efficient Streaming Speech-to-Speech Translation via Explicit Trajectory Supervision

    cs.SD 2026-07 conditional novelty 6.0 of 10

    A joint text-code trajectory supervision recipe lets a two-stream speech LM achieve competitive long-form streaming S2ST with ~2k hours of paired speech.

  3. Do LLMs Need Architectural Changes for Simultaneous Speech Translation? A Prefix-to-Prefix Data Driven Approach

    cs.CL 2026-07 conditional novelty 6.0 of 10

    Teacher-built bounded-waiting prefix targets let a chunked streaming speech LLM improve simultaneous translation quality by +1.54 COMETKiwi at +0.15 s latency.

  4. MFLA: Monotonic Finite Look-ahead Attention for Streaming Speech Recognition

    cs.CL 2025-06 conditional novelty 6.0 of 10

    MFLA adds finite look-ahead attention plus a CIF-based token counter to Whisper, enabling streaming recognition with a wait-k latency-quality trade-off.

Pith tools