Don't paraphrase, detect! Rapid and Effective Data Collection for Semantic Parsing

Jonathan Berant; Jonathan Herzig

REVIEW 2 major objections 4 minor 39 references

Don't paraphrase, detect! Rapid and Effective Data Collection for Semantic Parsing

T0 review · 2 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Crowd detection of grammar paraphrases lifts semantic parsing accuracy from 51.3 to 70.6.

desk verdict GRANNO is a solid, honest empirical contribution showing that detection-based annotation of unlabeled utterances beats paraphrase-based generation for semantic parsing; the main weakness is a missing self-training baseline that would isolate the source of the gain. read the letter →

arxiv 1908.09940 v2 pith:ES5COF4D submitted 2019-08-26 cs.CL cs.AI

Jonathan Herzig , Jonathan Berant This is my paper · ORCID

classification cs.CLcs.AI

keywords semanticparsingdatacollectiongrammar-drivenannotationparaphrasedetectioncrowdsourcingunlabeledutterancesOVERNIGHTdenotationaccuracy

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the standard way of collecting semantic parsing data—generating synthetic pseudo-language from a grammar and asking crowd workers to paraphrase it into natural language—fails because of two distinct mismatches: the generated logical forms differ from those real users produce, and the paraphrased language differs from how users actually speak. The authors propose GRANNO, which instead assumes access to unlabeled utterances from the true distribution, generates a large set of candidate canonical utterances from the grammar, and asks crowd workers to detect which candidate matches each real utterance. On two datasets, this detection-based procedure yields 70.6 average accuracy on the true distribution versus 51.3 for the paraphrase-based baseline, substantially narrowing the gap to a fully supervised parser at 84.9. A sympathetic reader would take the paper as showing that for rapid, low-cost semantic parser construction, collecting from the real distribution through detection is more effective than generating synthetic data and paraphrasing it.

What carries the argument

The central mechanism is the iterative detection loop: a grammar generates a large candidate set of canonical utterances paired with logical forms, and a trainable similarity scorer $s_t(x,c)$ ranks these candidates for each unlabeled utterance; crowd workers then detect, rather than write, the correct paraphrase from the top-$K$ candidates. The initial scorer is unsupervised, $s_0(x,c) = -WMD(x,c)$, where WMD is the Word Mover's Distance between the embedded words of the utterance and the canonical utterance, chosen because it performs word-level alignment and shared entities encourage low distance. The scorer is retrained each iteration with positive examples $(x,c_x)$ from detected paraphrases and negative examples $(x,c)$ from the top-$M$ incorrect candidates; the paper uses an ESIM natural-language-inference architecture with ELMo contextual embeddings. This detection setup also lets the grammar be generated to greater depth than in OVERNIGHT because no human paraphrase is needed for every generated canonical utterance, improving logical form template coverage from about 64-77% at $D=5$ to roughly 90% at $D=6$.

What would settle it

Run GRANNO on a new dataset where the Word Mover's Distance initial scorer places the gold canonical utterance in the top-5 for substantially fewer than half of the unlabeled utterances; if the iterative loop then fails to bootstrap and final accuracy remains near the OVERNIGHT baseline, the central claim would be contradicted.

Watch

Extended reading notes

Core claim

The paper's central claim is that the dominant OVERNIGHT data collection procedure for semantic parsing is limited by two separable sources of distribution mismatch, and that both can be mitigated by replacing paraphrase generation with paraphrase detection over unlabeled utterances from the target distribution. The authors decompose $p_{nat}(x,z) = p_{nat}(z)p_{nat}(x|z)$ and show that the logical form mismatch—insufficient coverage of templates that occur in real data, plus unlikely generated logical forms—and the language mismatch—crowd workers' paraphrases differ from natural user language even when the logical form is held fixed—together explain the performance gap. GRANNO generates canonical utterances up to depth $D=6$, uses an initial unsupervised scorer $s_0(x,c) = -WMD(x,c)$ (negative Word Mover's Distance) to rank candidate canonical utterances for each unlabeled utterance, has crowd workers select the correct candidate, and iteratively trains a neural paraphrase detector $s_t$ on the resulting positive and negative examples. Across GEOQUERY and SCHOLAR, GRANNO annotates 89% of the original training utterances and reaches 70.6 average denotation accuracy, compared to 51.3 for OVERNIGHT and 75.8 for a simulated perfect-worker version, while costing less ($155 versus $515) and converging in a few iterations.

Load-bearing premise

The initial scorer must rank the correct grammar-generated sentence among the top few candidates for enough real user utterances; the paper reports that this held for only about half of the utterances and explicitly warns that it is not guaranteed.

Editorial extensions

If this is right

If GRANNO is correct, developers of semantic parsers for new domains can build training data primarily from query logs and a grammar, rather than paying crowd workers to paraphrase synthetic language, and obtain dramatically better accuracy on real user utterances.
The 19-point accuracy improvement over OVERNIGHT closes roughly half the gap to a fully supervised parser, suggesting that the remaining gap comes from the errors in detection and from the coverage limit, both of which may be addressable with better initial scoring or deeper grammar generation.
Because GRANNO labels unlabeled utterances from the true distribution, the resulting training set has far less distribution mismatch, so the parser's performance on real data is not an artifact of development-set tuning.
The iterative procedure converges quickly, and crowd detection accuracy stays high (above 85%) across iterations, implying that detection is an easier and more reliable task for non-expert annotators than generation.
The method's cost is lower per task and requires fewer tasks than OVERNIGHT, so for a fixed annotation budget, GRANNO can produce more useful training signal.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The success of the whole pipeline rests on the initial unsupervised scorer's ability to rank the gold canonical utterance in the top few candidates for enough unlabeled utterances; in domains where user phrasing is highly idiomatic or contains rare entities, a retrieval-augmented initial scorer could improve the bootstrap and is a natural extension not explored in the paper.
The detection-based annotation scheme may transfer beyond semantic parsing to any setting where a grammar can enumerate candidate structured forms, such as intent classification, instruction following, or visual question generation, since the core requirement is only a candidate set and a crowd worker's ability to recognize a match.
The paper's quantified language-mismatch result (approximately 9 accuracy points lost even when logical forms are held fixed) implies that merely improving paraphrase models will not close the gap to real-data training; collecting from the true distribution appears necessary, which strengthens the case for approaches that assume unlabeled user utterances.
One could test whether the detected paraphrase annotations themselves are reusable: if the crowd-selected canonical utterances are stored, they could seed a larger paraphrase database or be used to train a generation model, potentially enabling a hybrid collection scheme in future work.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a real contribution, and worth a serious referee. The mismatch decomposition is genuinely useful; the GRANNO loop is simple and the gains are large. But the headline comparison conflates two changes at once — access to true-distribution unlabeled utterances and detection instead of paraphrasing — and the paper's only attempt to separate them is a self-training baseline that appears with no numbers in Section 5.2. That is the softest spot, and it is addressable rather than fatal.

What is actually new: previous OVERNIGHT extensions still paraphrase canonical utterances. GRANNO instead shows crowd workers can detect a correct paraphrase among top-k candidates, and re-trains the detection scorer iteratively. The quantitative split of logical-form mismatch vs language mismatch (Section 3, Table 4) is a clean, useful diagnostic. The paper is also honest about limitations: the initial WMD scorer ranks gold within top-5 for only about half of the unlabeled utterances, and the authors flag that this is not guaranteed. That is a real deployment risk, stated in the text.

Where it is soft: (1) The main 70.6 vs 51.3 comparison changes both unlabeled data and detection. A self-training baseline using the same unlabeled utterances would isolate the detection mechanism. The paper says they implemented one and couldn't obtain good performance, but gives no numbers, no iterations, no confidence threshold. For a claim titled 'Don't paraphrase, detect!', that is exactly the comparison reviewers will ask for. (2) No error bars or variance across seeds; with two datasets, the 19-point average could be driven by SCHOLAR's 40.8 floor. (3) The unlabeled set is the training split of the same benchmark, so the setup is optimistic relative to a true query-log scenario; grammar coverage at D=6 is 90%, which helps. (4) D=6 for GRANNO vs D=5 for OVERNIGHT means GRANNO also sees more logical forms; though the ORACLE-LF baseline partly controls for that, it does not fully match.

The math is straightforward, the evaluation is held-out, code and data are promised. The citation pattern is fine; self-citations are to the authors' own parser work and are relevant.

Who is this for: anyone building semantic parsers for new domains with access to query logs, and anyone thinking about data collection versus semi-supervised learning. It deserves a serious referee; the missing self-training numbers should be requested in revision.

Referee Report

2 major / 4 minor

Summary. This paper addresses the data collection bottleneck for semantic parsing. It first dissects the OVERNIGHT procedure into two mismatches relative to the true distribution: a logical-form mismatch (the grammar-generated logical forms differ in operator frequency and template coverage from real user queries) and a language mismatch (crowd paraphrases of canonical utterances differ from natural user language). The language-mismatch experiment (§3.2) shows about a 9-point drop even when the logical-form distribution is held fixed. To mitigate both mismatches, the authors propose GRANNO, an iterative annotation procedure that assumes access to unlabeled utterances from the target distribution. At each iteration, a scorer (initialized with word-mover distance, then an ESIM classifier) selects top-K canonical utterances from a grammar; crowd workers detect the correct one; the resulting pairs are used to train the scorer and, ultimately, a semantic parser. On GEOQUERY and SCHOLAR, GRANNO achieves 72.0 and 69.2 test accuracy, compared with 61.9 and 40.8 for the OVERNIGHT baseline (70.6 vs 51.3 average). The paper also reports a perfect-worker simulation, an analysis of crowd false positives, and a brief discussion of limitations.

Significance. Assuming the main comparison is internally valid, the contribution is practically significant: GRANNO obtains higher accuracy at lower annotation cost ($155 vs $515) on two benchmarks, and the code and collected data are publicly available. The mismatch analysis is also a useful contribution, quantifying the two failure modes of OVERNIGHT with a controlled experiment. However, the headline claim that detection is the cause of the gain is not yet established, because GRANNO and OVERNIGHT differ in two simultaneous ways—access to unlabeled target-distribution utterances and the detection interface—and the only control the paper mentions (a self-training baseline) is not reported with numbers. The method also depends on the initial unsupervised scorer succeeding, which the authors acknowledge is not guaranteed. These issues are addressable but must be fixed before the paper's central claim is fully supported.

major comments (2)

[§5.2 (Results), paragraph beginning 'We note in passing...'] The comparison between GRANNO and OVERNIGHT changes two variables at once: GRANNO has access to unlabeled utterances sampled from the target distribution (Xul = {x | (x,·) ∈ Dnat}), while OVERNIGHT starts from grammar-generated canonical utterances only. The paper's title and central claim attribute the 19.3-point average gain to detection, but the self-training baseline mentioned in §5.2 is the only experiment that could separate the contribution of unlabeled target-distribution data from the contribution of the detection interface, and no numbers or implementation details are given for it. Please report the self-training baseline's performance on the same two test sets, along with the confidence threshold, number of iterations, and the size of the resulting training set; if it recovers a large part of the 19.3-point gap, the detection thesis needs to be re-framed. In addition, the OVERNIGHT baseline uses grammar depth D=5 and GRANNO D=6, so the comparison also mixes in a grammar-depth effect; this should be acknowledged and, if possible, controlled.
[§4 (Initial Scoring Function) and 'Limitations' paragraph] The success of GRANNO depends on the initial scoring function s0(x,c) = -WMD: if the gold canonical utterance is not among the top-K candidates shown to the worker, the worker can never select it. The paper reports that s0 ranks gold paraphrases within the top-5 for roughly half of the unlabeled utterances and explicitly states that 'this is not guaranteed.' Because there is no diagnostic or threshold for deciding when s0 is good enough, the method's applicability beyond GEOQUERY and SCHOLAR is uncertain. Please provide an analysis of how coverage, crowd accuracy, and final parser accuracy vary with the quality of s0 (e.g., by comparing WMD to a stronger or weaker initialization, or by reporting the distribution of gold ranks in Xul), so a practitioner can judge when GRANNO will bootstrap.

minor comments (4)

[Table 5 and §5.2] The label 'GRANNO-ORCALE' appears to be a typo for 'GRANNO-ORACLE'; it should be corrected consistently in the table and text.
[Algorithm 1] The stopping criterion is merely stated as 'check for convergence'; please define it precisely, for example as an iteration with no newly labeled utterances, so the procedure is reproducible.
[§5.1 (Crowd Sourcing)] The number of paraphrases reported for OVERNIGHT (7,140) is not obviously consistent with the D=5 generation counts in Table 2 (2,283 for SCHOLAR plus 4,480 for GEOQUERY equals 6,763); please clarify whether the difference comes from additional examples, duplicate attempts, or a different generation setting.
[§3.2 (Language mismatch)] The text says the authors 'paraphrased their logical forms' to create Dlang, but the procedure paraphrases the corresponding canonical utterance templates Cnat; rewording to 'paraphrased the canonical utterances corresponding to the logical forms' would avoid confusion.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: GRANNO's central result is an externally measured outcome; the self-training omission is a comparison confound, not a circular derivation.

full rationale

GRANNO's derivation chain is self-contained and externally evaluated. The unlabeled set Xul is the original training utterances of GEOQUERY and SCHOLAR with labels stripped; the labels are produced by crowd detection against grammar-generated candidates, not read off from the benchmark, and the parser is scored on held-out test sets. The initial scorer s0(x,c) = -WMD is an unsupervised input assumption, not a fitted output, and the iteratively retrained st is trained on crowd annotations and used only to restrict later candidate sets, so no fitted parameter is renamed as a prediction. The self-citations to Wang et al. (2015) for the OVERNIGHT grammar and baseline, and to Herzig and Berant (2017, 2018) in related work, are not load-bearing circular dependencies: the OVERNIGHT baseline is re-implemented in this paper and compared against held-out test sets, and no uniqueness theorem is imported from the authors' prior work. The Section 5.2 sentence 'We note in passing that we also implemented a baseline that uses unlabeled examples in conjunction with OVERNIGHT through self-training... we were unable to obtain good performance with this method' is a genuine omission: without the self-training numbers, the 70.6-versus-51.3 improvement cannot be uniquely attributed to detection versus unlabeled-data access. That is a comparison confound, not a by-construction equivalence. The Limitations passage noting that s0 ranks gold paraphrases within the top-5 for roughly half of Xul and that 'this is not guaranteed' is an honest scope condition, not a circular step. Overall, no prediction reduces to its own input by definition or by fitted-value construction.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

All listed assumptions are stated or implicit design choices in Sections 4 and 5. The grammar, the unlabeled utterance assumption, the initial scorer, and the worker detection capability are inputs to the method, not outputs of the experiments. No invented theoretical entities are introduced.

free parameters (4)

Grammar depth D for GRANNO = 6
Chosen by hand: D=6 yields roughly 350K canonical utterances and about 90% template coverage of Xul in both datasets (Table 2); larger D increases candidate search cost. It is a method hyperparameter, not fitted to test accuracy.
Grammar depth D for OVERNIGHT baseline = 5
Set to a reasonable value because every generated canonical utterance must be paraphrased; affects coverage (63.7% for SCHOLAR, 76.5% for GEOQUERY).
Top-K candidate count for crowd workers = 5
Number of canonical utterances shown to each worker; chosen by hand. If the true candidate is outside the top-K, the worker cannot label correctly.
Top-M negative candidates for training st = 100
Number of negative canonical utterances sampled per annotated x to train the paraphrase detection model; chosen by hand.

assumptions (6)

domain assumption Access to unlabeled utterances from the true target distribution.
GRANNO requires Xul drawn from p_nat(x); in experiments Xul is all utterances in Dnat with labels removed (Section 4, Implementation details). This assumption is stated in the abstract and Section 4.
domain assumption The OVERNIGHT grammar covers the logical forms appearing in the target examples.
GRANNO can only label an utterance if its gold logical form can be expressed as a grammar-generated canonical utterance; only 99.3% of GEOQUERY and 96.7% of SCHOLAR examples were convertible (Section 5.1).
domain assumption Initial similarity function s0 = -WMD gives a workable bootstrap.
Section 4 states that s0 ranks the gold paraphrase in the top-5 for roughly half of Xul in both datasets, and the Limitations note this is not guaranteed; the whole iterative annotation process starts from s0.
domain assumption Crowd workers can reliably detect correct paraphrases among K candidates.
The method replaces generation with detection; Table 6 and Figure 4 show above 85% detection accuracy, but this is an empirical assumption about worker behavior outside the paper.
domain assumption Paraphrase detection model st trained on crowd annotations generalizes to remaining utterances.
GRANNO iteratively retrains st on Spos and Sneg; if the detection model does not rank gold candidates in the top-K for new utterances, coverage saturates at convergence.
domain assumption Original OVERNIGHT grammar and lambda-DCS conversion preserve dataset semantics.
The authors manually re-annotated GEOQUERY and SCHOLAR logical forms into lambda-DCS (Section 5.1); accuracy is measured against denotations of these converted forms, so the conversion must be faithful.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Don't paraphrase, detect! Rapid and Effective Data Collection for Semantic Parsing." pith.science (2026). https://pith.science/paper/ES5COF4D

@misc{pith2026190809940,
  author       = {Pith},
  title        = {Pith review of: Don't paraphrase, detect! Rapid and Effective Data Collection for Semantic Parsing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ES5COF4D}},
  note         = {Machine review of arXiv:1908.09940}
}

read the original abstract

A major hurdle on the road to conversational interfaces is the difficulty in collecting data that maps language utterances to logical forms. One prominent approach for data collection has been to automatically generate pseudo-language paired with logical forms, and paraphrase the pseudo-language to natural language through crowdsourcing (Wang et al., 2015). However, this data collection procedure often leads to low performance on real data, due to a mismatch between the true distribution of examples and the distribution induced by the data collection procedure. In this paper, we thoroughly analyze two sources of mismatch in this process: the mismatch in logical form distribution and the mismatch in language distribution between the true and induced distributions. We quantify the effects of these mismatches, and propose a new data collection approach that mitigates them. Assuming access to unlabeled utterances from the true distribution, we combine crowdsourcing with a paraphrase model to detect correct logical forms for the unlabeled utterances. On two datasets, our method leads to 70.6 accuracy on average on the true distribution, compared to 51.3 in paraphrasing-based data collection.

Figures

Figures reproduced from arXiv: 1908.09940 by the authors.

**Figure 1.** An overview of GRANNO, a method for annotating unlabeled utterances with their logical forms. haps the most pressing problem for scalable conversational interfaces. In recent years, many attempts aim to reduce the burden of data collection for semantic parsing, including training from denotations (Kwiatkowski et al., 2013; Artzi and Zettlemoyer, 2013), semisupervised learning (Kocisky et al. ´ , 2016; Yin et al.,… view at source ↗

**Figure 2.** OVERNIGHT: Canonical utterances are generated by a grammar and paraphrased by crowd workers. score the remaining unlabeled utterances. On two semantic parsing datasets, we show our procedure leads to annotation of 89% of the original training data. The accuracy of the resulting parser is 70.6 on average, well beyond the accuracy obtained through the original OVERNIGHT procedure at 51.3. This substantially closes … view at source ↗

**Figure 3.** Examples for canonical utterances (c) generated by the grammar, their paraphrase by crowd workers (xon), and their natural utterance in Dnat (xnat). While the paraphrases are correct, they are biased towards the language style in c. 3.2 Language mismatch The second mismatch in OVERNIGHT, demonstrated in [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Analysis of GRANNO across iterations. ment set, showing that detection is easier to model compared to generation. We note in passing that we also implemented a baseline that uses unlabeled examples in conjunction with OVERNIGHT through self-training (Konstas et al., 2…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 25 canonical work pages

[1]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Artzi and L

Y. Artzi and L. Zettlemoyer. 2013. Weakly supervised learning of semantic parsers for mapping instructions to actions. Transactions of the Association for Computational Linguistics (TACL), 1:49--62

work page 2013
[4]

Bahdanau, K

D. Bahdanau, K. Cho, and Y. Bengio. 2015. Neural machine translation by jointly learning to align and translate. In International Conference on Learning Representations (ICLR)

2015
[5]

Jonathan Berant, Daniel Deutch, Amir Globerson, Tova Milo, and Tomer Wolfson. 2019. Explaining queries over web tables to non-experts. International Conference on Data Engineering (ICDE)

work page 2019
[6]

Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2017. https://doi.org/10.18653/v1/P17-1152 Enhanced LSTM for natural language inference . In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1657--1668, Vancouver, Canada. Association for Computational Linguistics

doi:10.18653/v1/p17-1152 2017
[7]

Jianpeng Cheng, Siva Reddy, and Mirella Lapata. 2018. Building a neural semantic parser from a domain ontology. arXiv preprint arXiv:1812.10037

work page Pith review arXiv 2018
[8]

Marco Damonte, Rahul Goel, and Tagyoung Chung. 2019. Practical semantic parsing for spoken language understanding. In Human Language Technology and North American Association for Computational Linguistics (HLT/NAACL)

work page 2019

Show all 39 references

[9]

Dong and M

L. Dong and M. Lapata. 2016. Language to logical form with neural attention. In Association for Computational Linguistics (ACL)

2016
[10]

Kummerfeld, Li Zhang, Karthik Ramanathan, Sesh Sadasivam, Rui Zhang, and Dragomir Radev

Catherine Finegan-Dollak, Jonathan K. Kummerfeld, Li Zhang, Karthik Ramanathan, Sesh Sadasivam, Rui Zhang, and Dragomir Radev. 2018. https://www.aclweb.org/anthology/P18-1033 Improving text-to-sql evaluation methodology . In Proceedings of the 56th Annual Meeting of the Associ...

2018
[11]

Liu, Matthew Peters, Michael Schmitz, and Luke S

Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson F. Liu, Matthew Peters, Michael Schmitz, and Luke S. Zettlemoyer. 2017. http://arxiv.org/abs/arXiv:1803.07640 Allennlp: A deep semantic natural language processing platform

2017 arXiv
[12]

Sonal Gupta, Rushin Shah, Mrinal Mohit, Anuj Kumar, and Mike Lewis. 2018. https://www.aclweb.org/anthology/D18-1300 Semantic parsing for task oriented dialog using hierarchical representations . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Pro...

2018
[13]

Herzig and J

J. Herzig and J. Berant. 2017. Neural semantic parsing over multiple knowledge-bases. In Association for Computational Linguistics (ACL)

2017
[14]

Herzig and J

J. Herzig and J. Berant. 2018. Decoupling structure and lexicon for zero-shot semantic parsing. In Empirical Methods in Natural Language Processing (EMNLP)

2018
[15]

Drew A Hudson and Christopher D Manning. 2019. Gqa: a new dataset for compositional question answering over real-world images. arXiv preprint arXiv:1902.09506

2019 arXiv
[16]

S. Iyer, I. Konstas, A. Cheung, J. Krishnamurthy, and L. Zettlemoyer. 2017. Learning a neural semantic parser from user feedback. In Association for Computational Linguistics (ACL)

2017
[17]

Jia and P

R. Jia and P. Liang. 2016. Data recombination for neural semantic parsing. In Association for Computational Linguistics (ACL)

2016
[18]

Johnson, B

J. Johnson, B. Hariharan, L. van der Maaten, L. Fei-Fei, C. L. Zitnick, and R. Girshick. 2017. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In Computer Vision and Pattern Recognition (CVPR)

2017
[19]

Kocisk \'y , G

T. Kocisk \'y , G. Melis, E. Grefenstette, C. Dyer, W. Ling, P. Blunsom, and K. M. Hermann. 2016. Semantic parsing with semi-supervised sequential autoencoders. In Empirical Methods in Natural Language Processing (EMNLP), pages 1078--1087

2016
[20]

Konstas, S

I. Konstas, S. Iyer, M. Yatskar, Y. Choi, and L. Zettlemoyer. 2017. Neural AMR: sequence-to-sequence models for parsing and generation. CoRR, 0

2017
[21]

Matt Kusner, Yu Sun, Nicholas Kolkin, and Kilian Weinberger. 2015. From word embeddings to document distances. In International Conference on Machine Learning, pages 957--966

2015
[22]

Kwiatkowski, E

T. Kwiatkowski, E. Choi, Y. Artzi, and L. Zettlemoyer. 2013. Scaling semantic parsers with on-the-fly ontology matching. In Empirical Methods in Natural Language Processing (EMNLP)

2013
[23]

Igor Labutov, Bishan Yang, and Tom Mitchell. 2019. Learning to learn semantic parsers from natural language supervision. arXiv preprint arXiv:1902.08373

2019 arXiv
[24]

Carolin Lawrence and Stefan Riezler. 2018. https://www.aclweb.org/anthology/P18-1169 Improving a neural semantic parser by counterfactual learning from human bandit feedback . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1:...

2018
[25]

P. Liang. 2013. Lambda dependency-based compositional semantics. arXiv preprint arXiv:1309.4408

2013 arXiv
[26]

Liang, M

P. Liang, M. I. Jordan, and D. Klein. 2011. Learning dependency-based compositional semantics. In Association for Computational Linguistics (ACL), pages 590--599

2011
[27]

Nicholas Locascio, Karthik Narasimhan, Eduardo De Leon, Nate Kushman, and Regina Barzilay. 2016. https://doi.org/10.18653/v1/D16-1197 Neural generation of regular expressions from natural language with minimal domain knowledge . In Proceedings of the 2016 Conference on Empiric...

2016 doi
[28]

Pennington, R

J. Pennington, R. Socher, and C. D. Manning. 2014. Glo V e: Global vectors for word representation. In Empirical Methods in Natural Language Processing (EMNLP), pages 1532--1543

2014
[29]

M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, and L. Zettlemoyer. 2018. Deep contextualized word representations. In North American Association for Computational Linguistics (NAACL)

2018
[30]

Abhilasha Ravichander, Thomas Manzini, Matthias Grabmair, Graham Neubig, Jonathan Francis, and Eric Nyberg. 2017. https://doi.org/10.18653/v1/W17-5545 How would you say it? eliciting lexically diverse dialogue for supervised semantic parsing . In Proceedings of the 18th Annual...

2017 doi
[31]

u r, Gokhan T \

Pararth Shah, Dilek Hakkani-T \"u r, Gokhan T \"u r, Abhinav Rastogi, Ankur Bapna, Neha Nayak, and Larry Heck. 2018. Building a conversational agent overnight with dialogue self-play. arXiv preprint arXiv:1801.04871

2018 arXiv
[32]

Su and X

Y. Su and X. Yan. 2017. Cross-domain semantic parsing via paraphrasing. In Empirical Methods in Natural Language Processing (EMNLP)

2017
[33]

Sutskever, O

I. Sutskever, O. Vinyals, and Q. V. Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems (NeurIPS), pages 3104--3112

2014
[34]

Y. Wang, J. Berant, and P. Liang. 2015. Building a semantic parser overnight. In Association for Computational Linguistics (ACL)

2015
[35]

Wei Yang, Yuqing Xie, Aileen Lin, Xingyu Li, Luchen Tan, Kun Xiong, Ming Li, and Jimmy Lin. 2019. End-to-end open-domain question answering with BERT serini. arXiv preprint arXiv:1902.01718

2019 arXiv
[36]

Pengcheng Yin, Chunting Zhou, Junxian He, and Graham Neubig. 2018. https://www.aclweb.org/anthology/P18-1070 S truct VAE : Tree-structured latent variable models for semi-supervised semantic parsing . In Proceedings of the 56th Annual Meeting of the Association for Computation...

2018
[37]

Zelle and R

M. Zelle and R. J. Mooney. 1996. Learning to parse database queries using inductive logic programming. In Association for the Advancement of Artificial Intelligence (AAAI), pages 1050--1055

1996
[38]

L. S. Zettlemoyer and M. Collins. 2005. Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars. In Uncertainty in Artificial Intelligence (UAI), pages 658--666

2005
[39]

Victor Zhong, Caiming Xiong, and Richard Socher. 2017. Seq2sql: Generating structured queries from natural language using reinforcement learning. CoRR, abs/1709.00103

2017 arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Artzi and L

Y. Artzi and L. Zettlemoyer. 2013. Weakly supervised learning of semantic parsers for mapping instructions to actions. Transactions of the Association for Computational Linguistics (TACL), 1:49--62

work page 2013

[4] [4]

Bahdanau, K

D. Bahdanau, K. Cho, and Y. Bengio. 2015. Neural machine translation by jointly learning to align and translate. In International Conference on Learning Representations (ICLR)

2015

[5] [5]

Jonathan Berant, Daniel Deutch, Amir Globerson, Tova Milo, and Tomer Wolfson. 2019. Explaining queries over web tables to non-experts. International Conference on Data Engineering (ICDE)

work page 2019

[6] [6]

Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2017. https://doi.org/10.18653/v1/P17-1152 Enhanced LSTM for natural language inference . In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1657--1668, Vancouver, Canada. Association for Computational Linguistics

doi:10.18653/v1/p17-1152 2017

[7] [7]

Jianpeng Cheng, Siva Reddy, and Mirella Lapata. 2018. Building a neural semantic parser from a domain ontology. arXiv preprint arXiv:1812.10037

work page Pith review arXiv 2018

[8] [8]

Marco Damonte, Rahul Goel, and Tagyoung Chung. 2019. Practical semantic parsing for spoken language understanding. In Human Language Technology and North American Association for Computational Linguistics (HLT/NAACL)

work page 2019