Pith. sign in

REVIEW 5 major objections 6 minor 1 cited by

TOPJoin: A Context-Aware Multi-Criteria Approach for Joinable Column Search

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Combining overlap, embeddings, and table context in a TOPSIS ranking outperforms single-criterion column join search.

desk verdict A genuinely new framing for joinable column search with a solid system, but the benchmark never tests the low-overlap case the paper is motivated by. read the letter →

arxiv 2507.11505 v1 pith:WL3XTVM5 submitted 2025-07-15 cs.DB

Harsha Kokel , Aamod Khatiwada , Tejaswini Pedapati , Haritha Ananthakrishnan , Oktie Hassanzadeh , Horst Samulowitz , Kavitha Srinivas This is my paper · ORCID
classification cs.DB
keywords joinablecolumnsearchdatalakediscoveryTOPSISmulti-criteriadecisionmakingtablecontextsemanticjoinmetadataembeddingsminhash
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that finding joinable columns in enterprise data lakes requires more than value overlap or embedding similarity, because shared values can come from unrelated contexts. It defines context-aware joinability as a combination of a semantic relationship between tables and semantically joinable columns, and proposes TOPJoin, which retrieves candidates with three independent strategies and ranks them with TOPSIS across multiple preference criteria. The paper reports that TOPJoin outperforms LSH Ensemble and DeepJoin on both an enterprise operational data store and an annotated open-data benchmark, with the largest gains in recall at top-10 results. A memory-efficient minhash variant keeps most of the accuracy while cutting the inverted index from 2.6 GB to 508 KB.

What carries the argument

The load-bearing object is the TOPSIS ranking procedure, a standard multi-criteria decision-making method that scores each candidate by its distance to an ideal solution. TOPJoin feeds it candidates from three retrieval strategies: an inverted index or minhash index for syntactic overlap, a sentence-transformer embedding of column metadata, and a sentence-transformer embedding of column values. The criteria combined are unique-value ratio, estimated intersection size, join and reverse-join cardinality, value-semantics similarity, disjoint-value semantics similarity, and metadata-semantics similarity, with intersection size weighted most heavily.

What would settle it

Build a benchmark of column pairs with low but non-zero overlap that share domain vocabulary (e.g., county names from different states) alongside genuinely joinable high-overlap pairs, and check whether TOPJoin ranks the low-overlap impostors below the true joins; if it does not, the context-aware criteria are not doing the claimed rejection work.

Watch

Extended reading notes

Core claim

The central claim is that no single similarity signal is enough for joinable column search, and that combining syntactic overlap, metadata semantics, and value semantics with a multi-criteria ranking procedure gives a substantial improvement. On the CIO enterprise data lake, TOPJoin achieves MRR 0.39 and recall 0.68 at K=10, against 0.29 and 0.41 for LSH Ensemble and 0.20 and 0.29 for DeepJoin; on the OpenData benchmark it achieves MRR 0.51 and recall 0.65. The authors attribute the gains to criteria that single-criterion baselines ignore: the context of the table, the cardinality of the resulting join, and the semantics of values not shared between the two columns.

Load-bearing premise

The open-data evaluation is built only from column pairs that already have high value containment, so the experiments never directly test the paper's motivating scenario of rejecting a low-overlap false-positive join such as counties from a different state.

Editorial extensions

If this is right

  • If the central claim holds, join search systems can move from a single overlap or embedding score to a weighted multi-criteria ranking without losing efficiency, since all representations are precomputed.
  • The minhash variant shows that the syntactic index can be shrunk by several orders of magnitude (2.6 GB to 508 KB in their example) with modest accuracy loss, which matters for enterprise-scale data lakes.
  • The results imply that metadata (table descriptions, column names, tags) carries signal that value-only methods miss, so data lakes with richer curation benefit more.
  • Because value embeddings capture abbreviation and fuzzy matches (NY vs New York), TOPJoin can return joinable pairs that pure equality joins would miss.
  • The same pipeline is agnostic to the embedding model, so replacing the sentence transformer with a tabular-finetuned model could further improve the semantic criteria.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A test constructed from low-overlap pairs, like the Missouri county example, would be the cleanest check of whether the context criterion is actually doing the rejection work; the current benchmark draws only from high-containment pairs, so the reported gains may come mostly from the extra ranking criteria rather than from context rejection.
  • The authors use a 10% positive-agreement threshold over 6-15 annotations per pair; because only 42 of 471 pairs are labeled joinable, label noise alone could move recall by a few points, so re-running with stricter agreement thresholds would show how brittle the ranking is.
  • The design separates retrieval and ranking, so the same TOPSIS layer could be applied on top of any candidate generator (e.g., a fully embedding-based retrieval) to test whether the ranking criteria generalize beyond the specific indexes used here.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes TOPJoin, a method for joinable-column search that augments syntactic and semantic similarity with table context. It defines a context-aware joinable column as one whose tables are semantically related and whose columns can be combined to produce meaningful results, and it operationalizes this definition through a TOPSIS multi-criteria ranking over unique values, intersection size, join size, reverse join size, value semantics, disjoint value semantics, and metadata semantics. Candidates are retrieved from three indexes: a syntactic inverted index (or a minhash-based variant), a metadata-embedding index, and a value-embedding index. The experiments compare TOPJoin and an approximate variant TOPJoin-Minhash against LSH Ensemble and DeepJoin on an enterprise ODS (CIO) and a human-annotated OpenData benchmark, reporting higher MRR, MAP, and Recall@10 for TOPJoin.

Significance. TOPJoin addresses a practical problem: enterprise data lakes contain many columns that share values but are not meaningfully joinable, and the reported CIO recall gain (0.68 vs 0.41 and 0.29) is substantial. The paper's strengths include the use of a real enterprise ODS, the release of source code and data through a GitHub artifact, and the TOPJoin-Minhash variant that reduces the syntactic index from 2.6 GB to 508 KB while retaining most of the benefit. However, the evaluation does not currently establish the central context-aware-rejection claim: the OpenData benchmark is restricted to high-containment column pairs, which excludes the low-overlap false positives described in Example 1, and the intersection-size weight is chosen using the evaluation benchmarks. The paper is thus a useful engineering contribution whose claimed advantage over prior work needs additional experimental support.

major comments (5)
  1. [Section 4.1] The OpenData ground truth is built from 471 column pairs preselected for high containment, with only 42 labeled joinable. The motivating failure case in Example 1 is a low-containment pair (Texas county names vs. Missouri county names), so this design filters out exactly the false positives that TOPJoin is intended to reject. Table 2's OpenData results therefore measure re-ranking among high-containment candidates and do not test the paper's central claim of context-aware rejection of low-overlap false positives.
  2. [Section 4] The intersection-size weight is set to 0.5 'based on experimental results' while all other criteria receive 0.2, and the benchmarks used to choose that weight are the same ones reported in Table 2. This is a tuning-into-evaluation step, and no sensitivity analysis or held-out weight validation is reported. The reported superiority of TOPJoin may therefore be an artifact of this weight choice.
  3. [Section 4.2] The DeepJoin comparison is described as a replication trained on OpenData. If the same OpenData pairs are used for both training the replicated model and computing the OpenData results in Table 2, the comparison is circular; in any case the train/test split is not specified. The authors must clarify the split or compare against published DeepJoin results on a disjoint benchmark.
  4. [Table 2] All metrics are presented as single point estimates without confidence intervals or significance tests, and the OpenData positive set has only 42 pairs. Given the modest absolute differences in MRR (0.51 vs 0.44) and MAP (0.49 vs 0.37), the current tables do not demonstrate that the reported gains are statistically reliable.
  5. [Section 4.1] With 6-15 annotations per pair and a 10% positive-agreement threshold, one positive annotation is sufficient to label a pair joinable when only six annotations are available. The paper does not report inter-annotator agreement or the distribution of annotation counts, so the reliability of the 42 positive labels is unclear.
minor comments (6)
  1. [Section 4] The weighting scheme is not fully specified: a weight of 0.2 for every criterion and 0.5 for intersection size does not state whether the weights are normalized to sum to one or whether they are relative weights.
  2. [Section 3.2] The Value Semantics criterion does not state how many most frequent values are used to build the column sentence or which sentence-transformer model is used; these choices affect reproducibility.
  3. [Table 2] It is surprising that TOPJoin-Minhash achieves a higher OpenData MRR (0.60) than TOPJoin (0.51) while having a lower MAP (0.48 vs 0.49); the paper should explain this inversion.
  4. [Section 3.1] Because only the top-100 candidates from each index are retained, Recall@10 is bounded by the union recall of the candidate sets; reporting this upper bound would help interpret the recall numbers.
  5. [Section 4.1] The phrase 'only only 42' contains a typo that should be corrected.
  6. [Example 1] The claim that typical joinability criteria identify d.County as joinable with a.County is not demonstrated; adding a short explanation of the shared county names would make the example clearer.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: TOPJoin's evaluation is grounded in external SQL-log and human-annotated ground truth, so the reported gains do not reduce to the method's own inputs.

full rationale

The paper's derivation chain is self-contained against external evidence. TOPJoin's candidate retrieval (syntactic, metadata-semantic, value-semantic) and TOPSIS ranking are evaluated on ground truth that is not generated by TOPJoin itself: the CIO benchmark labels come from SQL query logs, and the OpenData benchmark labels come from 15 human annotators who saw table snippets and metadata. The 'context-aware joinable columns' definition (Definition 2) is not used to derive the labels by construction; instead, the definition motivates the criterion set and the annotation protocol, but the labels themselves are external. No load-bearing self-citation appears: references [12,19,20] motivate the application context, and [18] supplies an external cardinality estimator; none of them is invoked as a uniqueness theorem or as a substitute for the evaluation. The disclosed choice of a 0.5 weight for intersection size 'based on experimental results' is a tuning caveat that could optimistically bias the reported advantage, but it is not a fitted parameter renamed as a prediction, and the ground-truth labels remain external, so it does not rise to circularity. Likewise, the OpenData benchmark's restriction to high-containment pairs is a coverage limitation for the motivating low-overlap scenario, not a circular reduction of the method's outputs to its inputs. The central empirical claim is therefore independently testable rather than forced by definition or by self-reference.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The empirical claim rests on how the ground truth is defined (human annotations with a 10% threshold, SQL query logs), on embedding models capturing semantic relatedness, and on the chosen TOPSIS weights. No new physical entities or ad hoc constructs are introduced; the contribution is a weighted integration of existing signals.

free parameters (5)
  • intersection_size_weight = 0.5
    Set to 0.5, all other criteria to 0.2, 'based on experimental results' (Section 4). This is a benchmark-fitted ranking weight; changing it can change which columns are top-K.
  • positive_annotation_threshold = 10%
    Columns treated as joinable if at least 10% of human annotations are positive (Section 4.1). This threshold defines the ground truth and therefore the measured performance.
  • row_sample_size_for_inverted_index = 10K rows
    Inverted index built from 10K sampled rows per table (Section 4); affects syntactic candidate recall.
  • column_value_sample_size = 1M values
    Randomly selected 1 million values for columns with more values (Section 4); affects embeddings and minhash.
  • minhash_permutations = 100
    100 permutations used for minhash (Section 4); affects intersection size estimate.
assumptions (5)
  • domain assumption Human annotations for OpenData are a valid ground truth for context-aware joinability.
    Section 4.1 constructs labels from annotator agreement, with a 10% threshold, and these labels are the target the metrics are computed against.
  • domain assumption SQL query logs in the CIO ODS capture the set of meaningful joins.
    Section 4.1 derives ground truth from query logs; if the logs contain legacy or exploratory joins, the labels may not reflect ideal context-aware joins.
  • domain assumption Embeddings from sentence transformers capture semantic relatedness of column metadata and values.
    Section 3.1 uses Sentence-BERT embeddings without fine-tuning on tabular data; the authors note this as a limitation in the conclusion.
  • standard math Minhash hamming distance approximates Jaccard or intersection of column values.
    Section 3.2 uses 100 minhash permutations; this is a standard approximate set-similarity technique.
  • domain assumption TOPSIS with linear-weighted criteria is a valid aggregation for joinability preferences.
    Section 3.2 adopts TOPSIS; the quality of the ranking depends on criteria being roughly commensurable and weights being meaningful.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TOPJoin: A Context-Aware Multi-Criteria Approach for Joinable Column Search." pith.science (2026). https://pith.science/paper/WL3XTVM5

@misc{pith2026250711505,
  author       = {Pith},
  title        = {Pith review of: TOPJoin: A Context-Aware Multi-Criteria Approach for Joinable Column Search},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WL3XTVM5}},
  note         = {Machine review of arXiv:2507.11505}
}
read the original abstract

One of the major challenges in enterprise data analysis is the task of finding joinable tables that are conceptually related and provide meaningful insights. Traditionally, joinable tables have been discovered through a search for similar columns, where two columns are considered similar syntactically if there is a set overlap or they are considered similar semantically if either the column embeddings or value embeddings are closer in the embedding space. However, for enterprise data lakes, column similarity is not sufficient to identify joinable columns and tables. The context of the query column is important. Hence, in this work, we first define context-aware column joinability. Then we propose a multi-criteria approach, called TOPJoin, for joinable column search. We evaluate TOPJoin against existing join search baselines over one academic and one real-world join search benchmark. Through experiments, we find that TOPJoin performs better on both benchmarks than the baselines.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MosaicJoin: Compact Semantic Sketches for Value-Level Join Discovery

    cs.DB 2026-07 conditional novelty 6.0 of 10

    MosaicJoin finds semantically joinable columns in data lakes fast by scoring fixed-size k-center sketches of value embeddings with a Chamfer-style similarity and sampling query values.

Reference graph

Works this paper leans on

24 extracted references · 11 canonical work pages · cited by 1 Pith paper

  1. [1]

    Mayank Bawa, Tyson Condie, and Prasanna Ganesan. 2005. LSH forest: self- tuning indexes for similarity search. In Proceedings of the 14th international conference on World Wide Web, WWW 2005, Chiba, Japan, May 10-14, 2005 , Allan Ellis and Tatsuya Hagino (Eds.). ACM, 651–660. https://doi.org/10.1145/1060745. 1060840

    work page doi:10.1145/1060745 2005

  2. [2]

    Tianji Cong, James Gale, Jason Frantz, H. V. Jagadish, and Çagatay Demiralp

    work page

  3. [3]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Associ- ation for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, ...

    2019

  4. [5]

    Yuyang Dong, Chuan Xiao, Takuma Nozawa, Masafumi Enomoto, and Masafumi Oyamada. 2023. DeepJoin: Joinable Table Discovery with Pre-trained Language Models. Proc. VLDB Endow. 16, 10 (2023), 2458–2470. https://doi.org/10.14778/ 3603581.3603587

    arXiv 2023

  5. [6]

    1981.Multiple Attribute Decision Making: Methods and Applications - A State-of-the-Art Survey

    Ching-Lai Hwang and Kwangsun Yoon. 1981.Multiple Attribute Decision Making: Methods and Applications - A State-of-the-Art Survey. Lecture Notes in Economics and Mathematical Systems, Vol. 186. Springer. https://doi.org/10.1007/978-3- 642-48318-9

    doi:10.1007/978-3- 1981

  6. [7]

    1995.Building the operational data store

    William H Inmon, Claudia Imhoff, and Greg Battas. 1995.Building the operational data store. John Wiley & Sons, Inc

    work page 1995

  7. [8]

    Aamod Khatiwada, Roee Shraga, Wolfgang Gatterbauer, and Renée J. Miller

  8. [9]

    Christos Koutras, George Siachamis, Andra Ionescu, Kyriakos Psarakis, Jerry Brons, Marios Fragkoulis, Christoph Lofi, Angela Bonifati, and Asterios Katsifodi- mos. 2021. Valentine: Evaluating Matching Techniques for Dataset Discovery. In 37th IEEE International Conference on Data Engineering, ICDE 2021, Chania, Greece, April 19-22, 2021. IEEE, 468–479. ht...

    arXiv 2021

Show all 24 references
  1. [10]

    Oliver Lehmberg, Dominique Ritze, Petar Ristoski, Robert Meusel, Heiko Paul- heim, and Christian Bizer. 2015. The Mannheim Search Join Engine. J. Web Semant. 35 (2015), 159–166. https://doi.org/10.1016/J.WEBSEM.2015.05.001

    2015 doi

  2. [11]

    Malkov and Dmitry A

    Yury A. Malkov and Dmitry A. Yashunin. 2020. Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs.IEEE Trans. Pattern Anal. Mach. Intell. 42, 4 (2020), 824–836. https://doi.org/10.1109/ TPAMI.2018.2889473

    2020

  3. [12]

    Faisal Mahbub Chowdhury, Alfio Massimiliano Gliozzo, Ariel Farkash, Michael R

    Nandana Mihindukulasooriya, Sarthak Dash, Sugato Bagchi, Md. Faisal Mahbub Chowdhury, Alfio Massimiliano Gliozzo, Ariel Farkash, Michael R. Glass, Igor Gokhman, Oktie Hassanzadeh, Nhan Pham, Gaetano Rossiello, Boris Rozenberg, Yehoshua Sagron, Dharmashankar Subramanian, Toshih...

    2023

  4. [13]

    Tomás Mikolov, Edouard Grave, Piotr Bojanowski, Christian Puhrsch, and Ar- mand Joulin. 2018. Advances in Pre-Training Distributed Word Representations. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation, LREC 2018, Miyazaki, Japan, Ma...

    2018

  5. [14]

    Miller, Ken Q

    Fatemeh Nargesian, Erkang Zhu, Renée J. Miller, Ken Q. Pu, and Patricia C. Arocena. 2019. Data Lake Management: Challenges and Opportunities. Proc. VLDB Endow. 12, 12 (2019), 1986–1989. https://doi.org/10.14778/3352063.3352116

    2019

  6. [15]

    Pedregosa, G

    F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cour- napeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine Learning in Python. Journal of Machine ...

    2011

  7. [16]

    Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. CoRR abs/1908.10084 (2019). arXiv:1908.10084 http://arxiv.org/abs/1908.10084

    2019 arXiv

  8. [17]

    Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2020. MPNet: Masked and Permuted Pre-training for Language Understanding. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural In- formation Processing Systems 2020, NeurIPS 2020, Decem...

    2020

  9. [18]

    Swami and K

    Arun N. Swami and K. Bernhard Schiefer. 1994. On the Estimation of Join Result Sizes. In Advances in Database Technology - EDBT’94. 4th International Conference on Extending Database Technology, Cambridge, United Kingdom, March 28-31, 1994, Proceedings (Lecture Notes in Comput...

    1994 doi

  10. [19]

    Weidele, Mauro Martino, Abel N

    Daniel Karl I. Weidele, Mauro Martino, Abel N. Valente, Gaetano Rossiello, Hendrik Strobelt, Loraine Franke, Kathryn Alvero, Shayenna Misko, Robin Auer, Sugato Bagchi, Nandana Mihindukulasooriya, Md. Faisal Mahbub Chowd- hury, Gregory Bramble, Horst Samulowitz, Alfio Gliozzo, ...

    2024

  11. [20]

    Weidele, Gaetano Rossiello, Gregory Bramble, Abel N

    Daniel Karl I. Weidele, Gaetano Rossiello, Gregory Bramble, Abel N. Valente, Sugato Bagchi, Md. Faisal Mahbub Chowdhury, Mauro Martino, Nandana Mi- hindukulasooriya, Haritha Ananthakrishnan, Hendrik Strobelt, Hima Patel, Al- fio Massimiliano Gliozzo, Owen Cornec, Ankush Gupta,...

    2023

  12. [21]

    Erkang Zhu, Dong Deng, Fatemeh Nargesian, and Renée J. Miller. 2019. JOSIE: Overlap Set Similarity Search for Finding Joinable Tables in Data Lakes. In Proceedings of the 2019 International Conference on Management of Data, SIGMOD Conference 2019, Amsterdam, The Netherlands, J...

    2019

  13. [22]

    Pu, and Renée J

    Erkang Zhu, Fatemeh Nargesian, Ken Q. Pu, and Renée J. Miller. 2016. LSH Ensemble: Internet-Scale Domain Search. Proc. VLDB Endow. 9, 12 (2016), 1185–

    2016

  14. [1196]

    https://doi.org/10.14778/2994509.2994534 5

  15. [2022]

    Integrating Data Lake Tables. Proc. VLDB Endow. 16, 4 (2022), 932–945. https://doi.org/10.14778/3574245.3574274

    2022

  16. [2023]

    In 13th Conference on Innovative Data Systems Research, CIDR 2023, Amsterdam, The Netherlands, January 8-11, 2023

    WarpGate: A Semantic Join Discovery System for Cloud Data Warehouses. In 13th Conference on Innovative Data Systems Research, CIDR 2023, Amsterdam, The Netherlands, January 8-11, 2023 . www.cidrdb.org. https://www.cidrdb.org/ cidr2023/papers/p75-cong.pdf

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.