REVIEW 4 major objections 5 minor 30 references
Z-REx: Human-Interpretable GNN Explanations for Real Estate Recommendations
T0 review · 4 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read The paper introduces Z-REx, a perturbation-based explainer that identifies the features and graph connections a GNN recommender relies on, and reports it outperforms three prior explainers on fidelity.
desk verdict Plausible domain-specific perturbation explainer for GNN link prediction, but the paper's headline fidelity claim is circular as written and needs a rebuilt evaluation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central machinery is a two-stage perturbation search. Stage one, feature perturbation, zeroes out each city feature one at a time and records the change in nDCG@K (Eq. 3); features whose removal lowers ranking quality become the influential feature set. Stage two, structural perturbation, collapses the heterogeneous graph into a user-city graph, adds edges between co-clicked cities, extracts a k-hop subgraph around the target user, removes each edge in turn, and records the change in cosine similarity between user and target-city embeddings (Eq. 4). The co-clicked-city insight—cities that share a common predecessor user are proxies for shared preference—is the domain-knowledge constraint that keeps the search space sub-linear and makes whole-graph, instance-level explanation feasible at the scale of tens of millions of edges.
What would settle it
Train Z-REx on a synthetic heterogeneous graph with planted influential subgraphs and features, then check whether the edges and features Z-REx selects match the planted ones; if the match rate is no better than chance, or if removing randomly chosen edges of the same size produces a comparable drop in nDCG, the interpretability claim would be refuted.
Extended reading notes
Core claim
Z-REx's central claim is that a faithful explanation for a GNN link prediction is the smallest set of node features and graph edges whose perturbation most changes the model's ranking behavior. For a given user and recommended city, it ranks features by zeroing each one out and computing the change in nDCG@K (Eq. 3), and ranks edges by removing each from a k-hop user-city subgraph and computing the change in cosine similarity between user and city embeddings (Eq. 4); the two ranked lists together are the explanation. The paper's ground-truth comparison treats observed user-city interactions as positive and sampled non-interactions as negative, and evaluates fidelity by how much the identified features and edges reduce nDCG and similarity when removed. Z-REx is designed for whole heterogeneous graphs, avoiding path-based or ego-graph limitations, and its use of co-clicked city edges is what makes the edge search scalable. The paper reports gains of 29% over PaGE-Link, 85% over GNNExplainer, and 70% over SubgraphX in the Fidelity metric.
Load-bearing premise
The load-bearing premise is that the Fidelity metric—the drop in nDCG@K and cosine similarity when Z-REx's selected features and edges are removed—measures how much the model actually relies on those features and edges, and that observed positive versus negative edges are a valid ground truth for explanation quality.
Editorial extensions
If this is right
- For a given user-to-city recommendation, Z-REx outputs an ordered list of important city features and an ordered list of important graph edges, which can be rendered as a small subgraph of co-clicked cities for a user-facing explanation.
- Because the explanation search is restricted to co-clicked edges, the method remains tractable on month-long interaction graphs with tens of millions of edges, and the paper reports near-constant per-user inference time between 3-day and 30-day graphs.
- If the Fidelity gains are real, Z-REx gives system administrators a tool for auditing which features and connections drive a GNN recommender's ranking decisions.
- The same two-step feature-then-structure perturbation recipe transfers to other link-prediction recommenders, provided a domain-specific edge type is chosen to replace co-clicked cities, as the paper's generalizability section argues.
Reading between the lines
- The paper leaves implicit that its feature-importance lists could be validated against a full feature-ablation study of the recommender, which would test whether the explanation identifies causal drivers or merely high-leverage perturbations.
- A testable extension is to apply the same two-stage perturbation logic to edge-classification or community-detection tasks by swapping in a task-appropriate ranking metric.
- Because the Fidelity metric rewards explanations that maximally degrade recommendations, Z-REx may favor rare high-impact edges over the everyday connections users would recognize; a user study on whether the explanations feel intuitive would settle this.
- Since the paper treats observed interactions as ground-truth positive evidence, distinguishing weak negatives from dislikes could change which edges are considered important, and measuring that sensitivity is a direct follow-up.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Z-REx, a post-hoc explanation method for user-city link prediction in a heterogeneous GNN recommender (ZiGNN). It identifies important city features by zeroing out each feature and measuring the change in nDCG@K (Eq. 3), and important graph edges by removing candidate edges, including domain-derived co-clicked city edges, and measuring the change in cosine similarity (Eq. 4). The method is evaluated on Zillow interaction data and compared against PaGE-Link, GNNExplainer, and SubgraphX, with the claim that Z-REx outperforms these baselines by 61% in the Fidelity metric.
Significance. If the method were rigorously validated, this would be a useful practical contribution: Z-REx is one of few explainers targeted at heterogeneous link prediction, and the use of domain-specific co-clicked city edges to prune the search space is a sensible engineering idea. The paper also uses a real industrial dataset and reports a scaling analysis (Table 8). However, the central quantitative claim is undermined by a circular evaluation and by inconsistent reported numbers; the current evidence does not establish that Z-REx produces better explanations than the baselines.
major comments (4)
- [§5.1, Eq. (3); §5.2, Eq. (4); §6.1] The Fidelity metric is identical to the objective used for selection. Z-REx ranks features by ΔnDCG and edges by |Δsim|, and Table 2 then reports as “Fidelity” the drop in nDCG and Δsim after removing the selected items. Any method that greedily chooses high-impact perturbations will dominate this metric, so the reported 29/85/70 differences do not by themselves demonstrate superior explanation accuracy. A minimal control would be to compare against a perturbation-based explainer using the same greedy objective, or to evaluate fidelity on a held-out perturbation set not used during selection.
- [§6.4, Table 2; §6.5; §8] The reported numbers are internally inconsistent. Table 2 labels the row “nDCG (% decr.)” with Z-REx at 92%, but §6.4 says Z-REx “preserves 92% of the original nDCG”; §6.5 says PaGE-Link “retains only 63% of nDCG” while the table lists 63% in the same row. If the row is percent decrease, then Z-REx preserves 8%/6%, not 92%/94%; if it is retention, the differences in §8 (29%, 85%, 70%) do not follow from the table. Additionally, the 30-day table values imply an 83-percentage-point gap for GNNExplainer, not the 85% stated in §8 and used in the abstract’s 61% average. The headline numbers are therefore not reproducible from the paper’s own table.
- [§4.2, §6.4] The “ground truth” explanation is not established. §4.2 says ground-truth explanations are generated by identifying the most relevant subgraphs and comparing node features, but the actual evaluation treats observed positive edges as ground truth. Observed interactions are labels for link prediction, not evidence about which substructures the model used. Without a separate annotation study or a causal validation, the claim that Z-REx explanations are “aligned with ground truth” (Sections 1 and 8) is unsupported.
- [§6.6, Table 3] The hyperparameter sensitivity study does not validate the explainer. Table 3 reports nDCG@1 of the ZiGNN recommender under different settings, not explanation fidelity or human interpretability, so it cannot justify the choices of hop distance k or edge removal strategy in Z-REx. Moreover, all entries in Table 2 appear to be single-run values with no error bars, confidence intervals, or significance tests, so the reported margins (e.g., 9% vs 92% for GNNExplainer) cannot be distinguished from noise.
minor comments (5)
- [Algorithm 1] The function defined on line 17 is named “StructurePerturb” but is called as “StructuralPerturb” on line 3; the names should be consistent.
- [Table 2] The parenthetical values such as “(-13%)”, “(-73%)”, and “(-70%)” are not explained in the caption or text; clarify whether they denote relative differences from Z-REx or another quantity.
- [Reference [30]] The URL for the Zillow website is missing the protocol: “https:www.zillow.com” should be “https://www.zillow.com”.
- [§8, abstract] The phrase “outperforms them by 61%” should be stated as a percentage-point difference in the reported metric, not as a percent improvement, and should be made consistent with the corrected table values.
- [Figure 5] The axis label “Difference in nDCG ×10^2” is likely a scaling typo; as printed, the values would be implausibly large, so the axis scaling should be checked and corrected.
Circularity Check
Z-REx's reported Fidelity superiority is forced by construction: the evaluation metric is the same perturbation response the method optimizes.
-
fitted input called prediction
[Section 5.1 Eq. (3); Section 6.1 'Metric']
"First, we identify an important subset of features by measuring the change in nDCG@K score when individual features are zeroed out. Then, selecting those that produce a negative impact on nDCG, meaning those features were important for predicting relevant recommendations. To align with the recommendation explainability task, we redefine the traditional Fidelity metric [3]—originally based on change in prediction confidence—as the change in nDCG@K score due to perturbations."
Eq. (3) defines the feature-importance score as ΔnDCG@K(F) = nDCG@K(perturbed) − nDCG@K(original), and Section 5.1 states that a significant drop in the metric indicates the importance of the perturbed feature. Z-REx therefore selects features that maximize this exact drop. Section 6.1 then evaluates explanation performance based on the drop in nDCG@K score caused by structural perturbations and by only using the important features. The reported Fidelity is the selection objective itself, so the Table 2 nDCG-decrease comparison (94% vs. 81%, 21%, 47%) is an optimizer of ΔnDCG against methods optimizing different objectives (e.g., mutual information, Shapley values). The superiority claim is forced by construction.
-
fitted input called prediction
[Section 5.2 Eq. (4); Section 6.1 'Metric'; Table 2]
"Δsim = sim(h_u, h_c′_t) − sim(h_u, h_ct). ... Edges with the highest absolute Δsim values are identified as critical contributors to the recommendation."
Structural perturbation assigns importance by |Δsim| after edge removal, and the evaluation metric is also the change in cosine similarity (Δsim) between user and city embeddings caused by structural perturbations. Since Z-REx ranks edges to maximize |Δsim| and Table 2 reports that same quantity (-0.10 vs. -0.07, -0.02, -0.04), its larger drop is guaranteed by its selection rule. The baselines were not optimizing Δsim, so this is not independent evidence of better faithfulness to model behavior.
full rationale
The central claim of the paper is that Z-REx produces superior explanations, quantified by the Fidelity metric. However, the paper redefines Fidelity as the change in nDCG@K and the change in cosine similarity after perturbation, and those are exactly the objectives Z-REx uses to select important features and edges in Section 5.1 (Eq. 3) and Section 5.2 (Eq. 4). Consequently, the reported improvements over GNNExplainer, SubgraphX, and PaGE-Link are not independent measurements of explanation quality; they are the result of evaluating a method on the same objective it optimizes. The two author self-citations ([6], [29]) are background and domain-justification citations and are not load-bearing. The paper is not circular in its recommendation-accuracy evaluation or in its hyperparameter study, but the headline explanation-quality comparison reduces by construction to the selection objective, giving a partial-circularity score of 7.
Assumptions & free parameters
free parameters (6)
- Important feature subset size (top-k features) =
10 or 14
- Hop distance k for structural perturbation =
8 (best of 4, 8, 16)
- Edge removal strategy =
HID (High In-Degree) selected
- Negative graph size =
5 (best of 1, 3, 5)
- Output feature dimension =
512 (best of 128, 256, 512)
- Learning rate =
1e-2 (best of 1e-3, 1e-2, 1e-1)
assumptions (5)
- domain assumption Cosine similarity between user and city embeddings represents the GNN's predicted affinity (Eq. 1).
- domain assumption The interaction graph diameter is at most two, so a two-layer RGCN captures all needed information.
- domain assumption Non-interaction implies lack of interest (weak negative edges).
- ad hoc to paper Co-clicked cities indicate user groups with similar preferences.
- domain assumption Zeroing out a feature and measuring nDCG change reveals feature importance for a trained GNN.
invented entities (1)
-
Co-clicked city edges
Cite this review
Pith. "Pith review of Z-REx: Human-Interpretable GNN Explanations for Real Estate Recommendations." pith.science (2026). https://pith.science/paper/JQRTSCMD
@misc{pith2026250318001,
author = {Pith},
title = {Pith review of: Z-REx: Human-Interpretable GNN Explanations for Real Estate Recommendations},
year = {2026},
howpublished = {\url{https://pith.science/paper/JQRTSCMD}},
note = {Machine review of arXiv:2503.18001}
}
read the original abstract
Transparency and interpretability are crucial for enhancing customer confidence and user engagement, especially when dealing with black-box Machine Learning (ML)-based recommendation systems. Modern recommendation systems leverage Graph Neural Network (GNN) due to their ability to produce high-quality recommendations in terms of both relevance and diversity. Therefore, the explainability of GNN is especially important for Link Prediction (LP) tasks since recommending relevant items can be viewed as predicting links between users and items. GNN explainability has been a well-studied field, but existing methods primarily focus on node or graph-level tasks, leaving a gap in LP explanation techniques. This work introduces Z-REx, a GNN explanation framework designed explicitly for heterogeneous link prediction tasks. Z-REx utilizes structural and attribute perturbation to identify critical substructures and important features while reducing the search space by leveraging domain-specific knowledge. In our experimentation, we show the efficacy of Z-REx in generating contextually relevant and human-interpretable explanations for ZiGNN, a GNN-based recommendation engine, using a real-world real-estate dataset from Zillow Group, Inc. We compare against State-of-The-Art (SOTA) GNN explainers to show Z-REx outperforms them by 61% in the Fidelity metric by producing superior human-interpretable explanations.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
Gnnexplainer: Generating explanations for graph neural networks,
Z. Ying, D. Bourgeois, J. You, M. Zitnik, and J. Leskovec, “Gnnexplainer: Generating explanations for graph neural networks, ” inNeural Information Processing Systems , 2019
work page 2019
-
[2]
Parameterized explainer for graph neural network,
D. Luo, W. Cheng, D. Xu, W. Yu, B. Zong, H. Chen, and X. Zhang, “Parameterized explainer for graph neural network, ” in Neural Information Processing Systems , 2020
work page 2020
-
[3]
On explainability of graph neural networks via subgraph explorations,
H. Yuan, H. Yu, J. Wang, K. Li, and S. Ji, “On explainability of graph neural networks via subgraph explorations, ” in International Conference on Machine Learning , PMLR, 2021
work page 2021
-
[4]
Page-link: Path-based graph neural network explanation for heterogeneous link prediction,
S. Zhang, J. Zhang, X. Song, S. Adeshina, D. Zheng, C. Falout- sos, and Y. Sun, “Page-link: Path-based graph neural network explanation for heterogeneous link prediction, ” inProceed- ings of the ACM Web Conference 2023 , 2023, pp. 3784–3793
work page 2023
-
[5]
Collaboration- aware graph convolutional network for recommender sys- tems,
Y. Wang, Y. Zhao, Y. Zhang, and T. Derr, “Collaboration- aware graph convolutional network for recommender sys- tems, ” inProceedings of the ACM web conference , 2023
work page 2023
-
[6]
Explaining provenance-based gnn detectors with graph structural features,
K. Mukherjee, J. Wiedemeier, T. Wang, M. Kim, F. Chen, M. Kantarcioglu, and K. Jee, “Explaining provenance-based gnn detectors with graph structural features, ”arXiv preprint arXiv:2306.00934, 2023
-
[7]
L. Chen, L. Wu, R. Hong, K. Zhang, and M. Wang, “Revisiting graph based collaborative filtering: A linear residual graph convolutional network approach, ” inProceedings of the AAAI conference on artificial intelligence , vol. 34, 2020, pp. 27–34
work page 2020
-
[8]
Neighbor interaction aware graph convolution networks for recommendation,
J. Sun, Y. Zhang, W. Guo, H. Guo, R. Tang, X. He, C. Ma, and M. Coates, “Neighbor interaction aware graph convolution networks for recommendation, ” inProceedings of the 43rd international ACM SIGIR conference on research and develop- ment in information retrieval , 2020, pp. 1289–1298
work page 2020
Show all 30 references
-
[9]
Neural graph collaborative filtering,
X. Wang, X. He, M. Wang, F. Feng, and T.-S. Chua, “Neural graph collaborative filtering, ” inProceedings of the 42nd inter- national ACM SIGIR conference on Research and development in Information Retrieval, 2019, pp. 165–174
2019
-
[10]
Multi-graph convolution collaborative filtering,
J. Sun, Y. Zhang, C. Ma, M. Coates, H. Guo, R. Tang, and X. He, “Multi-graph convolution collaborative filtering, ” in 2019 IEEE International Conference on Data Mining (ICDM) , IEEE, 2019, pp. 1306–1311
2019
-
[11]
Deoscillated graph collaborative filtering,
Z. Liu, L. Meng, F. Jiang, J. Zhang, and P. S. Yu, “Deoscillated graph collaborative filtering, ”arXiv preprint 2011.02100, 2020
2011 arXiv
-
[12]
Hierarchi- cal representation learning for bipartite graphs.,
C. Li, K. Jia, D. Shen, C.-J. R. Shi, and H. Yang, “Hierarchi- cal representation learning for bipartite graphs., ” inIJCAI, vol. 19, 2019, pp. 2873–2879
2019
-
[13]
Graph convolutional neural networks for web- scale recommender systems,
R. Ying, R. He, K. Chen, P. Eksombatchai, W. L. Hamilton, and J. Leskovec, “Graph convolutional neural networks for web- scale recommender systems, ” inProceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, 2018, pp. 974–983
2018
-
[14]
Graph neural networks in recommender systems: A survey,
S. Wu, F. Sun, W. Zhang, X. Xie, and B. Cui, “Graph neural networks in recommender systems: A survey, ”ACM Com- puting Surveys, vol. 55, no. 5, pp. 1–37, 2022
2022
-
[15]
Inductive represen- tation learning on large graphs,
W. Hamilton, Z. Ying, and J. Leskovec, “Inductive represen- tation learning on large graphs, ”Advances in neural informa- tion processing systems, vol. 30, 2017
2017
-
[16]
Lightgcn: Simplifying and powering graph convolution net- work for recommendation,
X. He, K. Deng, X. Wang, Y. Li, Y. Zhang, and M. Wang, “Lightgcn: Simplifying and powering graph convolution net- work for recommendation, ” inProceedings of the 43rd Inter- national ACM SIGIR conference on research and development in Information Retrieval, 2020, pp. 639–648
2020
-
[17]
Explainable recommendation: A survey and new perspectives,
Y. Zhang, X. Chen, et al., “Explainable recommendation: A survey and new perspectives, ”Foundations and Trends® in Information Retrieval, vol. 14, no. 1, pp. 1–101, 2020
2020
-
[18]
Knowl- edge enhanced graph neural networks for explainable rec- ommendation,
Z. Lyu, Y. Wu, J. Lai, M. Yang, C. Li, and W. Zhou, “Knowl- edge enhanced graph neural networks for explainable rec- ommendation, ”IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 5, pp. 4954–4968, 2022
2022
-
[19]
The role of transparency in rec- ommender systems,
R. Sinha and K. Swearingen, “The role of transparency in rec- ommender systems, ” inCHI’02 extended abstracts on Human factors in computing systems , 2002, pp. 830–831
2002
-
[20]
Multi- level recommendation reasoning over knowledge graphs with reinforcement learning,
X. Wang, K. Liu, D. Wang, L. Wu, Y. Fu, and X. Xie, “Multi- level recommendation reasoning over knowledge graphs with reinforcement learning, ” inProceedings of the ACM Web Conference 2022, 2022, pp. 2098–2108
2022
-
[21]
Ex- plicit factor models for explainable recommendation based on phrase-level sentiment analysis,
Y. Zhang, G. Lai, M. Zhang, Y. Zhang, Y. Liu, and S. Ma, “Ex- plicit factor models for explainable recommendation based on phrase-level sentiment analysis, ” inProceedings of the 37th international ACM SIGIR conference on Research & develop- ment in information retrieval , 201...
2014
-
[22]
Trirank: Review- aware explainable recommendation by modeling aspects,
X. He, T. Chen, M.-Y. Kan, and X. Chen, “Trirank: Review- aware explainable recommendation by modeling aspects, ” in Proceedings of the 24th ACM international on conference on information and knowledge management , 2015
2015
-
[23]
Explanation mining: Post hoc in- terpretability of latent factor models for recommendation systems,
G. Peake and J. Wang, “Explanation mining: Post hoc in- terpretability of latent factor models for recommendation systems, ” inProceedings of the 24th ACM SIGKDD Interna- tional Conference on Knowledge Discovery & Data Mining , 2018, pp. 2060–2069
2018
-
[24]
Graphsvx: Shapley value expla- nations for graph neural networks,
A. Duval and F. D. Malliaros, “Graphsvx: Shapley value expla- nations for graph neural networks, ” inMachine Learning and Knowledge Discovery in Databases. Research Track: European Conference, ECML PKDD 2021, Bilbao, Spain, September 13–17, 2021, Proceedings, Part II 21 , Spri...
2021
-
[25]
Generative causal explanations for graph neural networks,
W. Lin, H. Lan, and B. Li, “Generative causal explanations for graph neural networks, ” inInternational Conference on Machine Learning, PMLR, 2021, pp. 6666–6679
2021
-
[26]
Explainability methods for graph convolutional neu- ral networks,
P. E. Pope, S. Kolouri, M. Rostami, C. E. Martin, and H. Hoff- mann, “Explainability methods for graph convolutional neu- ral networks, ” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2019
2019
-
[27]
" why should i trust you?
M. T. Ribeiro, S. Singh, and C. Guestrin, “" why should i trust you?" explaining the predictions of any classifier, ” in Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 2016, pp. 1135–1144
2016
-
[28]
A unified approach to interpreting model pre- dictions,
S. Lundberg, “A unified approach to interpreting model pre- dictions, ”arXiv preprint arXiv:1705.07874, 2017
2017 arXiv
-
[29]
Taxonomic recommendations of real estate properties with textual attribute information,
Z. Harrison and A. Khazane, “Taxonomic recommendations of real estate properties with textual attribute information, ” in Proceedings of the 16th ACM Conference on Recommender Systems, 2022, pp. 479–481
2022
-
[30]
Zillow group, inc website, https:www.zillow.com. 9 KDD ’25 MLoG-GenAI, August 06, 2025, Toronto, ON, Canada Kunal Mukherjee, Zachary Harrison, and Saeid Balaneshin A APPENDIX A.1 ZiGNN Architecture ZiGNN is a Graph Neural Network-based recommendation engine designed to operate...
2025
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.