Recognition: unknown
Which Discoveries Are Paradigm Shifting?
Pith reviewed 2026-05-10 15:31 UTC · model grok-4.3
The pith
Impact, novelty, and disruptiveness are strict complements for paradigm-shifting discoveries.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The authors create a unified measure that folds impact, novelty, and disruptiveness together, then calibrate it against known paradigm-shifting cases from the National Inventor Hall of Fame. The calibration reveals that the three attributes function as strict complements: greater impact cannot substitute for only moderate novelty, and the same holds for the other pairs.
What carries the argument
A single coherent score that integrates separate measures of impact, novelty, and disruptiveness extracted from patent records.
Load-bearing premise
The National Inventor Hall of Fame listings provide a valid ground truth for true paradigm-shifting discoveries and that the three dimensions can be reliably measured from patent data.
What would settle it
A well-documented discovery with very high impact and disruptiveness but only moderate novelty that experts still classify as paradigm-shifting would falsify the strict-complements result.
Figures
read the original abstract
To better align theories of paradigm shifting discoveries and empirics identifying them, we pro-pose a novel measure that incorporates a discovery impact, novelty, and tendency to break with the past into a single, coherent measure. Calibration using the National Inventor Hall of Fame data reveals that impact, novelty, and disruptiveness are strict complements meaning, for example, that greater impact cannot substitute for moderate novelty. We illustrate the workings of the measure using data on USPTO patents from 1982 to 2015.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a novel measure combining discovery impact, novelty, and disruptiveness into a single coherent metric for identifying paradigm-shifting discoveries. Calibration on National Inventor Hall of Fame data indicates that these three elements are strict complements (greater impact cannot substitute for moderate novelty, etc.). The measure is then illustrated using USPTO patent records from 1982 to 2015.
Significance. If the calibration is robust and the Hall of Fame labels align with theoretical criteria for paradigm shifts, the work could supply a practical empirical tool for distinguishing paradigm-shifting from incremental discoveries, helping bridge Kuhnian theory with quantitative science studies.
major comments (3)
- [Calibration procedure (implied in abstract)] The central claim that impact, novelty, and disruptiveness are strict complements rests on calibration to National Inventor Hall of Fame data, yet no equations, fitting procedure, or functional form are supplied in the abstract or visible text to show how the combined measure is constructed or why additive/substitutable alternatives fit worse.
- [Ground-truth construction] The National Inventor Hall of Fame is treated as ground truth for paradigm-shifting discoveries, but no independent validation is provided that these patents satisfy explicit Kuhn-style criteria (e.g., incommensurability or fundamental reorientation of the field) rather than selection on commercial success, citation volume, or inventor reputation.
- [Empirical illustration] Patent-derived proxies (citation discontinuities, keyword novelty, etc.) are mentioned for the 1982–2015 USPTO illustration, but without details on extraction, normalization, or robustness checks, it is impossible to assess whether the observed complementarity is an artifact of the chosen proxies or holds more generally.
minor comments (1)
- [Abstract] The abstract contains the apparent typo 'pro-pose' instead of 'propose'.
Simulated Author's Rebuttal
We thank the referee for their constructive comments, which have identified important areas for improving the clarity and rigor of our manuscript. We address each major comment below and outline the revisions we will make.
read point-by-point responses
-
Referee: The central claim that impact, novelty, and disruptiveness are strict complements rests on calibration to National Inventor Hall of Fame data, yet no equations, fitting procedure, or functional form are supplied in the abstract or visible text to show how the combined measure is constructed or why additive/substitutable alternatives fit worse.
Authors: We acknowledge that the submitted manuscript did not provide sufficient detail on the calibration in either the abstract or the main text. We will revise the manuscript to explicitly state the functional form as the product of the three normalized components (impact × novelty × disruptiveness) to enforce strict complementarity. The fitting procedure uses logistic regression on the Hall of Fame induction labels as the outcome variable, with model comparison showing superior fit (via AIC/BIC and predictive accuracy) relative to additive or substitutable specifications. The key equations and comparison results will be added to the abstract and a dedicated subsection in the Methods. revision: yes
-
Referee: The National Inventor Hall of Fame is treated as ground truth for paradigm-shifting discoveries, but no independent validation is provided that these patents satisfy explicit Kuhn-style criteria (e.g., incommensurability or fundamental reorientation of the field) rather than selection on commercial success, citation volume, or inventor reputation.
Authors: This is a fair critique of our ground-truth choice. The Hall of Fame is employed as an expert-curated benchmark for transformative inventions rather than a perfect operationalization of Kuhnian criteria. In the revision we will add a dedicated discussion subsection that maps specific Hall of Fame examples to Kuhnian notions of incommensurability and field reorientation, while explicitly noting that selection may also reflect commercial or reputational factors. We will frame this as a limitation and suggest qualitative case studies as future work to strengthen the link to theory. revision: partial
-
Referee: Patent-derived proxies (citation discontinuities, keyword novelty, etc.) are mentioned for the 1982–2015 USPTO illustration, but without details on extraction, normalization, or robustness checks, it is impossible to assess whether the observed complementarity is an artifact of the chosen proxies or holds more generally.
Authors: We agree that the empirical section requires greater transparency. The revised manuscript will expand the Data and Methods section to detail proxy construction (e.g., citation discontinuity measured as the ratio of forward citations in the five years after grant versus the prior average, keyword novelty via TF-IDF on abstracts), normalization (within-class z-scores), and robustness checks including alternative windows, semantic embedding-based novelty, and sensitivity analyses across technology classes. These additions will allow readers to evaluate whether the complementarity result is proxy-dependent. revision: yes
Circularity Check
No significant circularity: empirical calibration presented as independent finding
full rationale
The paper proposes a composite measure of paradigm-shifting discoveries by combining impact, novelty, and disruptiveness, then reports that calibration against National Inventor Hall of Fame labels shows these three dimensions behave as strict complements. No equations, functional forms, or fitting procedures are exhibited in the abstract or surrounding text that would allow the complementarity conclusion to be rewritten as a definitional identity or as a direct renaming of the calibration inputs. The Hall of Fame data functions as an external benchmark rather than an input that is algebraically rearranged into the result. Self-citation chains and ansatz smuggling are not invoked in the provided material. The derivation therefore remains self-contained against external labels and does not reduce to its own fitted parameters by construction.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
+! . +" . (2) !
ally disruptive (𝐷!_)* = 1). In contrast, the focal patent is fully consolidating if 𝐷!_)*=−1, since such a patent strengthens the ties of subsequent patents to the focal patent’s prior art. Subsequent literature has suggested several modifications to the CD index. For example, Bornmann et al. (2020) modify the CD index by excluding the term 𝑁' from its d...
2020
-
[2]
#$ ranges from zero to one. It is increasing in the propensity of a discovery to be paradigm shifting, with 𝐷!
on the normalization of citation-based measures, we standardize the dimensions – 𝑥?,𝑥8,𝑥B –, over which the generalized mean is calculated by using the percentages of their respective cumulative distributions (Bornmann & Williams, 2020), i.e., using 𝐺@(𝑥@). This normalization means that the variables over which the generalized mean is calculated are 𝑓@= 𝐺...
2020
-
[3]
a groundbreaking or a significant advancement
EMPIRICAL ANALYSIS 4.1. Data Data sources: We use patents granted by the United States Patent and Trademark Office (USPTO), for which an NBER (National Bureau of Economic Research) industry codes are available (obtained from patentsview.org). The patents are matched to text-based keyword information from Arts et al. (2021). Since the text-based keyword da...
2021
-
[4]
Table 1: Descriptive statistics. 4.2. Empirical properties of existing disruption measures Despite going through challenges with the existing disruption measures in this section, our inten-tion is not to criticize them. Quite the contrary, and as we wrote above, we suggest using them as an input to the proposed measure. Property #1 – Moderate impact and n...
2024
-
[5]
Specifically, of the 9 We winsorize the extreme values in the 99.9999% in the upper tail, i.e., replace the higher values of 4 observa-tions with the value at the 99.9999% percentile. Mean St. dev. p5 p25 p50 p75 p95 Min. Max. F 3.380 7.818 0 0 1 4 12 0 1,042 K 123.527 2,259.131 0 2 19 73 366 0 1,440,810 B 12.169 34.693 0 3 5 11 28 0 5,802 Ni 1.926 4.538 ...
2020
-
[6]
Among these patents, the difference 27 between the patent grant year and the induction year by NIHF is on average 19.8 years (median = 18). This means that the induction decision is based on a dispersed and rich set of qualitative and quantitative information that has cumulated over each discovery’s (patent’s) lifecycle and that has become available over ...
1996
-
[7]
Unlike standard applications of choice-based sampling in econometrics (e.g., Hsieh et al., 1985; Imbens,
– this is also called the case-population design. Unlike standard applications of choice-based sampling in econometrics (e.g., Hsieh et al., 1985; Imbens,
1985
-
[8]
0” vs. “1
and rare events analyses in statistics (e.g., King & Zeng, 2001), the case-population design consists of one sub-sample, selected fully on the outcome variable (“1”) for which also the relevant covariates are observed, and another subsample drawn randomly from the whole population, for which only the covariates ob-served. Because there is no information o...
2001
-
[9]
#$ helps in distinguishing between the two types of drug patents. In a Logit regression, the coefficient for 𝐷!
and their patents, we derive an estimation sample that consists of original and supplementary patents pro-tecting the drugs (see Appendix D for details of the data). Using the resulting sample and a dependent variable indicating an original patent within a drug (= 1; a supplementary patent = 0), we can analyze whether 𝐷!"#$ helps in distinguishing between...
2023
-
[10]
#$ in each NBER industry. Panel B focuses on the right tail of 𝐷!
Panels A–D in Figure 6 suggest the following: First, looking at Panel A, we find a steady down-ward trend in the annual means of 𝐷!"#$ in each NBER industry. Panel B focuses on the right tail of 𝐷!"#$ (𝐷!"#$ > 0.90 in each year), which arguably better identifies paradigm shifting patents per indus-try, each year. This criterion implies that we only look a...
1982
-
[11]
#$ to identify paradigm shifting technologies in real time or ex ante. A buffer period of five years is required to accumulate such citations, causing a lag before 𝐷!
DISCUSSION 5.1 Limitations A primary limitation of our empirical analysis is that we rely on patent data and use largely, but not solely, forward citations. To start with, using forward citations as inputs means that we cannot use 𝐷!"#$ to identify paradigm shifting technologies in real time or ex ante. A buffer period of five years is required to accumul...
2024
-
[12]
CONCLUSIONS Theory and empirics of economic growth show that longer-term improvements in human well-being are almost single-handedly driven by nurturing and applying new ideas. Whereas incrementally better new ideas and diffusion of old ideas are undoubtedly important, the role of paradigm shifting discoveries in the longer-term progress of humankind can ...
-
[13]
https://doi.org/10.1186/1745-6215-15-464 Brusoni, S., Henkel, J., Jacobides, M. G., Karim, S., MacCormack, A., Puranam, P., & Schilling, M. (2023). The power of modularity today: 20 years of “Design Rules”. Industrial and Corporate Change, 32(1), 1-10. https://doi.org/10.1093/icc/dtac054 Bu, Y ., Waltman, L., & Huang, Y . (2021). A multidimensional framew...
-
[14]
https://doi.org/10.3386/w8498 Harhoff, D., Narin, F., Scherer, F. M., & V opel, K. (1999). Citation Frequency and the Value of Patented Inventions. The Review of Economics and Statistics, 81(3), 511-515. https://doi.org/10.1162/003465399558265 Harhoff, D., Scherer, F. M., & V opel, K. (2003). Citations, family size, opposition and the value of patent righ...
-
[15]
https://doi.org/10.2193/0022-541X(2004)068[0774:UAIOLR]2.0.CO;2 Kelly, B., Papanikolaou, D., Seru, A., & Taddy, M. (2021). Measuring Technological Innovation over the Long Run. American Economic Review: Insights, 3(3), 303-320. https://doi.org/10.1257/aeri.20190499 Kim, J., Park, Y ., & Lee, Y . (2016). A visual scanning of potential disruptive signals fo...
-
[16]
http://search.ebscohost.com/login.aspx?direct=true&db=bch&AN=4504968&site=ehost-live O’Connor, G. C., & Veryzer, R. W. (2001). The nature of market visioning for technology-based radical innovation. The Journal of Product Innovation Management, 18(4), 231-246. https://doi.org/10.1016/S0737-6782(01)00092-3 Park, M., Leahey, E., & Funk, R. J. (2023). Papers...
-
[17]
https://doi.org/10.1186/s12961-016-0131-2 Sheng, L., Lyu, D., Ruan, X., Shen, H., & Cheng, Y . (2023). The association between prior knowledge and the disruption of an article. Scientometrics, 128(8), 4731-4751. https://doi.org/10.1007/s11192-023-04751-0 49 Sosa, M. L. (2011). From Old Competence Destruction to New Competence Access: Evidence from the Com...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.