Main
Although the past century witnessed an unprecedented expansion of scientific and technological knowledge, there is a problem that innovative activity is slowing18,19,20. Study documents declining research productivity in semiconductors, pharmaceuticals and other fields10,11. Papers, patents and even grant applications have become less novel relative to previous works and less likely to connect different areas of knowledge, both of which are precursors to innovation21,22. This may interest you : Inside the Army’s high-tech instrument testing facilities. The gap between the year of discovery and the awarding of the Nobel Prize also increased23,24, suggesting that today’s contribution does not measure up to the past. This trend has attracted greater attention from policy makers, as it poses a major threat to economic growth, human health and well-being, and national security, and global efforts to combat major challenges such as climate change25,26.
Several explanations for this slowdown have been proposed. Some point to the lack of ‘low-hanging fruit’ because the available productivity-enhancing innovations have been made19,27. Others emphasize the increasing burden of knowledge; scientists and inventors need ever more training to reach the frontiers of their fields, leaving less time to push those frontiers forward18,28. Still much remains unknown, not only about the cause of slowing innovative activity, but also the depth and breadth of the phenomenon. The decline is difficult to reconcile with centuries of observation by philosophers of science, who characterize the growth of knowledge as an endogenous process, where previous knowledge allows the discovery of the future, a view captured famously in Newton’s observation that if he has seen the next, it is by ‘standing on the shoulders of giants’3. Moreover, until now, the evidence points to the slowdown based on certain field studies, using disparate and domain-specific metrics10,11, so it is difficult to know why changes occur at similar rates throughout the science and technology area. Little is known about whether the patterns seen in aggregate indicators mask differences in the degree to which individuals work to push those limits.
We addressed this gap in understanding by analyzing 25 million papers (1945–2010) on the Web of Science (WoS) (Methods) and 3.9 million patents (1976–2010) in the United States Patent and Trademark Office’s (USPTO) Patent- Patent View database database. (Method). WoS data includes 390 million citations, 25 million paper titles and 13 million abstracts. Patent Display Data includes 35 million citations, 3.9 million patent titles and 3.9 million abstracts. Next, we replicated our core findings in four additional datasets—JSTOR, American Physical Society corpus, Microsoft Academic Graph and PubMed—covering 20 million papers. Using this data, we combine 12 new citation-based measures with textual analysis of titles and abstracts to understand why papers and patents forge new directions over time and across fields.
Measurement of disruptiveness
To characterize the nature of innovation, we draw on basic theories of scientific and technological change2,29,30, which distinguish two types of breakthroughs. First, some contributions increase the flow of existing knowledge, and therefore collect the status quo. Kohn and Sham (1965)31, a Nobel-winning paper used established theorems to develop a method to calculate the electron structure, which strengthened the value of previous research. See the article : China’s Barriers to Becoming a Science Superpower. Second, some contributions disrupt existing knowledge, make it obsolete, and push science and technology in new directions. Watson and Crick (1953)32, also a Nobel laureate, introduced a model of DNA structure that replaced previous approaches (for example, Pauling’s triple helix). Kohn and Sham and Watson and Crick are both important, but their implications for scientific and technological change are different.
We calculate this difference using a measure-CD index12-which characterizes the consolidating or disruptive nature of science and technology (Fig. 1). the intuition is that if a paper or patent is disruptive, the next work cited is less likely to also be cited as its predecessor; for future researchers, the ideas that went into his production are less relevant (for example, Pauling’s triple helix). If a paper or patent is consolidating, the next work cited is also easier to mention its predecessor; for future researchers, the knowledge on which the work is built is still (and perhaps more) relevant (for example, Kohn and Sham theorems are used). The CD index ranges from −1 (consolidating) to 1 (disruptive). We measure the CD index five years after the publication year of each of our papers (indicated by CD5, see Extended Data Fig. 1 for CD5 distribution among papers and patents and Extended Data Fig. 2 for analysis using alternative windows)33. For example, Watson and Crick and Kohn and Sham both received over a hundred citations within five years of publication. However, Kohn and Sham’s paper has a CD5 of -0.22 (indicating consolidation), while Watson and Crick’s paper has a CD5 of 0.62 (indicating disruption). The CD index has been extensively validated in previous research, including through correlation with expert assessments12,34.
This figure shows a schematic visualization of the CD index. a, CD index values of three Nobel Prize-winning papers31,32,58 and three famous patents59,60,61 in our sample, measured from five years after publication (indicated by CD5). b, Distribution of CD5 for papers from WoS (n = 24,659,076) between 1945 and 2010 and patents from Patent View (n = 3,912,353) between 1976 and 2010, where a single point represents a paper or patent. The vertical dimension (top-bottom) of each ‘strip’ corresponds to the CD index value (with the axis value shown in orange on the left). The horizontal (left-right) dimensions of each strip help to minimize overlapping points. Darker areas in each strip plot indicate areas of denser distribution (ie, more frequently observed CD5 values). Additional details on the CD index distribution are given in Extended Data Figs. 1. c, Three hypothetical reference networks, where the CD index is at the maximally disruptive value (CDt = 1), the midpoint value (CDt = 0), and the maximally consolidated value (CDt = −1). The panel also provides equations for the CD index and illustrative calculations.
Declining disruptiveness
Across the field, we find that science and technology are becoming less disruptive. Figure 2 plots the average CD5 over time for papers (Fig. 2a) and patents (Fig. 2b). For papers, the decline between 1945 and 2010 ranges from 91.9% (where the average CD5 dropped from 0.52 in 1945 to 0.04 in 2010 for ‘social science’) to 100% (where the average CD5 dropped from 0.36 in 1920 to 0.04 in 2010. for ‘physical sciences’); for patents, the decrease between 1980 and 2010 ranged from 78.7% (where the average CD5 decreased from 0.30 in 1980 to 0.06 in 2010 for ‘computers and communications’) to 91.5% ( where the average CD5 decreased from 0.38 in 0.38 in 1980). 2010 for ‘drugs and medical’). For both papers and patents, the rate of decline is greatest in the early parts of the time series, and for patents, they seem to have stabilized between 2000 and 2005. For papers, since about 1980, the rate of decline has increased. more modest in ‘life sciences and biomedicine’ and physical sciences, and most marked and persistent in social sciences and ‘technology’. Overall, however, relative to the past, recent papers and patents do less to push science and technology in new directions. The general similarity in trends that we see across fields is important in light of the ‘low-hanging fruit’ theory19,27, which seems to predict greater heterogeneity in decline, as it seems unlikely that fields will ‘consume’ the low-hanging fruit under conditions the same. rate or times.
a, b, Decrease in CD5 over time, separately for papers (a, n = 24,659,076) and patents (b, n = 3,912,353). For papers, lines correspond to WoS research areas; from 1945 to 2010, the amount of decline was between 91.9% (social sciences) to 100% (physical sciences). For patents, the line corresponds to the National Bureau of Economic Research (NBER) technology category; from 1980 to 2010 the magnitude of the decline varied from 93.5% (computer and communication) to 96. See the article : The Clean Air Panel takes an ‘unprecedented’ break for scientific review.4% (medicine and medical). Shaded bands correspond to 95% confidence intervals. As we elaborate in the Method, this pattern of decline is robust to adjustment for confounding from changes in publications, inclusion and author practices over time.
Linguistic change
The decline of disruptive science and technology can also be seen using alternative indicators. Because they make departures from the status quo, disruptive papers and patents are likely to introduce new words (for example, words used to create a new paradigm may be different from those used to develop an existing paradigm)35,36. Therefore, if disruptiveness has declined, we would expect a decline in the diversity of words used in science and technology. To evaluate this, Fig. 3a,d document the token-type ratio (ie, unique/total words) of paper titles and patents over time (Supplementary Information Section 1). We observed big declines, especially in the previous period, before 1970 for papers and 1990 for patents. For the title of the paper (Fig. 3a), the decrease (1945-2010) ranges from 76.5% (social sciences) to 88% (technology); for patent titles (Fig. 3d), decrease (1980-2010) from 32.5% (chemical) to 81% (computer and communication). For paper abstracts (Extended Data Fig. 3a), the decrease (1992-2010) ranges from 23.1% (life sciences and biomedicine) to 38.9% (social sciences); for patent abstracts (Extended Data Fig. 3b), the decrease (1980-2010) ranges from 21.5% (mechanical) to 73.2% (computer and communication). In fig. 3b,e, we demonstrate that declines in word diversity are accompanied by similar declines in combinatorial novelty; over time, specific words used by scientists and inventors in the titles of their papers and patents are increasingly used together in the titles of earlier works. Consistent with this trend in language, we also observed a decline in novelty in the combination of works previously cited by papers and patents, based on the previously established measure of ‘atypical combinations’14 (Extended Data Fig. 4).
a,d, Figures showing the decline in the diversity of languages used in science and technology based on unique words/total paper titles from 1945 to 2010 (a, n = 24,659,076) and patent titles from 1980 to 2010 (d , n = 3,912,353). b,e, Numbers that show the decrease in the novelty of the language used in science and technology based on the number of new word pairs/total word pairs introduced every year in the title of WoS papers from 1945 to 2010 (b) and in the title of patent Show Patent. from 1980 to 2010 (refs.1,17) (e). For papers in both a and b, the line corresponds to the WoS research area (n = 264 WoS research area × observation year). For patents in both d and e, the line corresponds to the NBER technology category (n = 229 NBER technology categories × observation year). c,f, Figures showing the frequency of the most frequently used verbs in paper titles for the first (red) and last (blue) decades of the observation period in papers (c, n = 24,659,076) and patents (f, n = 3,912 .353 ) the title.
The decrease in disruptive activity is also seen in the special words used by scientists and inventors. If disruptiveness has declined, we reason that verbs alluding to the creation, discovery or perception of new things should be used less often over time, while verbs alluding to improvement, application or assessment of existing things can be used more often35,36. Figure 3 shows the most common verbs in the paper (Fig. 3c) and patent titles (Fig. 3f) in the first and last decade of each sample (Supplementary information section 2). Although precisely and quantitatively characterizing words as ‘consolidating’ or ‘disruptive’ is challenging in the absence of context, the figure highlights a clear and qualitative shift in language. In previous decades, verbs evoking creation (for example, ‘produce’, ‘form’, ‘prepare’ and ‘make’), discovery (for example, ‘determine’ and ‘report’) and perception (for example, ‘measure.’) prevalent in title papers and patents. In the following decades, however, these verbs were almost entirely displaced by those that tended to be more evocative of improvement (for example, ‘to improve’, ‘to improve’ and ‘to improve’), application (for example, ‘to use’ and ‘to include’) or assessment (for example, ‘associate’, ‘mediation’ and ‘linking’) existing scientific and technological knowledge and artifacts. Taken together, this pattern shows a substantive shift in science and technology over time, and discoveries and inventions become less disruptive in nature, consistent with our results using the CD index.
Conservation of highly disruptive work
Our aggregate trends document considerable heterogeneity in the disruptiveness of individual papers and patents and exceptional stability in the absolute number of highly disruptive works (Method and Fig. 4). Specifically, despite the large increase in scientific productivity, the number of papers and patents with CD5 values in the far right tail of the distribution remains almost constant over time. This ‘conservation’ of the absolute number of papers is highly disruptive and the patents hold despite the considerable churn in the field of conditions responsible for producing their work (Extended Data Fig. 5, inset). These results show that the persistence of major breakthroughs—for example, the measurement of gravitational waves and the COVID-19 vaccine—does not correspond to a slowdown in innovative activity. In short, the decline of aggregate disruptiveness does not preclude individual highly disruptive work.
This figure shows the number of disturbing papers (a, n = 5,030,179) and patents (b, n = 1,476,004) in four different CD5 ranges (papers and patents with CD5 values in the range [−1.0, 0) are not represented in. figures). Lines correspond to different levels of disruptiveness as measured by CD5. Despite the large increase in the number of papers and patents published every year, there is little change in the number of highly disruptive papers and patents, as evidenced by the relatively flat red, green and orange lines. This pattern helps to calculate the simultaneous observation of both aggregate evidence of slowing innovative activity and breakthroughs seem to be major in many fields of science and technology. The inset plot shows the composition of the most disruptive papers and patents (defined as CD5 values > 0.25) over time. The stability observed in the absolute number of disruptive papers and patents holds despite the huge churn in the fields of science and technology responsible for producing those works. ‘Life Sciences’ refers to the life sciences and biomedical research areas; ‘electricity’ refers to the category of electrical and electronic technology; ‘drugs’ refers to the category of drugs and medical technology; and ‘computer’ denotes the category of computer and communication technology.
Alternative explanations
What caused the decline in disruptiveness? In advance, we suggest that our results are not consistent with the explanation that links slowing innovative activity to reduced ‘low-hanging fruit’. Expanding Data Fig. 5 shows that the decrease in disruptiveness is unlikely to be due to other field-specific factors by decomposing the variation in CD5 attributed to field, author and year effects (Method).
The decline in the level of disruptive activity does not seem to be due to the declining quality of science and technology22,37. If it is, then the pattern seen in Fig. 2 should be less visible in high quality work. However, when we limit our sample to articles published in premier publication places such as Nature, Proceedings of the National Academy of Sciences and Sciences or Nobel-winning discoveries38 (Fig. 5), the downward trend persists.
This figure shows changes in CD5 over time for papers published in Nature, Proceedings of the National Academy of Sciences (PNAS) and Science (inset plot, n = 223,745) and Nobel Prize winning papers (main plot, n = 635), with some examples notable31,32,58,62,63,64,65,66 are highlighted. Colors indicate the three different journals in the inset plot; colors indicate three different fields in which the Nobel Prize is awarded in the main plot. Shaded bands correspond to 95% confidence intervals. To complete the history, we plot the CD index scores for all Nobel papers back to 1900 (the first year the prize was awarded); however, our main analysis begins in the post-1945 era, when WoS data is generally more reliable. The figure suggests that changes in the quality of published science over time are unlikely to be responsible for the decline in disruption.
Furthermore, the trend is not driven by the characteristics of WoS and UPSTO data or our particular derivation of the CD index; we observe similar declines in disruptiveness when we calculate CD5 on papers in JSTOR, American Physical Society corpus, Microsoft Academic Graph and PubMed (Method), the results are shown in Extended Data Fig. 6. We further show that the decrease is not an artifact of the CD index by reporting the same pattern using an alternative derivation13,15 (Extended Methods and Data Fig. 7).
The decrease in disruptiveness is also not caused by changing publications, citations or author practices (Methods). First, using the approach from the bibliometrics literature39,40,41,42,43, we calculated some normalized versions of the CD index adjusted for the increasing tendency for papers and patents to mention previous work44,45. The result of using this alternative indicator (Extended Data Fig. 8a, d) is similar to what we reported in Fig. 2. Second, using regression, we estimate the CD5 model as a function of the indicator variable for each paper or patent publication year, throughout. with special control for field × year level-number of new papers/patents, average number of cited papers/patents, average number of authors or inventors per paper-and paper or patent-level-number of papers or patents -patent cited-factor. The prediction of this model shows a decline in disruptive papers and patents (Extended Data Fig. 8b, e and Supplementary Table 1) is consistent with our main results. Finally, using Monte Carlo simulation, we randomly rewired the observed citation network while preserving the key characteristics of the citation behavior of scientists ‘and inventors’, including the number of citations made and received by individual papers and patents and the age gap between citing and cited works. We found that the observed CD5 value was lower than the simulated network (Extended Data Fig. 8c,f), and the gap widened: over time, papers and patents became less disruptive than expected by chance. Taken together, these additional analyzes suggest that the decline in CD5 is unlikely to be driven by changing publication, citation or authorship practices.
Growth of knowledge and disruptiveness
We also considered how the decline of disruptiveness relates to the growth of knowledge (Extended Data Fig. 9). On the one hand, scientists and inventors face an increased burden of knowledge, which can inhibit discoveries and discoveries that disturb the status quo. On the other hand, as noted earlier, philosophers of science suggest that existing knowledge encourages discovery and discovery3,6,7. Using a regression model, we evaluated the relationship between the stock of papers and patents (a proxy for knowledge) in the field and its CD5 (Supplementary Information Section 3 and Supplementary Table 2). We found a positive effect of knowledge growth on disruptiveness for papers, consistent with previous work20; however, we find a negative effect for patents.
Given these conflicting results, we considered the possibility that the availability of knowledge may differ from its use. In particular, the growth of publishing and patents can lead scientists and inventors to focus on narrow slices of previous work18,46, thereby limiting the stock of ‘effective’ knowledge. Using three proxies, we document a decline in the use of prior knowledge among scientists and inventors (Fig. 6). First, we see a decline in the diversity of cited works (Fig. 6a, d), indicating that contemporary science and technology are involved with a narrow slice of existing knowledge. Moreover, the decrease in diversity is accompanied by an increase in the share of citations to the top 1% of cited papers and patents (Fig. 6a (i), d (i)), which also decreases semantic diversity (Fig. 6a (ii), d (ii)). Over time, scientists and inventors increasingly cited the same previous work, and that the previous work became more topically similar. Second, we see an increase in self-reference (Fig. 6b, e), a general proxy for the continuation of the flow of pre-existing research47,48,49, which is consistent with scientists and inventors relying more on familiar knowledge. Third, the average working age is cited, a common measure for the use of knowledge dates50,51,52, increasing (Fig. 6c, f), suggesting that scientists and inventors may be struggling to keep up with the pace of knowledge expansion. and instead of relying on old, familiar works. All three indicators point to a consistent story: the narrower scope of knowledge is informing contemporary discoveries and inventions.
a–f, Changes in the level of diversity in the use of scientific and technological knowledge among papers (a, n = 264 WoS research area × observation year; b and c, n = 24,659,076 papers) and patents (d, 229 NBER technology categories observation × year; e and f, n = 3,912,353 patents) based on the following measures: diversity of cited works (a and d), average number of self-citations (b and e) and average age works cited (c and f). Shaded bands (b,c,e and f) correspond to 95% confidence intervals. The inset plot of a and d shows the change in the citation share of the top 1% most cited papers (a(i) and d(i)) and the semantic diversity of the top 1% most cited over time (a). (ii) and d(ii)). The values of both measures are calculated in fields and years, and then averaged across fields for plotting. Semantic diversity based on papers and patent titles; the value corresponds to the ratio of the standard deviation and the average pairwise cosine similarity (that is, the coefficient of variation) among the titles of the 1% most cited papers and patents by field and year. To enable semantic comparison, the titles are vectorized using pre-trained word embeddings. For the paper, it will be shown for each WoS research area; for patents, lines are shown for each NBER technology category. In the subsequent regression analysis using this measure, we found that using less diverse work, more own work and older work associated with less disruptive papers and patents (Method and Expanded Data Table 1).
The results of a series of regression models further show that the use of less varied work, more self-employment and longer work are all associated with disruption (Methods, Extended Data Methods 1 and Supplementary Table 3), a pattern that persists even after accounting. for the average age and the number of previous works produced by team members. When the range of work used by scientists and inventors is narrowed, disruptive activity declines.
Discussion
In summary, we report a marked decline in disruptive science and technology over time. Our analysis suggests that this trend is unlikely to be driven by changes in citation practices or the quality of published work. Rather, the decline reflects a substantive shift in science and technology, which reinforces concerns about slowing innovative activity. We attribute this trend partly to the reliance of scientists and inventors on a narrow set of existing knowledge. Although the philosopher of science may be right that the growth of knowledge is an endogenous process – where the accumulation of understanding promotes future discovery and invention – Engagement with a wide range of extant knowledge is necessary for a process that can play out, the requirements that appear more difficult. with time. Relying on a narrower slice of knowledge benefits individual careers53, but not scientific progress in general.
Moreover, although the prevalence of disruptive work has decreased, we found that the sheer number has remained stable. On the one hand, these results may indicate that there is a fixed ‘carrying capacity’ for highly disruptive science and technology, in which case policy interventions aimed at increasing such work may be challenging. On the other hand, our observation of considerable churn in the field of responsibility for producing disruptive science and technology indicates the potential importance of factors such as shifting interests of funders and scientists and the ‘ripeness’ of scientific and technological knowledge for breakthroughs, which in the case of the production of disruptive works may be responsive to policy levers. In either case, the stability we observe in the sheer number of disruptive papers and patents shows that science and technology do not seem to have reached the end of the ‘endless frontier’. Room remains for regular rerouting that disruptive work contributes to scientific and technological progress.
Our study is not without limitations. In particular, although the research up to now supports the validity of the CD index12,34, it is a relatively new indicator of innovative activity and will benefit from future work on its behavior and properties, especially in data sources and contexts. Studies that systematically examine the effects of different citation practices54,55, which vary across fields, would be more informative.
Overall, our results deepen the understanding of knowledge evolution and can guide career planning and science policy. To promote disruptive science and technology, scholars can be encouraged to read widely and devote time to keep up with the rapidly expanding frontier of knowledge. Universities can abandon the focus on quantity, and more strongly reward the quality of research56, and perhaps more fully subsidize sabbaticals during the year. Federal agencies can invest in more risky and long-term individual awards that support careers and not just specific projects57, giving scholars the time they need to step outside the fray, inoculate themselves from the culture of spreading or perishing, and produce actual work. Understanding the decline of disruptive science and technology more fully enables the necessary rethinking of strategies to manage future science and technology production.
Methods
WoS data
We limited our focus to research papers published between 1945 and 2010. Although WoS data started in 1900, the scale and social organization of science shifted markedly in the post-war era, thereby making comparisons with today difficult and potentially misleading67, 68,69. We ended the analysis of our paper in 2010 because some of our measures require several years of data after the publication of the paper. WoS data archive 65 million documents published in 28,968 journals between 1900 and 2017 with 735 million citations among them. In addition, WoS data include titles and full-text abstracts for 65 and 29 million records, respectively, published between 1913 and 2017. After removing non-research documents (eg, book reviews and commentaries) and subset data into the window 1945– 2010, the analytical sample consisted of n = 24,659,076 papers.
Patents View data
We limit our focus to patents granted since 1976, which is the earliest year for which machine-readable records are available in the Patent View data. As we did with the paper, we ended our analysis in 2010 because some measures require data from later years for calculation. The Patents Data is the most comprehensive historical data source on inventions, with information on 6.5 million patents granted between 1976 and 2017 and 92 million citations. The Patent View data includes titles and abstracts for 6.5 million patents granted between 1976 and 2017. Following previous work12, we focused our attention on utility patents, which cover the majority (91% in our data) of patented inventions. After removing non-utility patents and subsetting the data to the 1976-2010 window, the analytical sample consisted of n = 3,912,353 patents.
Highly disruptive papers and patents
Observations (and claims) of slowing progress in science and technology are increasingly common, supported not only by the evidence we report, but also by previous research from various methodological and disciplinary perspectives10,11,18,19,20,21,22, 23, 24. But as noted in the main text, there is a tension between observing the slowing progress of aggregate data on the one hand, and continuing reports of seemingly major breakthroughs in many fields of science and technology – including everything from the measurement of gravitational waves to the sequencing of the human genome – on the other hand. In an effort to reconcile this tension, we considered the possibility that overall, discovery and invention may be less disruptive over time, high-level views taken in previous work may mask considerable heterogeneity. In contrast, the aggregate evidence of slowing progress does not preclude the possibility that some of the discoveries and discoveries are highly disruptive.
To evaluate this possibility, we plot the number of disruptive papers (Fig. 4a) and patents (Fig. 4b) over time, where disruptive papers and patents are defined by their CD5 & GT;0 value. In each panel, we plot four lines, corresponding to four equally spaced intervals-(0, 0.25], (0.25, 0.5], (0.5, 0.75], (0.75, 1 ,00] – more positive values of CD5. The interval therefore corresponds to papers and patents that are relatively weakly disruptive, while the latter two correspond to stronger ones (for example, where we can expect to see major breakthroughs such as some mentioned above). papers and patents published every year, we see little change in the number of highly disruptive papers and patents, as evidenced by the relatively flat red, green and orange lines over time in the composition of the scientific and technological fields responsible for producing the most disruptive works ( Fig. 4, inset plot).science and technology and aggregate evidence of slowing progress.
Relative contribution of field, year and author or inventor effects
Our results show a steady decline in science and technology disruption over time. Moreover, the patterns we see are generally similar across broad fields of study, which suggests that the factors driving the decline are not unique to the specific domains of science and technology. The decline may be driven by other factors, such as the state of science and technology at one time or the specific individuals who produced the science and technology. For example, exogenous factors such as economic conditions may encourage less disruptive research or discovery practices. Likewise, scientists and inventors of different generations may have different approaches, which may lead to a greater or lesser tendency to produce disruptive works. Therefore we sought to understand the relative contribution of the field, year and author (or inventor) factors to the decline of disruptive science and technology.
To do so, we decomposed the relative contribution of the field, year and author fixed effects on the predictive power of the regression model of the CD index. The unit of observation in this regression is author (or inventor) × year. We entered the field of fixed effects using granular subfield indicators (ie, 150 WoS subject areas for papers, 138 NBER subcategories for patents). For simplicity, we did not include additional covariates beyond fixed effects in our model. Field fixed effects capture all field-specific factors that do not vary by author or year (eg, underlying subject); year fixed effects capture all year-specific factors that do not vary by field or author (eg, state of communication technology); Author (or inventor) fixed effects capture all author-specific factors that do not vary by field or year (eg, year of PhD award). After defining our model, we determine the relative contribution of field, year and author fixed effects to the entire model adjusted R2 using Shapley-Owen decomposition. Specifically, given n = 3 groups of fixed effects (field, year and author) we evaluate the relative contribution of each set of fixed effects by estimating R2 adjusted separately for a 2n model using a subset of predictors. The relative contribution of each set of fixed effects is calculated using the Shapley value from game theory70.
The results of this analysis are shown in Extended Data Fig. 5, for both papers (upper bar) and patents (lower bar). The size of the total bar corresponds to the adjusted R2 value for the fully specified model (ie, with all three groups of fixed effects). Consistent with our observations from the plot of the CD index over time, we observed that for both papers and patents, field-specific factors make the lowest relative contribution to R2 adjusted (0.02 and 0.01 for papers and patents, respectively). Author fixed effects, in contrast, seem to contribute more to the predictive power of the model, for both papers (0.20) and patents (0.17). Researchers and inventors entering the field in recent years may face a greater burden of knowledge and thus choose to build narrower pieces of work (for example, due to more specialized doctoral training), which will generally lead to less disruptive science and technology produced in later years, consistent with our findings. The pattern is more complex for year fixed effects; although year-specific factors that do not vary by field or author hold more explanatory power than field for both papers (0.02) and patents (0.16), they seem to be substantially more important for the latter than before. Taken together, these findings indicate that relatively stable factors that vary across individual scientists and inventors may be particularly important for understanding changes in disruptiveness over time. The results also confirm that domain-specific factors in the field of science and technology play a very small role in explaining the decline in disruptiveness of papers and patents.
Alternative samples
We also consider whether the patterns we document may be artifacts of our choice of data sources. Although we see a consistent trend in WoS and Patent View data, and both databases are widely used by the Science community, our results may be driven by factors such as changes in coverage (for example, journals added or removed from WoS. over time) or even data errors rather than fundamental changes in science and technology. To assess this possibility, we calculated CD5 for papers in four additional databases—JSTOR, American Physical Society corpus, Microsoft Academic Graph and PubMed. We included all records from 1930 to 2010 from PubMed (16,774,282 papers), JSTOR (1,703,353 papers) and the American Physical Society (478,373 papers). JSTOR data was obtained by special request from ITHAKA, the data manager (http://www.ithaka.org), as well as American Physical Society data (https://journals.aps.org/datasets). We downloaded the Microsoft Academic Graph data from CADRE at Indiana University (https://cadre.iu.edu/). PubMed data were downloaded from the FTP server of the National Library of Medicine (ftp://ftp.ncbi.nlm.nih.gov/pubmed/baseline). Due to the incredibly large scale of the Microsoft Academic Graph and the associated computational load, we randomly extracted 1 million papers. As shown in Extended Data Fig. 6, the downward trend in disruptiveness is evident in all samples.
Alternative bibliometric measures
Some recent papers have introduced an alternative specification of the CD12 index. We evaluated whether the decrease in disruptiveness we observed was corroborated using two alternative variations. One criticism about the CD index is that the number of papers that are mentioned only as references of our focus papers dominates the size13. Bornmann et al.13 proposed ({{rm{DI}}}_{l}^{{rm{nok}}}) as a variant that is less susceptible to this problem. Another potential weakness of the CD index is that it can be very sensitive to small changes in the forward citation pattern of papers that make no backward citations15. Leydesdorff et al.15 suggested DI * as an alternative indicator of the disorder that addresses this problem. Therefore, we calculate ({{rm{DI}}}_{l}^{{rm{nok}}}) where l = 5 and DI* for 100,000 randomly drawn papers and patents from our analytical sample. . The results are presented in Extended Data Fig. 7a (paper) and b (patent). The blue line shows the interference according to Bornmann et al.13 and the orange line shows the interference according to Leydesdorff et al.15. Across science and technology, the two alternative measures both show declines in disruption over time, similar to the observed pattern with the CD index. Taken together, these results suggest that the decrease in interference we documented was not an artifact of our specific operationalization.
Robustness to changes in publication, citation and authorship practices
We also considered whether our results could be due to changes in publications, inclusion or author practices, rather than substantive shifts in discovery and discovery. Perhaps most critically, as noted in the main text, there was a marked expansion in publishing and patents during the period of our study. This expansion naturally increases the number of previous works related to current science and technology and thus the risk of being cited, a pattern reflected in the marked increase in the average number of citations made by papers and patents (that is, papers and patents that citing more previous jobs than in previous periods)44,45. Remember that the CD index calculates the degree to which future works cite the focal work and its predecessors (i.e., references in the bibliography of the focal work). Greater citations of independently focused works than their predecessors are considered evidence of a process of social disruption. As papers and patents are mentioned over previous works, however, the possibility of focal works cited independently of their predecessors may decrease mechanically; the more citations the focal work makes, the future work is more likely to be cited together with one of its predecessors, even by chance. Consequently, the increase in the number of papers and patents available for citation and the average number of citations made by scientists and inventors may contribute to the decrease in the value of the CD index. In short, given the marked changes in science and technology during our long study window, the index of paper CDs and patents published in the previous period may not be directly comparable to the more recent vintage, which may lead to our conclusion of a decline. in disruptive science and technology suspect. We addressed this problem using three typical but complementary approaches-normalization, adjustmentmen t regression and simulation.
Verification using normalization
First, following common practice in bibliometric research39,40,41,42,43, we developed two normalized versions of the CD index, with the goal of facilitating comparison across time. Among the various components of the CD index, we focus our attention on the number of papers or patents that only mention the reference of our focus work (Nk), as this term will seem to be the most likely scale and increase in publishing and patents and in the average number – average citations made by papers and patents to previous work13. A larger value of Nk leads to a smaller value of the CD index. Consequently, a marked increase in Nk over time, especially relative to other measurement components, can lead to a downward bias, thereby hampering our ability to accurately compare disruptive science and technology in future years with previous periods.
Our two normalized versions of the CD index aim to resolve this potential bias by reducing the effect of increasing Nk. In the first version, which we call ‘Paper dinormalisasi’, we subtract from Nk the number of citations made by the focus paper or patent to the previous work (Nb). The intuition behind this adjustment is that when a focal paper or patent cites more prior work, Nk is likely to be larger because there are more opportunities for future work to cite the focal paper or predecessor patent. This increase in Nk will lead to a lower value of the CD index, although not necessarily as a result of the focus paper or patent being less disruptive. In the second version, which we call ‘field × year normalized’, we reduce Nk by the average number of backward citations made by papers or patents in the focus paper or our patent WoS research area or NBER technology category, respectively during the year na. publication (we label this quantity ({N}_{{rm{b}}}^{{rm{m}}{rm{e}}{rm{a}}{rm{n} }})). The intuition behind this adjustment is that in the field and time period where there is a greater tendency for scientists and inventors to cite previous works, Nk is also likely to be larger, thereby leading to a lower value of the CD index, although again not necessarily. as a result of a focused paper or patent to be less disruptive. In the case where either Nb or ({N}_{{rm{b}}}^{{rm{m}}{rm{e}}{rm{a}}{rm{n } }}) exceeds the value of Nk, we set Nk to 0 (that is, Nk is never negative in the normalized size). Both adaptations of the CD index are inspired by the approach established in the scientometrics literature, and can be understood as a form of ‘side normalization’ (that is, normalization by correcting the effect of the difference in the length of the reference list)40.
In Extended Data Fig. 8, we plot the average value of both normalized versions of the CD index over time, separately for papers (Extended Data Fig. 8a) and patents (Extended Data Fig. 8d). In accordance with our findings reported in the main text, we continue to observe a decrease in the CD index over time, indicating that the pattern we observed in disruptive science and technology is unlikely to be driven by changes in citation practices.
Verification using regression adjustment
Second, we adjusted for potential confounding using a regression-based approach. This approach complements the bibliometric normalization that has just been described by allowing us to calculate a wider range of changes in publications, citations and author practices in general (the latter not directly accounted for in the normalization approach or the simulation approach described next), and increases the number of previous works which is relevant for science and technology today in particular. In Supplementary Table 1, we report the results of regression models predicting CD5 for papers (Models 1–4) and patents (Models 5–8), with indicator variables included for each year of our study window (reference categories are 1945 and 1980). for papers and patents, respectively). Models 1 and 4 are the base models, and do not include any other adjustments beyond the year indicator. In Models 2 and 5, we add subfield fixed effects (WoS subject area for papers and NBER technology subcategory for patents). Finally, in Models 3–4 and 7–8, we add control variables for several fields × year levels—the number of new papers or patents, the average number of cited papers or patents, the average number of authors or inventors per paper-and paper or Patent-level—the number of cited papers or patents—characteristics, thus allowing stronger comparisons in disruptive science and technology patterns over a long period of time harvested by our study. For the paper model, we also include a paper-level control for the number of unlinked references (that is, the number of citations of works that are not indexed in WoS). We found that the inclusion of this control improves the fit of the model, as indicated by the statistically significant Wald test presented below the relevant model.
Across the eight models shown in Supplement Table 1, we found that the coefficient on the year indicator is statistically significant and negative, and grows in magnitude over time, which is consistent with the pattern we reported based on the unadjusted CD5 value index in the main text. (Fig. 2). In Extended Data Fig. 8, we visualize the results of our regression-based approach by plotting the predicted CD5 value separately for each of the year indicators included in Models 4 (papers) and 8 (patents). To enable comparison with the raw CD5 value shown in the main text, we present a separate prediction made for each year as a line graph. As shown in the picture, we continue to observe the declining value of the CD index across papers and patents, even accounting for changes in publications, inclusions and author practices.
Verification using simulation
Third, following related works in the Science of Science14,71,72,73, we considered whether our results could be an artifact of changing patterns in publishing and citation practices by using a simulation approach. Essentially, the CD index measures interference by characterizing the network of citations around the focal paper or patent. However, many complex networks, even those generated by random processes, show structures that produce non-trivial values in the general network size (for example, clustering)74,75,76. During the period spanned by our study, the science and technology citation network experienced significant changes, with a marked increase in both the number of points (that is, papers or patents) and edges (that is, citations). Therefore, instead of representing a meaningful social process, the observed declines in the disorder may result from structural changes in the underlying network.
To assess this possibility, we followed standard techniques from network science75,77 and performed an analysis in which we recalculated the CD index in a randomly rewired citation network. If the pattern we see in the CD index is the result of structural changes in the science and technology citation network (for example, the growth of the number of nodes or edges) rather than a meaningful social process, then this pattern should also be visible. in a comparable random network that undergoes similar structural changes. Therefore, finding that the pattern we see in the CD index is different for the observational and random reference network will serve as evidence that the decrease in interference is not an artifact of the data.
We start by making a copy of the state template network in which the CD index value used in all analyzes is reported in the main text based, separately for papers and patents. For each citation network (one for papers, one for patents), we then rewired the citations using a degree-preserving randomization algorithm. In each iteration of the algorithm, two sides (for example, A-B and C-D) are selected from the basic citation network, after which the algorithm tries to change the two endpoints of the side (for example, A-B becomes A–D, and C–D becomes C–B). If the degree of centrality of A, B, C and D remains the same after the swap, the swap is retained; Otherwise, the algorithm discards the swap and moves on to the next iteration. When evaluating degree centrality, we considered ‘in-degree’ (ie, citations from other papers or patents to a focal paper or patent) and “out-degree” (ie, citations from a focal paper or patent to another paper or patent) separately. Furthermore, we also require that the age distribution of citations and citations of papers or patents is the same in the original and rewired networks. Specifically, the swap is only retained when the year of publication of the original citations and the same candidate. In light of this design choice, our rewiring algorithm should be seen as quite conservative, because it preserves the large structure of the original network. There is no scientific consensus on the number of swaps needed to ensure that native and wired networks are sufficiently distinct from one another; the rule we adopt here is 100 × m, where m is the number of edges in the re-embedded network.
Following the previous work14, we made ten rewired copies of the observed citation network for both papers and patents. After creating this rewired citation network, we then recomputed CD5. Due to the large scale of the WoS data, we based our analysis on a random subsample of ten million papers; CD5 was counted in the rewired network for all patents. For each paper and patent, we then calculated a z-score that compared the observed CD5 values with the same paper or patent in the ten reattached citation networks. A positive z score indicates that the observed CD5 value is greater (ie, more disturbing) than expected by chance; A negative z score indicates that the observed value is less (that is, more consolidating).
The results of this analysis are shown in Extended Data Fig. 8, separately for papers (Extended Data Fig. 8c) and patents (Extended Data Fig. 8f). The line corresponds to the average z score among the papers or patents published in the focal year. The plot shows the pattern of changes in the CD index over and beyond that ‘baked in’ to the changing structure of the network. We found that on average, papers and patents tend to be less disruptive than would be expected by chance, moreover, the gap between the observed CD index values and those from the rewired random network is increasing over time, which is consistent with our findings. from the disruptive decline of science and technology.
Taken together, the results of the above analysis show that although there are marked changes in science and technology during our long study window, especially in terms of publications, citations and authorship practices, the decline in disruptive science and technology that we document using the CD index is unlikely to be one artifact of this change, and instead represents a substantive shift in the nature of discovery and invention.
Regression analysis
We evaluate the relationship between disruptiveness and the use of prior knowledge using a regression model, predicting CD5 for individual papers and patents, based on three indicators of the use of prior knowledge-diversity of work cited, mean number of self-citations and average age of work. quoted. Our measure of the diversity of cited works is measured at the field × year level; all other variables including regressions are defined at the paper or patent level. To account for potential confounding factors, our model included year and field fixed effects. Year fixed effects account for time variance factors that affect all observations (papers or patents) equally (for example, global economic trends). Field fixed effects account for field-specific factors that do not change over time (eg, some fields may intrinsically value disruptive work over unifying ones). In contrast to our descriptive plot, for our regression model, we adjusted for field effects using 150 WoS ‘extended subjects’ more granular (for example, ‘biochemistry and molecular biology’, ‘biophysics’, ‘biotechnology and applied microbiology’, ‘. cell biology ‘, ‘developmental biology’, ‘evolutionary biology’ and ‘microbiology’ are extended subjects in the field of life sciences and biomedical research) and 38 NBER technology subcategories (eg, ‘agriculture’, ‘food’, ‘textile’; ‘coating’ ; ‘gas’; ‘organic’; and ‘resin’ are subcategories in the chemical technology category).
In addition, we also included controls for ‘average age of team members’ (that is, ‘career age’, defined as the difference between the year of publication of the focal paper or patent and the first year when each author or inventor published. paper or patent) and ‘ the average number of previous works produced by team members’. Although the increase in the rate of self-citations may indicate that scientists and inventors are more narrowly focused on their own work, this rate may also be driven in part by the number of previous works available for self-citation. Similarly, although the increase in working age cited in papers and patents can indicate that scientists and inventors are struggling to keep up, they may also be driven by a rapidly aging workforce in science and technology78,79. For example, old scientists and inventors may be more familiar with or more attentive to old works, or may actively resist change80. These control variables help account for these alternative explanations.
Supplementary table 3 shows summary statistics for the variables used in the ordinary-least-squares regression model. The diversity of cited works is measured by normalized entropy, which ranges from 0 to 1. The larger value in this measure indicates a more uniform distribution of citations to a wide range of existing works; lower values indicate a more concentrated distribution of citations to a small range of existing works. The table shows that the normalized entropy in a given field and year has an almost maximum average entropy of 0.98 for science and technology. About 16% of the papers cited in the paper were by the author of the focal paper; the corresponding number for patents is about 7%. Papers tend to rely on older works and works that vary greatly in age (measured by standard deviation) than patents. In addition, the average CD5 of papers is 0.04 while the average CD5 of patents is 0.12, meaning that average papers tend to be less disruptive than average patents.
We found that using more diverse work, less own work and older work tends to be associated with the production of more disruptive science and technology, even after accounting for the average age and number of previous works produced by team members. These findings are based on our regression results, shown in Extended Data Table 1. Models 6 and 12 present the full regression model. The model shows a consistent pattern for science and technology, where the coefficient for the diversity of works cited is positive and significant for papers (0.159, P < 0.01) and patents (0.069, P < 0.01), indicating that in the field of again the use of various works, there is a big problem. Holding all other variables in their means, the predicted CD5 of papers and patents increased respectively by 303.5% and 1.3%, if the diversity of works cited increased by 1 s.d. The coefficient of the ratio of self-citations to total cited works is negative and significant for papers (-0.011, P <0.01) and patents (-0.060, P <0.01), showing that when researchers or inventors rely more on them. own work, discoveries and inventions tend to be less disruptive. Again holding all other variables in their means, the predicted CD5 paper and patents decreased by 622.9% and 18.5%, respectively, with 1 s.d. ratio increase. The interaction coefficient between the average age of cited works and the dispersion in the age of cited works is positive and significant for papers (0.000, P < 0.01) and patents (0.001, P < 0.01), indicating that age dispersion constant cited work-papers and patents involved with older works are more likely to be disruptive. CD5 predicted from papers and patents increased by 2,072.4% and 58.4%, respectively, when the average age of pa cited works increased by 1 s.d. (about nine and eight years for papers and patents, respectively), again holding all other variables at their mean. In summary, the regression results show that changes in the use of prior knowledge can contribute to the production of less disruptive science and technology.
Reporting summary
More information about the research design is available in the Nature Portfolio Reporting Summary linked to this article.
Data availability
Data related to this study are freely available in the public repository at https://doi.org/10.5281/zenodo.7258379. Our study draws on data from six sources: American Physical Society, JSTOR, Microsoft Academic Graph, Patent View, PubMed and WoS. Data from Microsoft Academic Graph, Patent View and PubMed are publicly available, and our repository includes complete data for analysis from these sources. Data from the American Physical Society, JSTOR and WoS are not publicly available, and are used under license from their respective publishers. To facilitate replication, our repository includes a limited version of the data from this source, which will allow the calculation of basic descriptive statistics. The authors will provide full versions of these data upon request and permission from their respective publishers. Data sources are provided with this paper.
Code availability
Open-source code related to this study is available at https://doi.org/10.5281/zenodo.7258379 and http://www.cdindex.info. We use Python v.3.10.6 (pandas v.1.4.3, numpy v.1.23.1, matplotlib v.3.5.2, seaborn v.0.11.2, spacy v.2.2, jupyterlab v.3.4.4) wrangle , analyze and visualize data and to do statistical analysis. We use MariaDB v.10.6.4 to wrangle the data. We used Sundanese v.4.2.1 (ggplot2 v.3.36, ggrepel v.0.9.0) to visualize the data. We used StataMP v.17.0 (reghdfe v.5.7.3) to perform statistical analyses.
References
Fleming, L. Recombinant uncertainty in technological search. Manage. Sci. 47, 117-132 (2001).
article
Google Scholar
Schumpeter, J. Capitalism, Socialism and Democracy (Perennial, 1942).
Koyré, A. Robert Hooke’s unpublished letters to Isaac Newton. ISIS 43, 312–337 (1952).
article
MathSciNet
mathematics
Google Scholar
Popper, K. Conjectures and Refutations: The Growth of Scientific Knowledge (Routledge, 2014).
Fleck, L. Genesis and Development of a scientific fact (Univ of Chicago Press, 2012).
Acemoglu, D., Akcigit, U. & Kerr, W. R. Network Innovation. Proc. Natl Acad. Sci. AS 113, 11483–11488 (2016).
article
advertisement
charge
Google Scholar
Weitzman, M. L. Recombinant growth. Q. J. Econ. 113, 331–360 (1998).
article
MathSciNet
mathematics
Google Scholar
Tria, F., Loreto, V., Servedio, V. D. P. & Strogatz, S. H. Dynamics of correlated novelties. Sci. Rep. 4, 1-8 (2014).
Fink, T. M. A., Reeves, M., Palma, R& Farr, R. S. Serendipity and strategy in rapid innovation. Nat. Commune. 8, 1–9 (2017).
article
charge
Google Scholar
Pammolli, F., Magazzini, L. & Riccaboni, M. Productivity crisis in R & amp; D pharmaceutical. Nat. Rev. Drug Discov. 10, 428–438 (2011).
article
charge
Google Scholar
Bloom, N., Jones, C. I., Van Reenen, J. & Webb, M. Are ideas getting harder to find? Am. Econ. Revelation 110, 1104–1144 (2020).
article
Google Scholar
Funk, R. J. & Owen-Smith, J. A dynamic network measure of technological change. Manage. Sci. 63, 791-817 (2017).
article
Google Scholar
Bornmann, L., Devarakonda, S., Tekles, A. & Chacko, G. Is the disturbance index indicator convergently valid? Comparing several variants of indicators and assessments by peers. Amount. Sci. Stud. 1, 1242–1259 (2020).
article
Google Scholar
Uzzi, B., Mukherjee, S., Stringer, M. & Jones, B. Atypical combination and scientific impact. Science 342, 468-472 (2013).
article
advertisement
charge
Google Scholar
Leydesdorff, L., Tekles, A. & Bornmann, L. A proposal to revise the disturbance index. Prof. Inf. 30, e300121 (2021).
Lu, C. et al. Analyzing linguistic complexity and scientific impact. J. Informatr. 13, 817–829 (2019).
article
Google Scholar
Hofstra, B. et al. Paradox diversity-innovation in science. Proc. Natl Acad. Sci. AS 117, 9284–9291 (2020).
article
advertisement
charge
Google Scholar
Jones, B. F. The burden of knowledge and the ‘death of the renaissance man’: is innovation getting harder? Rev. Econ. Stud. 76, 283–317 (2009).
article
mathematics
Google Scholar
Gordon, R. J. The Rise and Fall of American Growth (Princeton Univ. Press, 2016).
Chu, J. S. G. & Evans, J. A. Slowing canonical progress in large fields of science. Proc. Natl Acad. Sci. US 118, e2021636118 (2021).
article
charge
Google Scholar
Packalen, M. & Bhattacharya, J. NIH funding and pursuit of cutting edge science. Proc. Natl Acad. Sci. US 117, 12011–12016 (2020).
article
advertisement
charge
Google Scholar
Jaffe, A. B. & Lerner, J. Innovation and its Discontents: How Our Broken Patent System Endangers Innovation and Progress, and What to Do About It (Princeton Univ Press, 2011).
Horgan, J. The End of Science: Confronting the Limits of Knowledge in the Twilight of the Scientific Age (Basic Books, 2015).
collision, P. & Nielsen, M. Science Gets Less Bang for its Buck (Atlantic, 2018).
Nolan, A. artificial intelligence and the future of science. oecd.ai, https://oecd.ai/en/wonk/ai-future-of-science (October 25, 2021).
Effective Policies for Fostering High-Risk/High-Reward Research. OECD Science, Technology, and Industry Policy Papers (OECD, 2021).
Cowen, T. The Great Stagnation: How America Ate All the Low-Hanging Fruit of Modern History, Got Sick, and Will (Eventually) Feel Better (Penguin, 2011).
Einstein, A. The World As I See It (Citadel Press, 1949).
Arthur, W. B. The structure of invention. Res. Policy 36, 274-287 (2007).
article
Google Scholar
Tushman, M. L. & Anderson, P. Technological discontinuities and the organizational environment. Adm. Sci. Q. 31, 439–465 (1986).
article
Google Scholar
Kohn, W. & Sham, LJ self-consistent equations including exchange and correlation effects. Phys. Rev. 140, A1133 (1965).
article
advertisement
MathSciNet
Google Scholar
Watson, J.D. & Crick, F. H. C. The molecular structure of nucleic acids: a structure for deoxyribose nucleic acid. Nature 171, 737–738 (1953).
article
advertisement
charge
Google Scholar
Bornmann, L. & Tekles, A. The disturbance index depends on the length of the quotation window. Prof. Inf. 28, e280207 (2019).
Wu, L., Wang, D. & Evans, J. A. Big teams thrive and small teams disrupt science and technology. Nature 566, 378–382 (2019).
article
advertisement
charge
Google Scholar
Kuhn, T. S. The Structure of Scientific Revolutions (Univ. Chicago Press, 1962).
Brad Wray, K. Kuhn and the discovery of paradigms. Philos. Soc. Sci. 41, 380-397 (2011).
article
Google Scholar
Ioannidis, J. P. A. Why most published research findings are false. PLoS Med. 2, e124 (2005).
article
Google Scholar
Li, J., Yin, Y., Fortunato, S. & Wang, D. A dataset of publication records for Nobel laureates. Sci. Data 6, 1–10 (2019).
article
Google Scholar
Bornmann, L. & Marx, W. Methods for the generation of normalized citation impact scores in bibliometrics: which method best reflects the judgments of experts? J. Informatr. 9, 408–418 (2015).
article
Google Scholar
Waltman, L. A literature review on reference impact indicators. J. Informatr. 10, 365-391 (2016).
Acknowledgements
article
Author information
Authors and Affiliations
Google Scholar
Waltman, L. & van Eck, N. J. in Springer Handbook of Science and Technology Indicators (eds. Glänzel, W. et al.) 281-300 (Springer, 2019).
Contributions
Bornmann, L. How can the impact of citation in bibliometrics be normalized? The new approach combines citation side normalization with citation percentile. Amount. Sci. Stud. 1, 1553–1569 (2020).
Corresponding author
article
Ethics declarations
Competing interests
Peer review
Peer review information
Google Scholar
Additional information
Petersen, A. M., Pan, R. K., Pammolli, F. & Fortunato, S. Methods to account for reference inflation in research evaluation. Res. Policy 48, 1855–1865 (2019).
Extended data figures and tables
Extended Data Fig. 1 Distribution of CD5.
article
Extended Data Fig. 2 CD index measured using alternative forward citation windows.
Extended Data Fig. 3 Diversity of language use in science and technology over time.
Google Scholar
Extended Data Fig. 4 Declining combinatorial novelty.
Bornmann, L. & Mutz, R. Growth rate of modern science: bibliometric analysis based on the number of publications and cited references. J. Assoc. Inf. Sci. Technology. 66, 2215–2222 (2015).
Extended Data Fig. 5 Contribution of field, year, and author effects.
article
Extended Data Fig. 6 CD index over time across data sources.
charge
Extended Data Fig. 7 Alternative measures of disruption.
Extended Data Fig. 8 Robustness to changes in publication, citation, and authorship practices.
Google Scholar
Extended Data Fig. 9 Growth of scientific and technological knowledge.
Bornmann, L., Haunschild, R& Mutz, R. Growth rate of modern science: latent piecewise growth curve approach to model publication numbers from established and new literature databases. Humanity. Soc. Sci. Commune. 8, 1–15 (2021).
Supplementary information
Rights and permissions
article
About this article
Cite this article
Google Scholar
