A dataset of AI in natural science research across 41.3 million papers

Dataset for “Artificial Intelligence Tools Expand Scientists’ Impact but Contract Science’s Focus” Background and Summary Recent advances in artificial intelligence (AI) have profound implications across multiple societal domains, ranging from finance, health to transportation, and social governance. Notably, the knowledge production process in the scientific research domain also deeply interweaves with AI. On the one hand, AI systems empowered by powerful algorithms and massive computing capabilities exhibit immense potential in incorporating existing knowledge and providing innovative solutions for unexplored scientific problems. For instance, DeepMind's AI techniques learn the patterns of millions of different knots and then serve as a guide for mathematicians' intuition, helping the discovery of new patterns worthy of investigation. In addition, AlphaFold revolutionizes structural biology research by accurately predicting 3D proteins structure, circumventing the need for costly conventional experimental techniques like X-ray crystallography and cryo-electron microscopy. On the other hand, generative AI possesses remarkable capability in natural language reasoning, greatly facilitating researchers in the retrieval of background literature and the compilation of their scientific findings. These transformative capabilities of AI in scientific research have induced structural change to the conventional paradigm of knowledge production, raising broadly concerned questions within the scientific community: how will AI shape the future of scientific research, and how should scientific participants seize opportunities in the co-evolution trend of AI and conventional science, promoting sustainable science development. Centered around the above questions, the impact of AI on scientific research has garnered enduring scholarly interest. However, existing studies confront the lack of large-scale dataset of AI usage in scientific research encompassing multiple disciplines, which is essential to facilitate systematic quantitative and statistical analyses. For example, previous researchers engaged in discussions regarding the potential substitution of human researchers by AI. While their work provided valuable perspectives, it lacked the support of quantitative evidence. Furthermore, an observational study investigated the potential social implications of AI research. Nevertheless, they only identified several AI-related research fields within the discipline of computer science based on the citation graph, without distinguishing AI usage across multiple disciplines on the fine-grained level of individual research papers. To bridge the data gap preventing researchers from better understanding the impact of AI on scientific research, we identify AI usage in 41,298,433 natural science research papers across four decades (1980-2025) and six scientific disciplines, namely biology, medicine, chemistry, physics, materials science, and geology.We extract the meta information of these papers from the OpenAlex dataset and design a two-stage fine-tuning process to adapt a pre-train language model to efficiently identify the usage of AI according to their titles and abstracts. To verify the accuracy of our identification, we enlist experts with substantial AI knowledge to scrutinize the results. In validation with expert-labeled data, the language model attains an F1-score of 0.875, confirming the reliability of our identification results. In addition to AI usage identification on the level of individual research papers, we also calculate the average probability for AI usage in 132,706 venues (journals and conferences) at the aggregated level. In summary, we present a large-scale dataset that includes the title, publication date, primary venue, and AI probability of papers, as well as the name, number of papers, and AI probability of venues. Among the included research papers, 310,957 are identified as having an AI adoption probability greater than 0.5, while 1,043 venues exhibit an average AI probability above 0.5. In addition, we showcase how our dataset enables investigations on the impact of AI on scientific innovation from the perspectives of research papers, research careers, research teams, etc. Therefore, our dataset holds great value in empowering the community to deeply understand and fully unleash the potential of AI in advancing scientific development. Data Records To facilitate data access and utilization, we organize the dataset into two files, i.e., “Paper information” and “Venue information”, where we show the details in the following table. File Format Size Brief description Paper information Table 41,298,433 lines Tabular file containing the IDs, titles, publishing dates, primary publication venues and AI usage identifications for papers Venue information Table 132,706 lines Tabular file containing the IDs, names, total paper numbers and probabilities for AI usage for venues “Paper information” contains 41,298,433 lines, where each line includes “PaperID”, “Title”, “PublishDate”, “PrimaryVenue” and “AIProb” for one research paper. “PaperID” refers to the unique ID of each paper in the OpenAlex database (the “https://openalex.org/W” prefix is omitted); “Title” and “PublishDate” correspond to the title and publication date of the paper; “PrimaryVenue” indicates the primary publication venue of the paper; and “AIProb” is our identified probability of AI usage in the paper. To reduce file sizes, we only include IDs of the primary venues rather than using the textual names. Index Type Brief description PaperID Integer Unique ID of the paper (the “https://openalex.org/W” prefix is omitted) Title String Title of the paper PublishDate String Publication date of the paper (YYYY-MM-DD) PrimaryVenue Integer ID of the primary publication venue of the paper AIProb Float Probability of AI usage in the paper On the other hand, “Venue information” contains 132,706 lines, where each line includes “VenueID”, “Name”, “NumberOfPapers” and “AvgAIProb” for one venue. “VenueID” refers to the unique ID of each venue in the OpenAlex database (the “https://openalex.org/S” prefix is omitted); “Name” is the formal texual name of the venue; “NumberOfPapers” corresponds to the total number of papers published in the venue; and “AvgAIProb” is the average probability for AI usage in papers published within the venue. Index Type Brief description VenueID Integer Unique ID of the venue (the “https://openalex.org/S” prefix is omitted) Name String Name of the venue NumberOfPapers Integer Total number of papers published in the venue AvgAIProb Float Average probability of AI usage in the papers within the venue We provide some examples for “Paper information”. For instance, paper NO. 3040660900, which is entitled “TE and TM guided modes in an air waveguide with negative-index-material cladding”, is published on 2005-04-05 in Physical Review E (venue ID: 35412551). Through our identification, the probability for this paper to use AI is almost zero. On the contrary, paper NO. 4327550249, namely “Evolutionary-scale prediction of atomic-level protein structure with a language model”, which is published on 2023-03-16 in Science (venue ID: 3880285), applied language models in protein structure prediction, and thereby is obviously identified to incorporate AI usage with probability higher than 95%. PaperID Title PublishDate PrimaryVenue AIProb 3040660900 TE and TM guided modes in an air wave guide with negative-index-materia lcladding 2005-04-05 35412551 0.000021 2109728051 An asymmetric model for XPS analysis 1982-06-01 114406758 0.000168 3177828909 Highly accurate protein structure prediction with AlphaFold 2021-07-15 137773608 0.889659 4327550249 Evolutionary-scale prediction of atomic-level protein structure with a language model 2023-03-16 3880285 0.950544 We also provide some cases of “Venue information”. For example, Nature (venue ID: 137773608) has 2794 recorded paper in total, where the average probability for AI usage among these papers is 0.023346. In contrast, the average probability of using AI in papers in Journal of Machine Learning Research (venue ID: 118988714) is above 90%, which is aligned with the specific scope of this journal. VenueID Name NumberOfPapers AvgAIProb 24807848 Physical Review Letters 116013 0.005508 3880285 Science 77478 0.015141 137773608 Nature 2794 0.023346 2607323502 Scientific Data 4066 0.075611 118988714 Journal of Machine Learning Research 404 0.931733 Usage Note We store our produced dataset in the form of binary-stream files, and we recommend researchers use the following Python packages for further processing of the dataset. The native pickle package in Python can be used to load the binary-stream files with the parameter “rb”, for which we provide an example program for data loading in our GitHub repository. The Pandas Library can be used to efficiently process the tabular data; the Numpy Library is suitable for large-scale statistical calculations; and the Matplotlib Library can generate statistical graphs in various styles. We keep the paper IDs and venue IDs in our dataset identical to the unique IDs in the original OpenAlex database. Therefore, besides the information we provided in “Paper information” and “Venue information”, researchers can further link our dataset with the OpenAlex database via the IDs and retrieve more information about the papers and venues. Thereby, our dataset supports extensive joint science of science analyses among various features of research papers and the usage of AI, paving the road toward a comprehensive understanding of the impact of AI on scientific research. Possible directions where this data can be applied include: Textual content. The textual content of research papers has long been investigated by the science of science research. Previous works have measured the cognitive extent of papers by evaluating the size of their vocabulary, and explored how the frequency of commonly used verbs reveals changes in the focus of scientific innovation over time. With our dataset, we examine how the adoption of AI in natural science research impacts the focus of research topics. We use a text embedding model to project the content of papers into a high-dimensional vector space and measure the spatial extent of both AI and non-AI research. Our comparison indicates that the focus of AI papers appears to be narrower across the natural science domain. This illustrates the potential of our dataset to reveal the evolving patterns of knowledge extent in scientific innovation as AI methods continue to be widely adopted and applied. Citation network. The citation network is a key object of analysis in the science of science. Previous works have computed the disruption score of papers on the citation network and analyzed how existing academic outcomes influence subsequent research. More specifically, another study has analyzed the relation of AI research to other fields on the citation network, revealing the increasing disconnect between AI and other areas. However, this study only identified a limited number of AI papers within specific topics based on keyword recognition. With our dataset, similar and further analyses can be generalized across the entire natural science domain, thereby providing a deeper understanding of how the adoption of AI methods in scientific innovation changes the ways of knowledge propagation on the citation network. Team innovation. The research team is the fundamental unit of scientific innovation. Prior studies have examined various characteristics of research teams, such as team size, member relationships, and team structure, and their influence on scientific outcomes. Using our dataset, we compare the composition of research teams for AI and non-AI papers. The results show that AI papers tend to employ smaller research teams, and the teams involve a lower proportion of junior researchers on average. Following this, our dataset enables further investigation into how the adoption of AI affects the roles of human scientists within research teams and transforms patterns of team collaboration. Researchers’ careers. Career development of researchers is a key topic in science of science research. Previous investigations have observed the phenomenon of a “temporary workforce” in scientific research, revealing the sharply declining career longevity of researchers. Using our dataset, we find that, compared to researchers in conventional fields, AI-adopted researchers are associated with greater opportunities to lead research teams, which in turn benefits their career development. This finding underscores the value of our dataset in understanding the impact of AI on researchers’ careers and provides insights for policies aimed at fostering the next generation of scientific talents and promoting sustainable scientific development. Moreover, as we illustrated, our method produces an identification model with generalization capability over AI methods in different eras. Therefore, future researchers can utilize our method to efficiently identify the usage of AI among more research papers beyond the extent of the currently provided dataset, enabling more extensive analyses of the relationship between AI and scientific development.