When AI starts to publish Nature, what can human scientists rely on to fight back?

Feb 26, 2025

In the journey of human exploration of the mysteries of nature, the innovation of scientific tools has always been the core driving force for breaking through the boundaries of cognition. The awarding of the 2024 Nobel Prize in Chemistry marks that artificial intelligence (AI) has officially entered the core stage of scientific research. The three winners have solved the problem that has plagued biology for half a century through AI-driven protein structure prediction and design, and achieved innovative protein design "from scratch". These discoveries not only deepen our understanding of life, but also provide practical solutions for the development of new drugs, vaccines and environmental protection technologies, and even solve global challenges such as antibiotic resistance and plastic degradation.

This milestone event has not yet dissipated its residual heat. The generative AI model MatterGen released by Microsoft in early 2025 has set off a storm in the field of materials: the new material TaCr₂O₆ generated by its reverse design has an experimental value of bulk modulus less than 20% error from the design target, shortening the research and development cycle of traditional materials from several years to several weeks. These breakthroughs reveal an irreversible trend: AI has evolved from an auxiliary tool for scientists to a "conspirator" in scientific discovery, and is reconstructing the underlying logic of scientific research.


> Inorganic materials designed by MatterGen. Source: @satyanadella


Facing this scientific revolution, researchers are facing unprecedented opportunities and challenges.

On the one hand, although many researchers have a solid professional knowledge background, they lack sufficient artificial intelligence knowledge and skills. They may feel confused and limited about the application of AI, and they don't know where to start to maximize the effectiveness of AI in specific scientific research tasks. On the other hand, the wet experimental methods relied on by many scientific research fields require high trial and error costs and a large number of repetitive experiments, coupled with the huge consumption of manpower and material resources, which also adds a lot of uncertainty to the scientific research process.

In this process, many seemingly useless data are abandoned, resulting in the failure to fully explore some potentially valuable information, resulting in a huge waste of resources. In the field of AI, data in the natural sciences and social sciences are often in short supply. Even if data is collected, it is inevitable that there will be problems of insufficient confidence or lack of interpretability (researchers will selectively select evaluation indicators when evaluating model performance). Especially in the application of large language models (LLMs), "fabrication" or misquotation often occurs, further deepening the doubts about the credibility of AI results.

The "black box" characteristics of AI technology also make many generated results lack transparency, and it is impossible to clearly explain the mechanism and logic behind them, which affects its trust and application depth in scientific research. More seriously, with the gradual development and popularization of AI technology, some tasks originally completed by human scientists have been gradually automated, and some positions are even at risk of being replaced. More and more scientific researchers are worried that the popularization of AI technology may lead to the weakening of human creative work. Ultimately, if the development direction of AI is not carefully grasped, this technological revolution may bring about profound changes in social structure, job market and scientific ethics.

This article provides some practical guidance and strategic suggestions for scientific researchers on how to cope with challenges and embrace artificial intelligence to help scientific research. This article will first clarify the key areas where artificial intelligence helps scientific research and explore the core elements required to achieve breakthroughs in these directions. It will then deeply analyze the common risks when using artificial intelligence in scientific research, especially the potential impact on scientific creativity and research reliability, and provide how to use artificial intelligence to bring the final overall net benefit through reasonable management and innovation. Finally, this article will propose three action guidelines to help scientific researchers actively embrace AI in this transformation and usher in a golden age of scientific exploration.


01 Key areas where AI helps scientific research


Changes in the way of acquiring, creating and disseminating knowledge

In order to make new breakthrough discoveries, scientists often have to face one increasingly huge peak of knowledge after another. Because new knowledge emerges in an endless stream and professional division of labor continues to deepen, the "knowledge burden" becomes heavier, resulting in the average age of scientists with significant innovation being older and older, and they are more inclined to interdisciplinary research, and are mostly rooted in top academic institutions. Even though small teams are generally more capable of promoting disruptive scientific ideas, the proportion of papers written by individuals or small teams is declining year by year. In terms of sharing scientific research results, most scientific papers are obscure and full of terminology, which not only hinders communication between researchers, but also makes it difficult to stimulate the interest of other researchers, the public, enterprises or policymakers in related scientific research.

However, with the development of artificial intelligence, especially large language models, we are discovering new ways to deal with the challenges in current scientific research. With the help of LLM-based scientific assistants, we can more efficiently extract the most relevant insights from massive literature, and directly ask questions about scientific research data, such as exploring the association between behavioral variables in the study. In this way, the tedious analysis, writing and review process is no longer the "must-go" to obtain new discoveries, and "extracting scientific discoveries from data" is also expected to significantly accelerate the scientific process.

With technological advances, especially by fine-tuning LLM on specific data in scientific fields, as well as breakthroughs in long context window processing capabilities and cross-literature citation analysis, scientists will be able to extract key information more efficiently, thereby significantly improving research efficiency. Going further, if a "meta-robot"-like assistant can be developed to integrate data across different research fields, it is expected to answer more complex questions and outline a panoramic view of disciplinary knowledge for humans. It is foreseeable that in the future, there may be researchers who design advanced queries specifically for these "paper robots" to push the boundaries of scientific exploration by intelligently splicing knowledge "fragments" from multiple fields.

Although these technologies provide unprecedented opportunities, they also come with certain risks. We need to rethink the nature of some scientific tasks, especially when scientists can rely on LLM to help critically analyze, adjust impact, or transform research into interactive papers, audio guides, etc. The definition of "reading" or "writing" scientific papers may change.


Generate, extract, annotate and create large scientific data sets

As scientific research data continues to increase, artificial intelligence is providing us with more and more help. For example, it can improve the accuracy of data collection and reduce errors and interference that may occur in processes such as DNA sequencing, cell type identification or animal sound collection. In addition, scientists can also use LLM's enhanced cross-image, video and audio analysis capabilities to extract hidden scientific data from more hidden resources such as scientific publications, archival materials and teaching videos, and convert them into structured databases for further analysis and use. Artificial intelligence can also add auxiliary information to scientific data to help scientists use this data better. For example, at least one-third of the functional details of microbial proteins cannot be reliably annotated. In 2022, DeepMind researchers used AI to predict protein functions, adding new entries to databases such as UniProt, Pfam, and InterPro.

When real data is not enough, verified AI models can also become an important source of synthetic scientific data. The AlphaProteo protein design model is trained based on more than 100 million AI protein structures generated by AlphaFold 2 and experimental structures from the Protein Database. These AI technologies can not only complement the existing scientific data generation process, but also significantly increase the returns of other scientific research efforts, such as archive digitization, or funding new data collection technologies and methods. Take the field of single-cell genomics as an example, huge single-cell data sets are being constructed with unprecedented precision to promote breakthroughs and progress in this field.


> Alphafold predicts a protein structure diagram, the bluer the color, the higher the credibility, and the redder the color, the lower the credibility. Image source: AFDB


Simulate, accelerate and provide information for complex experiments

Many scientific experiments are costly, complex and time-consuming, and some experiments cannot be carried out at all because researchers cannot obtain the necessary facilities, participants or inputs. Nuclear fusion is a typical example. Nuclear fusion promises to be a virtually inexhaustible, zero-emission energy source, and to scale up innovative technologies that consume high amounts of energy, such as desalination. To achieve nuclear fusion, scientists need to create and control plasma. However, the facilities required are extremely complex to build. Construction of the prototype tokamak reactor for the International Thermonuclear Experimental Reactor began in 2013, but plasma experiments will not begin until the mid-2030s at the earliest. Artificial intelligence can help simulate nuclear fusion experiments and make more efficient use of subsequent experimental time. Researchers can run reinforcement learning agents in simulations of physical systems to control the shape of plasma. Similar ideas can also be extended to large facilities such as particle accelerators, astronomical telescope arrays, or gravitational wave detectors.

Experiments simulated using artificial intelligence can vary greatly in different disciplines, but one thing in common is that these simulations usually provide information and guidance for physical experiments rather than replacing them. For example, the AlphaMissense model can classify 89% of 71 million potential human missense variants, helping scientists focus on those that may cause disease, thereby optimizing the allocation of experimental resources and improving research efficiency.


> The reactor chamber of DIII-D, an experimental tokamak fusion reactor operated by General Atomics in San Diego that has been used for research since its completion in the late 1980s. The typical doughnut-shaped chamber is covered with graphite to help it withstand the extreme heat. Image source: Wikipedia


Modeling complex systems and the interactions between their components

In a 1960 paper, Nobel Prize winner in physics Eugene Wigner marveled at the “unbelievable effectiveness” of mathematical equations in simulating natural phenomena such as the motion of planets. However, over the past half century, models that rely on systems of equations or other deterministic assumptions have struggled to fully capture the fast-changing dynamics and chaos of systems in biology, economics, weather, and other complex fields. The large number of components of these systems, their close interactions, and the random or chaotic behavior that can occur make it difficult for scientists to predict or control their responses in complex situations.

Artificial intelligence can improve modeling of these complex systems by acquiring more data about them and learning more powerful patterns and laws from them. For example, traditional numerical forecasts are mainly based on carefully defined physical equations, which have some explanatory power for atmospheric complexity, but are always inaccurate and computationally expensive. Deep learning-based forecasting systems can predict weather conditions 10 days in advance, outperforming traditional models in both accuracy and speed.

In many cases, AI does not replace traditional complex system modeling methods, but rather gives them richer tools. For example, agent-based modeling methods simulate interactions between individuals (such as companies and consumers) to study how these interactions affect larger and more complex systems such as the economy. Traditional methods require scientists to predefine the behavior of agents, such as "buy 10% less when you see a price increase" and "save 5% of your salary every month." However, the complexity of reality often makes these models "stretched to the limit" and difficult to accurately predict emerging phenomena (such as the impact of live streaming on the retail industry).

With the help of artificial intelligence, scientists can now create more flexible agents. These agents can communicate, take actions (such as searching for information or purchasing goods), and reason and remember these actions. Reinforcement learning can also allow these agents to learn and adapt in dynamic environments, and even adjust their behavior when faced with changes in energy prices or changes in epidemic policies. These new methods not only improve the flexibility and efficiency of simulations, but also provide scientists with more innovative tools to help them deal with increasingly complex research problems.


Finding innovative solutions to problems with vast search spaces

Many important scientific problems come with an astronomical number of potential solutions that are difficult to comprehend. When designing a small molecule drug, scientists have to screen from as many as 1060 possibilities; if designing a protein with 400 standard amino acids, the choice space is even as high as 1020400. Traditionally, scientists rely on a combination of intuition, trial and error, iteration, or brute force calculations to find the best molecule, proof, or algorithm. But these methods have difficulty exhausting the search space, and often miss the best solution. Artificial intelligence can open up new areas of these search spaces while focusing more quickly on the solutions that are most likely to be feasible and useful - this is a delicate balance.

Take the 2016 AlphaGo game against Lee Sedol as an example. The AI's placement of the pieces seemed unconventional and even beyond the traditional human chess path and experience, but it successfully disrupted Lee Sedol's thinking and made it easier for AlphaGo to control the situation. Lee Sedol later said that he was shocked by this move. This means that AlphaGo's move is completely beyond the thinking and experience of traditional human chess players. It also proves that AI can find solutions that humans have never thought of in a huge space of possibilities, thereby promoting strategic innovation.


> The second game between AlphaGo (black) and Lee Sedol (white), in which AlphaGo won the game. AlphaGo's 37th move landed on the fifth line, which exceeded the expectations of most chess players and experts. After the game, many people spoke highly of this move, believing that it showed AlphaGo's global judgment. Image source: Mustard Seed Viewing Xumi


02 Core Elements of AI-Driven Scientific Breakthroughs


Choice of Problems

As Heisenberg, the founder of quantum mechanics, said, "Asking the right question is often equivalent to solving most of the problem." So, how do we evaluate the quality of a problem? DeepMind CEO Demis Hassabis proposed a thinking model: If we regard the entire science as a knowledge tree, we should pay special attention to the roots of the tree - those basic "root node problems", solving these problems can unlock new research fields and applications. Second, to assess whether AI is applicable and can bring gains, we need to look for problems with specific characteristics, such as huge combinatorial search spaces, large amounts of data, and clear objective functions, to benchmark performance.

Often, a problem is theoretically suitable for AI, but because the input data is not yet in place, it may need to be put on hold and wait for the right time. In addition to choosing the right problem, specifying the difficulty level and feasibility of the problem is also crucial. AI's strong problem statement capabilities are often reflected in those that can produce intermediate results. If you choose a problem that is too difficult, you won't be able to generate enough signal to make progress. This requires relying on intuition and experimentation.


Choice of evaluation method

Scientists use a variety of evaluation methods, such as benchmarks, metrics, and competitions, to evaluate the scientific capabilities of AI models. Often, multiple evaluation methods are necessary. For example, weather forecast models start with an initial "progress measure" based on some key variables (such as surface temperature) to "climb" the model's performance. When the model reaches a certain level of performance, they use more than 1,300 metrics (inspired by the European Center for Medium-Range Weather Forecasts' evaluation scorecard) for a more comprehensive evaluation.

The AI ​​evaluation methods that have the most impact on science are often driven or recognized by the community. Community support also provides a basis for publishing benchmarks, which researchers can use, criticize, and improve. However, there is a hidden concern in this process: if the benchmark data is accidentally "absorbed" by the model's training process, the evaluation accuracy will be compromised. There is no perfect solution to this contradiction, but regularly launching new public benchmarks, establishing new third-party evaluation agencies, and holding various competitions are all feasible ways to continuously test and improve AI research capabilities.


Interdisciplinary collaboration

The application of artificial intelligence in science is often multidisciplinary by default, but to succeed, they need to truly transform into interdisciplinary collaboration. An effective start is to choose a scientific problem that requires a variety of expertise, and then provide it with enough time and energy to cultivate a team spirit of collaboration around the problem. For example, DeepMind's Ithaca project uses artificial intelligence to repair and classify damaged ancient Greek inscriptions to help scholars study the thoughts, languages, and history of past civilizations. To succeed, Yannis Assael, co-leader of the project, must understand epigraphy, the discipline of studying ancient written texts. The epigraphers in the project must learn how the AI ​​model works so that they can combine their professional intuition with the model output.

The development of this team spirit is inseparable from the right incentive mechanism. Empowering a small and close team to focus on solving problems rather than focusing on the authorship of the paper is the key to the success of AlphaFold 2. Such focus may be easier to achieve in industrial laboratories, but it also reiterates the importance of long-term public research funding, especially that such funding should not be too dependent on publication pressure.

Similarly, organizations need to create positions and career development paths for talents who can integrate different disciplines. For example, at Google, DeepMind's research engineers played a key role in promoting a benign interaction between research and engineering, and project managers helped to cultivate a teamwork atmosphere and promote communication and collaboration between teams. Those who can identify and connect the links between different disciplines and quickly improve their skills in new fields should be given more attention. In addition, to stimulate the exchange of ideas, organizations should encourage scientists and engineers to adjust projects regularly, establish a culture that promotes curiosity, humility and critical thinking, and enable practitioners from different fields to provide constructive opinions and feedback in open discussions.

Of course, building a partnership is never easy. When starting discussions, it is important to reach a consensus early on, clarify the overall goals, and resolve some thorny potential issues, such as the distribution of rights to the results among each party, whether to publish research, whether to open source models or datasets, and what type of license should be used. Disagreements are inevitable, but if public and private organizations with different incentive mechanisms can find clear and equal value exchange points, it is possible to achieve success together while fully leveraging their respective strengths.


The architecture of Ithaca. Damaged parts of the text are represented by a hyphen "-". In this case, the project team artificially damaged the character "δημ". Based on these inputs, Ithaca is able to recover the text and identify the time and place where the text was written. Image source: deepmind


03 Manage AI risks and improve scientific creativity and research reliability

A 2023 in-depth survey report published by Nature shows that 62% of scientific research teams around the world have used machine learning tools in data analysis, but 38% of these studies lack sufficient evidence for algorithm selection. This universality warns us that while AI is reshaping the scientific research paradigm, it is also creating new cognitive traps.

Although AI can help us extract useful rules from massive amounts of information, it often makes deductions based on existing data and knowledge, rather than creative thinking from a completely new perspective. This "imitation" innovation may make scientific research increasingly dependent on existing data and models, thereby limiting the breadth of scientific researchers' thinking. In the case of over-reliance on AI, we may overlook some original and non-traditional research methods that may open up new scientific fields. Especially when exploring unknown and cutting-edge fields, human intuition and independent thinking are still crucial.

In addition to the impact on scientific creativity, the popularity of AI may also pose a hidden danger to the reliability and understanding of research. When providing predictions and analysis, AI is often based on probability and pattern recognition rather than direct causal reasoning. Therefore, the conclusions given by AI may only be a statistical correlation, not necessarily a true causal relationship. In addition, the "black box" nature of AI algorithms also makes their decision-making process opaque. Therefore, it is crucial for researchers to understand the logic behind the conclusions drawn by AI, especially when the results need to be interpreted or applied to practical problems. If we blindly accept the results of AI without examining them, it may lead to misleading conclusions, which in turn affects the credibility of the research.

On the other hand, we believe that if the risks of AI can be properly managed, there will be an opportunity to deeply integrate this technology into scientific exploration, help address more challenges at all levels, and even bring far-reaching impacts.


Creativity

Scientific creativity refers to the ability of individuals or teams to propose novel hypotheses, theories, methods or solutions to promote scientific progress through unique thinking, methodology or perspective in scientific research. In practice, scientists usually judge whether a new idea, method or result is creative based on some subjective factors, such as its simplicity, counterintuitiveness or beauty. Some scientists worry that the large-scale use of AI may weaken the more intuitive, accidental discovery or eclectic research methods in science. This problem may manifest itself in different ways.

One concern is that AI models are trained to minimize outliers in training data, while scientists often respond to confusing data points by following their intuition and amplifying outliers. Another concern is that AI systems are usually trained to complete specific tasks, and relying on AI may miss more accidental breakthroughs, such as solutions to problems that have not been studied. At the community level, some people worry that if scientists embrace AI on a large scale, it may lead to the gradual homogenization of research results. After all, large language models may produce similar suggestions when responding to questions from different scientists, or scientists will over-focus on those disciplines and problems that are best suited to AI.

To mitigate such risks, researchers can flexibly adjust AI usage strategies while ensuring the depth of exploratory research. For example, by fine-tuning large language models to enable them to provide more personalized research ideas, or helping scientists better trigger their own ideas.

AI can also promote some scientific creativity that may not otherwise appear. One type of AI creativity is interpolation, that is, AI systems identify new ideas in their training data, especially when human capabilities are limited. For example, using AI to detect outliers in large data sets from the Large Hadron Collider experiment.

The second is extrapolation, in which the AI ​​model is able to generalize knowledge beyond its training data and come up with more innovative solutions.

The third is invention, in which the AI ​​system proposes a completely new theory or scientific system that is completely independent of its training data, similar to the initial development of general relativity or the creation of complex numbers. Although AI systems have not yet demonstrated such creativity, new methods are expected to unlock this ability, such as multi-agent systems that optimize for different goals (such as novelty and counterintuitiveness), or scientific problem generation models that are specifically trained to generate new scientific problems and inspire innovative solutions.


Reliability

Reliability refers to the degree of trust that scientists have when they rely on each other's research results. They need to ensure that these results are not accidental or wrong. There are currently bad practices in artificial intelligence research, and researchers should be highly vigilant when conducting scientific research. For example, researchers choose the criteria used to evaluate model performance based on their own preferences, and AI models, especially LLMs, are also prone to "hallucinations" that produce "illusions", that is, false or misleading outputs, including scientific citations. LLMs may also lead to a large number of low-quality papers that are similar to the works produced by "paper mills".

To address these issues, there are already some solutions, including developing checklists of good practices for researchers to follow, and different types of AI factual research, such as training AI models to align their outputs with trusted sources or helping to verify the outputs of other AI models.

On the other hand, researchers can also use AI to improve the reliability of the broader research base. For example, if AI can help automate parts of the process of data annotation or experimental design, this will provide much-needed standardization in these fields. As AI models continue to improve their ability to align their outputs with the reference literature, they can also help scientists and policymakers review the evidence base more systematically. Researchers can also use AI to help detect false or fake images or identify misleading scientific claims, such as the recent trial of an AI image analysis tool by the journal Science. AI may even play a role in peer review, especially considering that some scientists have used LLMs to help review their own papers and verify the outputs of AI models.


Explainability

In a recent survey by Nature, scientists believe that the biggest risk of using AI for scientific research is the reliance on pattern matching at the expense of deep understanding. One of the concerns about AI's potential to undermine scientific understanding is the questioning of the "theory-free" nature of modern deep learning methods. They do not contain or provide theoretical explanations for the phenomena they predict. Scientists also worry about the "uninterpretability" of AI models, that is, they are not based on explicit equations and parameter sets. Others worry that any way to explain the output of AI models will not be useful or easy to understand for scientists. After all, AI models may provide protein structures or weather predictions, but they may not tell us why proteins fold in a certain way, or how atmospheric dynamics lead to climate change.

In fact, people's concerns about "replacing "real theoretical science" with low-level calculations" are not new. Past techniques, such as Monte Carlo methods, have been similarly criticized. Fields that combine engineering and science, such as synthetic biology, have also been accused of prioritizing useful applications over deep scientific understanding. But history has shown that these methods and techniques ultimately promote the development of scientific understanding. Moreover, most AI models are not truly "theory-free." They usually build data sets and evaluation criteria based on prior knowledge, and some have a certain degree of interpretability.

Today's interpretability techniques are constantly developing, and researchers try to understand the inference logic of AI by identifying the "concepts" or internal structures learned in the model. Although these interpretability techniques have many limitations, they have enabled scientists to derive new scientific hypotheses from AI models. For example, studies have been able to predict the relative contribution of each base in a DNA sequence to the binding of different transcription factors and explain this result using concepts familiar to biologists. In addition, the "superhuman" strategy learned by AlphaZero when playing chess can be taught to human chess players after being parsed by another AI system. This means that the "new concepts" learned by AI may be able to feed back to human cognition.

Even without interpretability techniques, AI can improve scientific understanding by opening up new research directions that would not otherwise be possible. For example, by unlocking the ability to generate a large number of synthetic protein structures, AlphaFold enables scientists to search across protein structures, not just protein sequences. This approach was used to discover an ancient member of the Cas13 protein family that has potential in RNA editing, especially in helping to diagnose and treat diseases. This discovery also challenges previous assumptions about the evolution of Cas13. In contrast, attempts to modify the AlphaFold model architecture to incorporate more prior knowledge resulted in reduced performance. This highlights the trade-off between accuracy and interpretability. The "fuzziness" of AI comes from their ability to operate in high-dimensional spaces, which may be incomprehensible to humans, but are necessary for scientific breakthroughs.


04 Conclusion: Seize the opportunity, an action plan for AI to empower scientific research

Obviously, the potential of science and artificial intelligence in accelerating the scientific process should be highly valued by researchers. So, where should researchers start? In order to make full use of AI-driven scientific opportunities, it is necessary to embrace change with an active attitude. Perhaps there are some suggestions that can be adopted.

First, master the language of AI tools, such as understanding the technical principles of generative models and reinforcement learning, and skillfully use open source code libraries for customized exploration; secondly, build a closed loop of data and experiments, and quickly verify the results generated by AI through automated laboratories (such as A-Lab at the University of California, Berkeley), forming an iterative link of "hypothesis-generation-verification"; more importantly, reshape the imagination of scientific research-when AI can design proteins or superconductors beyond the scope of human experience, scientists should turn to more essential scientific problems, such as using AI to reveal the hidden variable relationship between material properties and microstructures, or explore the coupling mechanism of multi-scale cross-physical fields. As Nobel Prize winner David Baker said: "AI does not replace scientists, but gives us a ladder to touch the unknown." In this exploration of human-machine collaboration, only by deeply integrating human creative thinking with the computational violence of AI can we truly unleash the infinite possibilities of scientific discovery.


The picture is from the Internet.
If there is any infringement, please contact the platform to delete it.