Will AI help or hinder trust in science?

By means of
Jon Whittle

April 23, 2024
6 minutes reading

In the past year, generative artificial intelligence tools have emerged, such as ChatGPTTwinand Sora, OpenAI’s video generation tool – have captured the public’s imagination.

That’s all it takes to start experimenting with AI is an internet connection and a web browser. You can interact with AI as you would with a human assistant: by talking to it, writing to it, showing images or videos, or all of the above.

While this capability marks entirely new territory for the general public, scientists have been using AI as a tool for years.

But with greater public knowledge of AI will also come greater public scrutiny of how it is used by scientists.

AI is already revolutionizing science: six percent of all scientific work uses AI, not only in computer science, but also in chemistry, physics, psychology and environmental sciences.

Nature, one of the world’s most prestigious scientific journals, has included ChatGPT in its 2023 Nature’s 10 list of the most influential and, until then, exclusively human scientists in the world.

The use of AI in science is twofold.

At some level, AI can make scientists more productive.

When Google DeepMind released an AI-generated dataset of more than 380,000 new material compounds, Lawrence Berkeley Lab used AI to conduct compound synthesis experiments on a scale greater than what could be achieved by humans.

But AI has even greater potential: enabling scientists to make discoveries that would otherwise not be possible at all.

It was an AI algorithm that first found signal patterns in brain activity data that indicated the onset of seizures, a feat even the most experienced human neurologist cannot repeat.

Early success stories of using AI in science have led some to imagine a future where scientists will collaborate with research assistants on AI as part of their daily work.

That future is already here. CSIRO researchers are experimenting with AI scientists and have developed robots that can follow spoken language instructions to perform scientific tasks during field work.

While modern AI systems are impressively powerful – especially the so-called artificial general intelligence tools like ChatGPT and Gemini — they also have disadvantages.

Generative AI systems are prone to ‘hallucinations’‘where they make up facts.

Or they could be biased. Google’s Gemini shows America’s Founding Fathers as a diverse group is an interesting case of overcorrection for bias.

There is a very real danger of AI fabricating results and this has already happened. It’s relatively easy to get a generative AI tool that cites publications that don’t exist.

Furthermore, many AI systems cannot explain why they produce the output they do.

This is not always a problem. If AI generates a new hypothesis that is then tested using the usual scientific methods, no harm can be done.

However, for some applications a lack of explanation can be a problem.

Replication of results is a basic principle in science, but if the steps AI took to reach a conclusion remain opaque, replication and validation become difficult, if not impossible.

And that could damage people’s trust in the science produced.

A distinction must be made between general and narrow AI.

Narrow AI is AI trained to perform a specific task.

Narrow AI has already made great strides. AlphaFold from Google DeepMind This model has revolutionized the way scientists predict protein structures.

But there are also many other less well-publicized successes – such as AI being used at CSIRO to discover new galaxies in the night sky, and IBM Research developing AI that rediscovers Kepler’s third law of planetary motion.or Samsung AI building AI capable of reproducing Nobel Prize-winning scientific breakthroughs.

When it comes to narrow AI applied to science, trust remains high.

AI systems – especially those based on machine learning methods – rarely achieve 100 percent accuracy on any given task. (In fact, machine learning systems perform better than humans on some tasks, and humans perform better than AI systems on many tasks. People using AI systems generally perform better than people working alone, and they also perform better than AI if they work alone. There is a great scientific basis for this fact, including this study.)

AI working with an expert scientist, who confirms and interprets the results, is a perfectly legitimate way of working and is widely seen because it provides better performance than human scientists or AI systems working alone.

On the other hand, general AI systems are trained to perform a wide range of tasks, which are not specific to any particular domain or use case.

For example, ChatGPT can create a Shakespearean sonnet, suggest a recipe for dinner, summarize a range of academic literature, or generate a scientific hypothesis.

When it comes to general AI, the problems of hallucinations and bias are most acute and widespread. That doesn’t mean that general AI isn’t useful to scientists, but that it should be used with caution.

This means scientists need to understand and assess the risks of using AI in a specific scenario, and weigh these against the risks of not doing so.

Scientists now routinely use general AI systems to write papershelp review academic literature and even prepare experimental plans.

A danger could arise when it comes to these scientific assistants if the human scientist takes the outcomes for granted.

Well-trained, diligent scientists obviously won’t do this. But many scientists are just trying to survive in a tough industry of publish or perish. Scientific fraud is already increasingeven without AI.

AI could lead to new levels of scientific misconduct – either through deliberate misuse of the technology or through sheer ignorance, as scientists don’t realize that AI is making things up.

Both narrow and general AI have great potential to advance scientific discovery.

A typical scientific workflow conceptually consists of three phases: understanding which problem to focus on, running experiments related to that problem, and exploiting the results for real-world impact.

AI can help in all three of these phases.

However, there is a major caveat. Current AI tools are not suitable for being used naively out-of-the-box for serious scientific work.

Only if researchers responsibly design, build, and deploy the next generation of AI tools to support the scientific method will public trust in both AI and science be earned and maintained.

It’s worth getting this right: the possibilities for using AI to transform science are endless.

The iconic founder of Google DeepMind, Demis Hassabis, famously said it: “Building increasingly capable and general AI, safe and responsible, requires us to solve some of the toughest scientific and engineering challenges of our time.”

The converse conclusion is also true: Solving the toughest scientific challenges of our time requires building increasingly capable, secure, and responsible general AI.

Australian scientists are working on it.

This article was originally published by 360info under a Creative Commons license. Read the original article.

Professor Jon Whittle is Director of CSIRO’s Data61, Australia’s national center for R&D in data science and digital technologies. He is co-author of the book ‘Responsible AI: Best Practices for Creating Trustworthy AI Systems’.

Dr. Stefan Harrer is Program Director of AI for Science at CSIRO’s Data61 and leads a global innovation, research and commercialization program aimed at accelerating scientific discovery through the use of AI. He is the author of the Lancet article ‘Attention isn’t all you need: the complicated case of ethically using large language models in healthcare and medicine’.

Stefan Harrer is an inventor on several issued US and international patents related to the use of AI for science.