‘Godfather of AI’ now fears it’s unsafe. He has a plan to rein it in

The US Federal Bureau of Investigation recently disclosed that two individuals suspected of bombing a fertility clinic in California last month allegedly utilized artificial intelligence (AI) to acquire bomb-making instructions. The FBI did not reveal the identity of the AI program involved. This underscores the critical necessity to enhance the safety of AI. We currently find ourselves in an era comparable to the “wild west” of AI, where companies are fiercely competing to create the quickest and most captivating AI systems. Each company strives to surpass rivals and secure the top position. This intense competition frequently results in deliberate or unintentional shortcuts, particularly in terms of safety.

Around the same time as the FBI’s revelation, Canadian computer science professor Yoshua Bengio, a prominent figure in modern AI, established a nonprofit organization dedicated to constructing a new AI model expressly designed to be safer than existing models, focusing on those that cause societal harm. So, what exactly is Bengio’s new AI model, and can it genuinely shield the world from AI-related harm?

In 2018, Bengio, together with his colleagues Yann LeCun and Geoffrey Hinton, received the Turing Award for their groundbreaking research on deep learning. Deep learning, a subset of machine learning, seeks to imitate human brain processes by employing artificial neural networks to learn from computational data and make predictions.

Bengio’s nonprofit organization, LawZero, is developing “Scientist AI.” Bengio has indicated that this model will be “honest and non-deceptive,” integrating safety-by-design principles. According to a preliminary paper published online earlier this year, Scientist AI will diverge from current AI systems in two crucial ways. Firstly, it can evaluate and express its confidence level in responses, assisting in mitigating the issue of AI providing overly confident and inaccurate answers. Secondly, it can elucidate its reasoning to humans, enabling the evaluation and verification of its conclusions for accuracy.

Bengio also aims for “Scientist AI” to function as a safeguard against unsafe AI, potentially supervising other less dependable and harmful AI systems – essentially combating fire with fire. This strategy could be the only viable approach to enhance AI safety since humans are unable to effectively oversee systems like ChatGPT, which handle millions of queries daily. Only another AI system can manage such a scale.

Bengio’s team is also incorporating a “world model” into Scientist AI to provide certainty and explainability. Just as humans base decisions on their comprehension of the world, AI requires a comparable model to operate effectively. The absence of a world model in current AI systems is evident, leading to issues such as the “hand problem” and inefficiencies in tasks like chess.

Bengio’s pursuit of constructing safer, more reliable AI by merging large language models with other AI technologies is commendable. Nevertheless, this endeavor is anticipated to face challenges due to the comparatively modest funding of US$30 million allocated to LawZero, the dominance of major tech companies controlling most data essential for AI projects, and uncertainties regarding how Scientist AI will manage other harmful systems.

In spite of the obstacles, this project, supported by skilled researchers, has the potential to catalyze a movement toward a future where AI genuinely benefits human well-being. If successful, it could establish new standards for secure AI, motivating researchers, developers, and policymakers to prioritize safety. Such proactive measures, if taken during the emergence of social media, might have fostered a safer online environment for mental health, especially among young individuals. Had Scientist AI been operational earlier, it might have deterred individuals with malicious intents from accessing harmful information through AI systems.