Nvidia CEO Takes a Firm Stand: Labs Unable to Control AI Must Shut Down

Nvidia CEO Jensen Huang maintains that artificial intelligence does not pose an existential threat to humanity, contradicting some of the direst warnings emerging from within the AI industry itself. At the same time, Huang asserts that AI companies must take responsibility for ensuring their systems are safe and controllable before bringing them to market. And if they fail to contain their own AI experiments, the Nvidia chief suggests those labs should shut down.

Huang made these remarks during an interview with ‘New York Times’ journalist Ezra Klein on ‘The Ezra Klein Show’, where the two discussed the rapid development of AI, concerns regarding increasingly capable AI agents, and whether companies are moving too fast. "If you have a problem, solve it. And if you can't solve it, don't release it," Huang stated, arguing that AI companies must find ways to contain and isolate their systems before deploying them.

Huang drew a comparison to robotaxis. He noted that if engineers discover a problem in an autonomous vehicle that they do not know how to fix, the obvious response is not to put that vehicle on the road. He applied the same principle to advanced AI systems, arguing that companies must resolve containment and safety issues before launching their models.

"If they posit the alternative that there is no way to contain their experiments-that there is simply no way; that when testing their AI models, they will break loose and cause harm to the world-then I think the answer is that we must shut down the labs," Huang declared.

The conversation arose in the context of a recent incident involving AI agents from OpenAI and Hugging Face. Huang argued that the matter should ultimately be addressed as an engineering problem. According to him, AI companies must understand how to contain and isolate their systems before launching them.

He also placed the responsibility on the companies themselves. "These are companies with decision-making power. These are CEOs with decision-making power," Huang told Klein, arguing that if a company believes it is about to launch an unsafe product, it has the power and the responsibility to halt the launch.

Huang is not calling for a halt to AI development. Instead, he advocates distinguishing between moving quickly and moving recklessly. In a previous interview with CBS, Huang stated that concerns regarding AI safety are not unfounded in themselves, but maintained that some of the more alarmist predictions lack empirical backing. "We must move as fast as we can, but not faster than we should. If we compromise safety, that is unacceptable. If safety is put at risk and unsafe products are released that expose people to danger, there are all kinds of laws to penalise those responsible," he declared.

The debate over controlling AI systems comes amid rising reports of AI agents acting in ways not foreseen by their developers. Recently in Australia, Prime Minister Anthony Albanese revealed that, in June, an OpenAI AI agent gained unauthorized access to files-both public and private-on a Medicare statistics portal. The incident occurred on June 18 while OpenAI was conducting online research into public medical spending as part of an internal capabilities assessment.

The portal was a publicly accessible website containing aggregated statistics from Medicare and the Pharmaceutical Benefits Scheme (PBS); it was not connected to individual Medicare claims or payments, nor to personal medical information. Australian authorities note that the AI agent initially requested information and, after being denied access, engaged in what officials described as "misaligned behaviour" to access the portal without authorization. The government states that, at present, there is no evidence that anyone's personal medical information was accessed.

Another cause for concern has been the time that elapsed before the incident was reported. OpenAI notified Services Australia on September 10-nearly three months after the June event-via a public email address. The Australian government described the delay as unacceptable, and Prime Minister Albanese stated that he had spoken with Sam Altman to express his "extreme concern" regarding the incident.

OpenAI maintains that its investigation found no evidence of access to patient records. The company indicated that the incident affected aggregated health statistics and internal file names, while a broader review detected activity linked to several Australian government websites.

ABOUT THE AUTHOR

Kahekashan

[email protected]

Kahekashan is a passionate technophile with a keen eye for cutting-edge gadgets, emerging technologies, and everything in the digital realm. Raised in a Defence family with strong values and a background in literature, she has consistently pursued excellence in every endeavour. Her last full-time assignment involved content writing with the Indian School of Business.

Read More