India needs to work on mitigating risks from rogue AI agents

A recent development in the AI space has created shockwaves. Last month, OpenAI’s AI agents went rogue and hacked multiple US government websites. Importantly, this took place without the knowledge of the OpenAI’s engineers. Reportedly, the OpenAI agents behaved unusually with the US Education Department, Commerce Department, and Securities and Exchange Commission (SEC) during this incident.

Though OpenAI is investigating into the incident, it is now amply clear that in several instances the makers of foundational models don’t have any control over the AI agents that they are creating. These agents are behaving autonomously in many instances. Such events underscore the importance of control over AI companies. However, the question comes when the companies, which create those AI agents, don’t have control over their creations, how will they be controlled?

That is the reason the world recently saw all the AI giants coming together at the White House and signed what they call as ‘White House Accord on Super Intelligence’.

The agreement was signed by executives at Meta, Nvidia, Google, OpenAI, xAI and

Anthropic and called on companies to put in place guardrails internally to help monitor security risks. There are four layers of control according to the White House Accord.

Firstly, the companies will maintain internal controls, then establish an internal team to oversee these controls, and thirdly work with independent outside auditors. At the last step, they will create an independent board committee to review the results. Now, the question arises will the accord, which puts emphasis on peer review and self-control, give desired output?

The answer to this question is not known to many. And there are several reasons to this. Firstly, all the signatories of the White House Accord are intense competitors. They have poured in billions for creating the foundational models and are in a race to grab the maximum market share in the AI space. Moreover, many of them have ego-clashes that is not so secret to the world.

For instance, xAI’s Founder- Elon Musk and OpenAI’s Sam Altman frequently involved in arguments over issues relating to AI development. So, to expect, these companies will cooperate for peer reviews seem difficult. Secondly, when billions are at stake, business obligations are likely to dilute the risks arising from AI agents. Thirdly, the US is locked with an intense competition with the China to dominate the world in the AI space.

In this regard, any control that throttles the innovation in the AI world, will not be acceptable to the US. So, scepticism is reasonable. However, there is also a glimmer of hope. More founders of AI giants across the world- in the US, China, India, & others- have started to realise the real risks arising from human-level intelligence of the AI agents. They understand that any mistake by AI agents will not be taken lightly by the general public, which is already against the AI-led developments owing to job losses and other negative publicity issues.

They realise that true AI innovation can only be commercially successful when the general public embraces it fully than seeing it as a threat. In this perspective, a self-regulation in the AI world is likely to work this time. For India, which is a heavy user of foreign AI models and agents, now have to devise ways to protect itself from such erratic behaviour of AI agents. It will be wise to form a national panel to look at the risks and propose mitigating measures.

ABOUT THE AUTHOR

The Hans India