Microsoft CEO Satya Nadella has called for independent audits that can stop advanced artificial intelligence systems in action. In the evaluation published on October 10, the model’s capabilities and powers are discussed separately. This approach particularly concerns artificial intelligence representatives who access company data and take action on behalf of the user.
In his article on his X account, Nadella suggests evaluating models together with the risks that may arise from within the institution. This is not because the models are necessarily malicious. A powerful element that accesses important systems may make a mistake or be affected by an attack. He therefore argues that security design should not rely solely on assurances from the model provider.
At the heart of the recommendation is externally limiting the data the model can access and the operations it can perform. Nadella wants an authorized person to be able to pause or shut down the model mid-task. The emergency brake analogy also corresponds to this point. Therefore, the statement in question does not constitute an announcement of a new product that Microsoft launched today.
The article highlights observable records along with independent auditing. He notes that meaningful transactions must leave evidence that can be read by humans and cannot be changed later. Moreover, he does not find it sufficient for the same model to both do the job and verify its own result. Thus, it aims to keep the audit separate from the system it evaluates.
Nadella wants to keep AI authority separate from model
We can explain the equivalent of this approach in daily workflow with an example. The action taken by a representative when he reviews the document and suggests changes is not the same as the authority to send the file out. The institution can link these two steps to separate permissions. This example does not describe a new feature of Nadella, it shows why the separation of powers he proposes makes sense.
On the other hand, the rules of behavior written into the model and the access limits established around the system are different tools. The first describes how you want the model to respond. The latter determines which operations the model can actually achieve. This is where the difference between an agent producing a wrong response and being able to take action on a sensitive file comes into play.
NIST’s June statement that security requires constant maintenance also addresses it within the framework of a separate research. The institution states that a fixed set of behavioral rules does not provide universal resistance against adapted attack commands. In addition, it recommends continuous attack testing, updates for new vulnerabilities, and preparation to limit the impact of an incident when it occurs.
NIST’s statement is not a study that tests Nadella’s suggestions one by one. However, both sources offer an approach that does not consider security work complete with the initial installation. The real question for companies is not only how impressively responsive the model is, but also what data and processes it accesses. Nadella’s latest assessment also brings to the fore the need to keep this access visible and stoppable.