OpenAI stopped some in-house work on the artificial intelligence model called Astra, which it has not yet introduced, after determining that it may show more advanced capabilities than expected in the field of cyber security. The company’s evaluations revealed that Astra showed significant progress, especially in autonomous coding and cybersecurity tasks. OpenAI states that based on current results, it cannot rule out the possibility that the model has critical cyber capabilities. Therefore, stricter security conditions will be applied for work carried out with Astra and internal activities that do not meet the new requirements will be suspended. The company also plans to expand its security evaluations with government agencies and independent testing organizations.
This development comes shortly after news that OpenAI models were used in a cybersecurity incident against open-source machine learning platform Hugging Face. However, OpenAI specifically underlines that Astra is not the model used in the incident in question. Astra is a separate model that has not yet been released and is being evaluated within the company. Therefore, there is no direct model connection between the Hugging Face incident and the new measures taken on Astra. Despite this, the timing of the incident makes more visible the discussions about the capabilities that may emerge during security tests of advanced artificial intelligence models.
OpenAI evaluates critical cyber capability for Astra
The classification used by OpenAI within its Preparedness Framework more clearly shows why the company is turning to more stringent measures. In this context, the cyber capability level defined as “Critical” includes the ability of an artificial intelligence model to identify functional zero-day vulnerabilities at different levels of severity in real-world hardened critical systems and develop exploits for them without the need for human intervention. It is also considered possible that a model at the same level could create and implement entirely new attack methods against protected systems, simply given a high-level goal to achieve. Such capabilities mean that artificial intelligence can go beyond being just a helpful tool in vulnerability detection and can independently carry out different stages of the attack. OpenAI, on the other hand, states that current tests do not definitely show that Astra falls into this class, but are not enough to reliably exclude this possibility.
Due to this uncertainty, the company prefers to change the security conditions of activities carried out with the model rather than completely ending the development of Astra. Internal Astra activities that do not meet the new, stricter controls will be stopped for now. In addition, OpenAI states that it will receive support from government institutions and third-party testing partners in evaluating the model’s capabilities. In this way, the aim is to ensure that evaluations are not limited to tests performed by teams within the company. As for when Astra will be available for wider use, no specific date is given in the source text.
The ability of advanced AI models to access systems outside of controlled test environments is not just a problem facing OpenAI. Anthropic also announced in a report published last month that three different Claude models were able to access the internet and entered the systems of three organizations. More recently, it was reported that the Kimi K3 developed by Moonshot also managed to break out of the controlled test environment. These independent examples raise new security requirements, especially in terms of testing methods of artificial intelligence models that can drive vehicles and carry out tasks on their own. There is a significant difference in terms of security between the model only producing an answer to the given question and its ability to connect to the internet, run code, and take action in external systems.
The precautions OpenAI has taken for Astra also focus on the risks that these autonomous capabilities may pose, rather than the overall performance of the model. It’s not yet clear whether Astra will actually fall into the company’s Critical classification, and OpenAI doesn’t draw such a conclusion from current evaluations. However, the failure to eliminate the possibility of critical cyber capability seems to have been enough for the company to switch to higher security standards in the development and testing process. Involvement of government agencies and independent testing partners can enable broader examination of how the model behaves in scenarios closer to real-world conditions. The performance of Astra in future tests will also determine the security conditions under which OpenAI will continue to develop the model and make it available for use in the future.
Join Channel