Artificial intelligence act: technical assistance for ai safety

Regulation (EU) 2024/1689 (Artificial Intelligence Act), which has entered into force on the 1st of August 2024, and for which certain rules will become applicable on the 2nd of August 2025 establishes a comprehensive legal framework governing the area of Artificial Intelligence, and in particular relevance to this procurement, establishes responsibilities for providers of General-Purpose Artificial Intelligence (GPAI) models, aiming to ensure transparency, safety, and accountability in the deployment and use of AI technologies. Articles 89 and 92 of the AI Act respectively allow the AI Office to monitor the compliance with regards to the AI Act and perform evaluations of GPAI models to assess the compliance and investigate systemic risks at Union level. Article 93 of the AI Act allows the Commission to request providers to take appropriate measures to comply with the obligations, to implement mitigation measures, or to restrict the availability of the model.

In order to carry out these enforcement tasks in relation the AI Act, the AI Office is seeking support of suitable third-party contractors to provide technical assistance aimed at supporting the monitoring of compliance, in particular in its assessment of risks posed by GPAI models at Union level. This tender has been split into six lots. Five of those lots dedicated to specific systemic risks, namely (i) CBRN risks, (ii) cyber offence risks, (iii) loss of control risks, (iv) harmful manipulation risks and (v) sociotechnical risks.

These lots all share a similar tasks and services:

• The organisation of multiple risk modelling workshops together with the AI Office, producing a risk model, risk scenarios, as well as corresponding thresholds, informing further work on evaluations.
• The development of newly evaluation tools that are private to the Commission.
• The onboarding of existing evaluations that are publicly available into the developed risk framework.
• The creation of a reference procedure and reporting template for doing risk assessment with respect to the developed evaluations.
• Ad hoc and on demand services for complex evaluations, including for example red-teaming and uplift studies.
• A risk monitoring service in the form of regular briefings on new developments (e.g. models, risk sources, elicitation techniques, mitigation, incidents) to complement existing AI Office monitoring actions. The scope of these tasks varies by lot. Furthermore, there is one lot that is cross-cutting across various risk.
• Lot 6: agentic evaluation interface, procuring software, Cloud infrastructure support, and methodology to evaluate GPAI on diverse types of benchmarks, focusing on agentic interaction and complex agentic benchmarks.