Article by Dr Ian Gauci, managing partner GTG Advocates & Afilexion Alliance
The draft AI regulation has been drafted with the aim of addressing the use of a family of technologies in a manner to ensure that such technologies are not used to the detriment of society.
The ultimate objective of the regulation, that of protecting stakeholders particularly individual end-users and society as a whole is very much in line with the goal of technology regulation as implemented in Maltese law and as implemented through the Malta Digital Innovation Authority (MDIA), albeit there are also some cardinal differences as I will remark hereunder.
The tools used to achieve this goal also share much with the Maltese approach, however, unlike the Maltese model, which is a voluntary model, the draft regulation aims to put in place a mandatory regime for the captured AI and also classifies the types of AI, which will be banned as well as those which will require to follow a pre-set of obligations before being introduced to the European Market as well as during its existence.
The approach, which has been adopted by Malta, is that the technology due diligence processes, offered by the MDIA, although being voluntary, can only be mandated by another lead Authority, which regulates specific industries or sectors like Financial Services, Health, Electronic Communications, and so on. In this manner, domain specific risks are addressed in relevant law and, if need be, assessed by the Authority regulating such activity. This also allows for domain-specific control objectives to be assessed in conformity checks and monitoring to be identified by those regulating the domain – a process, which requires expertise, which a technology-centric authority would not have. Such an approach would also allow for the regulation of the use of any technology and not specifically AI-based systems.
The Draft Regulation introduces three categories of non-exhaustive High-Risk AI Systems and subjects providers and users, as well as importers and distributors of such AI Systems, to specific obligations. High-Risk AI Systems include:
1. AI Systems intended to be used as a product or as a component of products covered by a set of pre-existing EU Directives on, for example, machinery, safety of toys, lifts, radio equipment and medical devices. Concerning these AI Systems, the Draft Regulation largely refers to the provisions and conformity assessments under these specific Directives.
2. AI Systems intended to be used as a product or as a component of products covered by pre-existing EU Regulations on aviation, motor vehicle and railway safety.
3. AI Systems explicitly listed by the Draft Regulation, that are intended to be used to:
- Perform biometric identification and categorisation of natural persons.
- Work as safety components used in the management and operation of critical infrastructure (for example, for road traffic and the supply of water, gas or electricity) or to dispatch or establish priority in the dispatching of emergency first response services, for example fire-fighters and medical aid.
- Determine access to educational and vocational training institutions as well as for recruitment (for example advertising job vacancies, screening or filtering applications and evaluating candidates), make decisions on promotions, allocate tasks and monitor work performance.
- Evaluate the creditworthiness or establish the credit score of persons or evaluate their eligibility for public assistance benefits and services by public authorities or on their behalf.
- Make predictions intended to be used as evidence or information to prevent, investigate, detect or prosecute a criminal offense or adopt measures impacting the personal freedom of an individual; work with polygraphs or other tools to detect the emotional state of a person or predict the occurrence of crimes or social unrest in order to allocate patrols and surveillance.
- Process and examine asylum and visa applications to enter the EU or verify the authenticity of travel documents.
- Assist judges in court by researching and interpreting facts and the law and applying the law to a concrete set of facts.
In line with the Maltese approach, systems and solutions, which require technological assurances, will be required to:
- Carry out conformity assessment checks in order to ensure that the underlying technology is sound and safe; and
- Carry out continued monitoring of the use and outcome of the technology.
The regulation places a focus on high-risk applications and ones of a critical nature, very much in line with the recent widening of scope of the MDIA from addressing technological assurances for DLT-based systems to critical systems. Similarly, the regulation highlights the need to address start-ups and to set up sandbox environments to test technology needs identified by the MDIA to be priorities and which are being addressed through the launch of a technology-driven sandbox aimed primarily at start-ups in the coming months.
High-Risk AI Systems under the draft regulation must follow:
Technical parameters and transparency
(1) Risk management system: Providers must establish, implement, document and maintain a risk management system, including specific steps such as the identification of foreseeable risks of the AI System and analysis of data gathered from a post-market monitoring system. The risk management system must ensure that risks are eliminated or reduced as far as possible by the AI System’s design and development and adequately mitigate risks that cannot be eliminated.
(2) High quality data sets: The Draft Regulation requires High-Risk AI Systems to be trained, validated and tested by high quality data sets that are relevant, representative, free of errors and complete. This requirement must be ensured by appropriate data governance and data management.
(3) Technical documentation and record keeping: The design of High-Risk AI Systems must enable tracing back and verifying their outputs. For that purpose, the provider is obliged to retain technical documentation reflecting conformity of the AI System with the requirements of the Draft Regulation.
(4) Quality management system: The provider is required to put a quality management system in place.
(5) Transparency and information for Users: Users must be able to understand and control how a High-Risk AI System produces its output.
(6) Human oversight: High-Risk AI Systems must be designed in such a way that they can be effectively overseen by competent natural persons and introduces the notion and function of a kill switch.
(7) Robustness, accuracy and cybersecurity: High-Risk AI Systems must be resistant to errors as well as attempts to alter their performance by malicious third parties and meet a high level of accuracy.
(8) Authorised representative: Providers, established outside the EU, must appoint an authorised representative.
The draft regulation also introduces the concept of certification and registration like the Maltese Laws albeit it mandates certification which will have an EU dimension and will rely on the existing process for CE marking in the EU. It also mandates a centralised EU register. This implies that unlike the Maltese certification regime, which was not automatically recognised and endorsed outside of our shores, with the proposed EU model, the conformity and certification is imbued with a principle of EU equivalence as well as passport ability.
Under the draft regulation the provider must indicate the AI System’s conformity with the regulations by visibly affixing a CE marking so the AI System can operate freely within the EU. Before placing it on the market or putting it into service, the provider must also register the AI System in the newly set-up, publicly accessible EU database of High-Risk AI Systems.
Like the Maltese Law, the draft regulation also caters for post-market monitoring obligations. Providers must implement a proportionate post-market monitoring AI System to collect, document and analyse data provided by users or others on the performance of the AI System. This is also coupled with reporting obligations.
Unlike our Maltese regime, however, the draft regulation aside from covering the provider, also applies to the following:
(a) Users’ obligations for High-Risk AI Systems
(b) Importers’ obligations for High-Risk AI Systems
(c) Distributors’ obligations for High-Risk AI Systems
(d) Users, importers, distributors and third parties becoming providers
The draft regulations distinguish between national supervisory authority, which means the public authority to which a member state assigns the responsibility for the overall implementation and application of the Regulation, for coordinating the activities of other national competent authorities and for acting as the single contact point for the Commission and the European Artificial Intelligence Board and national competent authority, which means the public body to which a member state assigns the responsibility to carry out certain activities related to the implementation and application of this Regulation as well as market surveillance authorities and national accreditation bodies.
Malta stands to benefit here, having already set up the MDIA with the goal of regulating technology. Given the horizontal, cross-cutting nature of technology, having an authority entrusted with the regulation of technology, no matter the (vertical) operations domain, is foundational.
The draft regulation also allows subcontracting of functions of the respective notified bodies and this could further increase the prevalence and use of MDIA’s system auditors.
Human oversight and authorised representatives are also a welcome addition in the draft regulation. The Maltese Law (ITAS) also follows the same tenants as we have the role of the technology administrator as well as the notion of local representative. The draft regulation also speaks of traceability and transparency, particularly under Articles 12 & 13, which are also reflected in our local Forensic Node model. Our law also has clear provisions on transparency and user information, like draft regulation.
The Draft Regulation, like the Maltese Law also provides for administrative sanctions and fines, albeit the draft regulation provides for tougher measures as it provides for substantial fines in cases of non-compliance.