10 Things You Need to Know About the EU White Paper on Artificial Intelligence

On February 19, 2020, the European Commission (Commission) published proposals for the regulation of Artificial Intelligence (AI) with potentially far-reaching implications both for users and developers worldwide. The proposals are contained in a densely written 26-page “White Paper” which in this alert we distill down to 10 take-away points. The White Paper, together with the other strategy documents published at the same time¹, is vital reading for any business which participates in the data economy, and in particular companies that operate across borders.

At a very high level, the 10 take-away points include the fact that new, extensive risk-based regulation is on its way, which is broad in its material scope and might affect a number of different actors that are involved in the lifecycle of AI systems. The proposed framework would most certainly have extra-territorial impact. The new requirements will mainly affect AI which is considered “high-risk”, namely if it is deployed in health care, transport, energy and parts of the public sector, or if it is used in the employment sphere (for recruitment purposes or in situations impacting workers’ rights), or for remote biometric identification and “other intrusive surveillance technologies”. Non-high-risk AI will also be impacted, with an offer to comply voluntarily with certain requirements. The White Paper raises a number of questions, including in relation to global compliance issues such as export controls. It is important that stakeholders provide their comments on the White Paper by May 31, 2020. Please let us know if you would like us to assist with engagement with the Commission.

The White Paper is a statement of intent, setting out the Commission’s bold and ambitious vision of advancing as well as regulating AI. It offers some insights as to what the forthcoming new framework might include. This is the first publication that does so, and it is the first time that the Commission has confirmed that new regulation on AI will be adopted.

1. Extensive New Laws and Regulation of AI on the Way

The Commission recognized that AI is a powerful force for good, including in improving health care, contributing to climate change mitigation and adaptation, increasing the security of Europeans and improving the efficiency of production systems and transport. The White Paper states that AI is “one of the most important applications of the data economy” and a “critical enabler” for achieving the European Union’s (EU) goals of improving individuals’ lives and advancing society as a whole. However, the Commission also recognized that AI entails potential risks, mainly concerning the application of EU laws and rules designed to protect fundamental rights, including personal data and privacy rights and non-discrimination, as well as safety (including cybersecurity, issues concerning AI in critical infrastructure and malicious use of AI) and liability-related issues.

The Commission concluded that the current laws which impact AI are not sufficient to address the risks its development and deployment poses. For example, in relation to the protection of personal data and privacy, the General Data Protection Regulation (GDPR) and the ePrivacy Directive (with the new draft ePrivacy Regulation under negotiation) address these risks, but the Commission acknowledges a need to examine whether AI systems pose additional risks. The White Paper states that the current legislative framework could be improved by adjusting or clarifying existing legislation, but also that new legislation specifically on AI may be needed in order to make the EU legal framework fit for the current and anticipated technological and commercial developments.

We anticipate that, in addition to the array of currently applicable legislation, extensive new laws and regulation and adjustments to existing laws, will be adopted by the EU, aimed at creating a “unique ecosystem of trust” which supports a human-centric approach to AI and takes into account the Ethics Guidelines prepared by the EU High-Level Expert Group on AI (see our earlier post on the Guidelines). The new ecosystem of trust should, as stated in the White Paper, give EU citizens the confidence to take up AI applications and give companies and public organizations the legal certainty to innovate using AI.

2. What the Material and (Extra-)Territorial Scope of the Proposals Is

The Commission’s working assumption is that the proposed regulatory framework would apply “to products and services relying on AI”. The White Paper states that AI is a “collection of technologies that combine data, algorithms and computing power” and acknowledges that, in the new regulatory framework, the definition of AI will need to be sufficiently flexible to accommodate technical progress while being precise enough to provide the necessary legal certainty. For the purposes of what AI means in the White Paper, it essentially is defined as “systems that display intelligent behaviour by analysing their environment and taking actions – with some degree of autonomy – to achieve specific goals”. The Commission states that AI-based systems can be purely software-based, acting in the virtual world (e.g., voice assistants, image analysis software, search engines, speech and face recognition systems) or AI can be embedded in hardware devices (e.g., advanced robots, autonomous cars, drones or Internet of Things (IoT) applications).

In addition, in terms of the actors, or “economic operators”, that might have to comply with the new laws and regulations, the Commission recognizes that many actors are involved in the lifecycle of an AI system: the developer, the deployer (the person who uses an AI-equipped product or service) and potentially others such as the producer, distributor or importer, service provider, professional or private user. The Commission states that each obligation which the new framework would impose should be addressed to the actor(s) who is (are) best placed to address any potential risks. This might mean different obligations for different groups of economic operators. However, the White Paper does not provide details as to how these obligations will be delineated in the future framework.

Finally, the Commission states that an EU-wide approach to regulating AI must be provided, in order to minimize the real risk of fragmentation in the internal market. In addition to a common approach at the EU (rather than national) level, the Commission states that it is “paramount” that the anticipated new legislative requirements are applicable to all relevant economic operators providing AI-enabled products or services in the EU, regardless of whether such operators are established (i.e., incorporated or otherwise present) in the EU or not. The White Paper states that otherwise the objectives of the legislative intervention cannot fully be achieved. We anticipate that the new framework will likely have extra-territorial impact, much like the GDPR.

3. Risk-Based Approach, Focusing Primarily on Certain AI in Specified Industry Sectors, but also Non-Sector Specific “High-Risk” AI by Default

The White Paper states that the new regulatory framework should be effective to achieve its objectives while not being excessively prescriptive so that it could create a disproportionate burden, especially for small and medium-sized enterprises (SMEs). In order to achieve this, the Commission proposes to follow a risk-based approach. The new framework will, as a priority, regulate cases on a mandatory basis where an AI application is considered high-risk in light of what is at stake.

An AI system should be considered high-risk where it meets two cumulative criteria:

• First, where the AI application is employed in a sector in which significant risks can be expected to occur given the characteristics of the activities typically undertaken. It is proposed that those sectors (which will be exhaustively listed in the new framework) are (i) health care, (ii) transport, (iii) energy and (iv) parts of the public sector, which includes asylum, migration, border controls and judiciary, social security and employment services.
• Second, in addition to the above, where the AI application is used in such a manner that significant risks are likely to arise. This means that not every use of AI in the selected sectors would involve significant risks. For example, significant risks are likely to arise where AI applications affect the legal rights of individuals or companies, or pose risk of significant damage, material or otherwise.

The White Paper also confirms that there will be exceptional circumstances where the use of AI applications for certain purposes should be considered high-risk, irrespective of the sector concerned. These are, for example, the use of AI applications for recruitment purposes or in situations impacting workers’ rights or remote biometric identification and other intrusive surveillance technologies. Thus, facial recognition technology has been confirmed as a high-risk AI application, although there is no mention of a proposed five-year ban of the technology, contrary to earlier media reports on an allegedly leaked version of the White Paper.

Both high-risk and non-high-risk AI applications remain entirely subject to already existing EU-rules, as the White Paper emphasizes.

4. Some Insights into the Proposed New Requirements for High-Risk AI

The White Paper proposes that the new regulatory framework for AI may include mandatory requirements for high-risk AI applications. Such requirements could consist of the following key features, revolving around transparency, fairness, safety and security:

• Training data
• Data and record-keeping
• Information to be provided
• Robustness and accuracy
• Human oversight
• Specific requirements for certain AI applications, such as those used for remote biometric identification.

5. Pre-Market Assessment and Post-Market Compliance Checks for High-Risk AI

The White Paper states that in light of the high risk that certain AI applications pose, a prior conformity assessment, which would establish compliance with the laws, could be introduced. Procedures for testing, inspection and certification, including checks of the algorithms and the data sets used in the development phase, are envisaged. For economic operators established outside the EU, in cases when they need to comply with the prior conformity assessment, it is proposed that they could either make use of designated bodies established in the EU or, subject to mutual recognition agreements with third countries, have recourse to third-country bodies designated to carry out such assessment.

In addition to ex-ante conformity assessment, ongoing compliance will be monitored and enforced by competent national authorities, which would include, where appropriate, testing of the relevant AI applications by those authorities, enabled by adequate documentation of the relevant AI application. The Commission also refers to effective judicial redress, which should be ensured in the future framework, partly through the adjustments to the rules on safety and liability mentioned above.

6. Voluntary Labeling Scheme for Non-High-Risk AI

For those AI applications that do not pose a high risk, the Commission proposes to set up a voluntary labeling scheme, under which economic operators can signal that their AI-enabled products and services are trustworthy. The Commission believes that this would help enhance the trust of users and promote overall uptake in AI systems.

Developers or deployers of AI could decide to make themselves subject either to all the mandatory requirements applicable to high-risk AI applications, or to a specific set of similar requirements which would be established specifically for the purposes of the voluntary scheme.

7. Amendments to the Product Liability Regime, Applicable to e.g. Autonomous Vehicles

The Commission recognized, both in the White Paper and in the Report on Safety and Liability published with it, that the current product safety legislation already supports an extended concept of safety protecting against all kind of risks arising from the product according to its use. However, provisions explicitly covering new risks presented by the emerging digital technologies like AI, the IoT and robotics could be introduced to provide more legal certainty.

The White Paper provides an example with autonomous cars. Under the current Product Liability Directive, a manufacturer is liable for damage caused by a defective product. However, in the case of an AI-based system such as an autonomous car, it may be difficult to prove that there is a defect in the product because of the difficulty in proving the causal link between the damage caused and the product itself. In addition, there is some uncertainty about how and to what extent the Product Liability Directive applies in the case of certain types of defects, for example, where defects result from weaknesses in the cybersecurity of the product.

The Commission is seeking stakeholders’ views to assess what possible amendments to the Product Liability Directive and possible further targeted harmonization of national liability rules might be necessary.

8. Anticipated Stimulus Package for Investment in Research and Innovation

In addition to lack of trust, which the Commission aims to address through the proposed new legislative framework, the White Paper identifies lack of investment and skills as the main factors holding back a broader uptake of AI in the EU.

The Commission has acknowledged that the EU is in a weaker position in some applications of AI than its competitors, such as in consumer applications and online platforms. Although EU funding for research and innovation for AI has risen recently, investment in this area in Europe is still a fraction of the public and private investment in other regions of the world. The Commission has confirmed that its objective is to attract over €20 billion of annual public and private investment in the EU in AI over the next decade.

In addition, the Commission’s efforts will be focused on creating more synergies between European research and testing centers and developing a lighthouse center of research and innovation that has the ambition to be “a world reference of excellence in AI”. The Commission has also stated that developing proposals to improve education and training systems and to upskill the workforce to become fit for the AI-led transformation will be a priority.

9. Some Questions that the Proposals Raise

The White Paper and its accompanying strategy documents strive to achieve much: the Commission’s intention is to lead by example, by setting the global standard of AI regulation.

However, a few questions that the Commission’s proposals raise are worth noting:

• Providing feedback on the Commission’s proposals is important, as a means of influencing the content and scope of the new laws and ensuring they are workable for industry (see next section). For example, who of the various actors involved in a lifecycle of an AI application should be responsible for which obligations is an area where stakeholders should provide their feedback.
• There are other areas in the White Paper where further details may be sought. In that context, no information is provided, for example, as to what capabilities and expertise the envisaged competent bodies would have to carry out the pre-market conformity assessments for high-risk AI, such as the checks on algorithms and data sets.
• The way in which the balance will be struck between stimulating AI in EU and the EU’s strict privacy laws (including for example on explainability and consent under the GDPR) is currently unclear. In a similar vein, how the development of new technology will be safeguarded from the potential restrictive effect of new regulation is not outlined in detail.
• In addition, there is a potential question in relation to export controls on the underlying AI software or products. The EU is an observer in the Wassenaar Arrangement on Export Controls for Conventional Arms and Dual-Use Goods and Technologies. It therefore implements any amendments to the Wassenaar Arrangement Control Lists in its own control lists (i.e., the EU Common Military List and the EU Dual Use Regulation). On January 6, 2020, the U.S. Bureau of Industry and Security published an interim final rule classifying software specially designed to automate the analysis of geospatial imagery under the unilateral Export Control Classification Number 0D521. The U.S. government has already indicated that it currently plans to propose to the Wassenaar Arrangement that multilateral controls be placed on this item, which controls would, if adopted by the Wassenaar Arrangement, then become applicable in the EU as well.
• A related question is how global compliance with developing regulations with extra-territorial reach (which may be conflicting) can be achieved in an efficient and cost-effective manner. Regulation outside the EU is also on the rise: in January 2020, the White House released what it has described as a “first of its kind” set of principles that agencies must meet when drafting AI regulations. Post-Brexit, the data protection regulator in the U.K. issued draft guidance on AI Auditing framework (see our post).

10. Next Steps and Legislative Pipeline

The period of consultation on the White Paper will close on May 31, 2020 (it was extended from the initial deadline of May 19, 2020). Stakeholders should use the opportunity to provide comments on the proposals set out in the White Paper, including to address the questions which we identified in the section above. The follow-up legislative proposal to the White Paper is expected in the fourth quarter of 2020.

The Commission has also sought feedback on its European Strategy for Data, through an online consultation which will be available until May 31, 2020.

As mentioned above, the White Paper is only one of a number of publications and initiatives by the Commission in relation to new technologies and Big Data. A raft of policy and legislative actions are planned for this year, including:

• A European cybersecurity strategy, including the establishment of a joint Cybersecurity Unit and a review of the Security of Network and Information Systems (NIS) Directive ((EU) 2016/1148).
• European strategies on quantum and blockchain (Q2 2020), as well as a revised EuroHPC (Euro High-Performance Computing) Regulation on supercomputing, aimed at developing capabilities in the areas of AI, cyber, super- and quantum computing.
• A legislative proposal on crypto assets (Q3 2020).
• A legislative framework for data governance (Q4 2020) and a possible Data Act (2021), which could mandate business-to-business data sharing and enhanced data portability rights; these could lead to changes in the EU laws on trade secrets (among other things).
• An implementing act under the recently enacted Open Banking Directive, which is aimed at making high-value public sector data sets available across the EU for free, in machine-readable format and through standardized Application Programming Interfaces (APIs) (Q1 2021).
• A Digital Services Act package aimed at upgrading the EU’s liability and safety rules for digital platforms, services and products (Q4 2020).

The Akin Gump cross-practice AI team will continue to monitor, report and advise on developments. Please let us know if you would like assistance in engaging with the Commission on your behalf, including responding to the consultation.

¹The Communication about a European Strategy for Data, the Communication on Shaping Europe’s Digital Future and the Report on the Safety and Liability Implications of AI, the Internet of Things and Robotics.