Data Protection – KSLR EU Law Blog

by Vera Lúcia Raposo*

On 21 April 2021, the European Commission released a proposal for a future European Union (EU) regulation on Artificial Intelligence (hereafter, the ‘Proposal). While its structure sought inspiration in the General Data Protection Regulation (GDPR), the content was built upon other relevant EU interventions, namely the EC’s White Paper on Artificial Intelligence (AI), released in February 2020. The aim is to facilitate and develop the use of Artificial Intelligence within the EU to create a truly Digital Single Market while still protecting fundamental rights, EU values, and ethical principles that are potentially threatened by AI features such as its opacity (some have called it a black box), complexity, the demand for copious amounts of data (big data, with all the attached concerns), and the possibility of automation. So far, so good. We expected nothing less from the European Commission (EC). However, the proposed regulation might fall short regarding its aim.

The definition of AI used by the Proposal is presented as being broad enough to encompass future developments of this technology. However, the fast pace of technology advancement implies that it will eventually be outdated.

The proposal is based on a risk-assessment model, under which three categories of AI emerge:

Unacceptable risk AI, banned in Title II of the Proposal. This category includes AI that distorts behaviours (dark patterns), explores the vulnerabilities of specific groups (micro-targeting), scores people based on their behaviours (social scoring), and proceeds to real-time biometric identification for law enforcement purposes in public spaces (facial identification);
High-risk AI, as referred to in Title III of the Proposal are allowed, albeit under particularly strict standards;
Low-risk AI, for which only transparency requisites are demanded in some cases (namely chatbots and deep fakes), as established in Title IV of the Proposal.

The identification of a serious risk is based on the potential menace to health, safety, and/or fundamental rights. Notably, not only does the label of ‘high-risk’ depend on the specific task performed, but also on its purpose. Moreover, in case AI systems are erroneously classified, the consequences are not clearly defined in the Proposal. A correction mechanism should be in place to amend any wrong classification, but the Proposal remains silent on this issue.

High-risk AI systems are products or safe components of products, whose regime has been harmonised under EU law by the legislation listed in Annex II (medical devices, toys, elevators, and many others); or stand-alone AI systems that pose special threats to fundamental rights and that are referred in Annex III. These lists shall be updated in accordance with the latest technological developments.

For high-risk AI, a two-step system of control and monitoring is built. First, the control is performed through mandatory requirements to be complied with by AI systems before entering into the market. A conformity assessment is undertaken by a third party following harmonised standards, which may lead to the issuance of the ‘CE marking of conformity’. Second, an ex-post monitoring mechanism will be put in place by market surveillance authorities which use the powers conferred to them by Regulation (EU) 2019/1020 on market surveillance. Moreover, AI providers are also obliged to report severe incidents and malfunctions.

Low-risk AI systems almost go unnoticed in the Proposal, except for the provisions on transparency. The two-step procedure is not required in this case. However, providers can voluntarily accept to do so by creating their own codes of conduct (Article 69).

Such legal framework requires the set-up of specific organs at the national level (notifying authorities and notifying bodies), as well as the European level, particularly the European Artificial Intelligence Board. The latter is a supranational supervisory authority, similar to the European Data Protection Board (EDPB), created under the umbrella of the GDPR, and also to the recently proposed European Board for Digital Services established in the proposal of the Digital Services Act.

Another innovation is the EU database, in which stand-alone high-risk AI must be registered before entering into the market, to facilitate transparency and tracking. The information is supposed to be upload to the database by the provider of high-risk AI systems. Neutral entities, such as the notifying bodies, could be entrusted with this task for all AI systems. Such solution would lead to better results in terms of AI’s safety and transparency.

Accountability is another matter of concern. Indeed, compliance requirements are carried out by AI providers, i.e., the ones that develop the AI and put the AI system (or its outputs) on the EU market, even if they are not established within the EU territory. In addition to the above-mentioned tasks on control, monitoring and information update, their obligations include the implementation of risk management and quality management systems, the development of detailed technical documentation, the maintenance of automatically generated logs, and transparency obligations which require that the ones interacting with AI systems must be informed that they are. Moreover, compliance is also expected from AI users, i.e., people or entities established in the EU using AI in a professional context, or not established in the EU but whose outputs are used in the EU, along with distributors and importers. In the event of non-compliance, the Proposal foresees maximum administrative fines of up to €20m or 4% of total worldwide annual turnover, similarly to the GDPR clauses. However, no accountability mechanisms are in place. Individuals harmed by AI systems can barely obtain redress. The 2019 report on Liability for Artificial Intelligence and other Emerging Digital Technologies and the European Parliament Resolution of 20 October 2020 on a Civil Liability Regime for Artificial Intelligence suggest that such a mechanism might be put in place, as it is also supported by Coordinated Plan on Artificial Intelligence, released together with the Proposal.

source: https://digital-strategy.ec.europa.eu/

Some highly controversial topics, such as facial recognition technologies (FRT), are expected to raise discussion amongst experts. Interestingly, the 2020 White Paper did not ban FRT, although a draft version had suggested a time limited ban on the use of FRT in public spaces. The White Paper merely recommended the use of appropriate cautions, without giving much detail on such recommendations. Instead, Article 5 of the Proposal clearly prohibits the use of some forms of FRT. It is expected that Article 5/1/d, in particular, will be contested by the ones defying this technology. The norm bans ‘real-time remote biometric identification systems in publicly accessible spaces for law enforcement purposes, allegedly one of its more threatening uses from the perspective of fundamental human rights. Considering its potential added value for crime control, the proposal provides for some exceptions: the search of crime victims, including missing children, ‘specific, substantial and imminent threat to the life or physical safety of natural persons, or of a terrorist attack’, and the detection, localisation, identification, or prosecution of a perpetrator or suspect of crimes referred to in Article 2/2. The latter includes grievous crimes, such as terrorism, tariffing of humans beings, murder, rape, but also different crimes such corruption, fraud, facilitation of unauthorised entry and residency. Moreover, the ban leaves several loopholes: it does not cover the use of FRT for law enforcement purposes that does not take place in real-time or that is not carried out in public spaces; FRT by other public entities not related to law enforcement; and FRT used by private individuals and companies. Critics might argue that still many doors are left open.

Overall, the Proposal is more human-rights-friendly than the White Paper, but eventually also more conservative, a potential downside for EU’s competitive digital capacity at a global scale. At its core, the regulatory ideas – the categorisation of AI systems per levels of risk – are the same in the Proposal and the White Paper. However, the White Paper had a more pro-technology approach and unlike the Proposal it did not elaborate in detail the potential human rights violations. Some have even pointed out that the White Paper was more concerned with the economic outputs of a massive investiment in AI than with its consequences for human rights. Although the critic might be excessive, the White Paper did contain a stronger emphasis on technological development, as it results from the various sections dedicated to this aim (comparatively, there is proportionately more discussion on development and innovation than in the Proposal).

In contrast, the Proposal gives more space to the protection of fundamental rights (though many will argue that not enough), as expressed in multiple binding norms imposing risk assessments and safety requisites, whose violation can lead to severe economic penalties. Assuming this proposal becomes indeed the new AI Regulation, my guess is that European AI developers and manufacturers will be asked to comply with so many different requirements also coming from other norms, the GDPR, the Medical Devices Regulation, the In Vitro Diagnostic Devices Regulation, the Data Governance Act proposal, that AI in Europe will become a scientific topic, not a real industry though.

In sum, the Proposal essentially pivots around two core concepts. The first is compliance, based on a system of harmonised rules, monitoring, and good governance. Consequentially, the second is the principle of trust (‘trust’ and ‘trustwhorty’ are emphasised along the Proposal). On the one hand, developers of AI shall be able to rely on rules to carry out their businesses within the EU market by complying with a unified body of rules for all Member States. On the other hand, AI users should be able to rely on its safety.

Innovation should have been the third characteristic. The impetus to bring about innovation in AI technologies is restricted to Chapter V which, albeit introducing interesting provisions, falls short of what would be required for a truly digital single market. The most promising initiative is the creation of regulatory sandboxes to encourage innovation, though under strict (too strict?) requirements. AI investment in the EU might be hampered by such ‘innovation hole’ which could advantage other leading players. Given the outstanding Chinese technological development, including on AI, the EU might not be able to reach China in the near future (or ever). Whether the Proposal reached a fair balance between innovation and human rights, or conversely whether it will lead the EU to stagnation in the domain of AI remains to be seen.

*Vera Lúcia Raposo is Associate Professor at the Faculty of Law of the University of Macau, China, and Assistant Professor at the Faculty of Law of the University of Coimbra, Portugal. Her main lines of research are health law and digital law.

by Osal Stephen Kelly*

Introduction

In July 2020, the Court of Justice of the European Union (“CJEU”) delivered its judgment in the Schrems II case brought by the Austrian lawyer and activist Max Schrems, with far-reaching implications for data protection policy and practice. One question of particular urgency is what the consequences will be for the continued flow of personal data from the EU to the UK; while the EU-UK Trade and Cooperation Agreement temporarily allows these flows to continue on the same terms as between member states, this will end on 30^th June 2021. The purpose of this period is to allow for the EU Commission to determine whether or not to grant an “adequacy decision” that would confirm that the UK provides a level of protection essentially equivalent to that of member states, which would allow for these important transfers to continue indefinitely. While the Commission has issued a draft adequacy decision, some of the issues identified by the European Data Protection Board (“EDPB”) in its recent opinion on the draft expose frailties in these protections that could form the basis for a legal challenge in the future. It is submitted that there are two areas of particular vulnerability that would be key in any such challenge. First, there are serious unresolved questions around the powers of UK and US authorities to access data for security purposes. Second, the UK’s emerging post-Brexit constitutional and legal framework is likely to be somewhat less advantageous to data subjects vindicating their rights than was the case when EU law had direct effect.

Schrems II

Schrems II comes after another case brought forward by Mr Schrems who had already challenged the previous framework as well (Schrems I). The Schrems II case arose from a complaint concerning the transfer of his data from Facebook Ireland to Facebook Inc. (based in the United States). The complaint was made to the Irish Data Protection Commissioner and resulted in the Irish High Court making a preliminary reference to the CJEU. In its submissions, Facebook sought to justify these transfers as permitted by the EU Commission’s Privacy Shield decision, which set additional safeguards for data moving from the EU to the US. However, the Court found that the Privacy Shield was invalid as the protections offered by US law did not in fact afford the required level of protection. The Court stressed the importance of “effective and enforceable data subject rights” (para. 177 of judgment) and found that data subjects did not enjoy such rights under the Privacy Shield. Particular emphasis was placed on the lack of limits on the power of surveillance agencies to collect data on individuals held by companies such as Facebook (para. 180). While the Court recognised that data controllers could in principle rely on standard contractual clauses approved by the Commission to allow cross-border data transfers to continue, it noted that such clauses did not necessarily protect data from unlawful access by the authorities of the receiving country (para. 141).

from jonesday.com

Schrems III?

Although the UK ceased to be subject to EU law from 31^st December 2020, the GDPR has been incorporated (with amendments) into UK domestic law, in line with Section 3 of the European Union (Withdrawal) Act 2018. This amended version, referred to as the “UK GDPR”, now forms the basis of the UK’s legal framework for data protection, along with the UK’s existing Data Protection Act 2018 (draft adequacy decision, Recital 14), and this is the framework that was examined in the Commission’s draft adequacy decision, and, subsequently, the EDPB’s opinion, released on 13^th April 2021. Although important, the opinion in itself is non-binding and the final decision on adopting the adequacy decision rests with the Commission, so it is likely to be approved.

The EDPB opinion, read in light of Schrems II, would require the UK’s intelligence operations to apply particular scrutiny over the compliance with the (EU) GDPR. While the tone of the opinion as a whole is very measured, the EDPB nonetheless expresses “strong concerns” (para. 88 of opinion) over the data-sharing agreement between US and UK authorities pursuant to the US CLOUD Act. The Act requires US companies to disclose information stored on overseas-based servers on foot of a valid warrant. The EDPB notes that the Commission’s draft decision refers to non-binding “explanations” that were provided to it by UK authorities (para. 88 of opinion). Critically, however, the EDPB notes that these explanations did not seem to comprise “any concrete written assurance or commitment” on the part of the UK Government. It is difficult to see how mere explanations without substantive legal force could be relied upon by data subjects in enforcing their rights, which is concerning, given that the existence of “effective and enforceable data subject rights” was deemed vitally important in Schrems II.

Moreover, para. 189 of the opinion highlights how broad the general exemption is for intelligence-related processing, stating that “national security certificate DPA/S27/Security Service provides that until 24 July 2024, personal data processed ‘for, on behalf of, at the request of or with the aid or assistance of the Security Service or’ and ‘where such processing is necessary to facilitate the proper discharge of the functions of the Security Service described in section 1 of the Security Service Act 1989’ are exempted from the corresponding provisions in UK law to Chapter V GDPR in relation to transfers of personal data to third countries or international organisations”.

This provision is similarly open-ended to Section 702 of the US Foreign Intelligence Surveillance Act, which had been considered not to afford a sufficient level of protection to data flows in Schrems II (para. 180 of judgment). If Part V GDPR (and equivalent provisions in the UK GDPR) does not apply to intelligence processing, personal data would be transferred to US authorities and thus fall within the scope of the Court’s ruling in Schrems II.

Given that the UK is no longer a member of the EU and subject to the jurisdiction of the CJEU, issues also arise in relation to the UK’s overall legal framework (para. 54 of opinion). The Commission has placed great emphasis on the fact that the UK will continue to be a party to the European Convention on Human Rights (“ECHR”) and thus of the “European privacy family” (press release accompanying the adequacy decision). However, while the set of rights listed in the ECHR are also included in the EU’s Charter of Fundamental Rights, in Schrems II the Court notes that the ECHR is not part of the EU law acquis (paras. 98, 99 of judgment). Furthermore, the UK Government will review the Human Rights Act 1998 which implements the ECHR in the UK. The review will consider whether courts have been “unduly drawn into matters of policy”. Given that the CJEU identified “effective and enforceable data subject rights” as key in determining whether a country provided an adequate level of protection (para. 45 of judgment), any dilution of the rights of citizens to invoke their ECHR rights would be likely to count against the UK in the event of a legal challenge.

Conclusion

The foregoing indicates that a credible case could be brought before the Court to challenge the validity of the adequacy decision in the future. On a practical note, data controllers can at least be reassured by the CJEU’s clarification in Schrems II that an adequacy decision enjoys, in effect, a presumption of legality until it is successfully challenged (para. 156 of judgment), and accordingly they should not incur any liability for data transfers while the adequacy decision remains in place, for whatever period that may be.

*Osal Kelly is a postgraduate Law student in the Law Society of Ireland in Dublin and holds an undergraduate degree in Philosophy from Trinity College, Dublin. He currently works in the Irish public service. This article is written in a personal capacity.