EU Experts

​

1.    Scope of the EU AI Act

​

Angela Zhang: The EU AI Act is extensive, with over 400 pages, 113 provisions, and 13 annexes. Can you provide an overview of what the law covers and what exemptions exist under the law?

​

Carme Artigas: The EU AI Act is a groundbreaking legislation with international coverage, aimed at regulating artificial intelligence. It originated in June 2021 with the first version issued by the European Commission and is part of a broader legal framework that includes the GDPR, the Digital Services Act (DSA), the Digital Markets Act (DMA), and the Data Act. Some aspects not covered by the AI Act, such as data location or algorithmic transparency, are addressed in other parts of this framework.

​

The AI Act is designed to regulate high-risk AI use cases. It is product-based and risk-based, requiring a risk metric to identify prohibited and high-risk AI uses. The latter must undergo conformity assessments before and after market entry. Limited-risk applications only need to register in a central European database, while low-risk applications have no specific requirements. High-risk AI involves applications that can impact safety, health, or fundamental rights in domains such as education, workspaces, justice, and government use of AI. The law’s spirit is to regulate the harmful use of technology rather than the technology itself.

​

The legislative process in Europe adheres to particular procedures, leading to the creation of directives or acts. Directives provide general rules for Europe, but each member state must transpose them into national laws. This regulatory fragmentation has hindered Europe’s competitiveness in the digital world. To address this issue, the recent digital decade framework introduced acts such as the DSA, the DMA, and the AI Act. These acts establish a single, unified law for all member states without alterations, promoting a more cohesive digital market in Europe. 

​

The AI Act negotiations involve the European Commission, the Council representing the member states, and the Parliament representing individuals. The unforeseen ChatGPT moment led to a radically different proposal from the Parliament, sparking conflict. The challenge was to preserve the law’s spirit, which aimed to regulate only risky use cases rather than the technology itself, such as general-purpose AI (GPAI) models. The solution entails implementing minimum transparency rules for these models to enable the prevention of systemic risk throughout the value chain.

​

The most significant aspect of the AI Act is its prohibitions and law enforcement exemptions, rather than GPAI models. Europe has expressed a stance that certain AI applications, even if technically feasible, should not be allowed, such as using biometric identification to classify people. This crucial message from Europe highlights the importance of aligning AI with regional values, which may differ from perspectives in other parts of the world.

​

The AI Act aims to be future-proof, recognizing that it cannot be perfect due to the lack of past experiences to learn from. To stay relevant, the AI Act employs an innovative legislative technique, defining normative elements in the Articles and including specifics, such as high-risk domains, in the Annex. This approach allows for adaptability as risks evolve over time and contexts. A key feature of the law is the obligatory code of practice, negotiated with the private sector, aiming to cover high-risk cases while remaining practical and dynamic.

​

Philipp Hacker: The AI Act’s scope is broad, with a wide definition of AI to ensure future-proofing. It adopts the revised OECD definition, focusing on the concept of inference to distinguish AI from traditional software. The Act excludes basic data processing and human-crafted rule-based systems, such as auto-summation functions in Excel. However, it still covers many techniques, including long-standing statistical optimization methods used in finance.

​

The AI Act takes a two-tiered approach to GPAI models, with some minimum requirements for all GPAI models and additional rules for those with systemic risk. The presumptive threshold for systemic-risk GPAI models has been set at 10^25 FLOPs, which covers models like GPT-4 and above.

However, there are two issues with this approach. First, the threshold is quite high, and as models become smaller but more powerful, it will be challenging for regulators to address specific models below this threshold that still pose systemic risk. This could lead to lengthy legal battles, similar to those seen with very large online platforms designated under the DSA. 

​

Second, there is skepticism about the effectiveness of rules for systemic-risk GPAI models. The rules include cybersecurity, red teaming, incident reporting, and some evaluations, but content moderation is absent. Comparatively, the Chinese approach to AI regulation focuses on aligning model outputs with societal values and customs. It would have been beneficial for the AI Act to require state-of-the-art content moderation for GPAI models to reduce hate speech, insults, and fake news. Studies by the UN and investigative journalists show that this continues to be a huge problem.

​

Overall, the AI Act has a more positive potential than its current reputation suggests. Despite receiving negative reactions from businesses and venture capitalists, the law primarily focuses on prohibiting undesired practices in Europe and enforcing industry best practices in specific cases. For responsible developers, complying with these rules should not be difficult. It is crucial to reshape the narrative and emphasize that the AI Act’s obligations are achievable. With regulations in place, Europe must now focus on increasing investment in AI development and deployment, as it currently lags behind the US and China. The AI Act is a positive step towards achieving this goal in a responsible, sensible and sustainable way. 

Angele Zhang: I agree with Professor Philipp Hacker, as I’ve found more similarities than differences between the EU and US approaches to AI regulation. Many GPAI obligations in the previous Parliament’s version of the AI Act have been notably toned down, bringing the EU law closer to the US Executive Order. Additionally, I agree that even smaller AI models can be very powerful, as shown by Microsoft’s recent release of a small yet impressive image model.

​

Carme Artigas: Responding to Professor Philipp Hacker’s comments, the current AI definition has garnered the most consensus. While less advanced tools may fall under the AI Act, they will mostly be in the low-risk category and thus not affected. Secondly, the two-tier approach to GPAI models uses FLOPs as an indicator, which, although imperfect, distinguishes between large models like GPT-4 and others. This threshold primarily addresses energy consumption concerns, targeting companies that exceed it to provide more data and information. Large corporations can readily handle all the obligations. The challenge lies in helping small businesses comply with the law more easily.

​

The software industry, like other sectors, may not prefer regulation. However, regulation has been applied to various industries for safety and transparency. Until now, the software industry has remained largely unregulated. Introducing regulations for AI can foster consumer trust and market growth, as people are currently skeptical and fearful of AI. Providing certainty and support to businesses and citizens through measures like sandboxes and real-time testing scenarios can encourage AI adoption and innovation. Ultimately, building trust is essential for the successful integration of AI into the market.

​

Philipp Hacker: Transparency is crucial, but it is not enough. The systemic-risk threshold for GPAI models is linked to computing power, which translates into energy costs. However, the current threshold remains too high. As climate change affects the entire world, including China, the US, and the EU, coordinated action is needed to tackle the rising energy, water, and toxic material demands of GPAI models. The AI Act requires transparency in energy usage for GPAI models, but clear standards and additional measures are still needed. A “sustainability by design” approach, similar to “privacy by design,” could be beneficial. The AI Act includes risk mitigation assessment and treats environmental protection as a fundamental right, which could pave the way for incorporating environmental impact assessment into AI regulation.

​

2.    Regulation of Open-Source AI

​

Angela Zhang: Meta recently released the world’s most powerful open-source AI model, Llama 3. Throughout the AI Act’s legislative process, the regulation of open-source models has been a major point of debate. The discussion centered on preventing misuse by malicious actors without stifling innovation and competition. How did EU legislators develop their current approach to regulating open-source models?

​

Carme Artigas: In essence, open-source aims for transparency, allowing error detection in software. The AI Act addresses varying responsibilities for key players within the AI value chain, including providers of open-source models. Llama 3 is not considered open-source as it does not disclose its weights, which is required by the AI Act. The open-source community encourages the publishing of various parameters, but there’s a lack of consistency in reporting. Regardless of the specific approaches adopted, promoting transparency and accountability in AI is crucial, as it significantly impacts the economy and society, and should not develop without constraints.

​

Philipp Hacker: Open-source models in AI present both benefits and risks. While they promote competition, they can also cause huge damage if misused by terrorists or rogue nations. Some leading AI companies, including OpenAI and Meta, have pledged to submit their models for inspection to the UK AI Safety Institute, but this has not yet occurred. As AI models become more potent, there may be a need to prevent open sourcing and make it mandatory for models to be hosted via APIs for better control. Striking a balance between surveillance and control is challenging, but necessary to address potential threats like biological and chemical terrorism. Currently, the EU AI Act has few exemptions for open-source models. Future regulations should focus more on managing highly capable open-source models. This issue remains a central and unsolved aspect of AI policy debates.

​

3.    Risk-Based Regulatory Framework and Regulation of General Purpose AI (GPAI) Models

​

Angela Zhang: The EU AI Act adopts a risk-based framework, categorizing AI systems according to the risks they pose to the public, and includes specific provisions for GPAI models. How did European authorities arrive at these decisions? What have been the major controversies surrounding this regulatory model? 

​

Carme Artigas: The GPAI model and system are not considered high-risk by themselves. However, when utilized by millions of people, they can present unique systemic risks that may not directly threaten health and safety but could impact fundamental rights. Small and medium businesses that deploy these models—typically developed by well-resourced large corporations—in high-risk settings inherit both the risk and responsibility. To achieve a balance, it is crucial for model developers to supply pertinent information to downstream players, aiding them in meeting their obligations.

​

The fundamental rights impact assessment aims to identify risks that may not affect health or safety but are still undesirable. The AI Office’s role is to establish auditing mechanisms to address these concerns. Over the next two years, we expect to see multiple versions of the law, allowing the AI Act to be improved and incorporate the appropriate elements to ensure compliance and fulfill its intended purpose.

​

Reaching an agreement on the AI Act faced pressures from the market, governments, and public opinion. The challenge was to find a right balance between protecting citizens, fostering innovation, and regulating AI without hampering the market. Ultimately, having an 80% perfect law is better than no law at all, as it helps control technology, limit abuses by bad actors, and support innovation and freedom.

​

Philipp Hacker: The AI Act is commendable and is better than its reputation. A notable feature is the possibility of revising the Act through implementing or delegated acts, allowing for adjustments on a rolling basis, such as lowering the computing threshold for systemic-risk GPAI models. The Act can also revise the list of high-risk sectors to exempt ancillary applications with minor impacts on fundamental rights, health, or safety. For instance, AI systems used for scheduling job interviews rather than assessments are not considered high-risk. However, to prevent companies from exploiting loopholes, a reverse exemption exists: they are ineligible for the exemption if they engage in profiling, which involves surveying people and collecting extensive data for sophisticated inferences. Ultimately, the AI Act is a solid package that needs to be enforced effectively on a global scale, coordinating with partners in China and Europe to ensure the Act is considered and adhered to during the development process of AI systems.

​

4.    Enforcement Structure and Implementation Issues

​

Angela Zhang: How will the AI Act be implemented? Will it be enforced by the AI Office or national authorities? What lessons have been learned from GDPR enforcement, and what are the EU’s plans for the AI Act?

​

Carme Artigas: The AI Act aims to avoid the issues faced by GDPR implementation. Key phases of the AI Act include: retrieving prohibited items from the market within 6 months of publication, ensuring transparency for GPAI models and systems within 9 months, and establishing the AI Office within 3 months. There are two types of supervisory agencies: national bodies and the EU AI Office. National bodies handle high-risk use cases and conformity assessments, while the EU AI Office oversees horizontal regulations for GPAI models, threshold definitions, and codes of practice. The AI Office consists of several bodies, including a scientific panel, an AI board with representatives from national authorities, and an external consultancy body. Within two years, businesses must adapt to the AI Act, starting with larger companies. Compliance will be encouraged through transparency competition, with companies vying for a quality seal. Additionally, the AI sandbox will provide tools and best practices to help small and medium-sized enterprises (SMEs) comply with the AI Act. National agencies play a crucial role in providing SMEs with the tools and support needed for compliance.

​

Philipp Hacker: A crucial aspect of implementing the AI Act is hiring the right people and timely preparation. Recruitment faces competition from the industry, and it’s essential to synchronize AI regulators with existing regulatory authorities. To achieve this, a framework should be designed to prevent companies from navigating multiple, conflicting regulations. The AI Act currently includes language that allows the integration of compliance mechanisms, but having more comprehensive frameworks would be even more beneficial. For AI systems not previously regulated, companies are allowed to self-audit and certify their compliance. Market surveillance authorities oversee this process, but they do not examine every case, and regulatory approval is generally not required except in specific areas and circumstances. The AI Act’s effectiveness will depend on recruiting expertise from the tech sector and developing essential guidelines and codes of practice, especially for SMEs. Since SMEs are now in a heavily regulated industry, they require more guidance. 

​

Chinese Experts

​

1.    What are your thoughts on the scope of the EU AI Act? 

​

Weixing Shen: Europe has opted for an all-encompassing definition of AI, which could potentially cause the EU AI Act to be overly broad in its scope of application. While the Act primarily targets high-risk use cases of AI systems, the adoption of such an inclusive definition might inadvertently result in excessive regulation. This, in turn, could impede industrial development by imposing unnecessary restrictions.

​

Similar to the GDPR, the EU AI Act has extraterritorial effects. Many were surprised when the GDPR extended its reach to regulate global actors. However, due to Europe’s market size, many US and Chinese companies have complied with the GDPR in practice. As a result, GDPR-style long-arm jurisdiction has gained popularity. The effectiveness of this type of jurisdiction relies not solely on geopolitical power or the ability to legislate first, but more significantly on whether the law itself addresses specific needs. Countries often gain insights from each other’s experiences, yet they adapt such knowledge to their unique circumstances in order to refine their own legal systems, rather than simply “borrowing” without modification. Ultimately, the global influence of the AI Act depends on whether it aligns with the collective value, industrial development, technological advancements, and societal needs of various jurisdictions. If an EU law can fulfill these requirements, its strength would not be due to the “Brussels Effect”, but rather by effectively capturing the most resonant consensus.

​

Furthermore, AI models or systems that have not yet entered the market or are solely utilized for pure scientific research are not subject to the EU AI Act. On this matter, the legislative stances of China and Europe are in alignment.

​

2.    How do you assess the EU’s strategy for open-source AI governance?

​

Weixing Shen: The EU AI Act’s provision of exemptions for open-source AI tools is a prudent approach, as open-source resources play a crucial role in scientific research and technological innovation. However, they can also pose risks, and therefore, exemptions should not be granted in certain cases. As per the EU AI Act, for high-risk use cases and above, even freely available open-source models must assume liability if they cause harm. Similarly, China’s AI Law (Draft for Suggestions from Scholars) states that liability should be appropriately reduced for models provided for free as open-source, unless there is intent or gross negligence. The terms “intent or gross negligence” align with the risk levels outlined in the EU AI Act. In summary, if providing a model itself poses significant societal risks, regardless of whether it is free and open-source, the provider must assume corresponding responsibilities.

​

3.    How effective do you think a risk-based regulatory framework is for AI governance?

​

Linghan Zhang: The risk-based regulatory framework, popularized by the GDPR, has been widely adopted around the world. China has also proposed establishing a tiered, classified risk regulatory framework for AI and related components. Although this type of framework offers numerous benefits, such as shifting intervention points ex-ante and reducing supervision and compliance costs, there are notable distinctions between data risk governance and AI risk governance. Currently, AI governance is absorbing data and algorithmic governance, which may present some challenges in the future. Firstly, risk-based governance necessitates effective risk assessment as a foundation. Article 3 of the EU AI Act aptly defines risk as the combination of damage probability and severity. However, due to AI’s brief application and development history, the available data for risk assessment is limited in both quality and quantity. Secondly, the significance of technical factors in AI risk assessment is diminishing, while the importance of factors like ethics, policy, and culture is increasing. The US AI Risk Management Framework categorizes risks into technical, socio-technical, and guiding-principle risks; the UN High-level AI Advisory Body divides risks into risks to individuals, groups, and society. While these risk frameworks appear comprehensive, policy differences and institutional dynamics across countries may lead to oversights regarding AI activities that warrant concern.

​

Another challenge lies in categorizing high and low levels of risk in advance. The EU AI Act’s approach revolves around critical information infrastructure and use cases tied to fundamental human rights, addressing potential threats to health and safety, as well as substantial adverse impacts on basic rights, the environment, democracy, and the rule of law. However, these criteria remain highly flexible and ambiguous. In essence, the EU’s list of prohibited or restricted AI systems is more about importance than actual risk level. While an AI system’s use case may be significant, it does not necessarily imply a high risk. Besides, the risk associated with some potentially high-risk AI systems, such as job assessment and self-driving cars, may decrease as their usage increases. As these applications become more widespread and data more abundant, the likelihood and severity of harm caused by these systems could be mitigated. Labeling them as high-risk and restricting them from the outset may eliminate the opportunity to reduce risk later on. In future AI risk governance, balancing these factors will be crucial. China’s AI Law (Draft for Suggestions from Scholars) refrains from using terms like “unacceptable risk” or “high risk,” opting instead for descriptions like “critical AI systems” or those in “special application scenarios.” This approach seeks to avoid prematurely assigning values or negative evaluations, while incorporating dynamic evaluation into the governance of AI systems that pose genuine risks.

​

4.    What is your opinion on the regulation of AI systems with high impact and general-purpose AI (GPAI) models/systems?

​

Linghan Zhang: “General” refers to the scope within which a risk may occur, while “high” pertains to the severity of the risk. During the drafting of China’s AI Law (Draft for Suggestions from Scholars), debates arose concerning the regulation of GPAI. Firstly, what constitutes a GPAI model? Does its application in two or three fields or scenarios qualify it as such? Is a GPAI model truly more powerful when applicable to multiple scenarios? European experts also noted that smaller, specialized models may have greater capabilities, while models with broader applicability could experience diminished functionality and impact. Moreover, although GPAI models may present a wide array of risks, people are more concerned about AI systems that perform critical functions in critical scenarios. For instance, the development of smart courts in China is advancing rapidly, and the judicial context is undeniably a critical and potentially high-risk AI application scenario. Nevertheless, some AI systems that enhance the efficiency of judges’ case management are, in fact, quite low-risk. Further research and dialogue are needed to determine whether GPAI models should be classified as high-risk and whether their impact should be evaluated based on severity or scope.

​

5.    What do you consider the main implementation challenges faced by the EU AI Act? 

​

Weixing Shen: The EU AI Act has initiated a global competition in AI legislation, prompting everyone to consider the optimal timing for AI regulation. A primary motivation for legislation is to mitigate public distrust of AI and establish a trustworthy and responsible AI framework. To achieve this, laws may need to strike a balance between fostering innovation and imposing restrictions. In this context, the implementation of the EU AI Act faces several challenges. Firstly, the Act’s high legal liability may create a chilling effect on the market, disrupting the balance between industrial innovation and safety. Secondly, the rapid pace of technological change generates tension with legal stability. Thirdly, risk classification and tiering directly influence obligations, but the criteria are ambiguous and subject to change as technology iterates. Ultimately, the fundamental question remains: can legislation alone alleviate AI concerns? The answer is no. Overreliance on strict legislation to prevent potential AI risks is misplaced. Instead, the principle of “let he who tied the bell on the tiger take it off” should be adopted. In line with the “privacy by design” concept mentioned by European experts, legal values should be integrated into the design of AI models/systems from the outset, instead of imposing severe penalties after issues arise. Regulated self-regulation may be the most effective approach to reducing AI risks.

​

Responses from EU Experts

​

Philipp Hacker: Regarding the scope of the AI Act, there are two clarifications to be made. Firstly, the definition of AI does not include Excel and other automated processes, as explained in the Recitals. Automated systems with rules crafted by humans are not considered AI. Secondly, concerning extraterritorial effects, the EU casts a wide net similar to the GDPR. However, the AI Act only applies if you use your models or offer AI-based services within the EU. In essence, to conduct business in the EU, you must adhere to EU rules, and the same principle applies to EU companies operating in China, who must comply with Chinese regulations. This is a standard principle in market regulation. Notably, the AI Act’s extraterritorial effects are unique in their approach to copyright. Even if an AI model is trained in Beijing, it must still comply with EU TDM exemptions (authors’ right to object to their copyrighted material being used for AI training) when deployed or used in the EU. This is a departure from traditional copyright laws that followed a territoriality principle, where only local laws applied. Moving forward, regulators should exercise caution in enforcing the AI Act, particularly with companies unfamiliar with the regulation, such as those from China.

​

Carme Artigas: The effective negotiation is one where all parties are equally unsatisfied, and the AI Act has achieved this equilibrium with its crucial trade-offs. In the end, I want to make three notes. Firstly, the high-risk classification is well-defined in the AI Act, which takes into account the ever-changing nature of technology and allows for updates. Secondly, the Act’s extraterritorial effects apply to both European companies and external companies serving the European market. However, European companies selling products outside Europe, even in prohibited use cases, are not affected. For example, a European company could develop a social scoring technology and sell it to countries like China or Africa. Lastly, the AI Act does not directly address copyright issues in generative AI. Copyright law is indeed territorial, and there are 27 distinct copyright legislations in Europe. Besides, AI-generated content should be marked to avoid confusion. However, using generative AI to harm fundamental rights and infringe upon human dignity, even with a watermark, remains subject to regulation. Disclosure alone does not offer indemnity.