Regulation of Risks Associated with Usage of Artificial Intelligence: A Comparison of the Regulatory Regime between India and the European Union

I. Introduction

Artificial Intelligence (AI) has been a revolutionising phenomenon which has impacted several aspects of people’s lives including but not limited to the use of technology, the manner in which industries operate, educational tasks, storage of data, various healthcare facilities and even national administrative governance. With rise in tools such as Chat CPT, Bard AI and Midjourney, it has become very easy to find solutions to almost every technology related concern. However, with the rise in ease of access to a substantially self-learning tool such as AI, serious concerns arise with respect to the manner in which the dissemination of data, the storage of data as well as the creation of data will be regulated.² It has been pointed out by many literary sources that the continued use of unmonitored AI will significantly endanger the rights of both its users as well as who the tool is used against. Important rights such as right to privacy and free speech will take a significant setback if AI is continued to be used by people unregulated by effective legislative and governmental mechanisms.³ Set against the background of this concern, it is important to analyse the framework of AI regulation in India against the newly introduced (but not yet effectuated) Artificial Intelligence Act of the European Union (“EU”) introduced in 2021. The EU Artificial Intelligence Act 2021 is considered to be one of the world’s first AI focused legislative measures specifically tasked with regulating AI. This comparison will help identify any shortcomings with the Indian regime on AI so as to better enhance its efficacy in regulating any risks associated with AI.

II. Risks associated with usage of artificial intelligence

AI is the replication of human like intelligence through software incorporate in machines. To understand the areas impacted by AI which need to be regulated, it is first important to understand the risks associated with its usage.⁴ The software so incorporated is meant to function as a program that mimics human like thinking as well as learning abilities. To do this, computer systems are developed manually to help integrate AI so that it can undertake the traditional tasks that requires human intelligence, including the comprehension of natural human languages, forming patterns of recognition, integrate skills of problem-solving while also adapting to any new challenge that it faces.⁵ In this manner, all AI-based tools usually take on a self-learning aspect to them which helps them grow without any additional human assistance.⁶ Some of the risks associated with AI are as follows:

(i) An absence of transparency when AI tools are being used.

AI as well as deep learning models are quite difficult to comprehend, even when they work directly with the technology. As a result, users of such tools might find themselves at a situation where they do not find any transparency in how the AI has obtained data or come to conclusions.⁷ This creates a dearth of explanation on how the AI algorithms use the data and thus increase the scope for creating unsafe or even biased decisions. Concerns such as these give rise to the concept of ‘explainable AI’ which is the active process of creating transparent AI mechanisms that allow the user to see how data is used and processed. However, there is still a long way to go before proper mechanism of transparent AI systems become common.⁸

(ii) AI algorithms and the possibility of social manipulation

Non-transparent AI also results in the possibility of social manipulation if the data output creates biased and prejudiced results. This fear is quite prominent in light of the fact that several politicians often rely on various social media platforms backed by AI to promote their objectives. For instance, in the 2022 elections, a young politician Ferdinand Marcos, Jr created and worked through his TikTok army of trolls to secure his vote bank comprising of the younger populations.⁹ While TikTok is one such example that uses AL algorithms to create user manipulative content feed, there are several other regularly used social media apps such as Instagram and Facebook.¹⁰ AI makes it very easy to create hyper realistic photos as well as videos, audio clips while replacing them with the original ones. Thus, AI backed social media becomes a platform for the dissemination of false data, spreading misinformation and even pursue acts of horrific consequences such as war propagandas.¹¹

(iii) Social surveillance in violation of individual right to privacy

In addition to being an existential threat, a lot of warnings have also been presented towards the adverse impact of AI on an individual’s right to privacy and security. One example of this is use of facial recognition technology by China in various private spaces such as schools, offices and in public places. The Chinese government is known to track the movement of a person, can gather enough evidence to monitor the activity of a person, influence their relationship views and subsequently their political views.¹² The US government has also in a similar manner encouragingly used predictive policing algorithms to track people and their activities.¹³ Criticisms against the usage of Predictive policing algorithm include the disproportionate impact on the black and Hispanic communities and irregular arrest rates. Furthermore, even the data so provided to AI are not considered to be secure as evidence from an incident with ChatGPT which allowed several users to peek into the chat history of other active users in 2023.¹⁴ Even though there are laws that protect the dissemination and access to personal information almost in every country, there are little to no laws that explicitly protect harm caused to data privacy by AI. Additionally, the limited experiences and knowledge of legalities of the AI creators to a great extent limits the AI tools and sets it up for failure in speech-recognition when certain types of dialects and accents are unrecognizable. The developers as well as the businesses should take great care to avoid integrating unintentional biases and prejudices that puts minority populations at risk.¹⁵

III. Regulatory framework of risks associated with usage of artificial intelligence in india and its shortcomings

Currently, in India there are no specific legislations that focus only on AI, but the government has expressed its concerns on the absence of such a law. The government has expressed concerns about the lack of AI regulations to monitor the possible ethical and moral violations and the rising cases of such reported violations in India.¹⁶ The government of India has however, set up the MeiTY (The Ministry of Electronics and Information Technology) in 2016, which is tasked with addressing any IT related concern which also deals with concerns arising from and with the usage of AI.¹⁷ In pursuit of determining whether an AI based conduct or usage of AI can be unlawful within the India IT framework, the following legislative provisions are considered:

(i) Under the Information Technology Act, 2000:

The Information Technology Act, 2000 (“IT Act”) is probably the most important piece of legislation to govern any form of electronic transactions as well as digital data storage and dissemination. While the IT Act does not make an explicit references to AI, there are specific provisions whose interpretations allow them to apply to AI activities. For instance, IT Act s.32 provides users to be compensated if their data is breached in cases where their sensitive personal data has been negligently handled.¹⁸ Similarly, IT Act, s.73 penalises the electronic publishing of signature certificates. While such provisions are only applicable to IT transactions, they can be read in the context of AI systems too that process user data.¹⁹ In the case of Justice K.S. Puttaswamy (Retd.) v. Union of India,²⁰ the Supreme court of India’s recognition of right to privacy left little doubt that its application can extend to any platform, device tool or algorithm. Therefore, even in the absence of a specific legislation, the focus on right to privacy when safeguarding personal data on AI based systems is paramount.²¹

(ii) Under the Personal Data Protection Act, 2023

The personal data protection Act, 2023 was enforced with an aim to establish a comprehensive framework protecting personal and sensitive data. However, as recent the legislation may be, it also fails to make any reference to the term ‘Artificial intelligence’. The Act, however, prohibits the misuse of personal data collected both in digital or non-digital form or when non-digital data is subsequently digitized.²² In comparison to the IT Act, the Personal Data Protection Act, 2023 contains provisions that addresses personal profiling and even automated decision making and requires explicit and informed consent to be obtained from individuals prior to processing personal data using AI algorithms.²³ It defines automated as follow “means any digital process capable of operating automatically in response to instructions given or otherwise for the purpose of processing data.”²⁴ Thus, even though there is no explicit references to AI per se, this definition would to a great extent jurisdiction of the courts over AI-related misuse of personal data should it be found that the right to privacy of any user is infringed.²⁵

(iii) Under the Indian Copyright Act, 1957

The Indian Copyright Act, 1957 (Copyright Act 1957) was enforced with an intent to safeguard safeguards original literary, artistic, musical, and dramatic works, granting exclusive rights to creators and prohibiting unauthorized use or reproduction.²⁶ However, due to a rise in the AI-generated content, a lot of discussions have prompted the question on copyright ownership and infringement when they are AI created.²⁷ However, this issue was addressed in the case of Gramophone Company of India Ltd. v. Super Cassettes Industries Ltd. (2011)²⁸, where the Delhi High court had held that, any literary, artistic or musical composition created or generated by AI or a computer program lacks the required human creativity and will therefore be ineligible to be protected as copyright. Thus, the Indian Copyright Act, 1957 protects original human produced copyright-based work from being compared or overshadowed by AI based artistic creations.

(iv) Under the National E-governance plan

The National E-governance plan aims at digitally empowering the administrative and government departments that use AI and provide services online. It is undeniable that AI plays a vital role in how the efficiency and accessibility of e-governance is enhances and made more effective. As a result, various Indian government departments have either already integrated AI into their systems or are seeking to do so to improve decision-making and enhance the services to its citizens.²⁹

(v) AIRAWAT (AI Research, Analytics and knowledge Assimilation)

AIRAWAT (AI Research, Analytics and knowledge Assimilation) is an AI-based cloud computing infrastructure which is installed under the National program on AI by the Niti Ayog. This platform specifically focuses on the AI based requirements in India.³⁰

The shortcomings of the Indian regulatory framework dealing with AI. Some of the shortcomings identified are inclusive of but not limited to the following:

(i) A lack of AI-specific legislation

Currently India does not have any dedicated legislation that specifically focusses on AI. Even though as aforementioned there are several provisions within the existing framework such as the AI Act 2000 and the Personal Data Protection Bill, 2023 which extend interpretive support to AI based tools, there are no comprehensive provisions nor laws that address the unique challenges that come with AI or the complex nature of AI itself.³¹ Furthermore, even though a data protection board was established by way of the Personal Data Protection Act, 2023, there is still a need for a dedicated regulatory body that helps in the comprehensive oversight of AI technologies. The absence of such a body can and will create fragmented oversight of AI related activities and misuse in India.³²

(ii) Lack of transparent ethical guidelines to regulate AI

Currently, there are no well-defined nor any enforceable guidelines for companies to follow while developing AI based tools. This can and will lead to inconsistent practises on AI usage and also misuse of the AI systems.³³

(iii) Unaddressed concerns of bias and discrimination

After all, AI are also human created and therefore may inadvertently perpetuate bias or discrimination since they rely on the historical data fed to their systems which are already tainted with historical bias. No framework nor regulation in India explicitly addresses these potential issues of bias and discrimination.³⁴

(iv) Creation of accountability and liability

AI systems are very complex and even autonomous to a great extent, which makes it difficult to assign any liability to it in cases where there are harms or errors caused by such systems. Since they are created by humans, it would only seem logical that liability be assigned to the hands that shaped it, however, automated AI is quite complex and sometimes acts in contravention to what the creator intended. In such cases, the absence of a robust mechanism creates ambiguity in the assignment of liability of the entity that truly made the error.³⁵

(v) Ambiguity associated with intellectual property rights (IP)

Existing IP laws in India make little to no reference to the possibility of AI created content, innovation or inventions. As a result, if and when ambiguity arises on the nature of ownership of such AI generated product and the patentability of such content, there will be no simple solution to address such attribution issues.³⁶ Thus, with an increase in AI, it is important to recognise the significant nature of effectively regulating AI for both ethical as well as moral purposes. Yes, India has a lot of laws in place within its IT framework to address the associated challenges. However, in the absence of a comprehensive framework that specifically deals with AI, potential misuse of the existing provisions to argue that they do not apply to AI will be possible. Additionally, AI will continue to advance, and it has only become more crucial to monitor the legal developments to ensure that the laws have kept in pace with the right technological developments. while balancing individual rights.³⁷

IV. Regulation of risks associated with usage of artificial intelligence in the european union

Failures caused by reliance on Artificial Intelligence in the recent years have made a lot of headlines. For instance, a Tesla car that relied on autopilot crashed because of AI failure, the recruitment tool of Amazon which also ran on AI was found to show bias against women and similarly, TAY which is Microsoft’s AI chatbot, was shown to manipulate users to make sexist and racist remarks.³⁸ Such rising concerns have led to the European Union (EU) developing the which sought to establish a specifically focused governance and enforcement mechanism to protect human rights and the safety of the users when using AI. This Act has not yet come into effect, however, since relevant literary sources refer to it as the “AI Act” the same pattern will be followed here.³⁹ This AI Act will be the first major AI law by a major regulator. It seeks to ensure that AI is used safely and responsibly whole keeping the interests of the users as well as the enterprises in mind. It is the first step towards a focused and comprehensive regulatory framework for AI in the EU and is hoped that it creates the possibility of equal application of law and level playing field for enterprises dealing with EU.⁴⁰

 Risks identified and assessed under the AI Act

The AI Act identifies three categories of risks⁴¹:

(i) Applications and systems that create an unacceptable risk, these are inclusive of but not limited to the Chinese government-run social scoring app.

(ii) Application based risk. These include tools that help in scanning CV, place a rank on the job applicants and as such create a high-risk of bias and discriminatory output.

(iii) Risk arising from applications that are not explicitly banned but are still or listed as high-risk are largely left unregulated.

This legislation prohibits the creation of those AI systems which create an unacceptable risk from being used or deployed in EU while if the risks are lower, then it places varying levels of obligatory requirements on the enterprises that are dealing with them in the EU. While there are three types of risks that the AI Act deals with, it mostly classifies the AI and its usage as either ‘high risk’ or ‘limited risk’.⁴² Very soon, the AI Act will also be used to regulate the deployment of foundation models, that deal with the measures that are adopted to ensure compliance with EU copyright and other IP laws to publish detailed reports concerning the manner in which content is being fed into the system, the type of content being used and the extent to which the technical documentation are prepared for the use of such AI models. The Act will come into force only two years after its entry while some of the provisions might even come into force at a later date. The Act might thus come into effect in or around 2026.⁴³

 Applicability of the AI Act

The Act will govern both providers as well as deployers alike with respect to the manner in which the AI systems are used or produces an effect in the EU, regardless of their place of origin. It implies that the even the AI systems located in foreign or third countries outside the EU will have to comply with the EU AI act if they wish to use the system in EU. This would also apply to UK which has recently broken off from the EU.⁴⁴

The AI Act adopts the same definition of Artificial intelligence system which was proposed by the Organisation for Economic Co-operation and Development (OECD):"An AI system is a machine-based system that [...] infers from the input it receives how to generate outputs such as predictions, content, recommendations, or decisions that can affect physical or virtual environments."⁴⁵ The AI Act will, however, not be applicable to the following types of AI systems:

(i) used exclusively for military or defence purposes.

(ii) used solely for the purpose of research and innovation; and

(iii) used by people for non-professional reasons.

There are certain types of applications that are subjected to a complete ban under the EU AI act for engaging in use of features like emotion recognition especially, if they are carried out in public spaces.⁴⁶ These also include the usage of features such as scraping facial images, Closed-circuit television (CCTV) footage, using such systems to carry out remote biometric identification in public could be permitted to a certain extent, provided they are subject to strictly legal law enforcement objectives. No matter what the purpose, the Act mandates necessary safeguards in place to limit the use of such systems to carry out public searches for the people that could be suspected of crimes.⁴⁷

 Compliance requirements

Under the EU AI Act, there are various rrquirements for the companies to ensure and they all depend on the level of risk that the proposed AI system poses. For instance, the AI systems that present a limited risk are subject to a lighter obligatory requirements, such as informing uses of the content that they are engaging with is backed by AI or is generated by AI, the risks and liabilities associated with the use of the AI system or its generated data.⁴⁸ However, high risk-based AI systems would only be authorised if it they are subject to tougher requirements and obligations such as having the necessity to carry out mandatory fundamental rights impact assessment.⁴⁹ The citizens will thus have the right to obtain transparent explanation on the decisions carried out by the high-risk AI systems, how it arrived at that decision and the impact on their rights. However, AI systems that demonstrate an unacceptable level of risk would be completely prohibited. Some of the examples of such unacceptable level of risk-based AI systems are as follows⁵⁰:

(i) Limited risk: chat bots or deepfakes.

(ii) High risk: AI used in sensitive systems, such as welfare, employment, education, transport; and

(iii) Unacceptable risk: social scoring based on social behaviour or personal characteristics, emotion recognition in the workplace and biometric categorisation to infer sensitive data, such as sexual orientation.

Penalties

The Penalties under the EU AI Act are very similar to how they are calculated under the European General Data Protection Regulation, wherein fines are awarded if the EU AI Act are violated and the fine will be calculated as a percentage of the guilty party’s global annual turnover of the previous financial year or imposed a fixed sum of fine, whichever is higher⁵¹:

(i) €35 million or 7% for violations which involve the use of banned AI applications.

(ii) €15 million or 3% for violations of the Act's obligations; and

(iii) €7.5 million or 1.5% for the supply of incorrect information.

The EU AI Act does place proportionate caps in place when issuing administrative fines for small and medium enterprises as well as startups. The citizens will thus be able to file complaints about AI systems that negatively impact their rights. ⁵²

V. Comparative analysis of the regime governing risks associated with usage of artificial intelligence in india and eu

An analysis of the various legislative provisions pertaining to the information technology related issues in India and the upcoming EU AI Act identifies four distinct differences between the two systems. They are as follows:

1. The Indian regime on information technology in India does not refer to the term ‘AI’ which is now present in the soon to be introduced EU AI Act

As discussed above, currently, there are no laws in India that specifically refer to AI objectively. There are either passing references or merely references to the concept of automation. The closest that the Indian regime comes to any reference to AI based systems at all is under the Personal Data Protection Act, 2023 which defines what ‘automated’. It is this definition that permits the Data Protection body to apprehend possible perpetrators who misuse or are negligent with sensitive data. In comparison, the EU AI Act is more objective and specifically focuses only on AI-based risks. The EU AI Act also adopts a broad definition of AI System, which may be adopted into the Indian regime.

2. There are no discussions on the risks associated with AI-systems in India as compared to the EU AI Act which lays down various categories of risk.

While the Indian regime on IT does make passing references to AI based systems, there is literally no provision that deals with or identifies the risks associated with AI. Due to the complex nature of AI, one cannot assume that the risks associated with technology is the same with that of AI. The governance mechanism is supposed to ensure that it specifically deals with AI related issues and not just cover every other issue related to IT. Unlike the Indian system, the EU AI Act provides a strong framework for various types of risks, each type of risks carries with it different types of obligations and different types of threshold of safety measures to be taken. A framework such as this is not very complicated to instil in the Indian system.

3. There is no discussion nor provision for the establishment of a complaint mechanism in India for consumers to file concerns as compared to the EU AI Act which provides the scope for the establishment one.

As identified in the EU AI Act, the citizens will be able to file complaints about AI systems that negatively impact their rights with the relevant complaint authority. However, no such mechanism is presented here. As pointed out earlier, due to the complex nature of AI, one cannot assume that the risks associated with technology is the same with that of AI. Therefore, it is important to have a specific authority that focuses exclusively only on issues related to AI, so that they are equipped with people who are experts on such fields to better address such complex issues.

VI. Conclusion

A discussion on the various IT legislation in India dealing with technology related issues shows that there is little to no reference to AI or AI based system at all. In comparison, the EU AI Act, provides a plethora of significant provisions that would help establish a comprehensive network to govern and regulate the issues pertaining to AI. Therefore, even though, the EU AI Act is yet to be enforced, it would only benefit the Indian regime to borrow some of its features. For starters, a definition of AI or AI systems could be amended into the IT Act 2000 and provide for a discussion on the risks associated with it. In fact, even a simple introduction of a set of guidelines on AI would help enterprises carry out better AI management strategies that safeguard the rights of citizens involved.

*****