AI on Trial: Who's Guilty When Machines go Wrong

By Tarush Saitia and Pranjali Maithani 

I.   INTRODUCTION

The exponential rise of the Artificial Intelligence industry and its ever-increasing significance in our day-to-day lives bring forth a set of complex issues, one of them being the question of who will face the consequences of an algorithm’s failure. In a world where 50% of work today is automatable, decrees driven by AI carry immense significance, affecting careers, healthcare outcomes, financial futures, etc. Yet the contemporary legal and ethical frameworks barely manage to keep up with the ever-changing dimensions of AI.

The growing clout of AI in each and every sector has ultimately led to many opportunities accompanied by many unprecedented challenges, which makes it necessary to establish accountability frameworks for the time when these machines malfunction. This article dissects the nuances of the liability of AI and proposes feasible solutions for the same.

   II.   WHAT IS AI LIABILITY

Let’s try to comprehend this concept by painting a short scenario: While driving your brand new automated car, you divert your attention to something else, trusting the AI system to navigate the roads safely for you but unfortunately you end up in a deadly crash. Now the question arises: Who is to blame? Was it the AI's fault, or did the system fail because you weren't paying attention as the car requested? Should the company that designed the AI system be held responsible? Or is it you, the driver, who ultimately carries the burden?

The key question being: Who is at fault when AI goes south?

        III.   BARRIERS TO AI LIABILITY FRAMEWORKS

If the problem of accountability carries such immense significance, then why do we still not have proper legislation or a framework governing the same? Outlined below are the key reasons for the lack of proper legislation or framework:-

1. The Black Box Problem

Most AI systems have the feature of machine learning embedded in them, which means that they generate outputs to the inputs given and also tend to learn from the inputs. Hence, making the systems capable of generating different answers to the same question each time.

It is somehow similar to the way children are taught. The system is fed with certain precedents  to find a pattern. But the system develops its own neural network through its pattern-finding inclinations, making it barely possible to predict its response. This inability to see how deep learning systems make their decisions is defined as the Black Box Problem. This makes it difficult to trace how AI systems arrive at certain decisions, making it tough to assign liability.

One of the prominent examples emphasising on the significance of the Black Box Problem as a barrier is the backlash faced by BM’s Watson in 2017 for Oncology after recommending unsafe cancer treatments, highlighting the dangers of opaque AI decision-making. The black box nature of its system eroded trust among medical professionals and raised serious concerns about patient safety.

2. Lack of Frameworks

The traditional liability laws were crafted for a world of human decision-makers, typically built on the foundations of negligence, intent, and foreseeability, which crumble upon the application of AI systems. These systems operate on complex algorithms that their developer may not fully understand. In addition to this, AI systems, especially those that use machine learning, evolve as they interact with data, users, and various inputs; hence, a system might be safe at the time of deployment but could develop harmful behaviors over time.

In 2018, Tesla’s Autopilot system was involved in a fatal crash, where the driver, Walter Huang, died when his Model X collided with a highway barrier. Tesla denied liability, arguing the driver was aware of the system’s limitations. This case highlights the lack of a clear regulatory framework for autonomous vehicle liability, leaving accountability in question.

3. Fragmented Accountability

Due to the increasing automation of AI, the risk control has shifted from the users to its producers. The involvement of multiple stakeholders has rendered the ‘traditional liability determining’ methods quite inefficient.

In 2024, a class-action lawsuit was filed against Workday, alleging that its AI facilitated hiring software discriminated against applicants based on race, age, and disability. The lack of specific regulations governing AI led to the problem of fragmented accountability, limiting the ability to seek justice.

This is the dilemma of Fragmented Liability, which serves as another key reason for the lack of proper legislation in AI accountability.

IV. FEASIBLE SOLUTIONS

The rising complexity of AI has led to the creation of a landscape where no one solution is widespread enough to cover all ambiguities. The situation still remains dicey, but several promising approaches have emerged to tackle the intricate challenge of AI liability; the same are given below:

A.    The ‘Collective AI Agency’ Theory

To address the accountability and liability issues arising from autonomous actions, this theory proposes a blanket solution that vouches for granting limited legal personhood to artificial intelligence through piercing the veil akin to what’s done in company law. This model grants a unique legal status to AI systems imposing clear accountability by giving it a legal standing but without human-like rights. The model works by identifying the key stakeholders like developers, deployers, and operators and assigning them as official, legal representatives of the said AI. This could be similar to the legal standing corporations are given in civil law.Whenever a legal issue arises due to the autonomous decisions of the AI, the financial and legal responsibility gets divided among the representatives based on their roles and involvement. For instance, a developer might bear greater responsibility if a flaw in the coding led to the issue, while an operator might be more accountable if the problem arose from poor maintenance or misuse. This theory ensures accountability while encouraging strong safety measures.

One of the major perks of this solution is its attempt at maintaining a balance between risk and innovation by not holding a single party or a stakeholder completely liable for all the actions and potentially discouraging further AI development, erstwhile preventing the loophole of ‘AI did it’ to escape accountability by spreading liability appropriately.

Basically, imagine a company where shareholders are responsible for major decisions; this works similarly, but for AI systems and their actions. The people who build, deploy, and run these systems become like board members, sharing the responsibility when things go south.

B.    The Safety Nets

The inception of the unprecedented challenges of the autonomy of AI led to the emergence of the concept of The Safety Nets, which consist of mandatory insurance schemes and compensation funds, efficiently tackling the challenge of AI liability.

The idea of the establishment of various compensation funds involving the stakeholders is similar to an idea presented in the European Parliament in the year 2017. The idea proposed the forming of such compensation funds for either all AI systems or specific robot categories to limit the liability of the owner, manufacturer, programmer, or user by making significant contributions to the fund. This trade-off would not only benefit the victims of the accident but also the stakeholders at fault by discharging them from the punitive damages they might face because of the accident.

C.    Innovative Integrity

This concept puts forth the idea of making technological advancements in AI while making sure that the ethical standards are followed and accountability is taken into due consideration, which in turn gives rise to the concept of Sandbox Environments. A sandbox environment creates a space for the new AI systems to be tested multiple times and monitored before being rolled out in the real world. The advantage of running the tests in such a controlled environment is that it ensures that there are no security breaches. Hence, making sure that all the errors and possible issues are dealt with before they bring any real-world repercussions. The concept of a sandbox environment ultimately helps in mitigating the risks of a default in a system.

V. CONCLUSION

When glancing down at the challenges that the complex web of AI poses, the need for a robust framework is quite evident for safeguarding the society from potential harms.

The complexities of AI are coupled with multiple issues, such as the dearth of regulatory frameworks, the black box or AI accountability issues. The most pressing concern is its largely uncharted territory. People have been trying to figure out possible ways to know what comes next but the nuances of AI make it impossible to guess what’s in the store for us. The development of AI and the risks associated with it go hand in hand and it becomes imperative to brace the world for any potential challenge, be it the case of mishaps by self-driving cars or the robotic guards losing control of the ride. The world needs to be prepared for anything and everything. Concepts such as collective AI agency, along with the implementation of safety nets and sandbox environments, can significantly mitigate the risks associated with AI technologies. These strategies represent a proactive approach to achieving a balance between harnessing the benefits of artificial intelligence while ensuring accountability and safety.