Could AI Explainability Reinforce Existing Power Structures?

Transparency in artificial intelligence (AI) is often hailed as the key to fairness, trust, and accountability. The idea is that making AI systems more explainable will resolve their ethical issues. This perspective, however, is dangerously misleading. The rise of transparency requirements across healthcare, education, and public services has spurred a surge in algorithmic audits, explainability tools, and compliance reports. While these measures are presented as necessary for ethical AI, they have spawned an industry of consultants and technocrats who profit from managing compliance—without addressing the core ethical issues.

Reading settings
TL;DR / Summary: Transparency in artificial intelligence (AI) is often hailed as the key to fairness, trust, and accountability. The idea is that making AI systems more explainable will resolve their ethical issues. This perspective, however, is dangerously misleading. The rise of transparency requirements across healthcare, education, and public services has spurred a surge in algorithmic audits, explainability tools, and compliance reports. While these measures are presented as necessary for ethical AI, they have spawned an industry of consultants and technocrats who profit from managing compliance—without addressing the core ethical issues.

I argue that by creating bureaucratic hurdles and reducing ethics to metrics, transparency becomes a tool for maintaining corporate power while diverting attention from the deeper questions of control and governance.

Transparency in artificial intelligence (AI) is often hailed as the key to fairness, trust, and accountability. The idea is that making AI systems more explainable will resolve their ethical issues. This perspective, however, is dangerously misleading.

The rise of transparency requirements across healthcare, education, and public services has spurred a surge in algorithmic audits, explainability tools, and compliance reports. While these measures are presented as necessary for ethical AI, they have spawned an industry of consultants and technocrats who profit from managing compliance—without addressing the core ethical issues.

Public institutions, already stretched by budget constraints, must divert resources from essential services to meet transparency standards, often relying on expensive external firms. Yet these measures rarely tackle the most fundamental ethical questions: Who controls the data? Who profits from its collection? Who decides how AI systems are deployed? These questions become buried under layers of bureaucratic procedure, turning transparency into an end in itself.

A clear example is the 2016 partnership between Google’s DeepMind and the UK’s National Health Service (NHS). While this collaboration was praised for its transparency, it obscured the fact that vast amounts of sensitive patient data were being handed over to Google. The transparency measures focused on how the data was processed, ignoring the more important question of who controlled and benefited from this transfer. The illusion of accountability disguised a real ethical issue: ownership and control over public health data by a private corporation.

In practice, transparency often serves to maintain the status quo. It creates elaborate oversight mechanisms that simulate ethical responsibility while leaving the actual dynamics of power and control unchallenged. The illusion of accountability obscures deeper power structures that drive inequality, turning transparency into a tool for managing rather than addressing systemic harm.

The Problem with Metrics

One of the most concerning aspects of the transparency movement is the reduction of ethics to quantifiable metrics. Corporations now emphasize how many audits they conduct or how many compliance reports they file. This focus turns ethical oversight into a superficial exercise, diverting attention from the real human impacts of AI systems.

For instance, AI systems used in hiring processes often undergo audits to ensure they are free of bias. However, these audits tend to focus on surface-level adjustments—like removing specific biased variables—without questioning whether these systems should be making such critical decisions in the first place. In Amazon’s AI recruiting tool, which was found to be biased against women, the focus remained on how to make the tool more transparent rather than on whether it should be used for hiring decisions at all.

Similarly, Facebook’s 2019 civil rights audit revealed biases in its content moderation algorithms but failed to address whether algorithms should wield such immense power over public discourse. By reducing ethics to compliance checklists, organizations avoid confronting the deeper questions about AI’s role in perpetuating social and economic inequalities.

This reduction of ethics to metrics allows corporations to appear ethical without confronting the exploitation their systems perpetuate. Transparency, in these cases, distracts from the real human impacts of AI on marginalized groups and creates a superficial accountability that does little to address structural inequalities.

The Costs of Compliance: Burdening the Vulnerable

The financial and social costs of transparency-driven compliance are borne not by the tech giants creating AI systems, but by public institutions, workers, and marginalized communities. Hospitals, schools, and social services must divert funds from essential programs to comply with transparency mandates, reducing access to critical care and support for those most in need.

In the private sector, transparency sometimes justifies intensified surveillance and algorithmic management of workers. AI systems that monitor worker productivity in minute detail are marketed as “transparent” and “accountable.” However, these systems often enforce dehumanizing work conditions, pushing employees to meet unrealistic targets and causing stress and burnout. Consumers also bear the costs of compliance, as companies pass on these expenses through higher prices while smaller competitors struggle to keep up with costly regulatory demands.

Ultimately, transparency-driven bureaucracy reinforces the dominance of large corporations while shifting the burden of compliance onto those least able to bear it. This creates a system where the already oppressed—public institutions, workers, and marginalized communities—are disproportionately affected by transparency’s costs.

Selective Transparency: Manipulating Openness to Conceal Power

A particularly troubling aspect of the transparency agenda is its selective nature. Corporations often disclose just enough information to appear transparent without relinquishing real control. By sharing superficial details—such as basic algorithmic processes—while withholding essential insights—like data ownership or decision‑making criteria—companies manipulate public perception without addressing underlying power dynamics.

For instance, tech companies involved in extensive data collection may disclose how they gather data but remain silent on how that data is shared with third parties or used for profit. Uber, for example, has shared information about how its AI optimizes driver routes but is less forthcoming about how its algorithm determines driver wages or deactivates drivers based on performance metrics.

This selective transparency creates the illusion of openness while allowing corporations to maintain control over AI systems and prevent meaningful scrutiny. By controlling what is disclosed, selective transparency enables corporations to evade real accountability while continuing to profit from systems that entrench inequality and exploit vulnerable populations.

Reimagining Accountability: Empowering Communities and Decentralizing Control

If transparency merely reinforces existing power structures, how can we achieve genuine accountability in AI? A crucial step is shifting the focus from transparency to control—specifically, determining who designs, implements, and governs AI systems.

Empowering communities by involving them in decision‑making can help ensure that AI technologies serve the public interest rather than corporate agendas. This may involve participatory design methods, open‑source development, and policies that promote collective ownership and stewardship of AI systems. By decentralizing control, we can create technologies that respond to the needs and values of diverse groups instead of imposing a one‑size‑fits‑all model from powerful entities.

The DECODE project in Barcelona illustrates how decentralized governance can empower communities. Citizens control their own data, deciding who accesses it and for what purposes, creating a more equitable model of data governance. This demonstrates that decentralizing AI control can align systems with collective needs rather than reinforcing corporate power.

This approach also requires questioning whether AI is always the best solution. Rather than defaulting to technological fixes, we should consider alternatives that prioritize human judgment and community input over algorithmic decision‑making. By resisting AI’s encroachment into areas where it may cause more harm than good, we can promote fairness and equity in technology use.

Beyond Transparency: Building a Just and Equitable Future

Focusing solely on transparency is insufficient to address AI’s ethical challenges. To build systems that genuinely serve the public good, we must confront and dismantle the power structures governing AI development and deployment. This requires rethinking technology governance, moving away from centralized corporate control toward models that prioritize social justice, equity, and collective well‑being.

By challenging existing hierarchies and advocating for the democratization of AI, we can work toward a future where technology empowers many rather than reinforcing the privileges of few. This involves not only critiquing current practices but also actively developing and supporting alternatives that align with fairness and community empowerment.

Continue reading

Related essays

Next routes

Return to the archive, the guide, or a related route to keep the thread moving.

Version history

No prior versions in this archive snapshot.

    Get essays like this by email.

    Get new essays by email

    An occasional note when a new essay goes live.

    Get new essays by email

    An occasional note when a new essay goes live.