We’re often told that injustice “emerges” from complexity: that no one meant harm, no one can be held accountable. Yet from hospitals to welfare offices, disaster funds to high-school admissions, exclusion is engineered through proxies, thresholds, contracts, and automated rules. To dismantle systemic injustice today, move beyond mapping emergent patterns and toward reconstructing intentional design.
When an earthquake kills hundreds but the catastrophe bond covering it pays nothing because its magnitude fell below a contractual threshold, complexity is not the story. Contract design is.
In policy halls, academic journals, and tech boardrooms, emergence has become a shield. Models adapt. Networks shift. Feedback loops spiral. When harm follows, we’re reassured it was unintended, “just the way the system works.”
This sophistication obscures choice. Which proxy stood in for human need? Who drew the cutoff line? Who coded the filter to safeguard profit?
Patterned exclusion, traceable along racial and economic fault lines, is strong evidence of design responsibility: someone chose the proxy, the cutoff, and the filter. Whether those choices were commissioned or merely accepted once they proved profitable is what a forensic investigation would have to establish — the pattern alone does not decide it. The reconstruction should start there rather than assume the answer.
When the same communities are repeatedly cut out, and when denial flows directly into institutional profit, we don’t need theories of unpredictability. We need forensic reconstruction.
Exclusion is written in before launch
Across sectors, harm is embedded from the start in procurement, optimization, and risk modeling, long before anyone “discovers” it after deployment.
A 2019 Science study revealed that a widely used healthcare risk algorithm systematically under-prioritized Black patients. It used future healthcare spending as a proxy for medical need, baking historical inequities into resource allocation. (Obermeyer et al., Science, 2019)
The same pattern shows up in public services. Indiana’s $1.3 billion IBM contract introduced automated denial scripts that treated missing paperwork as a “failure to cooperate.” That denial rule was the design choice, and thousands lost benefits through it. A $78 million court judgment against IBM for breach of contract, final in 2019, showed the state had paid for a system that did not meet the contract it signed. (Associated Press, 2019)
Financial engineering does the same. Following two major earthquakes in 2017, Mexico’s World Bank–backed catastrophe bond paid out for an 8.1-magnitude quake—but not for a deadlier 7.1 quake near Puebla. The latter struck inside the covered zone but, at 7.1, fell short of the bond’s magnitude trigger. (Artemis.bm, 2017)
New York City’s high-school admissions algorithm weighted cut-scores and priorities that systematically filtered out Black and Latinx students. The system’s design, not its size, reproduced segregation patterns. (The Markup and THE CITY, 2021)
The instruments of exclusion are technically simple and politically potent: proxy variables that use biased stand-ins like spending instead of need, parametric triggers that withhold payouts even in the middle of a disaster, contractual exclusions that write out liability at the procurement phase, automated gatekeeping that codifies friction to deny access preemptively, and cut-score weights that embed advantage through numerical screens.
Someone picks each of these before launch, and the people they screen out lose the care, payout or school seat they sought.
A line of code, a contract clause and a numeric threshold all settle who gets what, which makes each of them a policy decision.
Calling harm conditionally emergent, an unintended consequence arising only under certain systemic constraints, changes nothing, because those constraints (proxy selections, trigger rules, threshold cutoffs) are themselves human decisions.
Conditional emergence is simply a rebranding of design as accident.
Follow the file trail
Mapping influence networks is necessary. But it is not sufficient.
Real accountability demands a forensic method:
- Follow the Procurement: Who issued the contract? What exclusions were priced in from the beginning?
- Inspect the Inputs: Which proxies were selected, and which forms of social knowledge were erased?
- Expose the Thresholds: Who set the numerical barriers, and to what end?
- Trace the Profits: Where does margin accumulate when access is denied?
Every system leaves a paper trail: contracts, risk models, marketing materials, optimization reports. Accountability comes from obtaining those documents, as in legal discovery, more than from diagnosing how the system behaves.
Complexity is real, and it can genuinely diffuse responsibility across many hands. But when harm is repeated, profitable, and predictable, invoking emergence to end the inquiry is a way of declining the reconstruction — and that refusal deserves scrutiny even where no single author intended the outcome.
Actor-Network Theory helped reveal that agency is distributed. But if critique stops at mapping flows without tracing commissions, it becomes a theory of mystification.
Harm doesn’t just happen. It’s scoped, signed, coded, and sold.
When denial repeats along the same lines and someone profits from it, start from the assumption that it was designed.
Treat this as an investigative hypothesis rather than a verdict: the pattern establishes that someone built the rule that produces it, and the profit establishes who benefits.
The evidence is in the file trail. Are we ready to follow it?
Notes & References
- Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019). “Dissecting racial bias in an algorithm used to manage the health of populations.” Science.
- Associated Press. (2019). “IBM owes Indiana $78 million over welfare automation services, court says.”
- Artemis.bm. (2017). “Mexico confirms $150m cat bond payout for quake” (11 October) and “Mexico City hit by deadly M7.1 quake, causes significant property damage” (20 September).
- The Markup and THE CITY. (2021). “NYC’s School Algorithms Cement Segregation. This Data Shows How.” 26 May.