A worldview should not have a gun.
Say a police department buys a platform. The platform ingests warrants, license plate reads, social media and unpaid fines, and produces a list of people worth attention. The department didn’t write the definition of “worth attention.” The vendor did, and it’s a trade secret. The officer who knocks on the door is acting on a classification nobody in the city can read, produced by a company that will never meet the person behind it.
Something has changed. Systems that sort people into “threats,” “militants” or “risks” used to be analysis, something a decision-maker consulted. Wired into police dispatch and drone tasking, and piloted in prison phone monitoring, they’re now the rails that decide where force lands, and the route has replaced the map.
Palantir Gotham is the clearest case. It does more than aggregate; it produces a picture in which “impactful targets” light up on a screen, and decides whose debts signify risk, whose associates imply complicity, whose presence on a street corner justifies suspicion. Police from Los Angeles to the German state of Hesse feed their databases into it and inherit the vendor’s assumptions under the name of “visibility.”
For deeper background, see how police use Palantir, coverage of Palantir’s expansion in Germany, and legal analysis of Palantir’s influence in policing.
Axon Fusus runs the same logic in real time, merging live camera feeds, drones, plate readers and body cams, with automated alerts that nudge officers toward intervention before anyone has hesitated. Axon markets it as a “real-time intelligence platform.” Reporting shows where officers look: in Toledo, two-thirds of live-feed viewing was at cameras in 12 subsidized housing developments, near which only a fifth of the city’s crime occurred (Gizmodo’s investigation).
In Gaza, +972 Magazine’s investigation of Lavender describes a system that rated almost everyone in Gaza from 1 to 100 for likely militant affiliation, flagged some 37,000 people as suspected militants and produced kill lists from them. A parallel system, Gospel, generated building targets at industrial tempo. Human review of a target often took about twenty seconds, enough to check that the target was male. At that pace the human in the loop works as a latency constraint on the machine.
To understand its commercial scale, see the private companies quietly building a police state.
View source
%% title: How the arrests prove the score right %% caption: A vendor’s secret definition scores a neighborhood high-risk, police flood it and make more arrests, and the database logs those arrests as proof the score was right. Each turn of the loop moves responsibility from officials to the system. flowchart TD classDef actor stroke-width:1.6px; classDef system stroke-width:1.2px; classDef gate stroke-width:1.6px; classDef binding stroke-width:3px,font-weight:bold; classDef status stroke-dasharray:4 3; classDef burden stroke-width:2px; classDef repair stroke-width:2.4px,font-weight:bold; classDef failure stroke-width:2.2px,stroke-dasharray:6 3; classDef concept stroke-dasharray:2 2; O["Vendor’s secret definition<br/>of who is worth attention"]:::concept --> C["Fines and plate reads score<br/>the neighborhood high-risk"]:::system C --> T["Neighborhood flagged<br/>for more patrols"]:::system T --> F(["Police flood it,<br/>make more arrests"]):::failure F --> D["Database logs the arrests<br/>as proof the score was right"]:::system D --> O
What the score does
These systems do more than predict. They distill a political judgment, who is dangerous, into a number, and sell the number back to the state as a fact.
In Gaza, according to +972’s sources, officers were given sweeping approval to adopt the kill lists and essentially treated the outputs “as if it were a human decision” (see the Lieber Institute’s symposium). In US prisons, Securus has trained an AI model on seven years of Texas inmate calls and is piloting it to flag crimes “being thought about or contemplated” (KSBY, MIT Technology Review). The people on those calls can’t opt out. Among defendants who did not reoffend, COMPAS labeled Black defendants higher risk at nearly twice the rate of white defendants, 45% against 23% (ProPublica). Chicago’s Strategic Subject List turned a history of over-policing in Black and Latino neighborhoods into a “heat list” of individual risk scores that justified more of it, for people with no violent record at all (legal critiques).
The mechanism is the same everywhere. The state targets a population; the population generates data; the model trains on the data and tells the state it was right to target them. Responsibility moves from officials to “the system.” And whether the vendor is Palantir or an open-source engineer matters less than the fact that one classification has been given the exclusive right to define who counts as dangerous.
More patrols, more arrests, higher scores
Police flood a “high-risk” zone. They make more arrests. The database logs the arrests and marks the zone higher risk. More patrols. In Chicago this was how the system was designed to work.
Proprietary complexity makes it nearly impossible to contest. What counts as a “gang indicator”? Which unpaid bill signals risk? Those definitions sit behind trade secrecy and NDAs. The reasons the data looks the way it does, poverty, segregation, coercion, colonial policy, disappear, and what remains is the line: the system found a pattern.
Marketing tidies up the rest. “Turnkey.” “Frictionless.” “Real-time.” Gotham promises seamlessness; Fusus sells speed. Friction is another word for due process, so a seamless system is one with the oversight removed.
For one legal-theoretical frame on this, see Artificial Intelligence Law through the Lens of Michel Foucault.
Langdon Winner warned that artifacts have politics. These dashboards are artifacts built to centralize the decision and treat the people at the margins as noise. The faster they run, the more of what doesn’t fit (duress, context, care) gets discarded as “unstructured” data. As the human loop shrinks and the appeal channel disappears, force becomes automatic.
The good guy with an ontology
One common response: we’re not against analytics, we just need better data, better models, a more ethical vendor.
That misses the problem. The danger is any single classification holding a monopoly on the path from data to force. A “good” stack run by well-meaning people still consolidates the power to decide what counts as a threat, what counts as order, and whose life is collateral. It replaces contestation with configuration. The engineers who tune it are legislators over life and death who are never named as such, and “good guy” ethics layered on top of that architecture is a fig leaf.
Break the monopoly
Chicago’s stalled consent decree shows how little leverage traditional oversight has over a force governed by black-box systems; the oversight reports document the department failing to meet basic reporting goals. Gaza shows what happens when one classification is fused to munitions at scale and called “precision.”
The work is structural. Ban turnkey pipelines that run data through one lens straight into lethal or carceral action. Put legal ceilings on automated tempo so systems cannot outrun the possibility of a human veto. Put competing perspectives, legal, historical, community, adversarial, into every chain from database to trigger, with real power to say no. As I argued in “Ask Any Ethics Board What It Has Stopped,” boards and principles without veto power only enable co-optation. You can’t add ethics to a machine whose function is moving faster than dissent.
Worldviews will always carve up reality, and guns aren’t going anywhere. But wiring one vendor’s classification directly to the trigger, the cell door and the drone is a choice, and it can be refused. Refusing it means that no model’s inference alone decides when force is used, and no single worldview settles who is allowed to live.