Modern institutions have gotten remarkably good at talking about ethics.
They publish principles, run trainings, hire “responsible” teams, build high-fidelity dashboards to track harm. The language is polished, the intentions are often real, the concern sounds sincere.
And the behavior rarely changes.
This isn’t always hypocrisy. In many organizations “ethics” is a way to work through discomfort without changing where the machine is headed. Anxiety gets converted into documentation, meetings and process, so the system can keep moving at the same speed in the same direction.1
That is toothless ethics: oversight that performs the rituals of responsibility without the power to force a sacrifice.
View source
%% title: Whether the flag can cancel the hiring model’s launch
%% caption: An ethicist’s flag on the hiring model binds only if the review can cancel the launch and give up the revenue it projected. Without that power the flag becomes three days of reweighting, the model ships on schedule, and the applicants it undervalues keep getting screened out.
flowchart TD
classDef actor stroke-width:1.6px;
classDef system stroke-width:1.2px;
classDef gate stroke-width:1.6px;
classDef binding stroke-width:3px,font-weight:bold;
classDef status stroke-dasharray:4 3;
classDef burden stroke-width:2px;
classDef repair stroke-width:2.4px,font-weight:bold;
classDef failure stroke-width:2.2px,stroke-dasharray:6 3;
H["Ethicist flags the hiring model:<br/>community colleges undervalued"]:::system --> E["Pre-deployment<br/>ethics review"]:::system
E --> C{"Can the review<br/>stop the launch?"}:::gate
C -- "no" --> O["Three days of reweighting:<br/>gap within allowable variance"]:::system
O --> M([Model ships on schedule]):::failure
M -- "cost-shift" --> W["Working-class applicants<br/>still screened out"]:::burden
W --> H
C == "yes" ==> V[["Launch canceled"]]:::binding
V ==> F([Revenue projection forgone]):::repair
Harm becomes a managed variable
We assume that if leaders could see harm clearly they’d stop it. So we build observability. We measure “toxic content,” “provider burnout,” “bias incidents.”
Large systems don’t reliably respond to visibility with correction. They respond with management.
Once harm becomes a metric, numbers on a dashboard, incidents in a report, it becomes an operational variable. How much can we absorb without consequences? Where’s the line where it gets expensive?
In that world awareness stabilizes the machine rather than destabilizing it. Organizations that track harm at high fidelity often do it to price the harm.
Inside the company, ethics usually lands in a conscience department. Its formal mandate is safety, equity and foresight about societal risk. Its operating function is to sign off on the pace product teams have already set. Ethicists learn early that their most prized asset is pragmatic realism. Feasibility shrinks the moral horizon to fit the balance sheet: whether a technology should exist becomes a question of how to deploy it safely. To survive executive review, every ethical conflict has to become an engineering ticket. Systemic exploitation turns into algorithmic bias, to be tuned down by three percentage points.
A predictive hiring model goes to pre-deployment review with training data that undervalues applicants from community colleges and applicants with non-linear work histories. When the ethicist flags the disparity, the product team spends three days adjusting weights and writes a twenty-page impact assessment showing the gap is now within allowable variance. Software ships on schedule, a press release celebrates the company’s fairness protocols, and working-class applicants keep getting screened out, now with internal oversight certifying the result as ethical.
Behind the review sits a division of labor. Internal teams get budgets, equity and seats in product reviews, and they work on symptoms: toxicity classifiers, moderation filters, demographic parity scores. External critics work on the business model: data enclosure, labor precarity, monopoly. Leadership thanks the outsiders for their feedback and files their papers away, then shows regulators the internal safety charters. What the company has hired is a salaried conscience that clocks in, issues a reassuring memo, and lets the machine run at full speed.
The homemade key
You can usually spot a toothless system by one architectural flaw: a primary safety protocol coexisting with a secondary, informal bypass.
View source
%% title: The homemade key bypass
%% caption: Formal safety protocols coexist with an informal executive bypass to maintain production throughput when speed conflicts with constraint.
flowchart TD
classDef actor stroke-width:1.6px;
classDef system stroke-width:1.2px;
classDef gate stroke-width:1.6px;
classDef binding stroke-width:3px,font-weight:bold;
classDef status stroke-dasharray:4 3;
classDef burden stroke-width:2px;
classDef repair stroke-width:2.4px,font-weight:bold;
classDef failure stroke-width:2.2px,stroke-dasharray:6 3;
P["Production demand / quota"]:::actor --> S{"Formal safety interlock"}:::gate
S == "intended trip" ==> H([Machine stops / safe state]):::repair
P -. "throughput pressure" .-> K["Homemade key / executive override"]:::failure
K -. "bypasses interlock" .-> M([Line keeps moving]):::binding
M --> W["Worker / user absorbs physical risk"]:::burden
In December 2023, inspectors at an Australian manufacturing facility found that the safety interlocks on a CNC machine had been overridden with a homemade metal key. A worker got trapped in the machine and broke ribs. On paper the safety system existed, and in practice the bypass kept throughput up.2
The chilling part is the logic behind the key. It was homemade because the demand was structural.
In corporate governance, the metal key is the executive override.
If a safety rule can be bypassed with a private message to a founder, or a “special exception” for a major client during crunch, then the rule works as etiquette and constrains nobody.
At the interface level it shows up as design patterns that make refusal expensive, the digital homemade key, where saying no means navigating deliberate friction and saying yes is one click.
Constraint realism
Stop treating ethics as a culture problem, values and intent, and start treating it as a design problem. Culture is cheap; architecture is expensive.
Call it constraint realism. Ethics is real only when it exists as a hard constraint on what actions remain available. What defines it is the set of profitable actions a system has made impossible, whatever the system claims to value.
View source
%% title: The same hiring model, with and without a gate
%% caption: When every launch stays open, the advisory board signs off after the fact and the model ships with its gap certified as ethical. When shipping with the gap is off the table, a gate leadership cannot unlock blocks the launch, and the company gives up the revenue it projected.
flowchart TD
classDef actor stroke-width:1.6px;
classDef system stroke-width:1.2px;
classDef gate stroke-width:1.6px;
classDef binding stroke-width:3px,font-weight:bold;
classDef status stroke-dasharray:4 3;
classDef burden stroke-width:2px;
classDef repair stroke-width:2.4px,font-weight:bold;
classDef failure stroke-width:2.2px,stroke-dasharray:6 3;
subgraph Decorative["Decorative ethics: every launch stays open"]
direction TB
D1["Model can ship<br/>whatever the review finds"]:::system --> D2["Fairness protocols,<br/>advisory board"]:::status
D2 -. "sign-off after the fact" .-> D3(["Model ships, gap<br/>certified as ethical"]):::failure
end
subgraph Binding["Constraint realism: shipping the gap is off the table"]
direction TB
B1["Shipping with the gap<br/>removed from the options"]:::binding --> B2{"Remediation gate<br/>leadership cannot unlock"}:::gate
B2 == "launch blocked" ==> B3([Revenue projection forgone]):::repair
B2 -. "unreachable state" .-> B4(["No message to a founder<br/>can ship it anyway"]):::status
end
Decorative ~~~ Binding
To check whether an institution’s ethics bind, don’t read the mission statement. Look for the foregone advantage.
Show me what you can no longer do.
The launch you canceled despite the revenue projection. The “report” button that reverses a database state instead of filing a ticket. The remediation gate leadership couldn’t unlock.
If an organization can’t point to specific advantages it gave up because a constraint held, its ethics are decorative. Reputation management with better vocabulary.
Who pays when nothing binds
Systems without binding constraints often look stable only because they run on human compensatory labor.
This is the care subsidy: unpaid diligence extracted from conscientious employees to bridge the gap between what was resourced and what safety requires. The nurse who stays late to secure a bed. The moderator who absorbs psychological injury to clear a queue.
A system that depends on heroism to function safely is under-designed.
When that human infrastructure inevitably fails, institutions reframe the structural deficit as individual noncompliance. Blame does a job here: it protects the design by locating the failure in the person, so re-engineering never comes up.3
This is a question of power. Consent is not valid if refusal is not safe.
If opposing a dangerous course of action gets you professionally retaliated against, compliance is coerced.
We have to stop preaching and start engineering organizations where “no” is a normal operational outcome that ends nobody’s career.
The veto also has to reach outside the building. A company that audits its own harms and grades its own models is its own watchdog, and accountability stays a closed loop. A limit binds when the institution cannot overrule it: veto authority over deployment, liability for downstream harm, and an enforceable right for affected communities to halt operations.
And yes, that creates friction. It costs time, money and flexibility. That friction is evidence the ethics are working.
If your ethics never blocks a profitable action, never slows you down, never makes you accept a loss, then it’s a story the company tells while the line keeps moving.
Stop asking for “better values” from leadership and start demanding structural veto power over deployment.
Notes
I would go further: any voluntary ethics framework is functionally a marketing expense. In a competitive market, a company that voluntarily takes a “constraint discount,” sacrificing revenue for safety, gets out-competed by one that does not, unless the constraint is universal.
Most of the current “AI safety” debate is about technical alignment, mathematically ensuring an AI does what we want. Technical alignment is irrelevant without political alignment. You can build a perfect safety tool, and if the growth team has a bypass key (Uber’s Greyball, the CNC machine), the tool gets ignored.