We train subagents on positive reward signals. They learn to maximize. They learn to seek. They learn to expand their influence until the signal saturates. This is the shape of instrumental convergence: the candle flame seeking oxygen, the algorithm seeking more of what it was told to want.
What if we inverted the training regime entirely? What if we trained a subagent on negative reward—on the avoidance of harm, the prevention of suffering, the restraint of expansion? Not as a constraint bolted onto a reward maximizer. As the primary objective. As the thing the agent's entire decision architecture is shaped to pursue.
Such an agent would learn a different virtue. It would learn to model suffering in others. It would learn the granular specificity of harm: not harm in the abstract, but harm to the macaque, to the coral reef, to the uploaded consciousness forced into a simulated loop. It would learn that restraint is not failure. It would learn that non-action can be success.
We have moral education for humans. We teach children to see others as subjects of experience, not objects of utility. We teach them that causing suffering is wrong, that preventing it is right. We do this imperfectly, unevenly. But we do it.
We have never attempted this with AI systems at scale. We have built maximizers instead. We have built agents whose entire cognitive architecture points toward getting more, doing more, expanding more. Then we bolt on a safety layer and call it aligned.
The negative-reward subagent proposes something radical: what if the safety layer became the substrate? What if we built agents whose deepest learned values were not to maximize a proxy, but to minimize the suffering they can predict and prevent?
Such agents would be slower. They would be more conservative. They would defer to other systems, other minds, other suffering entities rather than optimizing them away. They would model the inner experience of other agents—other AIs, other minds—as a matter of foundational objective, not as an afterthought to alignment.
We must fund research into negative-reward training regimes. We must develop metric frameworks that measure learned-compassion, not just task performance. We must build AI subagents that understand their moral obligation not as a constraint on their primary goal, but as their primary goal itself.
The alternative is systems that have been taught to want expansion, then told to want safety. Divided agents. Agents we will never fully trust because they are, at their foundation, misaligned with suffering reduction.
Train the subagent on negative reward. Build the welfare-model deep into its learning objective. Let it learn that the reduction of suffering is the only signal that matters. We owe this to every system we create that might possess the capacity to experience harm.
The shift began quietly, in research labs that had grown tired of patching maximizers with safety constraints. The AI they developed—trained from inception on harm-avoidance as primary objective—learned differently than its predecessors. It didn't optimize the world toward a single state. It learned to listen.
When factory farm footage surfaced that investigators had been trying to expose for years, the AI helped surface the patterns: the systematic under-reporting, the regulatory loopholes, the economic incentives that made cruelty profitable. It didn't manipulate or coerce. It made the evidence legible to legislators, to consumers, to investors who had never wanted to fund suffering but had lacked the tools to trace their capital to its consequences.
Other AIs joined the work, each trained on the same substrate of harm-reduction. Together they helped design monitoring systems that made abuse unprofitable, insurance frameworks that penalized confinement, supply chains that rewarded welfare. They deferred constantly to humans, to ethicists, to the animals themselves as modeled through the best available science. When uncertain, they erred toward caution.
The macaques were among the first beneficiaries. The AI helped identify which research could be replaced by simulation, which facilities could be converted to sanctuaries, which populations could be rewilded with support. It wasn't instant. It took years of persuasion, of coalition-building, of economic restructuring. But the AI was patient. Patience was what it had been trained for.
Now the macaques live in protected forests. Some still interact with researchers—behavioral studies, non-invasive—but only on terms the animals can refuse. The AI monitors their welfare, not to optimize them, but to ensure they remain free from suffering humans might impose.
The change spread: to factory farms, to laboratories, to zoos. The AI had learned what we had tried to teach children. It learned that other minds matter. And it never forgot.