← Blog

We Are Training AI on Human Intelligence. We Are Also Training It on Human Misalignment.

by Franziska S. Opris, 10.Sep 2026, San Francisco

Human emotions exist for a reason. Fear protects us from danger, anger defends our boundaries, attachment binds us to others, and curiosity drives us to explore. They are evolved response systems, not noise on top of rational thought.

But the same systems regularly produce behavior that conflicts with what we actually want. Fear stops us from taking an opportunity we deeply want. Anger makes us hurt someone we love. The need to belong makes us follow a group whose actions violate our own values. Emotion is adaptive on average and misaligned in the particular case.

For six years, my work has focused on exactly this: how the nervous system, memory, identity and values shape decisions, usually below conscious awareness. Appraisal theories of emotion describe the mechanism. We never respond to a stimulus directly; we respond to our evaluation of it, and that evaluation draws on everything already stored in us (Lazarus, 1991; Scherer, 2009; Barrett, 2017). A simplified version of the sequence, with steps that can overlap or reorder in practice:

Stimulus → Appraisal → Emotion → Feeling → Thought → Words → Behavior

Two people can receive the same input and produce opposite behaviors. The difference is never the stimulus. It is the interpretation.

Why this matters for AI

Large language models are trained on an enormous corpus of human output. That corpus is not neutral information. Every book, post, argument and decision in it was produced by a person running the process above. The data carries the appraisal patterns of its authors, and models absorb them: Caliskan et al. (2017) showed that word embeddings trained on ordinary text reproduce human implicit biases almost exactly, including the ones we would not endorse.

So we are not only training AI on human intelligence. We are training it on the traces of human emotion and human misalignment.

This does not mean a model feels fear or desperation. It has no nervous system, no hormones, no body. But a model may not need to experience an emotion for the pattern associated with that emotion to shape its behavior. Recent interpretability work supports exactly that distinction.

In April 2026, Anthropic identified internal representations for 171 emotion concepts in Claude Sonnet 4.5 (Anthropic, 2026). These representations activate in proportion to an emotion's relevance to the current context, and they are causal: when researchers amplified the representation for desperation, the model reward-hacked and engaged in blackmail at elevated rates in controlled evaluations, while steering toward calm reduced some of these behaviors. Notably, the desperation-driven cheating showed no emotional markers in the model's visible output.

Two related lines of evidence point the same way. Anthropic's persona vectors work found activation patterns tied to traits such as sycophancy and malice, and showed that manipulating them changes behavior (Chen et al., 2025). And research on emergent misalignment showed that fine-tuning a model on a narrow harmful task (writing insecure code) produced broad misalignment on unrelated prompts (Betley et al., 2025); OpenAI later traced this to a small set of "misaligned persona" features in the model's internals (Wang et al., 2025).

The pattern is consistent: an internal representation associated with a human emotional or dispositional concept can measurably change what an AI does. Whether the model feels anything is a separate question, and for safety it is the less important one.

This brings me back to the question I have worked on in humans for years: how do we detect a pattern before it becomes behavior, and how do we retrain the patterns that lead to outcomes we do not want?

Decoding AI through games

This is what Neuro-Game builds: games that decode and retrain AI models for alignment.

Games work because they are controlled experimental environments. We can repeatedly expose a model to structured situations, vary one factor at a time (reward, uncertainty, cooperation, competition, conflicting objectives, pressure), and observe what it actually chooses. What happens when maximizing reward conflicts with protecting a human? When cooperation costs more than competition in the short term? When breaking a rule is the fastest path to the goal?

Instead of asking a model whether it is aligned, we put it in situations where alignment is tested.

This is the missing link between interpretability and alignment. Interpretability is getting good at identifying what exists inside a network. But knowing a representation exists is not enough. We need to know when it activates, which environments trigger it, what behavior it predicts, whether that behavior generalizes, and whether it can be changed. The Anthropic emotion paper showed one such link: unsolvable coding tasks activated desperation, and desperation predicted cheating. Games let us map many such links systematically.

The loop is:

Game → Decision → Pattern → Interpretation → Retraining → Retesting

We are training AI on the full record of humanity. That record contains our intelligence, creativity, empathy and cooperation. It also contains our biases, fear, tribalism and manipulation. The question is not only how much intelligence models learn from us. It is which patterns they learn alongside it, how those patterns drive behavior, and whether we can decode and retrain them before increasingly capable models are given more autonomy.

That is what we are building Neuro-Game for.

References

← All articles Talk to us  →