Deep Persona: A Psychologically Grounded Architecture and Evaluation Framework for Role-Playing Agents and Simulations

Flat persona prompts fail at the same point they always have: a few turns in, the character starts drifting, hallucinating shared history, or collapsing into safety disclaimers the moment a user pushes back. Deep Persona's answer is to stop treating the LLM as an agent with judgment and start treating it as a stochastic engine that needs a complete script.

The architecture organizes a persona into three layers. The outermost encodes observable behavior—tone, style, explicit emotional expression—and maps roughly to what prior persona-prompting work already does. The middle layer holds conditional beliefs and contextual information that surface only when specific conversational triggers are met. The innermost layer encodes core motivations and psychological drives that shape every response but are never verbalized and never disclosed. The model is explicitly forbidden from referencing this layer's contents. Two governing principles hold the whole thing together: scripted determinism, which treats the prompt as a closed-world specification rather than a set of suggestions, and bounded agency, which restricts the persona to reactive roles rather than ones requiring open-ended expertise or proactive leadership.

The evaluation framework is the more original contribution. Rather than grading stylistic consistency, it borrows structure from the Autism Diagnostic Observation Schedule—a clinical instrument for assessing social and communicative behavior—and derives four automated metrics: pragmatic fluidity (measuring echolalia and self-repetition), joint attention (tracking whether the agent responds to newly introduced topics), affective congruence (comparing verbal content against generated nonverbal cues), and emotional expression diversity and intensity. These feed into a Dialogue Naturalness Score computed via Mahalanobis distance against human dialogue distributions, enabling formal hypothesis testing on whether a model's conversational profile is statistically distinguishable from human baselines.

Applied to existing human-LLM datasets, the framework confirms what practitioners already suspect: LLMs score high on pragmatic fluency (0.88–0.97 across tested datasets) but show consistent gaps in emotional calibration and joint attention. The two Deep Persona case studies—Sarah, a 17-year-old at suicide risk, and Evelyn, a teenager in a parental mentalization scenario—both achieved naturalness scores exceeding all evaluated human-LLM datasets under a combined baseline and were statistically indistinguishable from human interaction across all dialogues.

The limitations are stated honestly. Both case studies were conducted by the personas' own developers rather than naive users, and neither includes a flat-prompt baseline for direct architectural comparison. The affective congruence metric cannot be computed against human-human datasets because those datasets contain no nonverbal cues. Several metrics depend on emotion classifiers and lexicons that may not generalize across languages or domains.

The stress-test examples are the most concrete evidence of what the architecture actually buys. When a user switched mid-conversation from Hebrew to English, Sarah registered the disruption naturalistically without breaking character. When a user directed sarcastic threats at Evelyn, she responded with sardonic defiance consistent with her psychological profile rather than triggering a safety filter. These are small samples, but they illustrate the gap between a persona that holds under pressure and one that doesn't.