PADO: Personality-induced multi-Agents for Detecting OCEAN in human-generated texts

As personality can be useful in many cases — better understanding people's underlying contexts or providing personalized services — research has long focused on modeling personality from text. Personality detection, however, is hard: traits are latent and relative, vocabulary cues are context-dependent, and high-quality annotated data is scarce. We introduce PADO (Personality-induced multi-Agent framework for Detecting OCEAN Big Five traits), the first LLM-based multi-agent personality detection framework. PADO replaces a single-perspective judgment with a comparative judgment between agents that have been induced toward contrasting personality expressions, then aggregates their reasoning along psycholinguistic axes (emotion, cognition, sociality). Across two benchmarks (Essays, MyPersonality) and a range of LLMs from GPT-4o down to LLaMA3-8B-Instruct, PADO is consistently more accurate and generalizable than zero-shot, one-shot, and Chain-of-Thought prompting baselines, with particularly large gains on smaller models and on traits with subtle linguistic signatures.

Single-prompt LLM personality detection inherits the model's own biases and tends to over-rely on surface keywords. Naively stacking more agents helps little either, because similarly-prompted agents converge on similar verdicts — producing a multi-agent analogue of confirmation bias. PADO instead structures disagreement: it instantiates agents induced toward opposing endpoints of a trait, has them produce evidence-grounded explanations, and uses a separate judge agent to compare the two analyses. This mirrors LLM-as-Judge protocols (Zheng et al., 2024) but applies them to a task where the right answer depends on relative, not absolute, evidence — well-suited to personality.

The pipeline above is run independently for each of the Big Five trait dimensions. Each dimension gets its own pair of contrasting induced agents and its own comparative judgment pass.

Datasets

• Essays (Pennebaker & King, 1999) — 2,468 stream-of-consciousness essays labeled high/low on each OCEAN trait via standardized self-report. ~50 sentences per essay; 10% sampled for testing.
• MyPersonality (Celli et al., 2013) — 9,913 Facebook status messages from 250 users, each labeled high/low on OCEAN traits.

Baselines & Models Tested

• Encoder fine-tuning: BERT, RoBERTa (8:1:1 train/val/test split).
• Decoder LLMs with zero-shot, one-shot, Chain-of-Thought, and PADO: GPT-4o, GPT-3.5-turbo, Solar-10.7B-Instruct, LLaMA3-8B-Instruct.

Headline Results (F1)

On the Essays benchmark with GPT-4o, PADO lifts the average F1 from 0.53 (zero-shot) / 0.51 (one-shot) / 0.54 (CoT) to 0.66. The same backbone on MyPersonality reaches 0.83 F1 on Openness with PADO, well above all baselines. Smaller models (LLaMA3-8B-Instruct, Solar-10.7B) show similar directional improvements, confirming that the gains stem from the comparative-judgment structure rather than raw model scale.

F1 scores on the Essays benchmark with GPT-4o as backbone, excerpted from the paper. PADO improves the average and substantially closes the gap on Conscientiousness and Extraversion.

• A novel personality-induced multi-agent framework for OCEAN detection that leverages contrasting agent perspectives, the first such framework using LLM agents.
• Demonstration that PADO is backbone-agnostic across model scales (GPT-4o through LLaMA3-8B), with particularly strong improvements on smaller models.
• A psycholinguistically-grounded reasoning scheme that decomposes each agent's analysis into emotional, cognitive, and social axes — capturing implicit and relative trait signals without additional training.