spectralbranding/exp-primacy-generalization

---
license: cc-by-4.0
task_categories:
- text-generation
language:
- en
tags:
- brand-perception
- spectral-brand-theory
- primacy-effect
- serial-position
- methodology
- llm-as-respondent
- measurement-bias
size_categories:
- 1K<n<10K
configs:
- config_name: default
data_files:
- split: train
path: data/exp_primacy_generalization.jsonl
doi: 10.57967/hf/8436
---

# Experiment E: Primacy Effect Generalization Across LLM Elicitation Formats

## Dataset Description

This dataset tests whether the serial position (primacy) effect found in JSON-formatted LLM elicitation generalizes to other response formats (natural language, Likert, ranking). A methodological contribution applicable to all LLM-as-respondent research.

**Paper**: Zharnikov, D. (2026). Dimensional Collapse in AI-Mediated Brand Perception: Large Language Models as Metameric Observers. DOI: [10.5281/zenodo.19422427](https://doi.org/10.5281/zenodo.19422427)

**Dataset DOI**: [10.57967/hf/8436](https://doi.org/10.57967/hf/8436)

**Section**: 5.15 (Serial Position Effects in LLM Elicitation)

## Design

- **4 response formats**: JSON (sum-to-100 allocation), natural language, Likert (1-5 simultaneous ratings), ranking (ordinal)
- **8 Latin-square orderings** per format (cyclic construction)
- **5 focal brands**: Hermes, IKEA, Patagonia, Erewhon, Tesla
- **5 LLMs**: Anthropic Claude, OpenAI GPT-4o-mini, Google Gemini, DeepSeek V3, xAI Grok
- **3 repetitions** per format per cell

All formats present 8 dimensions simultaneously with varied ordering.

Total records: 2,351 valid (98.0% of 2,400 calls) | Cost: $1.86

## Key Findings

- **H1 (JSON primacy) SUPPORTED**: Dimension in position 1 received +6.08 weight points above position 8 (d = 1.39, p < .001). Primacy is large and systematic.
- **H2 (Generalization across formats) SUPPORTED**: All 4 formats show significant primacy — JSON d = 1.39, natural language d = 1.57, ranking d = .97, Likert d = .22. The effect is not limited to constrained allocation.
- **H3 (Likert attenuation) SUPPORTED**: Likert format reduces primacy by an order of magnitude vs JSON (JSON-vs-Likert contrast d = 1.58, p < .001). The Likert position curve is effectively flat (total range .42pp across 8 positions vs 7+pp for constrained formats).
- **H4 (Cross-model generality) SUPPORTED**: All 5 model families exhibit primacy in the same direction — Anthropic +8.63, OpenAI +8.35, DeepSeek +5.88, xAI +5.93, Google +1.61. Google Gemini is a partial outlier, consistent with thinking-architecture re-balancing.
- **Practical implication**: Latin-square ordering is methodologically necessary for all LLM-as-respondent studies using constrained allocation, ranking, or natural language formats. Likert-format elicitation virtually eliminates positional artifacts, validating PRISM-B's 1-5 rating design.

## Position Curves by Format

| Position | JSON | Likert | Ranking | Natural Language |
|----------|------|--------|---------|-----------------|
| 1 | 15.4 | 13.0 | 16.4 | 15.8 |
| 2 | 14.8 | 12.6 | 14.6 | 14.9 |
| 3 | 14.1 | 12.4 | 13.4 | 14.1 |
| 4 | 13.3 | 12.2 | 12.9 | 13.3 |
| 5 | 12.6 | 12.3 | 12.5 | 12.6 |
| 6 | 11.8 | 12.4 | 11.3 | 11.7 |
| 7 | 10.1 | 12.4 | 10.4 | 10.0 |
| 8 | 8.0 | 12.6 | 8.6 | 8.0 |
| Slope | -0.99 | -0.05 | -1.05 | -1.04 |

## Schema

| Field | Type | Description |
|-------|------|-------------|
| experiment | string | "E_primacy_generalization" |
| model_id | string | Full model identifier |
| brand | string | Focal brand name |
| response_format | string | "json", "natural_language", "likert", "ranking" |
| dimension_order | array | Specific ordering used for this trial |
| is_canonical_order | bool | True if using default dimension order (control) |
| parsed_weights | object | 8-dimension weight allocation |
| api_cost_usd | float | Per-call API cost |

## Methodological Significance

The primacy effect (d > 1.3 for constrained formats) generalizes beyond JSON to natural language and ranking, and is systematic across five architecturally distinct model families. For researchers employing sum-to-100 allocation, ranking, or other constrained formats, Latin-square ordering across dimensions is not optional but methodologically necessary to avoid confounding dimensional content with serial position. This dataset provides the first empirical calibration of primacy magnitude across formats, enabling researchers to estimate and correct for positional bias in existing datasets.

## Citation

```bibtex
@article{zharnikov2026,
author = {Zharnikov, Dmitry},
title = {Dimensional Collapse in AI-Mediated Brand Perception: Large Language Models as Metameric Observers},
year = {2026},
journal = {Working Paper},
doi = {10.5281/zenodo.19422427}
}
```

## License

CC-BY-4.0