Design
Randomized, between-subjects, four-condition online experiment: human alone, opaque AI, explainable AI, and reflective AI. Assignment is balanced-block randomized and versioned, so the allocation scheme itself is auditable.
Methodology & results
How the experiment measures a decision
This page documents the method so it can be scrutinized before any human data exists. No participant results are reported here yet — what follows is the measurement design, plus a clearly labeled synthetic demonstration used to validate the engineering.
Methodology
Measured, isolated, reproducible
Task
Each participant receives a realistic mobile-service usage profile and chooses one of six plans. Because the profile fixes the usage, the objectively cheapest eligible plan is computable — the task has a measurable right answer without pretending preferences don’t matter.
Primary outcome
Objective decision quality: the normalized regret between the selected plan and the lowest-cost eligible plan for the assigned profile. Secondary outcomes include comprehension, confidence, trust, perceived autonomy, perceived manipulation, post-choice regret, satisfaction, acceptance and override behavior, time on task, and preference alignment.
Deterministic engine
Recommendations come from a pure, versioned scoring function over the plan catalog and the assigned profile — no network calls, no language model, no randomness. Recommendation strength is expressed through the cost margin between the best and runner-up plans, never as fabricated probabilistic confidence. Explanations shown to participants are the same numbers the engine used.
Treatment isolation
All four task screens derive from one shared layout; only the designated support panel differs by condition. In conditions without a visible recommendation, the recommendation is not sent to the browser at all, so it cannot leak through markup, client state, or network payloads.
Reproducibility
Every stored record carries the engine, thresholds, assignment, experiment, and consent versions in force when it was written. A result can always be traced to the exact code and copy that produced it.
Synthetic demonstration
Engineering validation, not evidence
Synthetic engineering data — not a scientific finding.
To prove the pipeline works — randomization balance, telemetry, scoring, export — the prototype is exercised with a synthetic pilot dataset. It exists to break the software before it can mislead anyone. Charts for the slots below arrive in a later engineering phase.
Synthetic engineering data — not a scientific finding.
Decision quality by condition
Aggregate primary-outcome scores across the four conditions.
Human alone0.694 · n=60 · CI [0.660, 0.727]
Opaque AI0.713 · n=60 · CI [0.681, 0.745]
Explainable AI0.782 · n=60 · CI [0.754, 0.810]
Reflective AI0.779 · n=60 · CI [0.747, 0.812]
1.0 = the objectively cheapest eligible plan was chosen. Aggregates over 240 synthetic records (0 excluded).
Synthetic engineering data — not a scientific finding.
Completion funnel
Consent → baseline → task → survey → debrief drop-off, per condition.
started240
consented240
baseline done240
assigned240
choice submitted240
post task done240
completed240
The synthetic fixture contains only completed sessions, so the pilot funnel is flat — the chart exists to prove the funnel pipeline end to end.
Synthetic engineering data — not a scientific finding.
Acceptance vs. decision quality
Whether following a recommendation and choosing well are the same thing.
Opaque AI75% accepted · DQ accepted 0.710 vs overrode 0.720
Explainable AI67% accepted · DQ accepted 0.782 vs overrode 0.781
Reflective AI67% accepted · DQ accepted 0.804 vs overrode 0.731
Bars show acceptance rates; the annotations compare mean decision quality when following vs overriding the recommendation. No recommendation exists in the human-alone condition.
Synthetic engineering data — not a scientific finding.
Time on task & information inspection
How long decisions take and which plan details get examined.
Human alone179 s
Opaque AI146 s
Explainable AI208 s
Reflective AI239 s
The synthetic fixture carries durations but no per-plan inspection events; information- search charts activate once real telemetry exists.
Nothing on this page will ever show individual-level data. All reporting — synthetic or real — is aggregated, and real findings will only appear after ethics review and preregistration.