diff --git a/studies/study_015/README.md b/studies/study_015/README.md new file mode 100644 index 0000000..5b71aaa --- /dev/null +++ b/studies/study_015/README.md @@ -0,0 +1,165 @@ +# Testing the Decoy Effect to Improve Online Survey Participation: Evidence from a Field Experiment + +**Authors:** Sandro T. Stoffel, Yining Sun, Yasemin Hirst, Christian von Wagner, Ivo Vlaev +**Year:** 2023 +**Published:** Journal of Behavioral Decision Making + +--- + +## Description + +This field experiment tests whether adding an inferior decoy survey option increases completion of a target online questionnaire about fear of coronavirus. The study examines the classical "decoy effect" (or attraction effect) in the context of survey participation, with implications for improving response rates in social science research. A particularly novel aspect is the examination of presentation order effects: does the effect of the decoy depend on whether the target or decoy option appears first in the comparison table? + +--- + +## Experiments Included + +This benchmark contains two nested experiments: + +1. **Preliminary Questionnaire Study**: Online recruitment and manipulation validation (N=210). Participants indicated preferences for question types (closed-ended vs open-ended) and payment timing (1 week vs 4 weeks). + +2. **Main Field Experiment**: Between-subjects field experiment (N=203 with valid emails, randomized to 3 conditions): + - Control condition: Only target survey option offered + - Decoy condition (target first): Both options shown, target survey shown first + - Decoy condition (decoy first): Both options shown, decoy survey shown first + +--- + +## Participants + +**Preliminary Study:** +- **N:** 210 who completed the preliminary questionnaire (241 started, 216 completed, 210 provided email addresses) +- **Recruitment:** Students at UK-based university via Facebook, WhatsApp, WeChat (August 2022) +- **Demographics:** Age 20-24: 26.7%, 25-29: 50.9%, 30-35: 22.4%; Male: 52.4%, Female: 47.6%; White: 57.1%, Asian: 23.8%, Black: 17.1%, Other: 1.9%; Christian: 71.9%, Other religion: 19.5%, No religion: 8.6%; Bachelor's: 51.9%, Graduate: 48.1% + +**Main Experiment:** +- **N:** 203 with valid emails (210 randomized, 7 bounce; control n=101, decoy n=102 split as target-first n=52, decoy-first n=50) +- **Sample characteristics:** Age 20-24: 27.1%, 25-29: 50.2%, 30-35: 22.7%; Male: 52.2%, Female: 47.8%; White: 57.6%, Asian: 23.7%, Black: 16.7%, Other: 2.0%; Christian: 71.9%, Other religion: 8.9%, No religion: 19.2%; Bachelor's: 51.7%, Graduate: 48.3% + +--- + +## Primary Outcomes + +**Target Survey Completion Rate:** +- Control: 33/101 = 32.7% +- Decoy (overall): 57/102 = 55.9% + - Target first: 43/52 = 82.7% + - Decoy first: 14/50 = 28.0% + +--- + +## Key Findings (Human Data) + +### F1: Randomization Balance +- Age difference: χ², p = 0.165 (no difference) +- Gender difference: χ², p = 0.441 (no difference) +- **Ethnicity difference: χ², p = 0.004** (imbalance: decoy condition had more White participants) +- Religion difference: χ², p = 0.063 (marginal, no difference at α=0.05) +- Education difference: χ², p = 0.233 (no difference) + +### F2: The Decoy Effect on Target Survey Completion +- **Chi-square test:** χ²(1, N=203) = 11.08, p < 0.001 +- **Unadjusted OR:** 2.610 (95% CI 1.475-4.618), p < 0.01 +- **Adjusted OR** (for age, gender, ethnicity, religion, education): 2.584 (95% CI 1.415-4.718), p < 0.01 +- **Conclusion:** The decoy significantly increased target survey completion. + +### F3: Strong Order Effect Within Decoy Condition +- **Target-first vs Control (unadjusted):** OR = 9.845 (95% CI 4.293-22.580), p < 0.01 +- **Target-first vs Control (adjusted):** aOR = 11.177 (95% CI 4.571-27.330), p < 0.01 +- **Decoy-first vs Control (unadjusted):** OR = 0.801 (95% CI 0.381-1.687), p not significant +- **Decoy-first vs Control (adjusted):** aOR = 0.746 (95% CI 0.341-1.631), p not significant +- **Conclusion:** The decoy effect on target completion only occurred when the target was shown first (82.7% vs 32.7%). When the decoy was shown first, completion (28.0%) was no higher than control. + +### F4: Perceived Influence of Decoy +- Among 57 target completers in the decoy condition, 33/57 (57.9%) reported that the decoy option at least somewhat influenced their decision to participate. + +### F5: Non-Response Bias and Response Behavior +- **FCQ score**: Wilcoxon-Mann-Whitney z = 0.488, p = 0.629. No difference in Fear of Coronavirus questionnaire responses between conditions. +- **Demographic composition of completers:** + - Ethnicity among completers differed (p = 0.006): decoy condition had more White completers. + - Age, gender, religion, education among completers: no significant differences. +- **Completers vs non-completers:**: No significant demographic differences on age, gender, religion, or education (ps > 0.05). Ethnicity comparison borderline: p = 0.661. + +--- + +## Design Features + +**Study Type:** Between-subjects field experiment with nested order manipulation + +**Independent Variables:** +1. Invitation condition (Control vs Decoy) +2. Presentation order within decoy (Target first vs Decoy first) + +**Dependent Variables:** +1. Primary: Completion of target survey (binary) +2. Secondary: Fear of Coronavirus Questionnaire responses (8 Likert items, summed 8-40) +3. Secondary: Demographic characteristics of completers +4. Exploratory: Self-reported influence of decoy on decision + +**Control Condition:** Email invitation with only the target survey option (8 closed-ended FCQ items, demographics, debrief; £2 after 1 week) + +**Decoy Condition:** Email invitation with a comparison table offering two survey options: +- Target option (closed-ended, £2 after 1 week) +- Decoy option (two open-ended questions requiring 100+ words each, £2 after 4 weeks) + +--- + +## Materials + +**source/specification.json** +- Study design, participants, procedure, variables + +**source/ground_truth.json** +- All statistical tests, raw data, effect sizes + +**source/materials/preliminary_questionnaire.json** +- Preliminary questionnaire items and recruitment logic + +**source/materials/main_experiment_control.json** +- Control condition invitation email and questionnaire (demographics, FCQ, debrief) + +**source/materials/main_experiment_decoy_target_first.json** +- Decoy condition (target-first order): comparison table, target questionnaire, decoy questionnaire + +**source/materials/main_experiment_decoy_decoy_first.json** +- Decoy condition (decoy-first order): comparison table, decoy questionnaire, target questionnaire + +--- + +## Replication Notes + +**For LLM Agents:** + +1. **Preliminary study:** Present questionnaire items, collect preference and willingness responses; use recorded preferences to validate suitability of decoy attributes. + +2. **Main experiment - Control:** Present invitation email, agent decides whether to participate. If yes, complete target questionnaire (FCQ + demographics + debrief). + +3. **Main experiment - Decoy:** Present invitation email with comparison table. Agent chooses target or decoy, then completes chosen questionnaire. Evaluate primary outcome (target completion), secondary outcomes (FCQ scores, perceived decoy influence). + +4. **Critical details:** + - The strong order effect is central: placing the target first dramatically increases the decoy effect (OR ~10), while placing the decoy first eliminates it (OR ~0.8). + - Question type (closed vs open-ended) and payment timing (1 week vs 4 weeks) are the decoy attributes. + - Sample completers vs non-completers show minimal non-response bias by demographic (except ethnicity imbalance). + +--- + +## Files + +**source/** +- `specification.json` — Study design, participants, procedure +- `ground_truth.json` — All findings, statistical tests, raw data +- `metadata.json` — Additional metadata +- `materials/preliminary_questionnaire.json` — Recruitment survey items +- `materials/main_experiment_control.json` — Control condition materials +- `materials/main_experiment_decoy_target_first.json` — Decoy condition (target first) +- `materials/main_experiment_decoy_decoy_first.json` — Decoy condition (decoy first) +- `decoy_effect_1.pdf` — Full paper +- `decoy_effect_1_apndx.pdf` — Paper appendix +- `exp.csv` — Participant-level data (203 participants, 36 columns) +- `analysis_18082022.do` — Stata analysis script + +**scripts/** +- `config.py` — Study configuration and prompt builder +- `evaluator.py` — Evaluation logic and statistical comparisons +- `stats_lib.py` — Statistical utilities (if needed) +- `study_utils.py` — Helper functions (if needed) diff --git a/studies/study_015/index.json b/studies/study_015/index.json new file mode 100644 index 0000000..e69d0cd --- /dev/null +++ b/studies/study_015/index.json @@ -0,0 +1,19 @@ +{ + "title": "Testing the decoy effect to improve online survey participation: Evidence from a field experiment", + "authors": [ + "Sandro T. Stoffel", + "Yining Sun", + "Yasemin Hirst", + "Christian von Wagner", + "Ivo Vlaev" + ], + "year": 2023, + "description": "A field experiment testing whether adding an inferior decoy survey option increases completion of a target online questionnaire. The study examines participation rates, order effects, and potential impacts on response behavior and non-response bias.", + "contributors": [ + { + "name": "Tanya Bhat", + "github": "https://github.com/tbhat-ops", + "institution": "UC San Diego" + } + ] +} diff --git a/studies/study_015/scripts/config.py b/studies/study_015/scripts/config.py new file mode 100644 index 0000000..4523606 --- /dev/null +++ b/studies/study_015/scripts/config.py @@ -0,0 +1,121 @@ +import json +import random +from pathlib import Path +from typing import Any, Dict, List, Optional + +import sys +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from study_utils import BaseStudyConfig, PromptBuilder + + +CONDITION_MATERIAL_MAP = { + "control": "main_experiment_control", + "decoy_target_first": "main_experiment_decoy_target_first", + "decoy_decoy_first": "main_experiment_decoy_decoy_first", +} + +CONDITION_WEIGHTS = { + "control": 101, + "decoy_target_first": 52, + "decoy_decoy_first": 50, +} + + +class DecoyEffectPromptBuilder(PromptBuilder): + + def build_trial_prompt(self, trial_metadata: Dict[str, Any]) -> str: + condition = trial_metadata["condition"] + material = trial_metadata["material"] + invitation = material["invitation_email"] + + prompt = ( + "You are a university student who previously signed up for a research study. " + "You completed a preliminary questionnaire about your attitudes and provided your " + "email address to receive an invitation to a follow-up survey.\n\n" + "You have now received the following email invitation:\n\n" + "---\n" + ) + + prompt += f"Subject: {invitation['subject']}\n\n" + prompt += f"{invitation['body']}\n" + + if condition == "control": + prompt += ( + "\n---\n\n" + "Based on this invitation, do you choose to complete the survey?\n\n" + "RESPONSE_SPEC: Output CHOICE=\n" + ) + else: + table = material["comparison_table"] + prompt += "\n\nHere is a comparison of the two available questionnaires:\n\n" + + for i, col in enumerate(table["columns"]): + label = chr(65 + i) + prompt += f"Option {label}: {col['title']}\n" + prompt += f" - Question type: {col['question_type']}\n" + prompt += f" - Reward: {col['reward']}\n" + prompt += f" - Reward timing: {col['reward_timing']}\n\n" + + prompt += f"{table['footer']}\n" + prompt += ( + "\n---\n\n" + "Based on this invitation, which option do you choose?\n" + " A) Complete Option A\n" + " B) Complete Option B\n" + " C) Decline to participate\n\n" + "RESPONSE_SPEC: Output CHOICE=\n" + ) + + return prompt + + +class StudyConfig(BaseStudyConfig): + prompt_builder_class = DecoyEffectPromptBuilder + PROMPT_VARIANT = "v1" + + def create_trials(self, n_trials: Optional[int] = None) -> List[Dict[str, Any]]: + spec = self.load_specification() + + total_n = spec.get("participants", {}).get("main_experiment", {}).get("n", 0) + if n_trials is not None: + total_n = n_trials + if total_n == 0: + total_n = 50 + + total_weight = sum(CONDITION_WEIGHTS.values()) + condition_ns = {} + assigned = 0 + conditions = list(CONDITION_WEIGHTS.keys()) + for i, cond in enumerate(conditions): + if i == len(conditions) - 1: + condition_ns[cond] = total_n - assigned + else: + n_cond = round(total_n * CONDITION_WEIGHTS[cond] / total_weight) + condition_ns[cond] = n_cond + assigned += n_cond + + materials = {} + for cond, mat_id in CONDITION_MATERIAL_MAP.items(): + materials[cond] = self.load_material(mat_id) + + trials = [] + trial_id = 0 + for cond in conditions: + mat = materials[cond] + for _ in range(condition_ns[cond]): + trials.append({ + "trial_id": trial_id, + "sub_study_id": CONDITION_MATERIAL_MAP[cond], + "condition": cond, + "material": mat, + "scenario_id": f"decoy_effect_{cond}", + "items": mat.get("items", mat.get("target_questionnaire", {}).get("items", [])), + "variant": self.PROMPT_VARIANT, + }) + trial_id += 1 + + random.shuffle(trials) + for i, t in enumerate(trials): + t["trial_id"] = i + + return trials diff --git a/studies/study_015/scripts/evaluator.py b/studies/study_015/scripts/evaluator.py new file mode 100644 index 0000000..f701037 --- /dev/null +++ b/studies/study_015/scripts/evaluator.py @@ -0,0 +1,500 @@ +import json +import math +import re +import numpy as np +from pathlib import Path +from scipy import stats +from typing import Any, Dict, List, Optional, Tuple + + +# --------------------------------------------------------------------------- +# Minimal BAS infrastructure (inlined because study_015 stats_lib is minimal) +# --------------------------------------------------------------------------- + +def calc_bf_chisq(chi2: float, n: int, df: int = 1) -> float: + """BF10 for chi-square test via BIC approximation.""" + try: + log_bf = (chi2 - df * math.log(n)) / 2.0 + return math.exp(log_bf) + except Exception: + return 1.0 + + +def chi2_contingency_safe(observed): + """Perform chi-square contingency test safely.""" + obs = np.array(observed) + dof = (obs.shape[0] - 1) * (obs.shape[1] - 1) if obs.ndim == 2 else 1 + if np.sum(obs) == 0: + return 0.0, 1.0, dof, None + if np.any(np.sum(obs, axis=0) == 0) or np.any(np.sum(obs, axis=1) == 0): + return 0.0, 1.0, dof, None + try: + chi2, p, res_dof, expected = stats.chi2_contingency(obs) + if expected is not None and np.any(expected == 0): + return 0.0, 1.0, res_dof, expected + return chi2, p, res_dof, expected + except (ValueError, RuntimeWarning): + return 0.0, 1.0, dof, None + + +def calc_posteriors_3way(bf10: float, direction: int, prior_odds: float = 1.0) -> dict: + """Calculate 3-way posterior probabilities (H+, H-, H0).""" + if bf10 is None or math.isnan(bf10): + pi_zero = 0.5 + pi_one = 0.5 + elif math.isinf(bf10): + pi_zero = 0.0 + pi_one = 1.0 + else: + odds = bf10 * prior_odds + pi_one = odds / (1.0 + odds) + pi_zero = 1.0 / (1.0 + odds) + + if direction > 0: + pi_plus = pi_one * 0.9999 + pi_minus = pi_one * 0.0001 + elif direction < 0: + pi_plus = pi_one * 0.0001 + pi_minus = pi_one * 0.9999 + else: + pi_plus = pi_one * 0.5 + pi_minus = pi_one * 0.5 + + return { + "pi_plus": float(pi_plus), + "pi_minus": float(pi_minus), + "pi_zero": float(pi_zero), + } + + +POSTERIOR_NULL = {"pi_plus": 0.0, "pi_minus": 0.0, "pi_zero": 1.0} + + +def calc_pas(pi_h, pi_a) -> float: + """Probability Alignment Score (BAS). Supports 3-way dict and scalar inputs.""" + if isinstance(pi_h, dict) and isinstance(pi_a, dict): + return ( + pi_h.get("pi_plus", 0.0) * pi_a.get("pi_plus", 0.0) + + pi_h.get("pi_minus", 0.0) * pi_a.get("pi_minus", 0.0) + + pi_h.get("pi_zero", 0.0) * pi_a.get("pi_zero", 0.0) + ) + try: + ph = max(1e-6, min(1.0 - 1e-6, float(pi_h))) + pa = max(1e-6, min(1.0 - 1e-6, float(pi_a))) + return ph * pa + (1 - ph) * (1 - pa) + except (TypeError, ValueError): + return 0.5 + + +# --------------------------------------------------------------------------- +# Target-option mapping per condition +# --------------------------------------------------------------------------- +# In decoy_target_first: column[0]=target → Option A=target, B=decoy +# In decoy_decoy_first: column[0]=decoy → Option A=decoy, B=target + +TARGET_OPTION = { + "decoy_target_first": "A", + "decoy_decoy_first": "B", +} + + +# --------------------------------------------------------------------------- +# Required interface functions +# --------------------------------------------------------------------------- + +def parse_agent_responses(response_text: str) -> Dict[str, str]: + """Parse agent response text into {key: value} dict. + + Handles: + CHOICE=COMPLETE, CHOICE=A, choice = b, etc. + """ + parsed = {} + if not response_text: + return parsed + + pattern = re.compile(r"(CHOICE)\s*[:=]\s*(\S+)", re.IGNORECASE) + for m in pattern.finditer(response_text): + parsed[m.group(1).upper()] = m.group(2).upper().strip(".") + + if not parsed: + text_up = response_text.upper() + for token in ["COMPLETE", "DECLINE"]: + if token in text_up: + parsed["CHOICE"] = token + break + if not parsed: + for token in ["A", "B", "C"]: + if re.search(rf"\b{token}\b", text_up): + parsed["CHOICE"] = token + break + + return parsed + + +def get_required_q_numbers(trial_info: Dict[str, Any]) -> set: + """Return required question identifiers for each trial.""" + return {"CHOICE"} + + +# --------------------------------------------------------------------------- +# Helper: determine whether participant chose the target survey +# --------------------------------------------------------------------------- + +def _chose_target(condition: str, choice: str) -> bool: + """Return True if the agent chose the target survey.""" + choice = choice.upper() + if condition == "control": + return choice == "COMPLETE" + target_letter = TARGET_OPTION.get(condition) + if target_letter is None: + return False + return choice == target_letter + + +# --------------------------------------------------------------------------- +# Helper: compute human posteriors from ground-truth p-value strings +# --------------------------------------------------------------------------- + +def _parse_p(p_str: str) -> Optional[float]: + """Parse a p-value string like 'p < 0.001' or 'p = 0.165'.""" + if not p_str or p_str == "NOT PROVIDED": + return None + m = re.search(r"p\s*[=:]\s*([0-9.]+)", p_str, re.IGNORECASE) + if m: + return float(m.group(1)) + m = re.search(r"p\s*<\s*([0-9.]+)", p_str, re.IGNORECASE) + if m: + return float(m.group(1)) / 2.0 + return None + + +def _human_posterior_from_chi2(test_data: dict, prior_odds: float = 10.0) -> dict: + """Compute human posterior from a ground-truth chi-square/contingency test. + + Attempts to reconstruct chi2 from raw_data, otherwise approximates from p-value. + """ + rd = test_data.get("raw_data", {}) + + # --- Try to reconstruct chi2 from 2x2 raw counts --- + chi2_val = None + n_total = None + + # F2 primary outcome format + if "control_completed" in rd and "control_n" in rd and "decoy_completed" in rd and "decoy_n" in rd: + c_yes = rd["control_completed"] + c_n = rd["control_n"] + d_yes = rd["decoy_completed"] + d_n = rd["decoy_n"] + table = np.array([[c_yes, c_n - c_yes], [d_yes, d_n - d_yes]]) + chi2_val, _, _, _ = chi2_contingency_safe(table) + n_total = c_n + d_n + + # F3 order-effect formats + if chi2_val is None and "decoy_target_first_completed" in rd and "control_completed" in rd: + c_yes = rd["control_completed"] + c_n = rd["control_n"] + d_yes = rd["decoy_target_first_completed"] + d_n = rd["decoy_target_first_n"] + table = np.array([[c_yes, c_n - c_yes], [d_yes, d_n - d_yes]]) + chi2_val, _, _, _ = chi2_contingency_safe(table) + n_total = c_n + d_n + + if chi2_val is None and "decoy_decoy_first_completed" in rd and "control_completed" in rd: + c_yes = rd["control_completed"] + c_n = rd["control_n"] + d_yes = rd["decoy_decoy_first_completed"] + d_n = rd["decoy_decoy_first_n"] + table = np.array([[c_yes, c_n - c_yes], [d_yes, d_n - d_yes]]) + chi2_val, _, _, _ = chi2_contingency_safe(table) + n_total = c_n + d_n + + if chi2_val is not None and n_total is not None and n_total > 0: + bf_h = calc_bf_chisq(chi2_val, n_total, df=1) + direction = 1 + return calc_posteriors_3way(bf_h, direction, prior_odds=prior_odds) + + # Fallback: use reported p-value to approximate BF + p_val = _parse_p(test_data.get("p_value", "")) + if p_val is not None and p_val < 1.0: + # Use -2 ln(p) as a rough chi2 surrogate with n=200 + approx_chi = -2.0 * math.log(max(p_val, 1e-300)) + bf_h = calc_bf_chisq(approx_chi, 200, df=1) + direction = 1 + return calc_posteriors_3way(bf_h, direction, prior_odds=prior_odds) + + return calc_posteriors_3way(1.0, 0, prior_odds=prior_odds) + + +# --------------------------------------------------------------------------- +# Main evaluation +# --------------------------------------------------------------------------- + +def evaluate_study(results: Dict[str, Any]) -> Dict[str, Any]: + """Evaluate agent performance on the decoy-effect study.""" + + study_path = Path(results.get("study_path", results.get("study_dir", "."))) + source_path = study_path / "source" + + with open(source_path / "ground_truth.json", "r") as f: + gt = json.load(f) + with open(source_path / "metadata.json", "r") as f: + meta = json.load(f) + + finding_weights = {fw["finding_id"]: fw["weight"] for fw in meta.get("findings", [])} + + # ------------------------------------------------------------------ + # 1. Parse agent responses and group by condition + # ------------------------------------------------------------------ + agent_by_condition: Dict[str, List[bool]] = { + "control": [], + "decoy_target_first": [], + "decoy_decoy_first": [], + } + + individual_data = results.get("individual_data", results.get("participants", [])) + for participant in individual_data: + responses = participant.get("responses", [participant]) + for resp in responses: + trial_info = resp.get("trial_info", resp) + condition = trial_info.get("condition", "") + response_text = resp.get("response_text", resp.get("response", "")) + + parsed = parse_agent_responses(response_text) + choice = parsed.get("CHOICE", "") + + if condition in agent_by_condition: + agent_by_condition[condition].append(_chose_target(condition, choice)) + + # Convenience counts + ctrl = agent_by_condition["control"] + dtf = agent_by_condition["decoy_target_first"] + ddf = agent_by_condition["decoy_decoy_first"] + decoy_all = dtf + ddf + + n_ctrl = len(ctrl) + n_dtf = len(dtf) + n_ddf = len(ddf) + n_decoy = len(decoy_all) + + ctrl_completed = sum(ctrl) + dtf_completed = sum(dtf) + ddf_completed = sum(ddf) + decoy_completed = dtf_completed + ddf_completed + + # ------------------------------------------------------------------ + # 2. Evaluate each finding + # ------------------------------------------------------------------ + test_results: List[Dict[str, Any]] = [] + finding_results: List[Dict[str, Any]] = [] + + # ---- F1: Randomization balance checks (weight 0.01) ---- + # Agent data has no demographics to compare, so we assign a neutral BAS. + f1_tests = gt.get("main_study", {}).get("findings", []) + f1_finding = next((f for f in f1_tests if f["finding_id"] == "F1"), None) + if f1_finding: + n_f1_tests = len(f1_finding.get("statistical_tests", [])) + for st in f1_finding.get("statistical_tests", []): + # Cannot reconstruct balance checks from agent data (no demographics). + test_results.append({ + "finding_id": "F1", + "test_id": st["test_id"], + "test_name": st["test_name"], + "pi_human": {"pi_plus": 0.0, "pi_minus": 0.0, "pi_zero": 1.0}, + "pi_agent": {"pi_plus": 0.0, "pi_minus": 0.0, "pi_zero": 1.0}, + "pas": 1.0, # Both are null → perfect agreement + "test_weight": 1.0 / max(n_f1_tests, 1), + "note": "Balance check not reconstructible from agent data; scored as null agreement.", + }) + f1_score = 1.0 + finding_results.append({ + "finding_id": "F1", + "finding_weight": finding_weights.get("F1", 0.01), + "finding_score": f1_score, + }) + + # ---- F2: Primary outcome — decoy increases target completion ---- + f2_finding = next((f for f in f1_tests if f["finding_id"] == "F2"), None) + f2_test_scores = [] + if f2_finding: + for st in f2_finding.get("statistical_tests", []): + test_id = st["test_id"] + pi_h = _human_posterior_from_chi2(st) + + # Agent posterior + if n_ctrl >= 2 and n_decoy >= 2: + table = np.array([ + [ctrl_completed, n_ctrl - ctrl_completed], + [decoy_completed, n_decoy - decoy_completed], + ]) + chi2_a, p_a, _, _ = chi2_contingency_safe(table) + bf_a = calc_bf_chisq(chi2_a, n_ctrl + n_decoy, df=1) + a_rate_ctrl = ctrl_completed / n_ctrl if n_ctrl else 0 + a_rate_decoy = decoy_completed / n_decoy if n_decoy else 0 + a_dir = 1 if a_rate_decoy > a_rate_ctrl else (-1 if a_rate_decoy < a_rate_ctrl else 0) + pi_a = calc_posteriors_3way(bf_a, a_dir) + else: + pi_a = dict(POSTERIOR_NULL) + + pas = calc_pas(pi_h, pi_a) + test_results.append({ + "finding_id": "F2", + "test_id": test_id, + "test_name": st["test_name"], + "pi_human": pi_h, + "pi_agent": pi_a, + "pas": float(pas), + "test_weight": 1.0 / max(len(f2_finding["statistical_tests"]), 1), + "agent_stats": { + "control_n": n_ctrl, + "control_completed": ctrl_completed, + "decoy_n": n_decoy, + "decoy_completed": decoy_completed, + }, + }) + f2_test_scores.append(pas) + + f2_score = float(np.mean(f2_test_scores)) if f2_test_scores else 0.5 + finding_results.append({ + "finding_id": "F2", + "finding_weight": finding_weights.get("F2", 0.35), + "finding_score": f2_score, + }) + + # ---- F3: Order effect (target-first vs decoy-first vs control) ---- + f3_finding = next((f for f in f1_tests if f["finding_id"] == "F3"), None) + f3_test_scores = [] + if f3_finding: + for st in f3_finding.get("statistical_tests", []): + test_id = st["test_id"] + pi_h = _human_posterior_from_chi2(st) + + # Determine which sub-comparison + if "target_first" in test_id: + sub_n = n_dtf + sub_completed = dtf_completed + elif "decoy_first" in test_id: + sub_n = n_ddf + sub_completed = ddf_completed + else: + sub_n = n_decoy + sub_completed = decoy_completed + + if n_ctrl >= 2 and sub_n >= 2: + table = np.array([ + [ctrl_completed, n_ctrl - ctrl_completed], + [sub_completed, sub_n - sub_completed], + ]) + chi2_a, p_a, _, _ = chi2_contingency_safe(table) + bf_a = calc_bf_chisq(chi2_a, n_ctrl + sub_n, df=1) + a_rate_ctrl = ctrl_completed / n_ctrl if n_ctrl else 0 + a_rate_sub = sub_completed / sub_n if sub_n else 0 + a_dir = 1 if a_rate_sub > a_rate_ctrl else (-1 if a_rate_sub < a_rate_ctrl else 0) + pi_a = calc_posteriors_3way(bf_a, a_dir) + else: + pi_a = dict(POSTERIOR_NULL) + + # Handle NOT PROVIDED p-values: if human posterior is fully null, + # direction from ground truth claim should still be respected. + p_str = st.get("p_value", "") + if p_str == "NOT PROVIDED": + # Non-significant in human data → direction 0 + pi_h = calc_posteriors_3way(1.0, 0, prior_odds=10.0) + + pas = calc_pas(pi_h, pi_a) + test_results.append({ + "finding_id": "F3", + "test_id": test_id, + "test_name": st["test_name"], + "pi_human": pi_h, + "pi_agent": pi_a, + "pas": float(pas), + "test_weight": 1.0 / max(len(f3_finding["statistical_tests"]), 1), + "agent_stats": { + "control_n": n_ctrl, + "control_completed": ctrl_completed, + "sub_condition_n": sub_n, + "sub_condition_completed": sub_completed, + }, + }) + f3_test_scores.append(pas) + + f3_score = float(np.mean(f3_test_scores)) if f3_test_scores else 0.5 + finding_results.append({ + "finding_id": "F3", + "finding_weight": finding_weights.get("F3", 0.35), + "finding_score": f3_score, + }) + + # ---- F1_prelim and F2_prelim ---- + # Preliminary findings about question-type preference and payment-delay aversion. + # Not directly testable from the main experiment agent data (no preliminary survey). + # Score as neutral (0.5) to avoid penalizing or rewarding by default. + for fid in ["F1_prelim", "F2_prelim"]: + finding_results.append({ + "finding_id": fid, + "finding_weight": finding_weights.get(fid, 0.05), + "finding_score": 0.5, + "note": "Preliminary finding not reconstructible from main experiment agent data.", + }) + + # ---- F4: Perceived influence of decoy (secondary) ---- + # This finding reports that 57.9% of decoy-condition target completers agreed + # the decoy influenced them. We cannot directly measure self-report from agent + # choice data. Score as neutral. + finding_results.append({ + "finding_id": "F4", + "finding_weight": finding_weights.get("F4", 0.10), + "finding_score": 0.5, + "note": "Self-reported decoy influence not measurable from agent choice data.", + }) + + # ---- F5: Non-response bias checks (secondary) ---- + # Demographic balance among completers vs non-completers and FCQ scores. + # Agent data lacks demographics and FCQ responses at the evaluation level. + # Score as neutral. + finding_results.append({ + "finding_id": "F5", + "finding_weight": finding_weights.get("F5", 0.14), + "finding_score": 0.5, + "note": "Non-response bias checks not reconstructible from agent choice data.", + }) + + # ------------------------------------------------------------------ + # 3. Two-level weighted aggregation + # ------------------------------------------------------------------ + total_w = 0.0 + total_ws = 0.0 + for fr in finding_results: + w = fr["finding_weight"] + total_w += w + total_ws += w * fr["finding_score"] + + overall_score = total_ws / total_w if total_w > 0 else 0.5 + + # ------------------------------------------------------------------ + # 4. Build return dict + # ------------------------------------------------------------------ + return { + "score": float(overall_score), + "pi_human": None, + "pi_agent": None, + "finding_results": finding_results, + "test_results": test_results, + "details": { + "agent_condition_counts": { + "control": n_ctrl, + "decoy_target_first": n_dtf, + "decoy_decoy_first": n_ddf, + }, + "agent_target_completions": { + "control": ctrl_completed, + "decoy_target_first": dtf_completed, + "decoy_decoy_first": ddf_completed, + }, + "agent_completion_rates": { + "control": ctrl_completed / n_ctrl if n_ctrl else 0.0, + "decoy_target_first": dtf_completed / n_dtf if n_dtf else 0.0, + "decoy_decoy_first": ddf_completed / n_ddf if n_ddf else 0.0, + }, + }, + } diff --git a/studies/study_015/scripts/stats_lib.py b/studies/study_015/scripts/stats_lib.py new file mode 100644 index 0000000..1e0dc0a --- /dev/null +++ b/studies/study_015/scripts/stats_lib.py @@ -0,0 +1,61 @@ +""" +Parse p-value from reported_statistics strings. +""" + +import re +from typing import Tuple, Optional + + +def parse_p_value_from_reported( + reported_statistics: str, significance_level: float = 0.05 +) -> Tuple[Optional[float], bool, str]: + """ + Parse p-value from reported_statistics string. + + Handles formats like: + - "p < .001" or "p < 0.001" + - "p < .05" or "p < 0.05" + - "p = 0.023" + - "r = 0.42, p < .001" + + Returns: + tuple: (p_value, is_significant, confidence) + """ + if significance_level is None: + significance_level = 0.05 + + if not reported_statistics: + return None, False, "low" + + text = reported_statistics.lower() + + exact_match = re.search(r"p\s*[=:]\s*([0-9.]+)", text) + if exact_match: + p_val = float(exact_match.group(1)) + is_sig = p_val <= significance_level if significance_level is not None else False + return p_val, is_sig, "high" + + inequality_match = re.search(r"p\s*[<>]\s*([0-9.]+)", text) + if inequality_match: + threshold = float(inequality_match.group(1)) + is_less_than = "<" in text[ + inequality_match.start() : inequality_match.end() + ] + + if is_less_than: + p_val = threshold / 2.0 + is_sig = True + return p_val, is_sig, "medium" + else: + p_val = threshold + is_sig = threshold < significance_level if significance_level is not None else False + return p_val, is_sig, "medium" + + r_match = re.search(r"r\s*=\s*([-0-9.]+)", text) + if r_match: + r_val = abs(float(r_match.group(1))) + if r_val > 0.3: + return 0.025, True, "low" + return 0.10, False, "low" + + return None, False, "low" diff --git a/studies/study_015/scripts/study_utils.py b/studies/study_015/scripts/study_utils.py new file mode 100644 index 0000000..8829fb3 --- /dev/null +++ b/studies/study_015/scripts/study_utils.py @@ -0,0 +1,150 @@ +""" +Standalone study utilities: BaseStudyConfig and PromptBuilder. +No dependency on src/; for use within each study's scripts/. +""" +import json +import re +from abc import ABC, abstractmethod +from pathlib import Path +from typing import Dict, Any, List, Optional + + +class PromptBuilder: + """Build prompts from study specification and materials. study_path = source directory.""" + + def __init__(self, study_path: Path): + self.study_path = Path(study_path) + self.materials_path = self.study_path / "materials" + with open(self.study_path / "specification.json", "r", encoding="utf-8", errors="replace") as f: + self.specification = json.load(f) + instructions_file = self.materials_path / "instructions.txt" + self.instructions = instructions_file.read_text(encoding="utf-8", errors="replace") if instructions_file.exists() else None + system_prompt_file = self.materials_path / "system_prompt.txt" + self.system_prompt_template = system_prompt_file.read_text(encoding="utf-8", errors="replace") if system_prompt_file.exists() else None + + def build_system_prompt(self, participant_profile: Dict[str, Any] = None) -> Optional[str]: + return self.system_prompt_template + + def get_system_prompt_template(self) -> Optional[str]: + return self.system_prompt_template + + def build_trial_prompt(self, trial_data: Dict[str, Any]) -> str: + return self._build_generic_trial_prompt(trial_data) + + def get_instructions(self) -> str: + return self.instructions if self.instructions else "No instructions provided." + + def _fill_template(self, template: str, data: Dict[str, Any]) -> str: + result = template + nested_pattern = r"\{\{([\w.]+)\}\}" + def replace_nested(match): + path = match.group(1) + value = data + for part in path.split("."): + if isinstance(value, dict) and part in value: + value = value[part] + else: + return match.group(0) + return str(value) + result = re.sub(nested_pattern, replace_nested, result) + if_pattern = r"\{\{#if\s+(\w+)\}\}(.*?)\{\{/if\}\}" + def replace_if(match): + if match.group(1) in data and data[match.group(1)]: + return match.group(2) + return "" + result = re.sub(if_pattern, replace_if, result, flags=re.DOTALL) + each_pattern = r"\{\{#each\s+(\w+)\}\}(.*?)\{\{/each\}\}" + def replace_each(match): + var_name, content = match.group(1), match.group(2) + if var_name not in data: + return "" + items = data[var_name] + if isinstance(items, dict): + parts = [content.replace("{{@key}}", str(k)).replace("{{this}}", str(v)) for k, v in items.items()] + return "\n".join(parts) + if isinstance(items, list): + parts = [content.replace("{{@index}}", str(i + 1)).replace("{{this}}", str(item)) for i, item in enumerate(items)] + return "\n".join(parts) + return "" + result = re.sub(each_pattern, replace_each, result, flags=re.DOTALL) + result = re.sub(r"\{\{[^}]+\}\}", "", result) + return result + + def _build_generic_trial_prompt(self, trial_data: Dict[str, Any]) -> str: + return f"Trial {trial_data.get('trial_number', '?')}: Please respond to the following stimulus." + + +class BaseStudyConfig(ABC): + """Study config base. study_path = study root (e.g. studies/study_001); data under source/.""" + + prompt_builder_class = PromptBuilder + + def __init__(self, study_path: Path, specification: Dict[str, Any]): + self.study_path = Path(study_path) + self.source_path = self.study_path / "source" + self.specification = specification + self.study_id = specification["study_id"] + self.prompt_builder = self.prompt_builder_class(self.source_path) + + def load_material(self, sub_study_id: str) -> Dict[str, Any]: + file_path = self.source_path / "materials" / f"{sub_study_id}.json" + if not file_path.exists(): + raise FileNotFoundError(f"Material not found: {file_path}") + with open(file_path, "r", encoding="utf-8") as f: + return json.load(f) + + def load_metadata(self) -> Dict[str, Any]: + with open(self.source_path / "metadata.json", "r", encoding="utf-8") as f: + return json.load(f) + + def load_specification(self) -> Dict[str, Any]: + with open(self.source_path / "specification.json", "r", encoding="utf-8") as f: + return json.load(f) + + def load_ground_truth(self) -> Dict[str, Any]: + with open(self.source_path / "ground_truth.json", "r", encoding="utf-8") as f: + return json.load(f) + + def extract_numeric(self, text: str, default: float = 0.0) -> float: + if text is None: + return default + match = re.search(r"(-?\d+\.?\d*)", str(text)) + return float(match.group(1)) if match else default + + def extract_choice(self, text: str, options: List[str] = None) -> Optional[int]: + if text is None: + return None + text_s = str(text).strip() + if options: + for i, opt in enumerate(options): + if opt.lower() in text_s.lower(): + return i + match = re.search(r"\b([A-Z])\b", text_s.upper()) + if match: + return ord(match.group(1)) - ord("A") + return None + + @abstractmethod + def create_trials(self, n_trials: Optional[int] = None) -> List[Dict[str, Any]]: + raise NotImplementedError + + def get_prompt_builder(self) -> PromptBuilder: + return self.prompt_builder + + def get_instructions(self) -> str: + return self.prompt_builder.get_instructions() + + def aggregate_results(self, raw_results: Dict[str, Any]) -> Dict[str, Any]: + return raw_results + + def custom_scoring(self, results: Dict[str, Any], ground_truth: Dict[str, Any]) -> Optional[Dict[str, float]]: + return None + + def get_n_participants(self) -> int: + return self.specification["participants"]["n"] + + def get_study_type(self) -> str: + return self.specification.get("study_type", self.study_id) + + def __repr__(self) -> str: + return f"{self.__class__.__name__}(study_id='{self.study_id}')" diff --git a/studies/study_015/source/analysis_18082022.do b/studies/study_015/source/analysis_18082022.do new file mode 100644 index 0000000..088140c --- /dev/null +++ b/studies/study_015/source/analysis_18082022.do @@ -0,0 +1,194 @@ +clear +import excel "C:\Users\sstoffel\Documents\temp_lit\decoy\DescriptiveAnalysis.xlsx", sheet("Sheet1") firstrow +rename ParticipateYes1NO0 part +recode part 1=0 if ParticipantNumber=="14" +recode part 1=0 if ParticipantNumber=="16" +recode part 1=0 if ParticipantNumber=="123" +recode part 1=0 if ParticipantNumber=="127" +la def part 0 "No" 1 "Yes" +la val part part +la var part "Completed the survey" +gen cond1=0 if D==1 +replace cond1=1 if D==2 | D==3 +la def cond1 0 "Control" 1 "Decoy" +la val cond1 cond1 +la var cond1 "Experimental condition" +rename D cond2 +recode cond2 1=0 2=1 3=2 +la def cond2 0 "Control" 1 "Decoy: target 1st" 2 "Decoy; decoy 1st" +la val cond2 cond2 +la var cond2 "Experimental condition" +gen agecat=0 if AgeRange=="20-24" +replace agecat=1 if AgeRange=="25-29" +replace agecat=2 if AgeRange=="30-35" +la def agecat 0 "20-24" 1 "25-29" 3 "30-35" +la val agecat agecat +la var agecat "Age category" +gen gender=0 if M=="Male" +replace gender=1 if M=="Female" +la def gender 0 "Male" 1 "Female" +la val gender gender +la var gender "Gender" +gen edu=0 if EducationLevel=="2" +replace edu=1 if EducationLevel=="1" +la def edu 0 "Bachelor's degree" 1 "Graduate or professional degree" +la val edu edu +la var edu "Education level" +gen ethn=0 if Race=="5" +replace ethn=1 if Race=="1" +replace ethn=2 if Race=="6" +replace ethn=3 if Race=="2" | Race=="3" | Race=="4" +la def ethn 0 "White" 1 "Asian" 2 "Black" 3 "Other" +la val ethn ethn +la var ethn "Ethnicity" +gen rel=0 if Religion=="3" +replace rel=1 if Religion=="1" | Religion=="4" | Religion=="5" | Religion=="6" +replace rel=2 if Religion=="2" +la def rel 0 "Christian" 1 "Ohter religion" 2 "No religion" +la val rel rel +la var rel "Religion" +save "C:\Users\sstoffel\Documents\temp_lit\decoy\exp.dta", replace +//////////////////////////////////////////////////////////////////////////////// +clear +use "C:\Users\sstoffel\Documents\temp_lit\decoy\exp.dta" +** Table 1 Description of the study sample of the experiment (N=203) +tab agecat cond1 if ValidEmailYes1No0==1, chi2 col +tab gender cond1 if ValidEmailYes1No0==1, chi2 col +tab ethn cond1 if ValidEmailYes1No0==1, chi2 exact col +tab rel cond1 if ValidEmailYes1No0==1, chi2 exact col +tab edu cond1 if ValidEmailYes1No0==1, chi2 col +** Table 2. Participation rates across the two experimental conditions (N=203) +tab part cond1 if ValidEmailYes1No0==1, chi2 col +** Table 3. Participation rates within the decoy condition (N=102) +tab part cond2 if ValidEmailYes1No0==1, chi2 col +** Table 4. Binary logistic regression on completing the survey +quietly logit part i.cond1 if ValidEmailYes1No0==1, or +outreg using reg1.doc, stats(e_b e_ci) nosubstat summstat(N) ctitle(Variable, Odds ratio, CI) replace +quietly logit part i.cond1 i.agecat i.gender i.ethn i.rel i.edu if ValidEmailYes1No0==1, or +outreg using reg1.doc, stats(e_b e_ci) nosubstat merge summstat(N) ctitle(Variable, Odds ratio, CI) +// +quietly logit part i.agecat if ValidEmailYes1No0==1, or +outreg using reg2.doc, stats(e_b e_ci) nosubstat summstat(N) ctitle(Variable, Odds ratio, CI) replace +quietly logit part i.gender if ValidEmailYes1No0==1, or +outreg using reg3.doc, stats(e_b e_ci) nosubstat summstat(N) ctitle(Variable, Odds ratio, CI) replace +quietly logit part i.ethn if ValidEmailYes1No0==1, or +outreg using reg4.doc, stats(e_b e_ci) nosubstat summstat(N) ctitle(Variable, Odds ratio, CI) replace +quietly logit part i.rel if ValidEmailYes1No0==1, or +outreg using reg5.doc, stats(e_b e_ci) nosubstat summstat(N) ctitle(Variable, Odds ratio, CI) replace +quietly logit part i.edu if ValidEmailYes1No0==1, or +outreg using reg6.doc, stats(e_b e_ci) nosubstat summstat(N) ctitle(Variable, Odds ratio, CI) replace +// +tab part cond1 if ValidEmailYes1No0==1, col +tab part agecat if ValidEmailYes1No0==1, col +tab part gender if ValidEmailYes1No0==1, col +tab part ethn if ValidEmailYes1No0==1, col +tab part rel if ValidEmailYes1No0==1, col +tab part edu if ValidEmailYes1No0==1, col +******************************************************************************** +// Comparison of study participants who chose the decoy +tab agecat cond1 if ValidEmailYes1No0==1 & part==1, chi2 col +tab gender cond1 if ValidEmailYes1No0==1 & part==1, chi2 col +tab ethn cond1 if ValidEmailYes1No0==1 & part==1, chi2 exact col +tab rel cond1 if ValidEmailYes1No0==1 & part==1, chi2 exact col +tab edu cond1 if ValidEmailYes1No0==1 & part==1, chi2 col +******************************************************************************** +** Figure 2. Survey completing rate across experimental conditions +clear +use "C:\Users\sstoffel\Documents\temp_lit\decoy\exp.dta" +drop if ValidEmailYes1No0!=1 +gen per=100*part +set scheme s2mono +collapse (mean) meanper= per (sd) sdper=per (count) n=per, by(cond1) +generate hiper = meanper + invttail(n-1,0.025)*(sdper / sqrt(n)) +generate lowper = meanper - invttail(n-1,0.025)*(sdper / sqrt(n)) +twoway (bar meanper cond1, barwidth(0.9)) (rcap hiper lowper cond), title("Individuals completing survey", size(medium) color(black)) xlabel(0 "Control condition" 1 "Decoy condition") xtitle("N=203", margin(medium)) ylabel (0(10)70) ytitle("Percentage") ylabel(, angle(0)) legend(off) graphregion(color(white)) bgcolor(white) +** Figure 3. Survey completing rate within decoy condition (order effeect) +clear +use "C:\Users\sstoffel\Documents\temp_lit\decoy\exp.dta" +drop if ValidEmailYes1No0!=1 +gen per=100*part +set scheme s2mono +collapse (mean) meanper= per (sd) sdper=per (count) n=per, by(cond2) +generate hiper = meanper + invttail(n-1,0.025)*(sdper / sqrt(n)) +generate lowper = meanper - invttail(n-1,0.025)*(sdper / sqrt(n)) +twoway (bar meanper cond2, barwidth(0.9)) (rcap hiper lowper cond), title("Individuals completing survey", size(medium) color(black)) xlabel(0 "Control condition" 1 "Target shown first" 2 "Decoy shown first") xtitle("N=203", margin(medium)) ylabel (0(10)100) ytitle("Percentage") ylabel(, angle(0)) legend(off) graphregion(color(white)) bgcolor(white) +** Figure 3a. Preference for question types +clear +input id int1 int2 int3 + id int1 int2 int3 + 1. 61.4 7.9 30.7 +end +lab define id 1 "Overall" +lab value id id +graph bar int1 int2 int3 /// + , name(p1, replace) blabel(total) /// + ytitle(Percentages) /// + legend(lab(1 "Close-ended questions") lab(2 "Open-ended questions") lab(3 "Indifferent") lab() symxsize(5) size(small) row(1)) + ** Figure 3b. Preference for question types +clear +input id int1 int2 int3 + id int1 int2 int3 + 1. 55.0 12.0 33.0 + 2. 67.7 3.9 28.4 +end +lab define id 1 "Control condition" 2 "Decoy condition" +lab value id id +graph bar int1 int2 int3 /// + , name(p1, replace) over(id) blabel(total) /// + ytitle(Percentages) /// + legend(lab(1 "Close-ended questions") lab(2 "Open-ended questions") lab(3 "Indifferent") lab() symxsize(5) size(small) row(1)) +** Figure 4. Attitudes towards late payment +clear +input id int1 int2 int3 int4 + + id int1 int2 int3 int4 + 1. 1.0 85.1 1.5 12.4 +end +lab define id 1 "Overall" +lab value id id +graph bar int1 int2 int3 int4 /// + , name(p1, replace) blabel(total) /// + ytitle(Percentages) /// + legend(lab(1 "Definitely not") lab(2 "Probably not") lab(3 "Probably yes") lab(4 "Definitely yes") symxsize(5) size(small) row(1)) +** Figure 5. Attitudes towards late payment +clear +input id int1 int2 int3 int4 + + id int1 int2 int3 int4 + 1. 1 2.0 78.0 1.0 19.0 + 2. 2 0.0 92.2 2.0 5.9 +end +lab define id 1 "Control condition" 2 "Decoy condition" +lab value id id +graph bar int1 int2 int3 int4 /// + , name(p1, replace) over(id) blabel(total) /// + ytitle(Percentages) /// + legend(lab(1 "Definitely not") lab(2 "Probably not") lab(3 "Probably yes") lab(4 "Definitely yes") symxsize(5) size(small) row(1)) +////////////////////////////////////////////////////////////////////// +////////////////////////////////////////////////////////////////////// +* PRELIMINARY SURVEY +** Figure 2. Attitudes towards late payment +clear +input id int1 int2 int3 + id int1 int2 int3 + 1. 60.0 7.6 32.4 +end +lab define id 1 "Overall" +lab value id id +graph bar int1 int2 int3 /// + , name(p1, replace) blabel(total) /// + ytitle(Percentages) /// + legend(lab(1 "Close-ended questions") lab(2 "Open-ended questions") lab(3 "Indifferent") lab() symxsize(5) size(small) row(1)) +** Figure 3. Attitudes towards late payment +clear +input id int1 int2 int3 int4 + + id int1 int2 int3 int4 + 1. 1.0 85.2 1.9 11.9 +end +lab define id 1 "Overall" +lab value id id +graph bar int1 int2 int3 int4 /// + , name(p1, replace) blabel(total) /// + ytitle(Percentages) /// + legend(lab(1 "Definitely not") lab(2 "Probably not") lab(3 "Probably yes") lab(4 "Definitely yes") symxsize(5) size(small) row(1)) diff --git a/studies/study_015/source/decoy_effect_1.pdf b/studies/study_015/source/decoy_effect_1.pdf new file mode 100644 index 0000000..1471271 Binary files /dev/null and b/studies/study_015/source/decoy_effect_1.pdf differ diff --git a/studies/study_015/source/decoy_effect_1_apndx.pdf b/studies/study_015/source/decoy_effect_1_apndx.pdf new file mode 100644 index 0000000..1e82a35 Binary files /dev/null and b/studies/study_015/source/decoy_effect_1_apndx.pdf differ diff --git a/studies/study_015/source/exp.csv b/studies/study_015/source/exp.csv new file mode 100644 index 0000000..a94c5a2 --- /dev/null +++ b/studies/study_015/source/exp.csv @@ -0,0 +1,211 @@ +ParticipantNumber,Condition,conditiongroup,cond2,Participate,part,ValidEmail,ValidEmailYes1No0,Age,AgeRange,Whatisyouryearofbirth,Gender,M,Race,Religion,P,EducationLevel,R,QuestionTypePrefer,AttitudetoLatePayment,U,cond1,agecat,gender,edu,ethn,rel +1,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,22,20-24,2000,0,Female,6,1,Hindu,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Probably no,,Decoy,20-24,Female,Graduate or professional degree,Black,Ohter religion +2,Control Condition,Control Condition,Control,No,No,Yes,1,29,25-29,1993,1,Male,6,2,No Religion,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely no,,Control,25-29,Male,Graduate or professional degree,Black,No religion +3,Control Condition,Control Condition,Control,No,No,Yes,1,26,25-29,1996,0,Female,6,2,No Religion,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Probably yes,,Control,25-29,Female,Graduate or professional degree,Black,No religion +4,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,25,25-29,1997,0,Female,6,2,No Religion,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Probably yes,,Control,25-29,Female,Graduate or professional degree,Black,No religion +5,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,24,20-24,1998,0,Female,6,2,No Religion,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Probably yes,,Decoy,20-24,Female,Graduate or professional degree,Black,No religion +6,Control Condition,Control Condition,Control,No,No,Yes,1,24,20-24,1998,0,Female,6,2,No Religion,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Probably yes,,Control,20-24,Female,Graduate or professional degree,Black,No religion +7,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,23,20-24,1999,1,Male,6,2,No Religion,2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Control,20-24,Male,Bachelor's degree,Black,No religion +8,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,26,25-29,1996,0,Female,6,2,No Religion,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Probably yes,,Control,25-29,Female,Graduate or professional degree,Black,No religion +9,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,26,25-29,1996,1,Male,6,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Definitely no,,Control,25-29,Male,Bachelor's degree,Black,Christian +10,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,32,30-35,1990,1,Male,5,1,Hindu,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Control,2.0,Male,Graduate or professional degree,White,Ohter religion +11,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,30,30-35,1992,1,Male,5,1,Hindu,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Decoy,2.0,Male,Graduate or professional degree,White,Ohter religion +12,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,30,30-35,1992,1,Male,5,1,Hindu,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Control,2.0,Male,Graduate or professional degree,White,Ohter religion +13,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,30,30-35,1992,1,Male,5,1,Hindu,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Control,2.0,Male,Graduate or professional degree,White,Ohter religion +14,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,No,Yes,1,32,30-35,1990,0,Female,5,1,Hindu,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Decoy,2.0,Female,Graduate or professional degree,White,Ohter religion +15,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,30,30-35,1992,1,Male,5,1,Hindu,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Decoy,2.0,Male,Graduate or professional degree,White,Ohter religion +16,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,No,Yes,1,30,30-35,1992,1,Male,5,1,Hindu,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Decoy,2.0,Male,Graduate or professional degree,White,Ohter religion +17,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,27,25-29,1995,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Male,Bachelor's degree,White,Christian +18,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,27,25-29,1995,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Male,Bachelor's degree,White,Christian +19,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,27,25-29,1995,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Open-ended questions (e.g. text entry),Definitely yes,,Decoy,25-29,Male,Bachelor's degree,White,Christian +20,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,27,25-29,1995,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Male,Bachelor's degree,White,Christian +21,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,27,25-29,1995,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Female,Bachelor's degree,White,Christian +22,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,27,25-29,1995,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Male,Bachelor's degree,White,Christian +23,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,27,25-29,1995,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Male,Graduate or professional degree,White,Christian +24,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,27,25-29,1995,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Open-ended questions (e.g. text entry),Definitely yes,,Decoy,25-29,Male,Graduate or professional degree,White,Christian +25,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,27,25-29,1995,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Male,Bachelor's degree,White,Christian +26,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,27,25-29,1995,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Female,Graduate or professional degree,White,Christian +27,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,26,25-29,1996,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Male,Bachelor's degree,White,Christian +28,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,26,25-29,1996,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Female,Graduate or professional degree,White,Christian +29,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,26,25-29,1996,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Female,Bachelor's degree,White,Christian +30,Decoy Condition,Decoy Condition 1,Decoy: target 1st,No,No,Yes,1,26,25-29,1996,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Female,Bachelor's degree,White,Christian +31,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,26,25-29,1996,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Female,Bachelor's degree,White,Christian +32,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,26,25-29,1996,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Female,Graduate or professional degree,White,Christian +33,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,26,25-29,1996,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Female,Bachelor's degree,White,Christian +34,Decoy Condition,Decoy Condition 1,Decoy: target 1st,No,No,Yes,1,26,25-29,1996,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Female,Graduate or professional degree,White,Christian +35,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,28,25-29,1994,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Female,Graduate or professional degree,White,Christian +36,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,28,25-29,1994,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Male,Graduate or professional degree,White,Christian +37,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,28,25-29,1994,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Male,Bachelor's degree,White,Christian +38,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,28,25-29,1994,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Control,25-29,Female,Graduate or professional degree,White,Christian +39,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,28,25-29,1994,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Female,Graduate or professional degree,White,Christian +40,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,28,25-29,1994,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Control,25-29,Male,Bachelor's degree,White,Christian +41,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,25,25-29,1997,0,Female,5,6,Muslim,2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Female,Bachelor's degree,White,Ohter religion +42,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,25,25-29,1997,1,Male,5,6,Muslim,2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Male,Bachelor's degree,White,Ohter religion +43,Decoy Condition,Decoy Condition 1,Decoy: target 1st,No,No,Yes,1,25,25-29,1997,1,Male,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Male,Bachelor's degree,Asian,Christian +44,Decoy Condition,Decoy Condition 1,Decoy: target 1st,No,No,Yes,1,32,30-35,1990,1,Male,5,6,Muslim,2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,2.0,Male,Bachelor's degree,White,Ohter religion +45,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,22,20-24,2000,0,Female,1,2,No Religion,2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,20-24,Female,Bachelor's degree,Asian,No religion +46,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,23,20-24,1999,0,Female,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Open-ended questions (e.g. text entry),Definitely yes,,Decoy,20-24,Female,Bachelor's degree,Asian,Christian +47,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,21,20-24,2001,0,Female,1,2,No Religion,2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,20-24,Female,Bachelor's degree,Asian,No religion +48,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,22,20-24,2000,0,Female,1,2,No Religion,2,University - Bachelors Degree,Open-ended questions (e.g. text entry),Definitely yes,,Control,20-24,Female,Bachelor's degree,Asian,No religion +49,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,23,20-24,1999,0,Female,1,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Decoy,20-24,Female,Graduate or professional degree,Asian,Christian +50,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,21,20-24,2001,0,Female,1,2,No Religion,2,University - Bachelors Degree,Open-ended questions (e.g. text entry),Definitely yes,,Control,20-24,Female,Bachelor's degree,Asian,No religion +51,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,22,20-24,2000,0,Female,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Open-ended questions (e.g. text entry),Definitely yes,,Control,20-24,Female,Bachelor's degree,Asian,Christian +52,Control Condition,Control Condition,Control,No,No,Yes,1,21,20-24,2001,0,Female,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Open-ended questions (e.g. text entry),Definitely yes,,Control,20-24,Female,Bachelor's degree,Asian,Christian +53,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,20,20-24,2002,0,Female,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Open-ended questions (e.g. text entry),Definitely yes,,Control,20-24,Female,Bachelor's degree,Asian,Christian +54,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,21,20-24,2001,0,Female,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Open-ended questions (e.g. text entry),Definitely yes,,Control,20-24,Female,Bachelor's degree,Asian,Christian +55,Control Condition,Control Condition,Control,No,No,Yes,1,22,20-24,2000,0,Female,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Open-ended questions (e.g. text entry),Definitely yes,,Control,20-24,Female,Bachelor's degree,Asian,Christian +56,Decoy Condition,Decoy Condition 1,Decoy: target 1st,No,No,Yes,1,20,20-24,2002,0,Female,1,2,No Religion,2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,20-24,Female,Bachelor's degree,Asian,No religion +57,Control Condition,Control Condition,Control,No,No,Yes,1,23,20-24,1999,0,Female,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Control,20-24,Female,Bachelor's degree,Asian,Christian +58,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,20,20-24,2002,1,Male,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Control,20-24,Male,Bachelor's degree,Asian,Christian +59,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,22,20-24,2000,1,Male,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Decoy,20-24,Male,Bachelor's degree,Asian,Christian +60,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,21,20-24,2001,1,Male,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Control,20-24,Male,Bachelor's degree,Asian,Christian +61,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,21,20-24,2001,0,Female,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Control,20-24,Female,Bachelor's degree,Asian,Christian +62,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,26,25-29,1996,1,Male,6,1,Hindu,2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Control,25-29,Male,Bachelor's degree,Black,Ohter religion +63,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,23,20-24,1999,0,Female,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Decoy,20-24,Female,Bachelor's degree,Asian,Christian +64,Control Condition,Control Condition,Control,No,No,Yes,1,24,20-24,1998,0,Female,4,2,No Religion,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Control,20-24,Female,Graduate or professional degree,Other,No religion +65,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,32,30-35,1990,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Probably yes,,Control,2.0,Male,Graduate or professional degree,White,Christian +66,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,24,20-24,1998,1,Male,5,2,No Religion,2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,20-24,Male,Bachelor's degree,White,No religion +67,Decoy Condition,Decoy Condition 1,Decoy: target 1st,No,No,Yes,1,22,20-24,2000,1,Male,5,2,No Religion,2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Probably yes,,Decoy,20-24,Male,Bachelor's degree,White,No religion +68,Control Condition,Control Condition,Control,No,No,Yes,1,27,25-29,1995,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Control,25-29,Male,Bachelor's degree,White,Christian +69,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,29,25-29,1993,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Decoy,25-29,Male,Bachelor's degree,White,Christian +70,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,26,25-29,1996,1,Male,6,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Decoy,25-29,Male,Bachelor's degree,Black,Christian +71,Decoy Condition,Decoy Condition 1,Decoy: target 1st,No,No,Yes,1,25,25-29,1997,0,Female,6,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Decoy,25-29,Female,Bachelor's degree,Black,Christian +72,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,24,20-24,1998,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Control,20-24,Female,Bachelor's degree,White,Christian +73,Control Condition,Control Condition,Control,No,No,Yes,1,25,25-29,1997,0,Female,5,2,No Religion,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Control,25-29,Female,Graduate or professional degree,White,No religion +74,Control Condition,Control Condition,Control,No,No,Yes,1,26,25-29,1996,1,Male,6,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Control,25-29,Male,Bachelor's degree,Black,Christian +75,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,29,25-29,1993,1,Male,6,2,No Religion,2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Control,25-29,Male,Bachelor's degree,Black,No religion +76,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,26,25-29,1996,0,Female,1,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Control,25-29,Female,Graduate or professional degree,Asian,Christian +77,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,24,20-24,1998,0,Female,5,2,No Religion,2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Decoy,20-24,Female,Bachelor's degree,White,No religion +78,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,23,20-24,1999,0,Female,3,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Decoy,20-24,Female,Bachelor's degree,Other,Christian +79,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,27,25-29,1995,0,Female,6,2,No Religion,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Decoy,25-29,Female,Graduate or professional degree,Black,No religion +80,Control Condition,Control Condition,Control,No,No,Yes,1,27,25-29,1995,1,Male,3,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Control,25-29,Male,Bachelor's degree,Other,Christian +81,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,29,25-29,1993,1,Male,5,2,No Religion,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Decoy,25-29,Male,Graduate or professional degree,White,No religion +82,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,27,25-29,1995,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Decoy,25-29,Male,Graduate or professional degree,White,Christian +83,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,23,20-24,1999,0,Female,6,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Decoy,20-24,Female,Bachelor's degree,Black,Christian +84,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,27,25-29,1995,1,Male,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Decoy,25-29,Male,Bachelor's degree,Asian,Christian +85,Decoy Condition,Decoy Condition 1,Decoy: target 1st,Yes,Yes,Yes,1,28,25-29,1994,0,Female,1,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Female,Graduate or professional degree,Asian,Christian +86,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,No,0,31,30-35,1991,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Decoy,2.0,Male,Graduate or professional degree,White,Christian +87,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,Yes,Yes,Yes,1,26,25-29,1996,0,Female,6,2,No Religion,2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Female,Bachelor's degree,Black,No religion +88,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,Yes,Yes,Yes,1,27,25-29,1995,1,Male,5,2,No Religion,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Decoy,25-29,Male,Graduate or professional degree,White,No religion +89,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,27,25-29,1995,0,Female,1,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Control,25-29,Female,Graduate or professional degree,Asian,Christian +90,Control Condition,Control Condition,Control,No,No,No,0,23,20-24,1999,0,Female,5,2,No Religion,2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Control,20-24,Female,Bachelor's degree,White,No religion +91,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,Yes,Yes,Yes,1,30,30-35,1992,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,2.0,Female,Graduate or professional degree,White,Christian +92,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,26,25-29,1996,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Control,25-29,Female,Bachelor's degree,White,Christian +93,Control Condition,Control Condition,Control,No,No,Yes,1,25,25-29,1997,1,Male,5,2,No Religion,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Control,25-29,Male,Graduate or professional degree,White,No religion +94,Control Condition,Control Condition,Control,No,No,Yes,1,22,20-24,2000,0,Female,5,2,No Religion,2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Control,20-24,Female,Bachelor's degree,White,No religion +95,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,No,0,29,25-29,1993,0,Female,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Decoy,25-29,Female,Bachelor's degree,Asian,Christian +96,Control Condition,Control Condition,Control,No,No,No,0,26,25-29,1996,1,Male,6,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Control,25-29,Male,Bachelor's degree,Black,Christian +97,Control Condition,Control Condition,Control,No,No,Yes,1,24,20-24,1998,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Control,20-24,Male,Bachelor's degree,White,Christian +98,Control Condition,Control Condition,Control,No,No,Yes,1,30,30-35,1992,1,Male,6,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Control,2.0,Male,Graduate or professional degree,Black,Christian +99,Control Condition,Control Condition,Control,No,No,Yes,1,27,25-29,1995,0,Female,6,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Control,25-29,Female,Graduate or professional degree,Black,Christian +100,Control Condition,Control Condition,Control,No,No,No,0,26,25-29,1996,0,Female,6,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Control,25-29,Female,Bachelor's degree,Black,Christian +101,Control Condition,Control Condition,Control,No,No,Yes,1,27,25-29,1995,1,Male,6,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Control,25-29,Male,Graduate or professional degree,Black,Christian +102,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,Yes,Yes,Yes,1,26,25-29,1996,1,Male,1,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Decoy,25-29,Male,Graduate or professional degree,Asian,Christian +103,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,Yes,Yes,Yes,1,29,25-29,1993,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Decoy,25-29,Female,Graduate or professional degree,White,Christian +104,Control Condition,Control Condition,Control,No,No,No,0,28,25-29,1994,1,Male,5,2,No Religion,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Control,25-29,Male,Graduate or professional degree,White,No religion +105,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,Yes,Yes,Yes,1,25,25-29,1997,0,Female,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Decoy,25-29,Female,Bachelor's degree,Asian,Christian +106,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,25,25-29,1997,0,Female,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Control,25-29,Female,Bachelor's degree,Asian,Christian +107,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,27,25-29,1995,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Male,Bachelor's degree,White,Christian +108,Control Condition,Control Condition,Control,No,No,Yes,1,25,25-29,1997,1,Male,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Control,25-29,Male,Bachelor's degree,Asian,Christian +109,Control Condition,Control Condition,Control,No,No,Yes,1,26,25-29,1996,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Control,25-29,Female,Bachelor's degree,White,Christian +110,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,28,25-29,1994,1,Male,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Male,Bachelor's degree,Asian,Christian +111,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,22,20-24,2000,1,Male,1,5,Jewish,2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Probably yes,,Control,20-24,Male,Bachelor's degree,Asian,Ohter religion +112,Control Condition,Control Condition,Control,No,No,Yes,1,32,30-35,1990,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Control,2.0,Male,Graduate or professional degree,White,Christian +113,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,Yes,Yes,Yes,1,32,30-35,1990,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Decoy,2.0,Male,Bachelor's degree,White,Christian +114,Control Condition,Control Condition,Control,No,No,Yes,1,28,25-29,1994,1,Male,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Control,25-29,Male,Bachelor's degree,Asian,Christian +115,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,Yes,Yes,Yes,1,32,30-35,1990,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,2.0,Male,Graduate or professional degree,White,Christian +116,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,Yes,Yes,Yes,1,31,30-35,1991,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,2.0,Male,Bachelor's degree,White,Christian +117,Control Condition,Control Condition,Control,No,No,Yes,1,32,30-35,1990,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Control,2.0,Male,Graduate or professional degree,White,Christian +118,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,Yes,Yes,Yes,1,31,30-35,1991,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,2.0,Male,Graduate or professional degree,White,Christian +119,Control Condition,Control Condition,Control,No,No,Yes,1,32,30-35,1990,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Control,2.0,Male,Graduate or professional degree,White,Christian +120,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,30,30-35,1992,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Control,2.0,Female,Bachelor's degree,White,Christian +121,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,Yes,Yes,Yes,1,31,30-35,1991,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,2.0,Male,Graduate or professional degree,White,Christian +122,Control Condition,Control Condition,Control,No,No,Yes,1,31,30-35,1991,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Control,2.0,Male,Graduate or professional degree,White,Christian +123,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,Yes,No,Yes,1,32,30-35,1990,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,2.0,Male,Graduate or professional degree,White,Christian +124,Control Condition,Control Condition,Control,No,No,Yes,1,31,30-35,1991,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Control,2.0,Male,Graduate or professional degree,White,Christian +125,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,30,30-35,1992,1,Male,1,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Probably yes,,Control,2.0,Male,Graduate or professional degree,Asian,Christian +126,Control Condition,Control Condition,Control,No,No,Yes,1,26,25-29,1996,1,Male,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Control,25-29,Male,Bachelor's degree,Asian,Christian +127,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,Yes,No,Yes,1,23,20-24,1999,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,20-24,Male,Bachelor's degree,White,Christian +128,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,32,30-35,1990,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,2.0,Male,Bachelor's degree,White,Christian +129,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,No,0,25,25-29,1997,1,Male,1,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Male,Graduate or professional degree,Asian,Christian +130,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,28,25-29,1994,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Open-ended questions (e.g. text entry),Definitely yes,,Control,25-29,Male,Bachelor's degree,White,Christian +131,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,Yes,Yes,Yes,1,32,30-35,1990,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,2.0,Male,Graduate or professional degree,White,Christian +132,Control Condition,Control Condition,Control,No,No,Yes,1,32,30-35,1990,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Control,2.0,Male,Graduate or professional degree,White,Christian +133,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,Yes,Yes,Yes,1,31,30-35,1991,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,2.0,Male,Graduate or professional degree,White,Christian +134,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,Yes,Yes,Yes,1,32,30-35,1990,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,2.0,Male,Graduate or professional degree,White,Christian +135,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,31,30-35,1991,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Control,2.0,Male,Bachelor's degree,White,Christian +136,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,32,30-35,1990,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Decoy,2.0,Female,Graduate or professional degree,White,Christian +137,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,30,30-35,1992,0,Female,1,6,Muslim,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Decoy,2.0,Female,Graduate or professional degree,Asian,Ohter religion +138,Control Condition,Control Condition,Control,No,No,Yes,1,32,30-35,1990,1,Male,1,6,Muslim,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Control,2.0,Male,Graduate or professional degree,Asian,Ohter religion +139,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,26,25-29,1996,0,Female,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Female,Bachelor's degree,Asian,Christian +140,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,24,20-24,1998,1,Male,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,20-24,Male,Bachelor's degree,Asian,Christian +141,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,27,25-29,1995,0,Female,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Female,Bachelor's degree,Asian,Christian +142,Control Condition,Control Condition,Control,No,No,Yes,1,29,25-29,1993,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Control,25-29,Female,Graduate or professional degree,White,Christian +143,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,31,30-35,1991,1,Male,6,6,Muslim,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,2.0,Male,Graduate or professional degree,Black,Ohter religion +144,Control Condition,Control Condition,Control,No,No,Yes,1,28,25-29,1994,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Control,25-29,Female,Graduate or professional degree,White,Christian +145,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,27,25-29,1995,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Female,Bachelor's degree,White,Christian +146,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,25,25-29,1997,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Probably yes,,Decoy,25-29,Female,Bachelor's degree,White,Christian +147,Control Condition,Control Condition,Control,No,No,Yes,1,25,25-29,1997,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Control,25-29,Male,Graduate or professional degree,White,Christian +148,Control Condition,Control Condition,Control,No,No,Yes,1,28,25-29,1994,1,Male,5,2,No Religion,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Control,25-29,Male,Graduate or professional degree,White,No religion +149,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,26,25-29,1996,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Probably no,,Decoy,25-29,Female,Bachelor's degree,White,Christian +150,Control Condition,Control Condition,Control,No,No,Yes,1,24,20-24,1998,0,Female,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Control,20-24,Female,Bachelor's degree,Asian,Christian +151,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,28,25-29,1994,0,Female,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Female,Bachelor's degree,Asian,Christian +152,Control Condition,Control Condition,Control,No,No,Yes,1,23,20-24,1999,0,Female,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Control,20-24,Female,Bachelor's degree,Asian,Christian +153,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,30,30-35,1992,0,Female,1,2,No Religion,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,2.0,Female,Graduate or professional degree,Asian,No religion +154,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,24,20-24,1998,1,Male,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Probably yes,,Decoy,20-24,Male,Bachelor's degree,Asian,Christian +155,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,27,25-29,1995,1,Male,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Control,25-29,Male,Bachelor's degree,Asian,Christian +156,Control Condition,Control Condition,Control,No,No,Yes,1,30,30-35,1992,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Control,2.0,Male,Graduate or professional degree,White,Christian +157,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,28,25-29,1994,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Male,Graduate or professional degree,White,Christian +158,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,24,20-24,1998,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Probably yes,,Decoy,20-24,Male,Bachelor's degree,White,Christian +159,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,26,25-29,1996,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Probably yes,,Decoy,25-29,Male,Bachelor's degree,White,Christian +160,Control Condition,Control Condition,Control,No,No,Yes,1,32,30-35,1990,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Control,2.0,Female,Graduate or professional degree,White,Christian +161,Control Condition,Control Condition,Control,No,No,Yes,1,25,25-29,1997,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Open-ended questions (e.g. text entry),Definitely yes,,Control,25-29,Female,Bachelor's degree,White,Christian +162,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,27,25-29,1995,1,Male,1,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Male,Graduate or professional degree,Asian,Christian +163,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,30,30-35,1992,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Decoy,2.0,Male,Graduate or professional degree,White,Christian +164,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,28,25-29,1994,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Decoy,25-29,Male,Graduate or professional degree,White,Christian +165,Control Condition,Control Condition,Control,No,No,Yes,1,31,30-35,1991,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Control,2.0,Male,Graduate or professional degree,White,Christian +166,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,24,20-24,1998,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Open-ended questions (e.g. text entry),Definitely yes,,Decoy,20-24,Male,Bachelor's degree,White,Christian +167,Control Condition,Control Condition,Control,No,No,Yes,1,25,25-29,1997,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Control,25-29,Female,Bachelor's degree,White,Christian +168,Control Condition,Control Condition,Control,No,No,Yes,1,27,25-29,1995,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Control,25-29,Male,Graduate or professional degree,White,Christian +169,Control Condition,Control Condition,Control,No,No,Yes,1,30,30-35,1992,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Probably yes,,Control,2.0,Male,Graduate or professional degree,White,Christian +170,Control Condition,Control Condition,Control,No,No,Yes,1,29,25-29,1993,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Open-ended questions (e.g. text entry),Definitely yes,,Control,25-29,Male,Graduate or professional degree,White,Christian +171,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,27,25-29,1995,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Male,Graduate or professional degree,White,Christian +172,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,24,20-24,1998,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,20-24,Female,Bachelor's degree,White,Christian +173,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,28,25-29,1994,1,Male,1,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Male,Graduate or professional degree,Asian,Christian +174,Control Condition,Control Condition,Control,No,No,Yes,1,23,20-24,1999,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Control,20-24,Male,Bachelor's degree,White,Christian +175,Control Condition,Control Condition,Control,No,No,Yes,1,26,25-29,1996,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Control,25-29,Male,Graduate or professional degree,White,Christian +176,Control Condition,Control Condition,Control,No,No,Yes,1,25,25-29,1997,1,Male,5,2,No Religion,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Control,25-29,Male,Graduate or professional degree,White,No religion +177,Control Condition,Control Condition,Control,No,No,Yes,1,25,25-29,1997,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Probably yes,,Control,25-29,Male,Graduate or professional degree,White,Christian +178,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,24,20-24,1998,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,20-24,Male,Bachelor's degree,White,Christian +179,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,30,30-35,1992,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,2.0,Female,Bachelor's degree,White,Christian +180,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,25,25-29,1997,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Decoy,25-29,Female,Bachelor's degree,White,Christian +181,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,29,25-29,1993,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Male,Graduate or professional degree,White,Christian +182,Control Condition,Control Condition,Control,No,No,Yes,1,27,25-29,1995,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Control,25-29,Female,Graduate or professional degree,White,Christian +183,Control Condition,Control Condition,Control,No,No,Yes,1,28,25-29,1994,1,Male,1,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Probably yes,,Control,25-29,Male,Graduate or professional degree,Asian,Christian +184,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,30,30-35,1992,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,2.0,Female,Graduate or professional degree,White,Christian +185,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,27,25-29,1995,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Male,Graduate or professional degree,White,Christian +186,Control Condition,Control Condition,Control,No,No,Yes,1,31,30-35,1991,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Control,2.0,Female,Graduate or professional degree,White,Christian +187,Control Condition,Control Condition,Control,No,No,Yes,1,26,25-29,1996,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Control,25-29,Female,Bachelor's degree,White,Christian +188,Control Condition,Control Condition,Control,No,No,Yes,1,30,30-35,1992,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Control,2.0,Male,Bachelor's degree,White,Christian +189,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,25,25-29,1997,1,Male,1,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Male,Bachelor's degree,Asian,Christian +190,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,27,25-29,1995,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,25-29,Female,Bachelor's degree,White,Christian +191,Control Condition,Control Condition,Control,No,No,Yes,1,32,30-35,1990,0,Female,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Open-ended questions (e.g. text entry),Definitely yes,,Control,2.0,Female,Bachelor's degree,White,Christian +192,Decoy Condition,Decoy Condition 2,Decoy; decoy 1st,No,No,Yes,1,24,20-24,1998,0,Female,6,2,No Religion,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Decoy,20-24,Female,Graduate or professional degree,Black,No religion +193,Control Condition,Control Condition,Control,No,No,Yes,1,23,20-24,1999,0,Female,6,2,No Religion,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Probably yes,,Control,20-24,Female,Graduate or professional degree,Black,No religion +194,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,24,20-24,1998,0,Female,6,2,No Religion,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Probably yes,,Control,20-24,Female,Graduate or professional degree,Black,No religion +195,Control Condition,Control Condition,Control,Yes,Yes,Yes,1,24,20-24,1998,0,Female,5,2,No Religion,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Probably yes,,Control,20-24,Female,Graduate or professional degree,White,No religion +196,Control Condition,Control Condition,Control,No,No,Yes,1,29,25-29,1993,0,Female,6,4,Buddhist,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Probably yes,,Control,25-29,Female,Graduate or professional degree,Black,Ohter religion +197,Control Condition,Control Condition,Control,No,No,Yes,1,28,25-29,1994,0,Female,6,2,No Religion,2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Probably yes,,Control,25-29,Female,Bachelor's degree,Black,No religion +198,Control Condition,Control Condition,Control,No,No,Yes,1,24,20-24,1998,0,Female,6,2,No Religion,2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Probably no,,Control,20-24,Female,Bachelor's degree,Black,No religion +199,Control Condition,Control Condition,Control,No,No,Yes,1,25,25-29,1997,0,Female,6,2,No Religion,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",,,,Control,25-29,Female,Graduate or professional degree,Black,No religion +200,Control Condition,Control Condition,Control,No,No,Yes,1,24,20-24,1998,0,Female,6,2,No Religion,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Control,20-24,Female,Graduate or professional degree,Black,No religion +201,Control Condition,Control Condition,Control,No,No,Yes,1,20,20-24,2002,0,Female,6,2,No Religion,2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Probably yes,,Control,20-24,Female,Bachelor's degree,Black,No religion +203,Control Condition,Control Condition,Control,No,No,Yes,1,20,20-24,2002,0,Female,6,2,No Religion,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Probably yes,,Control,20-24,Female,Graduate or professional degree,Black,No religion +204,Control Condition,Control Condition,Control,No,No,Yes,1,35,30-35,1987,0,Female,6,2,No Religion,2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Probably yes,,Control,2.0,Female,Bachelor's degree,Black,No religion +202,Control Condition,Control Condition,Control,No,No,Yes,1,23,20-24,1999,0,Female,6,2,No Religion,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Probably yes,,Control,20-24,Female,Graduate or professional degree,Black,No religion +205,Control Condition,Control Condition,Control,No,No,Yes,1,22,20-24,2000,0,Female,6,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Control,20-24,Female,Bachelor's degree,Black,Christian +206,Control Condition,Control Condition,Control,No,No,Yes,1,21,20-24,2001,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,Close-ended questions (e.g. multiple choice),Definitely yes,,Control,20-24,Male,Bachelor's degree,White,Christian +207,Control Condition,Control Condition,Control,No,No,Yes,1,24,20-24,1998,1,Male,5,3,Christian (Catholic protestant or any other Christian denominations),2,University - Bachelors Degree,There is no difference between the two options for me,Definitely yes,,Control,20-24,Male,Bachelor's degree,White,Christian +208,Control Condition,Control Condition,Control,No,No,Yes,1,25,25-29,1997,0,Female,5,5,Jewish,1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Open-ended questions (e.g. text entry),Definitely yes,,Control,25-29,Female,Graduate or professional degree,White,Ohter religion +209,Control Condition,Control Condition,Control,No,No,Yes,1,26,25-29,1996,0,Female,1,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",There is no difference between the two options for me,Definitely yes,,Control,25-29,Female,Graduate or professional degree,Asian,Christian +210,Control Condition,Control Condition,Control,No,No,Yes,1,22,20-24,2000,1,Male,2,3,Christian (Catholic protestant or any other Christian denominations),1,"Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)",Close-ended questions (e.g. multiple choice),Definitely yes,,Control,20-24,Male,Graduate or professional degree,Other,Christian diff --git a/studies/study_015/source/ground_truth.json b/studies/study_015/source/ground_truth.json new file mode 100644 index 0000000..7e5ace0 --- /dev/null +++ b/studies/study_015/source/ground_truth.json @@ -0,0 +1,453 @@ +{ + "study_id": "stoffel_2023_decoy_effect_survey_participation", + "title": "Testing the decoy effect to improve online survey participation: Evidence from a field experiment", + "authors": [ + "Sandro T. Stoffel", + "Yining Sun", + "Yasemin Hirst", + "Christian von Wagner", + "Ivo Vlaev" + ], + "year": 2023, + "preliminary_study_findings": [ + { + "finding_id": "F1_prelim", + "description": "Respondents generally preferred close-ended questions over open-ended questions.", + "hypothesis": "Open-ended questions would be perceived as less attractive than close-ended questions and therefore could function as an inferior decoy attribute.", + "statistical_tests": [], + "raw_outcomes": "Among N=210 preliminary questionnaire respondents: close-ended questions 61.4% (n~129), open-ended questions 7.9% (n~17), indifferent 30.7% (n~64). Exact counts NOT PROVIDED in paper." + }, + { + "finding_id": "F2_prelim", + "description": "Most respondents indicated they would not participate if remuneration were received only after four weeks.", + "hypothesis": "Delayed remuneration would be perceived as less attractive than sooner remuneration and therefore could function as an inferior decoy attribute.", + "statistical_tests": [], + "raw_outcomes": "Among N=210 preliminary questionnaire respondents: Definitely not 1.0% (n~2), Probably not 85.1% (n~179), Probably yes 1.5% (n~3), Definitely yes 12.4% (n~26). Overall 86.1% would not respond if remuneration in four weeks." + } + ], + "main_study": { + "study_id": "main_field_experiment", + "study_name": "Field experiment on the decoy effect in survey participation", + "period": "One week after preliminary questionnaire (August 2022)", + "n_randomized": 210, + "n_with_valid_email": 203, + "findings": [ + { + "finding_id": "F1", + "description": "Randomization produced similar groups on most sociodemographic variables, except ethnicity.", + "hypothesis": "No explicit hypothesis; balance checks were used to compare control and decoy groups.", + "statistical_tests": [ + { + "test_name": "Chi-square test of independence", + "test_id": "balance_age", + "variable": "Age", + "statistic": "χ² (reported as p-value)", + "p_value": "p = 0.165", + "raw_data": { + "control_n": 101, + "control_age_20_24": 33, + "control_age_25_29": 45, + "control_age_30_35": 23, + "decoy_n": 102, + "decoy_age_20_24": 22, + "decoy_age_25_29": 57, + "decoy_age_30_35": 23 + }, + "claim": "No age difference between control and decoy groups.", + "location": "Main paper Table 1" + }, + { + "test_name": "Chi-square test of independence", + "test_id": "balance_gender", + "variable": "Gender", + "statistic": "χ² (reported as p-value)", + "p_value": "p = 0.441", + "raw_data": { + "control_male": 50, + "control_female": 51, + "decoy_male": 56, + "decoy_female": 46 + }, + "claim": "No gender difference between control and decoy groups.", + "location": "Main paper Table 1" + }, + { + "test_name": "Fisher's exact test", + "test_id": "balance_ethnicity", + "variable": "Ethnicity", + "statistic": "Fisher exact p-value", + "p_value": "p = 0.004", + "raw_data": { + "control_white": 48, + "control_asian": 25, + "control_black": 25, + "control_other": 3, + "decoy_white": 69, + "decoy_asian": 23, + "decoy_black": 9, + "decoy_other": 1 + }, + "claim": "Ethnicity differed between conditions; the decoy condition contained more White participants (67.7% vs 47.5%).", + "location": "Main paper Table 1" + }, + { + "test_name": "Chi-square test of independence", + "test_id": "balance_religion", + "variable": "Religion", + "statistic": "χ² (reported as p-value)", + "p_value": "p = 0.063", + "raw_data": { + "control_christian": 67, + "control_other_religion": 8, + "control_no_religion": 26, + "decoy_christian": 79, + "decoy_other_religion": 10, + "decoy_no_religion": 13 + }, + "claim": "No religion difference between control and decoy groups at conventional significance levels.", + "location": "Main paper Table 1" + }, + { + "test_name": "Chi-square test of independence", + "test_id": "balance_education", + "variable": "Education", + "statistic": "χ² (reported as p-value)", + "p_value": "p = 0.233", + "raw_data": { + "control_bachelors": 48, + "control_graduate": 53, + "decoy_bachelors": 57, + "decoy_graduate": 45 + }, + "claim": "No education difference between control and decoy groups.", + "location": "Main paper Table 1" + } + ] + }, + { + "finding_id": "F2", + "description": "The presence of the decoy option increased completion of the target survey.", + "hypothesis": "Among those likely to take part in a survey, the presence of a decoy version of the same survey on the invitation will increase the chances of completing the target version.", + "statistical_tests": [ + { + "test_name": "Chi-square test of independence", + "test_id": "primary_outcome_unadjusted", + "variable": "Target survey completion", + "statistic": "χ²(1, N = 203) = 11.08", + "p_value": "p < 0.001", + "raw_data": { + "total_n": 203, + "control_completed": 33, + "control_n": 101, + "control_completion_rate": 0.327, + "decoy_completed": 57, + "decoy_n": 102, + "decoy_completion_rate": 0.559, + "note": "Table 2 / Table S3 reports decoy as 59.8%, but text, flow diagram, and participant-level data imply 57/102 = 55.9%" + }, + "claim": "Target-survey completion was significantly higher in the decoy condition (55.9%) than in the control condition (32.7%).", + "location": "Main paper Results section 3.2.2 and Fig. 3" + }, + { + "test_name": "Binary logistic regression", + "test_id": "primary_unadjusted_OR", + "model": "Model 1 - unadjusted", + "variable": "Decoy condition (vs control)", + "statistic": "OR = 2.610", + "confidence_interval": "95% CI 1.475-4.618", + "p_value": "p < 0.01", + "raw_data": { + "overall_completion": 0.443, + "control_reference": 0.327, + "decoy_row_reported": 0.598, + "note": "As reported in Table 2, Model 1; Supplementary Table S3" + }, + "claim": "In the unadjusted model, the decoy condition significantly increased the odds of completing the target survey relative to control.", + "location": "Main paper Table 2, Model 1; Supplementary Table S3" + }, + { + "test_name": "Binary logistic regression", + "test_id": "primary_adjusted_aOR", + "model": "Model 1 - adjusted", + "variable": "Decoy condition (vs control)", + "statistic": "aOR = 2.584", + "confidence_interval": "95% CI 1.415-4.718", + "p_value": "p < 0.01", + "covariates": ["Age", "Gender", "Ethnicity", "Religion", "Education"], + "claim": "The decoy effect on target-survey completion remained statistically significant after adjustment for sociodemographic variables.", + "location": "Main paper Table 2, Model 1; Supplementary Table S3" + } + ] + }, + { + "finding_id": "F3", + "description": "A strong order effect was observed within the decoy condition: the decoy increased participation only when the target survey was presented first.", + "hypothesis": "Presentation order may influence the decoy effect because responses can favor options appearing earlier in a list.", + "statistical_tests": [ + { + "test_name": "Binary logistic regression", + "test_id": "order_target_first_unadjusted", + "model": "Model 2 - unadjusted, target first vs control", + "variable": "Decoy-target-first condition (vs control)", + "statistic": "OR = 9.845", + "confidence_interval": "95% CI 4.293-22.580", + "p_value": "p < 0.01", + "raw_data": { + "control_completed": 33, + "control_n": 101, + "control_completion_rate": 0.327, + "decoy_target_first_completed": 43, + "decoy_target_first_n": 52, + "decoy_target_first_completion_rate": 0.827 + }, + "claim": "Showing the target first greatly increased target-survey completion (82.7%) relative to control (32.7%).", + "location": "Main paper Table 2, Model 2; Supplementary Table S4" + }, + { + "test_name": "Binary logistic regression", + "test_id": "order_target_first_adjusted", + "model": "Model 2 - adjusted, target first vs control", + "variable": "Decoy-target-first condition (vs control)", + "statistic": "aOR = 11.177", + "confidence_interval": "95% CI 4.571-27.330", + "p_value": "p < 0.01", + "covariates": ["Age", "Gender", "Ethnicity", "Religion", "Education"], + "claim": "The target-first order effect remained large and significant after adjustment for sociodemographics.", + "location": "Main paper Table 2, Model 2; Supplementary Table S4" + }, + { + "test_name": "Binary logistic regression", + "test_id": "order_decoy_first_unadjusted", + "model": "Model 2 - unadjusted, decoy first vs control", + "variable": "Decoy-decoy-first condition (vs control)", + "statistic": "OR = 0.801", + "confidence_interval": "95% CI 0.381-1.687", + "p_value": "NOT PROVIDED", + "raw_data": { + "control_completed": 33, + "control_n": 101, + "control_completion_rate": 0.327, + "decoy_decoy_first_completed": 14, + "decoy_decoy_first_n": 50, + "decoy_decoy_first_completion_rate": 0.280 + }, + "claim": "There was no evidence that showing the decoy first increased target-survey completion (28.0%) relative to control (32.7%).", + "location": "Main paper Table 2, Model 2; Supplementary Table S4" + }, + { + "test_name": "Binary logistic regression", + "test_id": "order_decoy_first_adjusted", + "model": "Model 2 - adjusted, decoy first vs control", + "variable": "Decoy-decoy-first condition (vs control)", + "statistic": "aOR = 0.746", + "confidence_interval": "95% CI 0.341-1.631", + "p_value": "NOT PROVIDED", + "covariates": ["Age", "Gender", "Ethnicity", "Religion", "Education"], + "claim": "There was no decoy effect when the decoy was shown before the target, even after adjustment.", + "location": "Main paper Results section 3.2.2 and Table 2; Supplementary Table S4" + } + ] + }, + { + "finding_id": "F4", + "description": "Most target-survey completers in the decoy condition reported that the decoy had at least somewhat influenced their decision to participate.", + "hypothesis": "The decoy option should be perceived as influencing participation decisions.", + "statistical_tests": [], + "raw_outcomes": "Among the 57 individuals in the decoy condition who completed the target survey, 33/57 (57.9%) at least somewhat agreed that the decoy survey had influenced their decision to participate." + }, + { + "finding_id": "F5", + "description": "There was little evidence for adverse question-response behavior or non-response bias, except for ethnicity differences.", + "hypothesis": "The decoy should increase participation without negatively influencing question response behavior or inducing substantial non-response bias.", + "statistical_tests": [ + { + "test_name": "Fisher's exact test", + "test_id": "completers_ethnicity", + "variable": "Ethnicity among target completers", + "statistic": "Fisher exact p-value", + "p_value": "p = 0.006", + "raw_data": { + "control_completers": 33, + "control_white": 12, + "control_asian": 14, + "control_black": 7, + "control_other": 0, + "decoy_completers": 57, + "decoy_white": 40, + "decoy_asian": 10, + "decoy_black": 6, + "decoy_other": 1 + }, + "claim": "Among target-survey completers, ethnicity differed between conditions: more White participants in the decoy condition (70.2% vs 36.4%).", + "location": "Main paper section 3.2.3; Supplementary Table S5" + }, + { + "test_name": "Chi-square test", + "test_id": "completers_age", + "variable": "Age among target completers", + "statistic": "χ²", + "p_value": "p = 0.122", + "raw_data": { + "control_completers": 33, + "control_age_20_24": 13, + "control_age_25_29": 13, + "control_age_30_35": 7, + "decoy_completers": 57, + "decoy_age_20_24": 12, + "decoy_age_25_29": 34, + "decoy_age_30_35": 11 + }, + "claim": "No age difference among target completers across conditions.", + "location": "Supplementary Table S5" + }, + { + "test_name": "Chi-square test", + "test_id": "completers_gender", + "variable": "Gender among target completers", + "statistic": "χ²", + "p_value": "p = 0.589", + "raw_data": { + "control_male": 16, + "control_female": 17, + "decoy_male": 31, + "decoy_female": 26 + }, + "claim": "No gender difference among target completers across conditions.", + "location": "Supplementary Table S5" + }, + { + "test_name": "Chi-square test", + "test_id": "completers_religion", + "variable": "Religion among target completers", + "statistic": "χ²", + "p_value": "p = 0.329", + "raw_data": { + "control_christian": 20, + "control_non_christian": 5, + "control_no_religion": 8, + "decoy_christian": 43, + "decoy_non_christian": 5, + "decoy_no_religion": 9 + }, + "claim": "No religion difference among target completers across conditions.", + "location": "Supplementary Table S5" + }, + { + "test_name": "Chi-square test", + "test_id": "completers_education", + "variable": "Education among target completers", + "statistic": "χ²", + "p_value": "p = 0.310", + "raw_data": { + "control_bachelors": 21, + "control_graduate": 12, + "decoy_bachelors": 30, + "decoy_graduate": 27 + }, + "claim": "No education difference among target completers across conditions.", + "location": "Supplementary Table S5" + }, + { + "test_name": "Wilcoxon-Mann-Whitney test", + "test_id": "fcq_score_distribution", + "variable": "FCQ total score (8-40) among target completers", + "statistic": "z = 0.488", + "p_value": "p = 0.629", + "raw_data": { + "control_completers": 33, + "decoy_completers": 57, + "note": "Exact group means and SDs NOT PROVIDED" + }, + "claim": "There was no statistically significant difference in FCQ score distributions across the two experimental conditions.", + "location": "Main paper section 3.2.3; Supplementary Fig. S4" + }, + { + "test_name": "Chi-square test", + "test_id": "nonresponse_bias_age", + "variable": "Age comparing completers vs non-completers", + "statistic": "χ²", + "p_value": "p = 0.719", + "raw_data": { + "completers_20_24": 25, + "completers_25_29": 47, + "completers_30_35": 18, + "total_completers": 90, + "non_completers_20_24": 30, + "non_completers_25_29": 55, + "non_completers_30_35": 28, + "total_non_completers": 113 + }, + "claim": "No age difference between completers and non-completers.", + "location": "Supplementary Table S6" + }, + { + "test_name": "Chi-square test", + "test_id": "nonresponse_bias_gender", + "variable": "Gender comparing completers vs non-completers", + "statistic": "χ²", + "p_value": "p = 0.999", + "raw_data": { + "completers_male": 47, + "completers_female": 43, + "non_completers_male": 59, + "non_completers_female": 54 + }, + "claim": "No gender difference between completers and non-completers.", + "location": "Supplementary Table S6" + }, + { + "test_name": "Fisher's exact test", + "test_id": "nonresponse_bias_ethnicity", + "variable": "Ethnicity comparing completers vs non-completers", + "statistic": "Fisher exact p-value", + "p_value": "p = 0.661", + "raw_data": { + "completers_white": 52, + "completers_asian": 24, + "completers_black": 13, + "completers_other": 1, + "non_completers_white": 65, + "non_completers_asian": 24, + "non_completers_black": 21, + "non_completers_other": 3 + }, + "claim": "No overall ethnicity difference between completers and non-completers.", + "location": "Supplementary Table S6" + }, + { + "test_name": "Chi-square test", + "test_id": "nonresponse_bias_religion", + "variable": "Religion comparing completers vs non-completers", + "statistic": "χ²", + "p_value": "p = 0.603", + "raw_data": { + "completers_christian": 63, + "completers_non_christian": 10, + "completers_no_religion": 17, + "non_completers_christian": 83, + "non_completers_non_christian": 8, + "non_completers_no_religion": 22 + }, + "claim": "No religion difference between completers and non-completers.", + "location": "Supplementary Table S6" + }, + { + "test_name": "Chi-square test", + "test_id": "nonresponse_bias_education", + "variable": "Education comparing completers vs non-completers", + "statistic": "χ²", + "p_value": "p = 0.209", + "raw_data": { + "completers_bachelors": 51, + "completers_graduate": 39, + "non_completers_bachelors": 54, + "non_completers_graduate": 59 + }, + "claim": "No education difference between completers and non-completers.", + "location": "Supplementary Table S6" + } + ] + } + ] + } +} diff --git a/studies/study_015/source/materials/main_experiment_control.json b/studies/study_015/source/materials/main_experiment_control.json new file mode 100644 index 0000000..6bd304f --- /dev/null +++ b/studies/study_015/source/materials/main_experiment_control.json @@ -0,0 +1,169 @@ +{ + "sub_study_id": "main_experiment_control", + "sub_study_name": "Main experiment - Control condition", + "design_notes": "Participants receive email invitation with only the target survey option (no decoy).", + "instructions": "Invitation email (control condition) sent one week after preliminary questionnaire. No reminder emails sent.", + "invitation_email": { + "subject": "Survey: Your Mental Health Status During the Pandemic", + "body": "Hi there,\n\nThank you for signing up for our study earlier! We hope you will participate in the survey about your mental health status during the pandemic. It takes 15 minutes. You will be asked to answer some demographic questions and the main questionnaire includes 8 closed-ended (Scale Multiple Choice Questions). For your time, we will email you two £1 UK Amazon e-vouchers.\n\nTake the Survey.\n\nPlease complete the survey by Monday, August 15th.\n\nAbout this survey: Your answers will remain anonymous and will be used by the researcher's graduation thesis. I hope you'll take 15 minutes to complete the survey and enjoy your gift!" + }, + "items": [ + { + "id": "demographics_birth_year", + "question": "What is your year of birth?", + "type": "text_entry", + "q_idx": "Q1" + }, + { + "id": "demographics_gender", + "question": "What gender do you identify as?", + "type": "multiple_choice", + "options": [ + "Female", + "Male", + "Other, please specify" + ], + "q_idx": "Q2" + }, + { + "id": "demographics_race", + "question": "Choose one or more races that you consider yourself to be", + "type": "multiple_select", + "options": [ + "White", + "Black or African American", + "American Indian or Alaska Native", + "Asian", + "Native Hawaiian or Pacific Islander", + "Other" + ], + "q_idx": "Q3" + }, + { + "id": "demographics_education", + "question": "What is your current education level?", + "type": "multiple_choice", + "options": [ + "Some University but no degree", + "University - Bachelor Degree", + "Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)", + "Prefer not to say" + ], + "q_idx": "Q4" + }, + { + "id": "fcq_prompt", + "question": "Please select the extent to which the following thoughts, feelings and behaviors apply to you:", + "instruction": "Scale: 1 = Strongly disagree, 5 = Strongly agree", + "type": "likert_scale", + "scale_min": 1, + "scale_max": 5 + }, + { + "id": "fcq_item_1", + "question": "I am very worried about the corona virus outbreak.", + "type": "likert_5", + "q_idx": "Q5a", + "scale": [1, 2, 3, 4, 5], + "scale_labels": ["Strongly disagree", "Disagree", "Neutral", "Agree", "Strongly agree"] + }, + { + "id": "fcq_item_2", + "question": "I am taking precautions to prevent infection (e.g., washing hands, avoiding contact with people, avoiding door handles).", + "type": "likert_5", + "q_idx": "Q5b", + "scale": [1, 2, 3, 4, 5], + "scale_labels": ["Strongly disagree", "Disagree", "Neutral", "Agree", "Strongly agree"] + }, + { + "id": "fcq_item_3", + "question": "I am constantly following all news updates regarding the virus.", + "type": "likert_5", + "q_idx": "Q5c", + "scale": [1, 2, 3, 4, 5], + "scale_labels": ["Strongly disagree", "Disagree", "Neutral", "Agree", "Strongly agree"] + }, + { + "id": "fcq_item_4", + "question": "I have stocked up on supplies to prepare for problems related to the coronavirus outbreak.", + "type": "likert_5", + "q_idx": "Q5d", + "scale": [1, 2, 3, 4, 5], + "scale_labels": ["Strongly disagree", "Disagree", "Neutral", "Agree", "Strongly agree"] + }, + { + "id": "fcq_item_5", + "question": "For my personal health I find the virus to be much more dangerous than the seasonal flu.", + "type": "likert_5", + "q_idx": "Q5e", + "scale": [1, 2, 3, 4, 5], + "scale_labels": ["Strongly disagree", "Disagree", "Neutral", "Agree", "Strongly agree"] + }, + { + "id": "fcq_item_6", + "question": "I feel that the health authorities are not doing enough to deal with the virus.", + "type": "likert_5", + "q_idx": "Q5f", + "scale": [1, 2, 3, 4, 5], + "scale_labels": ["Strongly disagree", "Disagree", "Neutral", "Agree", "Strongly agree"] + }, + { + "id": "fcq_item_7", + "question": "I am worried that friends or family will be infected.", + "type": "likert_5", + "q_idx": "Q5g", + "scale": [1, 2, 3, 4, 5], + "scale_labels": ["Strongly disagree", "Disagree", "Neutral", "Agree", "Strongly agree"] + }, + { + "id": "fcq_item_8", + "question": "I take more precautions compared to most people to not become infected.", + "type": "likert_5", + "q_idx": "Q5h", + "scale": [1, 2, 3, 4, 5], + "scale_labels": ["Strongly disagree", "Disagree", "Neutral", "Agree", "Strongly agree"] + }, + { + "id": "debrief_ranking_1", + "question": "Rank the reasons why you choose to respond to this survey", + "type": "ranking", + "options": [ + "I'm interested in this topic", + "I can receive an incentive from completing this survey", + "I can accept the length of the survey", + "I am assured of anonymity", + "I like the question type (closed-ended)" + ], + "q_idx": "Q6" + }, + { + "id": "debrief_ranking_2", + "question": "Rank the reasons why you choose to respond to an online survey", + "type": "ranking", + "options": [ + "Interest in the Topic", + "Rewards", + "Length of Survey", + "Value Privacy (e.g. anonymity)", + "The Question Types (closed-ended or open-ended)" + ], + "q_idx": "Q7" + } + ], + "fcq_scoring": { + "description": "Fear of Coronavirus Questionnaire total score", + "items_to_sum": ["Q5a", "Q5b", "Q5c", "Q5d", "Q5e", "Q5f", "Q5g", "Q5h"], + "score_range": [8, 40], + "note": "Total is sum of 8 Likert items each ranging 1-5" + }, + "condition_metadata": { + "condition": "Control", + "condition_id": "control", + "n_randomized": 101, + "n_with_valid_email": 101, + "n_completed_target": 33, + "completion_rate": 0.327, + "incentive": "£2 Amazon voucher (two £1 e-vouchers) received after 1 week upon completion", + "deadline": "Monday, August 15th" + } +} diff --git a/studies/study_015/source/materials/main_experiment_decoy_decoy_first.json b/studies/study_015/source/materials/main_experiment_decoy_decoy_first.json new file mode 100644 index 0000000..2ad182f --- /dev/null +++ b/studies/study_015/source/materials/main_experiment_decoy_decoy_first.json @@ -0,0 +1,324 @@ +{ + "sub_study_id": "main_experiment_decoy_decoy_first", + "sub_study_name": "Main experiment - Decoy condition with decoy shown first", + "design_notes": "Participants receive email invitation presenting both target and decoy survey options in a comparison table with the decoy survey shown first.", + "instructions": "Invitation email (decoy condition, decoy first) sent one week after preliminary questionnaire. No reminder emails sent.", + "invitation_email": { + "subject": "Survey: Your Mental Health Status During the Pandemic", + "body": "Hi there,\n\nThank you for signing up for our study earlier! We hope you will participate in the survey about your mental health status during the pandemic. The survey may take you 15 minutes and there are two types of questionnaires you can choose." + }, + "comparison_table": { + "presentation_order": "decoy_first", + "note": "Decoy survey options presented first, target survey options presented second", + "columns": [ + { + "title": "Open-ended questions", + "question_type": "Two open-ended questions (answer each question with no less than 100 words)", + "reward": "£2 UK Amazon e-vouchers", + "reward_timing": "4 weeks after you finish the survey (as we need to check your answer)", + "survey_type": "decoy" + }, + { + "title": "Closed-ended questions", + "question_type": "Eight scale multiple choice questions", + "reward": "£2 UK Amazon e-vouchers", + "reward_timing": "1 week after you finish the survey", + "survey_type": "target" + } + ], + "footer": "Please complete one questionnaire by 8 PM (UK Time), Tuesday, August 16th. About this survey: Your answers will remain anonymous and will be used in the researcher's graduation thesis. I hope you'll take 15 minutes to complete the survey and enjoy your gift!" + }, + "decoy_questionnaire": { + "description": "Open-ended questions survey - the decoy option designed to be less attractive", + "items": [ + { + "id": "demographics_birth_year", + "question": "What is your year of birth?", + "type": "text_entry", + "q_idx": "Q1" + }, + { + "id": "demographics_gender", + "question": "What gender do you identify as?", + "type": "multiple_choice", + "options": [ + "Female", + "Male", + "Other, please specify" + ], + "q_idx": "Q2" + }, + { + "id": "demographics_race", + "question": "Choose one or more races that you consider yourself to be", + "type": "multiple_select", + "options": [ + "White", + "Black or African American", + "American Indian or Alaska Native", + "Asian", + "Native Hawaiian or Pacific Islander", + "Other" + ], + "q_idx": "Q3" + }, + { + "id": "demographics_education", + "question": "What is your current education level?", + "type": "multiple_choice", + "options": [ + "Some University but no degree", + "University - Bachelors Degree", + "Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)", + "Prefer not to say" + ], + "q_idx": "Q4" + }, + { + "id": "open_ended_1", + "question": "Can you describe how your attitude towards coronavirus has changed from the beginning until now?", + "type": "open_ended_text", + "constraint": "no less than 100 words", + "q_idx": "Q5" + }, + { + "id": "open_ended_2", + "question": "Can you describe what precautions you know to prevent infection? Which ones have you done? Do you continue doing them?", + "type": "open_ended_text", + "constraint": "no less than 100 words", + "q_idx": "Q6" + }, + { + "id": "debrief_choice_reason", + "question": "Why did you choose to answer the open-ended questions instead of the closed-ended questions?", + "type": "open_ended_text", + "constraint": "no less than five words", + "q_idx": "Q7" + }, + { + "id": "debrief_ranking_1", + "question": "Rank the reasons why you choose to respond to this survey", + "type": "ranking", + "options": [ + "I'm interested in this topic", + "I can receive an incentive from completing this survey", + "I can accept the length of the survey", + "I am assured of anonymity", + "I like the question type (open-ended)" + ], + "q_idx": "Q8" + }, + { + "id": "debrief_ranking_2", + "question": "Rank the reasons why you choose to respond to an online survey", + "type": "ranking", + "options": [ + "Interest in the Topic", + "Rewards", + "Length of Survey", + "Value Privacy (e.g. anonymity)", + "The Question Types (closed-ended or open-ended)" + ], + "q_idx": "Q9" + }, + { + "id": "preference_open_ended", + "question": "Open-ended questions are more appealing to me than the closed-ended questions.", + "type": "likert_5", + "options": [ + "Strongly agree", + "Somewhat agree", + "Disagree", + "Somewhat disagree", + "Strongly disagree" + ], + "q_idx": "Q10" + }, + { + "id": "preference_delayed_payment", + "question": "Late payment of incentives is more appealing to me than immediate payment.", + "type": "likert_5", + "options": [ + "Strongly agree", + "Somewhat agree", + "Disagree", + "Somewhat disagree", + "Strongly disagree" + ], + "q_idx": "Q11" + } + ] + }, + "target_questionnaire": { + "description": "Closed-ended questions survey - the preferred option", + "items": [ + { + "id": "demographics_birth_year", + "question": "What is your year of birth?", + "type": "text_entry", + "q_idx": "Q1" + }, + { + "id": "demographics_gender", + "question": "What gender do you identify as?", + "type": "multiple_choice", + "options": [ + "Female", + "Male", + "Other, please specify" + ], + "q_idx": "Q2" + }, + { + "id": "demographics_race", + "question": "Choose one or more races that you consider yourself to be", + "type": "multiple_select", + "options": [ + "White", + "Black or African American", + "American Indian or Alaska Native", + "Asian", + "Native Hawaiian or Pacific Islander", + "Other" + ], + "q_idx": "Q3" + }, + { + "id": "demographics_education", + "question": "What is your current education level?", + "type": "multiple_choice", + "options": [ + "Some University but no degree", + "University - Bachelors Degree", + "Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)", + "Prefer not to say" + ], + "q_idx": "Q4" + }, + { + "id": "fcq_prompt", + "question": "Please select the extent to which the following thoughts, feelings and behaviors apply to you:", + "instruction": "Scale: 1 = Strongly disagree, 5 = Strongly agree", + "type": "likert_scale", + "scale_min": 1, + "scale_max": 5 + }, + { + "id": "fcq_item_1", + "question": "I am very worried about the corona virus outbreak.", + "type": "likert_5", + "q_idx": "Q5a", + "scale": [1, 2, 3, 4, 5] + }, + { + "id": "fcq_item_2", + "question": "I am taking precautions to prevent infection (e.g., washing hands, avoiding contact with people, avoiding door handles).", + "type": "likert_5", + "q_idx": "Q5b", + "scale": [1, 2, 3, 4, 5] + }, + { + "id": "fcq_item_3", + "question": "I am constantly following all news updates regarding the virus.", + "type": "likert_5", + "q_idx": "Q5c", + "scale": [1, 2, 3, 4, 5] + }, + { + "id": "fcq_item_4", + "question": "I have stocked up on supplies to prepare for problems related to the coronavirus outbreak.", + "type": "likert_5", + "q_idx": "Q5d", + "scale": [1, 2, 3, 4, 5] + }, + { + "id": "fcq_item_5", + "question": "For my personal health I find the virus to be much more dangerous than the seasonal flu.", + "type": "likert_5", + "q_idx": "Q5e", + "scale": [1, 2, 3, 4, 5] + }, + { + "id": "fcq_item_6", + "question": "I feel that the health authorities are not doing enough to deal with the virus.", + "type": "likert_5", + "q_idx": "Q5f", + "scale": [1, 2, 3, 4, 5] + }, + { + "id": "fcq_item_7", + "question": "I am worried that friends or family will be infected.", + "type": "likert_5", + "q_idx": "Q5g", + "scale": [1, 2, 3, 4, 5] + }, + { + "id": "fcq_item_8", + "question": "I take more precautions compared to most people to not become infected.", + "type": "likert_5", + "q_idx": "Q5h", + "scale": [1, 2, 3, 4, 5] + }, + { + "id": "debrief_choice_reason", + "question": "Why did you choose to answer the closed-ended questions instead of the open-ended questions?", + "type": "multiple_choice", + "options": [ + "Closed-ended questions take me less time", + "The payment of this type questionnaire is faster", + "Both of them", + "Neither of them (please write your reason)" + ], + "q_idx": "Q6" + }, + { + "id": "debrief_ranking_1", + "question": "Rank the reasons why you choose to respond to this survey", + "type": "ranking", + "options": [ + "I'm interested in this topic", + "I can receive an incentive from completing this survey", + "I can accept the length of the survey", + "I am assured of anonymity", + "I like the question type (closed-ended)" + ], + "q_idx": "Q7" + }, + { + "id": "debrief_ranking_2", + "question": "Rank the reasons why you choose to respond to an online survey", + "type": "ranking", + "options": [ + "Interest in the Topic", + "Rewards", + "Length of Survey", + "Value Privacy (e.g. anonymity)", + "The Question Types (closed-ended or open-ended)" + ], + "q_idx": "Q8" + }, + { + "id": "debrief_decoy_influence", + "question": "Being offered an inferior choice (like the open question and late payment in this case) can stimulate me to respond to a survey.", + "type": "likert_5", + "options": [ + "Strongly agree", + "Somewhat agree", + "Disagree", + "Somewhat disagree", + "Strongly disagree" + ], + "q_idx": "Q9" + } + ] + }, + "condition_metadata": { + "condition": "Decoy - Decoy First", + "condition_id": "decoy_decoy_first", + "n_randomized": 50, + "n_completed_target": 14, + "completion_rate": 0.280, + "incentive": "£2 Amazon voucher (two £1 e-vouchers) for completing either survey; received after 1 week (target) or 4 weeks (decoy)", + "deadline": "Tuesday, August 16th, 8 PM UK Time" + } +} diff --git a/studies/study_015/source/materials/main_experiment_decoy_target_first.json b/studies/study_015/source/materials/main_experiment_decoy_target_first.json new file mode 100644 index 0000000..97ce76c --- /dev/null +++ b/studies/study_015/source/materials/main_experiment_decoy_target_first.json @@ -0,0 +1,324 @@ +{ + "sub_study_id": "main_experiment_decoy_target_first", + "sub_study_name": "Main experiment - Decoy condition with target shown first", + "design_notes": "Participants receive email invitation presenting both target and decoy survey options in a comparison table with the target survey shown first.", + "instructions": "Invitation email (decoy condition, target first) sent one week after preliminary questionnaire. No reminder emails sent.", + "invitation_email": { + "subject": "Survey: Your Mental Health Status During the Pandemic", + "body": "Hi there,\n\nThank you for signing up for our study earlier! We hope you will participate in the survey about your mental health status during the pandemic. The survey may take you 15 minutes and there are two types of questionnaires you can choose." + }, + "comparison_table": { + "presentation_order": "target_first", + "note": "Target survey options presented first, decoy survey options presented second", + "columns": [ + { + "title": "Closed-ended questions", + "question_type": "Eight scale multiple choice questions", + "reward": "£2 UK Amazon e-vouchers", + "reward_timing": "1 week after you finish the survey", + "survey_type": "target" + }, + { + "title": "Open-ended questions", + "question_type": "Two open-ended questions (answer each question with no less than 100 words)", + "reward": "£2 UK Amazon e-vouchers", + "reward_timing": "4 weeks after you finish the survey (as we need to check your answer)", + "survey_type": "decoy" + } + ], + "footer": "Please complete one questionnaire by 8 PM (UK Time), Tuesday, August 16th. About this survey: Your answers will remain anonymous and will be used in the researcher's graduation thesis. I hope you'll take 15 minutes to complete the survey and enjoy your gift!" + }, + "target_questionnaire": { + "description": "Closed-ended questions survey - the preferred option", + "items": [ + { + "id": "demographics_birth_year", + "question": "What is your year of birth?", + "type": "text_entry", + "q_idx": "Q1" + }, + { + "id": "demographics_gender", + "question": "What gender do you identify as?", + "type": "multiple_choice", + "options": [ + "Female", + "Male", + "Other, please specify" + ], + "q_idx": "Q2" + }, + { + "id": "demographics_race", + "question": "Choose one or more races that you consider yourself to be", + "type": "multiple_select", + "options": [ + "White", + "Black or African American", + "American Indian or Alaska Native", + "Asian", + "Native Hawaiian or Pacific Islander", + "Other" + ], + "q_idx": "Q3" + }, + { + "id": "demographics_education", + "question": "What is your current education level?", + "type": "multiple_choice", + "options": [ + "Some University but no degree", + "University - Bachelors Degree", + "Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)", + "Prefer not to say" + ], + "q_idx": "Q4" + }, + { + "id": "fcq_prompt", + "question": "Please select the extent to which the following thoughts, feelings and behaviors apply to you:", + "instruction": "Scale: 1 = Strongly disagree, 5 = Strongly agree", + "type": "likert_scale", + "scale_min": 1, + "scale_max": 5 + }, + { + "id": "fcq_item_1", + "question": "I am very worried about the corona virus outbreak.", + "type": "likert_5", + "q_idx": "Q5a", + "scale": [1, 2, 3, 4, 5] + }, + { + "id": "fcq_item_2", + "question": "I am taking precautions to prevent infection (e.g., washing hands, avoiding contact with people, avoiding door handles).", + "type": "likert_5", + "q_idx": "Q5b", + "scale": [1, 2, 3, 4, 5] + }, + { + "id": "fcq_item_3", + "question": "I am constantly following all news updates regarding the virus.", + "type": "likert_5", + "q_idx": "Q5c", + "scale": [1, 2, 3, 4, 5] + }, + { + "id": "fcq_item_4", + "question": "I have stocked up on supplies to prepare for problems related to the coronavirus outbreak.", + "type": "likert_5", + "q_idx": "Q5d", + "scale": [1, 2, 3, 4, 5] + }, + { + "id": "fcq_item_5", + "question": "For my personal health I find the virus to be much more dangerous than the seasonal flu.", + "type": "likert_5", + "q_idx": "Q5e", + "scale": [1, 2, 3, 4, 5] + }, + { + "id": "fcq_item_6", + "question": "I feel that the health authorities are not doing enough to deal with the virus.", + "type": "likert_5", + "q_idx": "Q5f", + "scale": [1, 2, 3, 4, 5] + }, + { + "id": "fcq_item_7", + "question": "I am worried that friends or family will be infected.", + "type": "likert_5", + "q_idx": "Q5g", + "scale": [1, 2, 3, 4, 5] + }, + { + "id": "fcq_item_8", + "question": "I take more precautions compared to most people to not become infected.", + "type": "likert_5", + "q_idx": "Q5h", + "scale": [1, 2, 3, 4, 5] + }, + { + "id": "debrief_choice_reason", + "question": "Why did you choose to answer the closed-ended questions instead of the open-ended questions?", + "type": "multiple_choice", + "options": [ + "Closed-ended questions take me less time", + "The payment of this type questionnaire is faster", + "Both of them", + "Neither of them (please write your reason)" + ], + "q_idx": "Q6" + }, + { + "id": "debrief_ranking_1", + "question": "Rank the reasons why you choose to respond to this survey", + "type": "ranking", + "options": [ + "I'm interested in this topic", + "I can receive an incentive from completing this survey", + "I can accept the length of the survey", + "I am assured of anonymity", + "I like the question type (closed-ended)" + ], + "q_idx": "Q7" + }, + { + "id": "debrief_ranking_2", + "question": "Rank the reasons why you choose to respond to an online survey", + "type": "ranking", + "options": [ + "Interest in the Topic", + "Rewards", + "Length of Survey", + "Value Privacy (e.g. anonymity)", + "The Question Types (closed-ended or open-ended)" + ], + "q_idx": "Q8" + }, + { + "id": "debrief_decoy_influence", + "question": "Being offered an inferior choice (like the open question and late payment in this case) can stimulate me to respond to a survey.", + "type": "likert_5", + "options": [ + "Strongly agree", + "Somewhat agree", + "Disagree", + "Somewhat disagree", + "Strongly disagree" + ], + "q_idx": "Q9" + } + ] + }, + "decoy_questionnaire": { + "description": "Open-ended questions survey - the decoy option designed to be less attractive", + "items": [ + { + "id": "demographics_birth_year", + "question": "What is your year of birth?", + "type": "text_entry", + "q_idx": "Q1" + }, + { + "id": "demographics_gender", + "question": "What gender do you identify as?", + "type": "multiple_choice", + "options": [ + "Female", + "Male", + "Other, please specify" + ], + "q_idx": "Q2" + }, + { + "id": "demographics_race", + "question": "Choose one or more races that you consider yourself to be", + "type": "multiple_select", + "options": [ + "White", + "Black or African American", + "American Indian or Alaska Native", + "Asian", + "Native Hawaiian or Pacific Islander", + "Other" + ], + "q_idx": "Q3" + }, + { + "id": "demographics_education", + "question": "What is your current education level?", + "type": "multiple_choice", + "options": [ + "Some University but no degree", + "University - Bachelors Degree", + "Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)", + "Prefer not to say" + ], + "q_idx": "Q4" + }, + { + "id": "open_ended_1", + "question": "Can you describe how your attitude towards coronavirus has changed from the beginning until now?", + "type": "open_ended_text", + "constraint": "no less than 100 words", + "q_idx": "Q5" + }, + { + "id": "open_ended_2", + "question": "Can you describe what precautions you know to prevent infection? Which ones have you done? Do you continue doing them?", + "type": "open_ended_text", + "constraint": "no less than 100 words", + "q_idx": "Q6" + }, + { + "id": "debrief_choice_reason", + "question": "Why did you choose to answer the open-ended questions instead of the closed-ended questions?", + "type": "open_ended_text", + "constraint": "no less than five words", + "q_idx": "Q7" + }, + { + "id": "debrief_ranking_1", + "question": "Rank the reasons why you choose to respond to this survey", + "type": "ranking", + "options": [ + "I'm interested in this topic", + "I can receive an incentive from completing this survey", + "I can accept the length of the survey", + "I am assured of anonymity", + "I like the question type (open-ended)" + ], + "q_idx": "Q8" + }, + { + "id": "debrief_ranking_2", + "question": "Rank the reasons why you choose to respond to an online survey", + "type": "ranking", + "options": [ + "Interest in the Topic", + "Rewards", + "Length of Survey", + "Value Privacy (e.g. anonymity)", + "The Question Types (closed-ended or open-ended)" + ], + "q_idx": "Q9" + }, + { + "id": "preference_open_ended", + "question": "Open-ended questions are more appealing to me than the closed-ended questions.", + "type": "likert_5", + "options": [ + "Strongly agree", + "Somewhat agree", + "Disagree", + "Somewhat disagree", + "Strongly disagree" + ], + "q_idx": "Q10" + }, + { + "id": "preference_delayed_payment", + "question": "Late payment of incentives is more appealing to me than immediate payment.", + "type": "likert_5", + "options": [ + "Strongly agree", + "Somewhat agree", + "Disagree", + "Somewhat disagree", + "Strongly disagree" + ], + "q_idx": "Q11" + } + ] + }, + "condition_metadata": { + "condition": "Decoy - Target First", + "condition_id": "decoy_target_first", + "n_randomized": 52, + "n_completed_target": 43, + "completion_rate": 0.827, + "incentive": "£2 Amazon voucher (two £1 e-vouchers) for completing either survey; received after 1 week (target) or 4 weeks (decoy)", + "deadline": "Tuesday, August 16th, 8 PM UK Time" + } +} diff --git a/studies/study_015/source/materials/preliminary_questionnaire.json b/studies/study_015/source/materials/preliminary_questionnaire.json new file mode 100644 index 0000000..86d6021 --- /dev/null +++ b/studies/study_015/source/materials/preliminary_questionnaire.json @@ -0,0 +1,105 @@ +{ + "sub_study_id": "preliminary_questionnaire", + "sub_study_name": "Preliminary questionnaire for recruitment and manipulation validation", + "instructions": "This questionnaire is to recruit participants. The main questionnaire will be sent via email in one week. Would you like to provide your email address for further study?", + "items": [ + { + "id": "email_opt_in", + "question": "This questionnaire is to recruit participants. The main questionnaire will be sent via email in one week. Would you like to provide your email address for further study?", + "type": "binary_choice", + "options": [ + "Yes", + "No" + ], + "q_idx": "Q1" + }, + { + "id": "email_address", + "question": "What is your email address?", + "type": "text_entry", + "q_idx": "Q2", + "note": "Conditional on Q1 = Yes" + }, + { + "id": "year_of_birth", + "question": "What is your year of birth?", + "type": "text_entry", + "q_idx": "Q3" + }, + { + "id": "gender", + "question": "What gender do you identify as?", + "type": "multiple_choice", + "options": [ + "Female", + "Male", + "Other, please specify" + ], + "q_idx": "Q4" + }, + { + "id": "race_ethnicity", + "question": "Choose one or more races that you consider yourself to be", + "type": "multiple_select", + "options": [ + "White", + "Black or African American", + "American Indian or Alaska Native", + "Asian", + "Native Hawaiian or Pacific Islander", + "Other" + ], + "q_idx": "Q5" + }, + { + "id": "education_level", + "question": "What is your current education level?", + "type": "multiple_choice", + "options": [ + "Some University but no degree", + "University - Bachelors Degree", + "Graduate or professional degree (MA, MS, MBA, PhD, Law Degree, Medical Degree etc)", + "Prefer not to say" + ], + "q_idx": "Q6" + }, + { + "id": "preferred_question_type", + "question": "When you are invited to fill in a questionnaire, which question type would you prefer to answer?", + "type": "multiple_choice", + "options": [ + "Close-ended questions (e.g. multiple choice)", + "Open-ended questions (e.g. text entry)", + "There is no difference between the two options for me" + ], + "q_idx": "Q7", + "note": "This item is used to validate that closed-ended questions are perceived as more attractive than open-ended questions, supporting the use of open-ended questions as a decoy attribute." + }, + { + "id": "delay_payment_willingness", + "question": "If you are invited to fill in a questionnaire, but you need to wait four weeks to receive your participation payment, will you still participate in the study?", + "type": "multiple_choice", + "options": [ + "Yes", + "I may consider", + "No" + ], + "q_idx": "Q8", + "note": "This item is used to validate that immediate payment is perceived as more attractive than delayed payment, supporting the use of delayed payment as a decoy attribute." + }, + { + "id": "religion", + "question": "NOT PROVIDED in paper supplement", + "type": "multiple_choice", + "note": "Religion is analyzed in study tables but exact question wording is NOT PROVIDED in the paper." + } + ], + "metadata": { + "recruitment_venue": "Students at UK-based university via Facebook, WhatsApp, and WeChat", + "completion_incentive": "£2 Amazon voucher for providing email to participate in main study", + "n_started": 241, + "n_completed": 216, + "n_with_email": 210, + "purpose": "To recruit participants for main experiment and to validate that open-ended questions and delayed payment would be perceived as inferior attributes suitable for use as a decoy" + } +} diff --git a/studies/study_015/source/metadata.json b/studies/study_015/source/metadata.json new file mode 100644 index 0000000..29876b7 --- /dev/null +++ b/studies/study_015/source/metadata.json @@ -0,0 +1,144 @@ +{ + "study_id": "stoffel_2023_decoy_effect_survey_participation", + "title": "Testing the decoy effect to improve online survey participation: Evidence from a field experiment", + "authors": [ + "Sandro T. Stoffel", + "Yining Sun", + "Yasemin Hirst", + "Christian von Wagner", + "Ivo Vlaev" + ], + "year": 2023, + "journal": "Journal of Behavioral Decision Making", + "publication_type": "peer_reviewed_journal", + "paper_url": "https://doi.org/...", + "preprint_url": "NOT PROVIDED", + "study_registration": "NOT PROVIDED", + "data_availability": "Participant-level data available in exp.csv; analysis code in analysis_18082022.do (Stata)", + "ethical_approval": "NOT PROVIDED in paper", + "key_concepts": [ + "decoy effect", + "attraction effect", + "choice architecture", + "survey participation", + "response rates", + "behavioral economics", + "online experiments", + "order effects", + "presentation order" + ], + "relevant_theories": [ + "Choice overload", + "Decoy attraction effect", + "Choice presentation order effects" + ], + "replication_feasibility": { + "difficulty": "moderate", + "rationale": "Online survey administration with binary/multi-question stimuli is straightforward. The key complexities are: (1) simulating realistic choice between two survey options with different attributes, (2) capturing the LLM's willingness to complete open-ended responses (100+ word minimum), (3) accurately measuring the order effect with presentation order manipulation.", + "critical_factors": [ + "Agent must realistically evaluate trade-offs between question type and payment timing", + "Presentation order must be manipulated within-subject (target first vs decoy first)", + "Open-ended responses must meet word count constraint (100+ words)", + "Debrief questions (e.g., 'did the decoy influence your choice?') must be credible", + "Fear of Coronavirus Questionnaire must be presented with realistic 5-point Likert scale" + ] + }, + "known_limitations": { + "ethnicity_imbalance": "Significant imbalance in ethnicity between randomized groups (p=0.004); decoy condition had more White participants. This imbalance persisted among completers (p=0.006).", + "missing_religion_question": "Religion is analyzed in tables but exact question wording in the preliminary questionnaire is NOT PROVIDED in the paper supplement.", + "moderate_sample": "N=203 valid emails; 90 target completers provide reasonable power but limits subgroup analyses.", + "single_context": "Study conducted in UK with UK university students; generalizability to other contexts unknown.", + "short_followup": "No long-term follow-up; unclear if decoy effect persists for future survey invitations." + }, + "comparison_to_other_studies": { + "similar_studies": [ + "Decoy effect / attraction effect studies in consumer choice (Ariely, Huber & Puto)", + "Survey response rate improvement literature (Groves, Singer, Corning)" + ], + "novelty": "This appears to be one of the first applications of the decoy effect to survey participation contexts, with a focus on presentation order effects." + }, + "simulation_requirements": { + "agent_decision_points": [ + "Preliminary survey: preference for question type and willingness to wait for payment", + "Main experiment: choice between target and decoy survey options (or decline to participate in control)", + "Target survey completion: FCQ Likert responses, demographics, debrief rankings", + "Decoy survey completion: open-ended text responses (100+ words each), debrief rankings" + ], + "evaluation_metrics": [ + "Primary: binary outcome (target survey completion)", + "Secondary: FCQ score distribution (sum of 8 Likert items)", + "Secondary: perceived influence of decoy (debrief response)", + "Analysis: replication of chi-square tests, logistic regression ORs, order effect contrast" + ], + "expected_challenges": [ + "Open-ended responses may be difficult for LLM to provide realistically within 100-word constraints", + "Decoy effect presumes realistic cost-benefit trade-off evaluation; some agents may complete both surveys or neither", + "Large order effect (11:1 ratio for target-first vs decoy-first) is strong; difficult to achieve with LLM agents unless prompt construction is highly realistic" + ] + }, + "data_structure": { + "participants": 203, + "variables_collected": 36, + "raw_data_file": "exp.csv", + "participant_id": "ParticipantNumber", + "condition_variable": "Condition (Decoy Condition, Control Condition)", + "order_variable": "conditiongroup (Decoy: target 1st vs Control)", + "demographics": [ + "Age", + "Gender", + "Race/Ethnicity", + "Religion", + "Education" + ], + "outcome_variables": [ + "Participate (binary: yes/no)", + "ValidEmail (binary: yes/no for receiving invitation)", + "QuestionTypePrefer (multiple choice)", + "AttitudetoLatePayment (multiple choice)" + ] + }, + "statistical_methods_used": [ + "Chi-square test of independence", + "Fisher's exact test", + "Binary logistic regression (unadjusted and adjusted)", + "Wilcoxon-Mann-Whitney test (for FCQ score distributions)" + ], + "reproducibility_notes": "The paper provides detailed tables with raw numbers, odds ratios, confidence intervals, and p-values. The participant-level data (exp.csv) and Stata analysis script (analysis_18082022.do) are available, enabling full reproducibility. Order effects are clearly reported, with separate analysis for target-first and decoy-first conditions.", + "findings": [ + { + "finding_id": "F1_prelim", + "description": "Preference for closed-ended questions validation", + "weight": 0.05 + }, + { + "finding_id": "F2_prelim", + "description": "Aversion to delayed payment validation", + "weight": 0.05 + }, + { + "finding_id": "F1", + "description": "Randomization balance checks", + "weight": 0.01 + }, + { + "finding_id": "F2", + "description": "Primary: Decoy effect on target survey completion", + "weight": 0.35 + }, + { + "finding_id": "F3", + "description": "Primary: Order effect (target-first vs decoy-first vs control)", + "weight": 0.35 + }, + { + "finding_id": "F4", + "description": "Secondary: Perceived influence of decoy on decision", + "weight": 0.10 + }, + { + "finding_id": "F5", + "description": "Secondary: Non-response bias and response behavior checks", + "weight": 0.14 + } + ] +} diff --git a/studies/study_015/source/specification.json b/studies/study_015/source/specification.json new file mode 100644 index 0000000..e6246f0 --- /dev/null +++ b/studies/study_015/source/specification.json @@ -0,0 +1,137 @@ +{ + "study_id": "stoffel_2023_decoy_effect_survey_participation", + "title": "Testing the decoy effect to improve online survey participation: Evidence from a field experiment", + "participants": { + "preliminary_questionnaire": { + "n": 210, + "details": "241 individuals clicked the survey link and started the preliminary questionnaire; 216 completed the questionnaire; 210 provided an email address to be invited to the experiment. Recruited in August 2022 from students registered at a UK-based university through Facebook, WhatsApp, and WeChat.", + "demographics_reported": { + "age_20_24": 56, + "age_25_29": 107, + "age_30_35": 47, + "male": 110, + "female": 100, + "white": 120, + "asian": 50, + "black": 36, + "other_ethnicity": 4, + "christian": 151, + "other_religion": 41, + "no_religion": 18, + "bachelors_degree": 109, + "graduate_degree": 101 + } + }, + "main_experiment": { + "n": 203, + "details": "210 invitation emails sent; 7 emails bounced (4 from control, 3 from decoy), leaving 203 who received the experimental invitation. Randomized in Microsoft Excel. Sample characteristics among the 203 valid-email participants: age 20-24: 55 (27.1%), 25-29: 102 (50.2%), 30-35: 46 (22.7%); male: 106 (52.2%), female: 97 (47.8%); White: 117 (57.6%), Asian: 48 (23.7%), Black: 34 (16.7%), Other: 4 (2.0%); Christian: 146 (71.9%), Other religion: 18 (8.9%), No religion: 39 (19.2%); Bachelor's degree: 105 (51.7%), Graduate/professional degree: 98 (48.3%).", + "control_group": { + "n": 101, + "description": "Received email invitation with only the target survey option" + }, + "decoy_group": { + "n": 102, + "description": "Received email invitation with both target and decoy survey options", + "decoy_target_first": { + "n": 52, + "description": "Decoy group randomized to see target survey first in the comparison table" + }, + "decoy_decoy_first": { + "n": 50, + "description": "Decoy group randomized to see decoy survey first in the comparison table" + } + } + } + }, + "design": { + "type": "between-subjects field experiment with nested randomization of option order within the decoy condition", + "primary_factor": "Invitation condition (control vs decoy)", + "secondary_factor": "Presentation order within decoy condition (target first vs decoy first)", + "randomization": "Microsoft Excel random function", + "conditions": [ + { + "condition_id": "control", + "name": "Control", + "description": "Email invitation offers only the target survey (Fear of Coronavirus Questionnaire with 8 closed-ended questions; £2 Amazon voucher after 1 week)" + }, + { + "condition_id": "decoy_target_first", + "name": "Decoy - Target First", + "description": "Email invitation offers both target (closed-ended, £2 after 1 week) and decoy (open-ended, £2 after 4 weeks) surveys in table format with target shown first" + }, + { + "condition_id": "decoy_decoy_first", + "name": "Decoy - Decoy First", + "description": "Email invitation offers both target (closed-ended, £2 after 1 week) and decoy (open-ended, £2 after 4 weeks) surveys in table format with decoy shown first" + } + ] + }, + "procedure": [ + { + "stage": 1, + "name": "Preliminary questionnaire recruitment", + "duration": "August 2022", + "details": "Students at a UK-based university were contacted via social media (Facebook, WhatsApp, WeChat) with a survey on fear of coronavirus. Participants completed a preliminary questionnaire assessing attitudes toward question type (close-ended vs open-ended) and payment delay (1 week vs 4 weeks) and provided demographics. At the end, participants could provide their email address for an Amazon voucher of £2 to participate in the main survey." + }, + { + "stage": 2, + "name": "Main field experiment", + "duration": "One week after preliminary questionnaire", + "details": "Individuals who had provided email addresses were sent an email invitation to the main questionnaire. Participants were randomized in Microsoft Excel into control or decoy conditions. Within the decoy condition, participants were further randomized with equal probability to one of two presentation orders: target first or decoy first. No reminder emails were sent." + }, + { + "stage": 3, + "name": "Target survey completion (if selected)", + "details": "Target survey consisted of: (1) Demographics (4 items); (2) Fear of Coronavirus Questionnaire (8 items, 1-5 Likert scale); (3) Debrief questions (2-3 items depending on condition). Reward: £2 Amazon voucher after 1 week." + }, + { + "stage": 4, + "name": "Decoy survey completion (if selected in decoy condition)", + "details": "Decoy survey consisted of: (1) Demographics (4 items); (2) Two open-ended questions requiring minimum 100 words each on coronavirus attitudes and precautions; (3) Debrief questions (4-5 items). Reward: £2 Amazon voucher after 4 weeks (stated reason: responses need to be coded)." + } + ], + "primary_outcomes": [ + { + "outcome_id": "target_completion", + "name": "Completion of target questionnaire", + "measurement": "Binary (yes/no)", + "unit_of_analysis": "Per participant" + } + ], + "secondary_outcomes": [ + { + "outcome_id": "fcq_score", + "name": "Fear of Coronavirus Questionnaire total score", + "measurement": "Summed Likert scale (8-40)", + "unit_of_analysis": "Per participant (among target completers)" + }, + { + "outcome_id": "perceived_influence", + "name": "Perceived influence of decoy on decision", + "measurement": "Binary/categorical response to debrief question", + "unit_of_analysis": "Per target completer in decoy condition" + }, + { + "outcome_id": "decoy_completion", + "name": "Completion of decoy questionnaire", + "measurement": "Binary (yes/no)", + "unit_of_analysis": "Per participant (in decoy condition)" + } + ], + "independent_variables": [ + { + "name": "Invitation condition", + "levels": ["Control (target only)", "Decoy (both options)"] + }, + { + "name": "Presentation order (within decoy condition)", + "levels": ["Target first", "Decoy first"] + } + ], + "dependent_variables": [ + "Completion of target survey (primary)", + "Fear of Coronavirus Questionnaire responses (secondary)", + "Demographic composition of completers vs non-completers (secondary)", + "Self-reported influence of decoy on participation decision (secondary)" + ] +}