1 A Motivating Example: Hiring Discrimination

Do employers discriminate against racial minority job applicants? A survey asking employers "Would you hire a Black applicant?" suffers from social desirability bias—respondents know the "right" answer and give it. An alternative approach is an audit study: send thousands of fictitious CVs with racially distinct names to real job postings and measure callback rates. Bertrand and Mullainathan (2004) did exactly this, finding that CVs with names associated with Black applicants received 50% fewer callbacks than otherwise identical CVs with names associated with White applicants.  

But audit studies are expensive and can only vary one or two attributes at a time. Suppose we want to know simultaneously: how much do race, gender, education level, and experience each affect hiring preferences? A conjoint experiment can answer all these questions at once, from a single survey.  

2 The Fundamental Problem Conjoint Solves

Observational data on hiring or immigration attitudes conflate multiple attributes. A highly educated immigrant is also likely to be from a wealthier country, speak better English, and have a higher-skilled occupation—these attributes are correlated in the real world, so disentangling their individual effects is impossible without experimental variation.

The fundamental insight of conjoint analysis is to randomise the attributes of hypothetical profiles shown to survey respondents. Because each attribute is assigned randomly and independently, the difference in responses between profiles differing only in one attribute is a valid causal estimate of that attribute's effect.  

3 The Method: Factorial Conjoint Analysis

3.1 Design

A conjoint survey presents respondents with pairs (or sets) of hypothetical profiles. Each profile has several attributes (e.g., a job candidate's race, education, experience, salary history), each with multiple levels (e.g., race: Black / White / Hispanic / Asian). The attribute levels of each profile are randomly and independently assigned to each respondent.

Respondents are asked to make a binary choice ("Which candidate would you prefer to hire?") or to rate each profile on a scale. The randomisation of attributes across respondents creates experimental variation that identifies causal effects.  

3.2 The Key Estimand: AMCE

Hainmueller et al. (2014) formalise the identifying estimand as the Average Marginal Component Effect (AMCE):  

where Yi(Dk = d) is the respondent's choice if attribute k is set to level d (all other attributes being marginalised over their randomised distribution). The expectation is over both respondents and the marginal distribution of the other attribute levels.

The AMCE is a causal quantity because attribute assignment is randomised: by the Fisher randomisation argument, changing attribute k from d to d' while keeping all others fixed yields a causal comparison. Averaging over all profile configurations gives the AMCE—the average causal effect of changing one attribute, integrated over the full profile distribution.  

3.3 Estimation

The AMCE is estimated by linear regression of the choice variable on dummy variables for each attribute level (Hainmueller et al., 2014):  

where Yijt is the choice indicator (1 if profile j is chosen by respondent i in task t), and βkl is the AMCE of setting attribute k to level l relative to the omitted base level. Standard errors are clustered by respondent to account for multiple observations per person.  

4 A Numerical Example

Hainmueller et al. (2014) apply conjoint analysis to immigration preferences in the United States. They present respondents with pairs of hypothetical immigrants described by attributes including education level (no degree / high school / college / graduate), profession, country of origin, language skills, and reason for applying. Selected findings from their AMCE estimates:  

• A college-educated immigrant is preferred over a high-school educated immigrant with AMCE = 0.12 (12 percentage points higher probability of selection).  

• A graduate-degree immigrant relative to no education: AMCE = 0.21.  

• Country of origin matters: immigrants from Iraq are preferred with probability 0.08 lower than immigrants from Germany (AMCE = -0.08), holding all other attributes constant.  

• Reason for immigration (refugee vs. economic migrant) has a smaller effect than education.These estimates isolate each attribute's causal contribution, something impossible with observational data where education, country, and reason for immigrating are all correlated.  

5 Extensions

• Subgroup AMCEs: The AMCE can be estimated separately for subgroups of respondents (e.g., do Republicans and Democrats weight education differently?). These are simply interaction terms between attribute levels and respondent characteristics in (2).  

• Marginal means: Leeper et al. (2020) argue that AMCEs depend on the distribution of other attribute levels (which is set by the researcher), making comparisons across studies with different designs difficult. They propose marginal means—the average probability of choosing a profile at a given attribute level, marginalised over all other attributes—as a more interpretable alternative.  

• Interaction effects: If two attributes interact (e.g., the effect of race depends on the education level of the profile), interaction terms can be added to (2). These require larger samples for stable estimation.  

6 Conjoint vs. Audit Studies

Conjoint experiments and audit studies are complements, not substitutes. Conjoint analysis efficiently estimates the AMCE for many attributes simultaneously; audit studies provide behavioural realism with actual decision-makers in real settings.  

7 Common Mistakes

1. Not clustering standard errors by respondent. Multiple responses per person are correlated; treating them as independent underestimates standard errors.  

1. Confusing AMCE with subgroup effects. The AMCE is an average over the full profile distribution; it does not tell you the effect when all other attributes are held at specific values.  

1. Ignoring carryover and order effects. Responses to the second profile in a pair may be influenced by the first; randomly vary the order of profiles across respondents.  

1. Profile realism. If the attribute combinations are unrealistic (e.g., a doctor with no education), respondents may respond inconsistently. Screen profiles for realism or add profile constraints.  

8 Software

The cregg and cjoint packages in R estimate AMCEs from conjoint data with minimal code. Specify the choice variable, respondent ID, and attribute columns:  

9 Where to Learn More

Hainmueller et al. (2014) is the foundational reference. Leeper et al. (2020) develop marginal means. For randomisation-based inference in conjoint experiments (avoiding parametric assumptions), see Bansak et al. (2023) .  

References

1. Bansak, K., Hainmueller, J., Hopkins, D. J., Yamamoto, T., and Hangartner, D. (2023). Conjoint Survey Experiments. Cambridge University Press.   
2. Bertrand, M. and Mullainathan, S. (2004). Are Emily and Greg more employable than Lakisha and Jamal? A field experiment on labor market discrimination. American Economic Review, 94(4):991-1013.   
3. Hainmueller, J., Hopkins, D. J., and Yamamoto, T. (2014). Causal inference in conjoint analysis: Understanding multidimensional choices via stated preference experiments. Political Analysis, 22(1):1-30.   
4. Imbens, G. W. and Rubin, D. B. (2015). Causal Inference for Statistics, Social, and Biomedical Sciences. Cambridge University Press.   
5. Leeper, T. J., Hobolt, S. B., and Tilley, J. (2020). Measuring subgroup preferences in conjoint experiments. Political Analysis, 28(2):207-221.