Model Building Strategy

Overview

Effective model building combines statistical rigor with business judgment. This guide outlines strategic approaches to building robust, interpretable, and actionable marketing mix models that drive business decisions.

Core Principles

Principle 1: Start Simple, Add Complexity

Begin with Baseline:

• Include only top 5-10 most important variables

• Use OLS (faster iteration)

• No transformations initially

• Establish performance benchmark

Add Incrementally:

• Add one variable or transformation at a time

• Evaluate impact on R², coefficients, diagnostics

• Keep changes that meaningfully improve model

• Document rationale for each addition

Why This Works:

• Easier to diagnose problems

• Understand contribution of each element

• Avoid over-fitting

• Build confidence progressively

Principle 2: Theory Before Statistics

Business Logic First:

• Include variables that should matter theoretically

• Don't remove variables solely because p-value >0.05

• Consider business context and domain knowledge

• Balance statistical significance with practical significance

Example:

TV_Spend: p=0.08 (not "significant" at 0.05)
But: TV is major channel, large budget, known to drive sales
→ Keep in model, investigate why significance is marginal

Principle 3: Diagnostics are Not Optional

Always Validate:

• Run diagnostics on every model candidate

• Don't trust results from models that fail tests

• Address issues before interpreting coefficients

• Document which tests passed/failed

Minimum Required Tests:

• Multicollinearity (VIF)

• Residual normality

• Autocorrelation

• Heteroscedasticity

Act on Results: Failed tests indicate model problems that must be fixed

Principle 4: Iterate and Compare

Build Multiple Specifications:

• Baseline model

• Model with adstock

• Model with saturation curves

• Model with interactions

• Simplified "parsimonious" model

Compare Systematically:

• R², Adjusted R², AIC, BIC

• Coefficient stability

• Diagnostic test performance

• Business reasonableness

Select Best Overall: Not just highest R², but best balance of fit, parsimony, and interpretability

Step-by-Step Strategy

Step 1: Build Baseline Model

Include:

• KPI as dependent variable

• Top 5-10 marketing channels (spend or activity)

• Basic controls (seasonality if obvious)

Exclude (for now):

• Transformations

• Interactions

• Minor variables

• Complex features

Evaluate:

• R²: Expect 0.50-0.70 for baseline

• Coefficients: Check signs make sense

• Significance: Note which variables are significant

Example Baseline:

Revenue ~ TV_Spend + Digital_Spend + Print_Spend + Radio_Spend + 
          Promo_Flag + Trend

Step 2: Add Control Variables

Seasonality:

• Month indicators or quarter indicators

• Holiday flags (Thanksgiving, Christmas, etc.)

• Day-of-week effects (if daily data)

Trends:

• Linear time trend

• Year indicators

• Macro factors (if available)

External Factors:

• Weather (if relevant)

• Competitive activity

• Economic indicators

Impact: Usually increases R² by 0.05-0.15

Step 3: Apply Adstock Transformations

For Media Variables:

• TV, Radio, Print: Test 50-70% decay

• Digital, Social: Test 30-50% decay

• Out-of-Home: Test 60-80% decay

Process:

1. Go to Variable Testing

2. Test multiple adstock rates per variable

3. Select rate with highest t-statistic or contribution

4. Apply to main model

Evaluation:

• Does adstock improve fit (R²)?

• Do coefficients make more sense?

• Are effects more significant?

Typical Improvement: +0.03 to +0.10 in R²

Step 4: Test Saturation Effects

For High-Spend Channels:

• Channels with >20% of total budget

• Channels suspected of diminishing returns

• Channels with wide spending range

Curves to Test:

• S-curve (ICP): For awareness to consideration

• Concave (ADBUG): For direct response

• CDR (Constant Diminishing Returns): General case

Decision:

• Compare linear vs curved specification

• Use curve if:

  • Significantly better fit

  • Passes diagnostic tests

  • Makes business sense

• Otherwise stick with linear

Advanced: Combine adstock + saturation for same variable

Step 5: Explore Interactions

Test When:

• Theoretical synergy suspected

• Channels often used together

• Campaign data suggests interaction

Create:

• Multiply variables: TV × Digital

• Test in Variable Testing first

• Add to model if significant and improves fit

Be Cautious:

• Interactions increase multicollinearity

• Harder to interpret

• Can overfit with small samples

• Only include if strong evidence

Step 6: Refine and Simplify

Remove Weak Variables:

• Non-significant with p >0.10

• Coefficients near zero

• High VIF (>10) with other variables

• Don't contribute to business decisions

Combine Similar Variables:

• Multiple social platforms → Total_Social

• Similar media → Combined category

• Reduces multicollinearity

• Improves stability

Final Model Should Have:

• 10-30 variables (typical)

• All VIF <10 (ideally <5)

• Most variables significant (p <0.10)

• Clear business interpretation

Step 7: Validate Thoroughly

Run All Diagnostics:

• Normality: Shapiro-Wilk, Jarque-Bera

• Autocorrelation: Durbin-Watson

• Heteroscedasticity: Breusch-Pagan

• Multicollinearity: VIF

• Influential points: Cook's Distance

Sanity Checks:

• Coefficients have expected signs

• Magnitudes are reasonable

• Seasonality aligns with business knowledge

• Top channels match expectations

Stress Tests:

• Remove one variable at a time - are results stable?

• Split sample in half - consistent estimates?

• Compare to previous models - similar findings?

Model Selection Criteria

Statistical Fit

**R² and Adjusted R²