# Pre-Testing Variables

### What is Pre-Testing?

Pre-testing evaluates how variables would perform in your model WITHOUT actually adding them. It answers: "If I add this variable, what happens?"

**Key benefit:** Risk-free evaluation of candidates

### How Pre-Testing Works

#### The Process

1. Select variables to test
2. System temporarily adds each to model
3. Runs regression with variable included
4. Calculates performance metrics
5. Removes variable (model unchanged)
6. Returns results for comparison

**Result:** Performance preview without commitment

### Pre-Testing Workflow

#### Step 1: Preparation

**Have a stable baseline model:**

* Working model with core variables
* Passing diagnostics
* Reasonable R²

**Identify candidates:**

* Variables you're considering
* New marketing channels
* Control factors
* Transformations to test

#### Step 2: Selection

**In Variable Testing interface:**

1. Select baseline model
2. Check variables to pre-test
3. Configure adstock if applicable
4. Click "Test Variables"

#### Step 3: Analysis

**Review results table:**

* Sorted by T-statistic
* Shows coefficient, significance, VIF
* R² increase for each

**Identify winners:**

* T-stat > 2.0
* P-value < 0.05
* Correct sign
* VIF < 10

#### Step 4: Decision

**Add winners to model:**

* Navigate to Model Builder
* Add top 2-3 performers
* Check diagnostics
* Validate results

### What to Look For

#### Statistical Criteria

**Significance:**

* T-statistic > 2.0
* P-value < 0.05
* 95% confidence it matters

**Model Improvement:**

* R² increase > 0.01 (1%)
* Meaningful contribution
* Worth the complexity

**No Multicollinearity:**

* VIF < 10
* Not redundant
* Adds unique information

#### Business Criteria

**Correct Sign:**

* Marketing spend: Positive
* Price: Usually negative
* Matches expectations

**Reasonable Magnitude:**

* Coefficient makes business sense
* Neither too large nor too small
* Validates with domain knowledge

**Relevance:**

* Variable available going forward
* Data quality acceptable
* Actionable insight

### Pre-Testing Strategies

#### Compare Similar Variables

**Scenario:** Have TV\_GRPs and TV\_Spend

**Test both:**

* See which performs better
* Check VIF (likely correlated)
* Choose one, not both

**Decision:** Keep higher T-stat, lower VIF

#### Find Optimal Transformation

**Test multiple versions:**

* Raw variable
* With adstock
* With saturation curve
* With both

**Decision:** Use best-performing transformation

#### Screen Many Candidates

**Test 10-20 variables:**

* Rank by performance
* Identify top 3-5
* Add only winners

**Efficiency:** Pre-test before trial-and-error

#### Category Evaluation

**Test all from one category:**

* All digital channels
* All control variables
* All seasonality indicators

**Decision:** Add top performers from each category

### Common Pre-Testing Scenarios

#### New Variable Evaluation

**Question:** Should I add Radio\_Spend?

**Pre-test:**

* Radio\_Spend with multiple adstock rates
* Check significance and R² increase

**Decision criteria:**

* If t > 2.0 and R² +0.02: Add
* If t < 1.5 and R² +0.001: Skip

#### Transformation Comparison

**Question:** Raw or transformed?

**Test:**

* TV\_Spend (raw)
* TV\_Spend\_adstock\_70
* TV\_Spend\_adstock\_70|ICP\_ATAN

**Compare:**

* Statistical significance
* R² improvement
* Business interpretability

**Choose:** Best combination of fit and interpretability

#### Multicollinearity Resolution

**Problem:** Multiple correlated variables

**Pre-test all:**

* Check individual VIF
* Compare T-statistics
* Review R² contributions

**Solution:** Keep best, drop others

### Interpreting Pre-Test Results

#### Good Candidate

✅ T-stat: 3.5 (highly significant)\
✅ P-value: 0.001 (< 0.05)\
✅ VIF: 2.3 (< 5, excellent)\
✅ R² increase: +0.03 (meaningful)\
✅ Coefficient: +450 (positive, reasonable)

**Action:** Strong candidate, add to model

#### Marginal Candidate

⚠️ T-stat: 2.1 (barely significant)\
⚠️ P-value: 0.04 (< 0.05 but close)\
⚠️ VIF: 7.8 (< 10 but moderate)\
⚠️ R² increase: +0.008 (small)\
⚠️ Coefficient: +120 (positive)

**Action:** Consider if theoretically important, otherwise skip

#### Poor Candidate

❌ T-stat: 0.8 (not significant)\
❌ P-value: 0.42 (> 0.05)\
❌ VIF: 15 (> 10, multicollinear)\
❌ R² increase: +0.002 (negligible)\
❌ Coefficient: -30 (wrong sign)

**Action:** Do not add

### Best Practices

#### Pre-Testing Principles

**Test before committing:**

* Never add without pre-testing
* Systematic > trial-and-error
* Data-driven decisions

**Test in context:**

* Use realistic baseline model
* Test against actual model you'll use
* Consider existing variables

**Test multiple options:**

* Don't stop at first decent result
* Compare alternatives
* Find truly best option

#### What NOT to Do

❌ Add variable because "it might help"\
❌ Skip pre-testing to save time\
❌ Add all variables with t > 1.5\
❌ Ignore VIF warnings\
❌ Add variables with wrong signs

✅ Pre-test systematically\
✅ Use statistical AND business criteria\
✅ Add only clear winners\
✅ Check multicollinearity\
✅ Validate coefficient signs

### Integration with Workflow

#### Phase 1: Baseline Model

Build core model without pre-testing:

* Obvious variables (main media channels)
* Add directly in Model Builder
* Establish foundation

#### Phase 2: Expansion via Pre-Testing

Use Variable Testing to grow model:

* Pre-test 10-20 candidates
* Identify top 3-5
* Add winners to Model Builder
* Validate with diagnostics

#### Phase 3: Optimization

Fine-tune using pre-testing:

* Test transformed versions
* Optimize adstock rates
* Compare specifications
* Select best configuration

#### Phase 4: Final Validation

After pre-testing and addition:

* Run full diagnostics
* Check decomposition
* Validate business logic
* Export final model

### Key Takeaways

* Pre-testing evaluates variables without changing model
* Enables risk-free comparison of many candidates
* Look for t > 2.0, p < 0.05, VIF < 10, positive R² increase
* Use to compare transformations and optimize adstock
* Test systematically before adding to model
* Add only variables that pass both statistical and business criteria
* Combine pre-testing with Model Builder for efficient development
