Credible Intervals
Overview
Credible intervals are the Bayesian equivalent of confidence intervals, but with a crucial difference: they have a direct probability interpretation. A 95% credible interval means "there is a 95% probability that the true parameter value lies within this range," given your data and priors.
Unlike frequentist confidence intervals (which have a more complex interpretation), credible intervals provide intuitive, actionable insights for business decision-making.
What Are Credible Intervals?
Definition
A credible interval contains a specified percentage of the posterior probability mass. For a 95% credible interval, 95% of the posterior samples fall within the interval bounds.
Types of Credible Intervals
Equal-Tailed Interval (ETI): Excludes 2.5% from each tail of the distribution. Simple but may not capture the most probable values for skewed distributions.
Highest Density Interval (HDI): Contains the most probable parameter values. For skewed distributions, HDI is narrower and more informative than ETI. MixModeler uses HDI by default.
HDI vs ETI Example
For a right-skewed posterior distribution:
HDI 95%: [1.2, 4.5] - narrowest interval containing 95% probability
ETI 95%: [0.8, 4.8] - wider interval with 2.5% in each tail
HDI provides a tighter, more meaningful range by focusing on the high-density regions.
Reading Credible Intervals in MixModeler
Coefficient Output Format
When you view Bayesian model results, each coefficient displays:
TV_Advertising
Posterior Mean: 3.45
Posterior Std: 0.82
95% HDI: [1.95, 5.12]Interpretation: "Given our data and priors, we are 95% certain the TV advertising coefficient is between 1.95 and 5.12, with the most likely value around 3.45."
Interval Width and Uncertainty
Narrow Intervals: High certainty about parameter value
Example: 95% HDI [2.8, 3.2] - very precise estimate
Indicates strong data signal or informative priors
Wide Intervals: High uncertainty about parameter value
Example: 95% HDI [0.5, 6.8] - substantial uncertainty
Indicates limited data, weak signal, or uninformative priors
Zero-Crossing Intervals
Interval Does Not Cross Zero: [1.5, 4.2]
Strong evidence of an effect in one direction
High probability the coefficient is positive (or negative if both bounds negative)
Actionable insight for business decisions
Interval Crosses Zero: [-0.8, 3.5]
Uncertainty about direction of effect
Coefficient could be positive, negative, or zero
May indicate need for more data or model refinement
Common Credible Interval Widths
95% Credible Interval (Default)
Most commonly reported interval in research and business contexts.
Use Cases:
Standard model reporting
Business presentations
Comparing with frequentist 95% confidence intervals
General-purpose uncertainty quantification
Interpretation: "We are 95% certain the true value is in this range."
90% Credible Interval
Slightly narrower interval for less conservative estimates.
Use Cases:
Less critical decisions
When slightly higher risk is acceptable
Industry standards in some domains
Interpretation: "We are 90% certain the true value is in this range."
99% Credible Interval
Very wide interval for high-confidence decisions.
Use Cases:
Critical business decisions
Regulatory compliance
Risk-averse contexts
Publishing rigorous results
Interpretation: "We are 99% certain the true value is in this range."
50% Credible Interval
Captures the most likely half of the distribution.
Use Cases:
Quick sense of most probable values
Comparing central tendencies across parameters
Technical discussions with analysts
Interpretation: "There's a 50% chance the true value is in this range - it's equally likely to be inside or outside."
Interpreting Credible Intervals
Positive Evidence
When the entire 95% HDI is positive:
Example: Digital_Marketing: [0.5, 2.8]
Interpretation:
Very high confidence (>97.5%) that this channel has a positive effect
Minimum plausible effect is 0.5
Maximum plausible effect is 2.8
Can confidently invest in this channel
Business Action: Increase budget allocation to this channel
Negative Evidence
When the entire 95% HDI is negative:
Example: Print_Advertising: [-1.8, -0.3]
Interpretation:
Very high confidence (>97.5%) that this channel has a negative effect
This is unusual and warrants investigation
May indicate measurement issues, cannibalization, or budget waste
Business Action: Investigate data quality, consider reducing or eliminating this channel
Uncertain Evidence
When the 95% HDI crosses zero:
Example: Radio_Advertising: [-0.5, 1.2]
Interpretation:
Cannot confidently determine if effect is positive or negative
Coefficient could be zero (no effect)
Need more data or better measurement
Business Action: Consider A/B testing, gathering more data, or using informative priors based on similar channels
Strong vs Weak Effects
Compare interval widths and means:
Strong Effect: Mean = 4.2, HDI = [3.5, 5.0]
Large mean, narrow interval
High confidence in substantial impact
Weak Effect: Mean = 0.8, HDI = [0.1, 1.6]
Small mean, narrow interval
High confidence in minimal impact
Uncertain Effect: Mean = 2.5, HDI = [-0.5, 5.8]
Moderate mean, very wide interval
Low confidence, need more data
Probability Statements
One of the key advantages of credible intervals is the ability to make direct probability statements.
Probability Above/Below Threshold
MixModeler calculates probabilities for various thresholds:
Probability > 0: Chance the coefficient is positive
Example: P(TV_Coef > 0) = 98.5%
Strong evidence of positive effect
Probability > 1: Chance the coefficient exceeds a specific value
Example: P(Digital_Coef > 1) = 75.2%
Can inform whether effect size meets business requirements
Probability in Range: Chance coefficient falls in specific range
Example: P(2 < Social_Coef < 4) = 60%
Useful for scenario planning
Calculating Custom Probabilities
To calculate probability of any condition:
Access the Bayesian Results panel
Click on a specific coefficient
View the posterior distribution plot
Click "Calculate Probability"
Enter your threshold or range
System computes probability from posterior samples
Business Applications
Budget Allocation: "There's an 85% chance our TV ROI is above $2 per dollar spent."
Risk Assessment: "There's only a 5% chance the new channel will have negative ROI."
Scenario Planning: "There's a 50% chance the effect is between 2 and 3, and a 25% chance it exceeds 4."
Competitive Comparison: "We're 95% confident our social media effect is between 2x and 5x stronger than email."
Visualizing Credible Intervals
Coefficient Plot with Error Bars
Horizontal error bars show HDI for each variable:
Variable |----------o----------|
TV_Advertising |--------o--------|
Digital |-----o------|
Print |---o----|
Social |------o---------|The circle represents the posterior mean, and bars extend to HDI bounds.
Quick Interpretation:
Longer bars = more uncertainty
Bars not crossing zero = significant effects
Position of circle = expected effect size
Posterior Distribution Plot
Shaded area under the curve shows the HDI:
Density
^
|
| /\
| / \
| / \____
|___/ \___
|
+-----------------> Parameter Value
[ HDI ]The shaded region contains 95% of probability mass.
Comparison Plots
When comparing multiple channels, overlapping intervals suggest similar effects:
No Overlap: [2.0, 3.5] vs [4.5, 6.0]
Clear difference between channels
High confidence they have different effects
Partial Overlap: [2.0, 4.0] vs [3.0, 5.5]
Some uncertainty about relative performance
May have similar effects
Complete Overlap: [2.0, 5.0] vs [2.5, 4.8]
Cannot distinguish effect sizes
Insufficient data to compare
Credible Intervals vs Confidence Intervals
Confidence Intervals (Frequentist)
Interpretation: "If we repeated this experiment many times, 95% of the computed intervals would contain the true value."
Challenge: For a single experiment, the true value either is or isn't in the interval - no probability statement possible.
Focus: Long-run frequency properties of the procedure
Credible Intervals (Bayesian)
Interpretation: "There is a 95% probability the true value is in this interval."
Advantage: Direct probability statement for your specific dataset
Focus: Quantifying uncertainty about parameters given observed data
Practical Difference
For most well-powered analyses, confidence and credible intervals are numerically similar. The key difference is interpretability:
Business Question: "What's the probability our TV advertising coefficient is above 2?"
Frequentist: Cannot answer directly (no probability statement about parameters)
Bayesian: Can calculate precisely from posterior samples (e.g., 76%)
Advanced Applications
Sequential Testing
As you gather more data:
Month 1: 95% HDI = [-0.5, 4.5] (wide, uncertain) Month 3: 95% HDI = [0.8, 3.2] (narrowing, emerging pattern) Month 6: 95% HDI = [1.5, 2.7] (narrow, high confidence)
Watch intervals narrow over time as evidence accumulates.
Interval Width as Information Gain
Compare prior and posterior interval widths:
Prior 95% Interval: [-10, 10] (very uncertain) Posterior 95% HDI: [2.0, 4.5] (much narrower)
Information Gain: Data reduced uncertainty by 87.5%
Larger information gain indicates data was highly informative.
ROI Credible Intervals
Transform coefficient intervals into ROI intervals:
Coefficient HDI: [2.0, 4.5] Cost per Impression: $0.05 Average Order Value: $100
ROI HDI: [(2.0 × 100 / 0.05) - 100%, (4.5 × 100 / 0.05) - 100%] = [3,900%, 8,900%]
Interpretation: "We're 95% confident ROI is between 39x and 89x."
Portfolio Optimization
Use credible intervals to assess risk-return tradeoffs:
Channel A: Mean ROI = 5x, HDI = [4x, 6x] (high return, low risk) Channel B: Mean ROI = 8x, HDI = [2x, 14x] (high return, high risk)
Intervals inform risk-adjusted allocation decisions.
Best Practices
Report Intervals Always: Never report just point estimates. Always include credible intervals to communicate uncertainty.
Use HDI by Default: HDI is more informative than ETI, especially for skewed distributions. MixModeler uses HDI by default.
Match Width to Decision: Use 95% for standard reporting, 99% for critical decisions, 90% for less critical contexts.
Check Zero Crossing: Always note whether intervals cross zero - this fundamentally affects interpretation.
Compare Interval Widths: Narrow intervals indicate high precision, wide intervals suggest need for more data or better priors.
Calculate Probabilities: Use posterior samples to calculate probabilities of business-relevant hypotheses, not just report intervals.
Visualize Distributions: Don't rely solely on intervals. Plot full posterior distributions to see skewness, multimodality, or other features.
Document Interpretation: When presenting to stakeholders, clearly explain what credible intervals mean in non-technical language.
Common Mistakes
Ignoring Interval Width: Focusing only on point estimates without considering uncertainty.
Misinterpreting Zero-Crossing: Concluding "no effect" when interval crosses zero without considering probability mass on each side.
Over-Confident Conclusions: Making strong business decisions based on wide, uncertain intervals.
Comparing Without Overlap Analysis: Assuming different means imply different effects without checking interval overlap.
Forgetting Prior Influence: Not considering how informative priors might be narrowing intervals artificially.
Next Steps: Learn about Convergence Diagnostics to ensure your credible intervals are based on reliable MCMC samples, or explore how to export and present your Bayesian results.
Last updated