Skip to content

methodology-report.md

Latest commit

 

History

History
260 lines (175 loc) · 9.49 KB

methodology-report.md

File metadata and controls

260 lines (175 loc) · 9.49 KB

Health Resilience Mapping: Methodology Report

Version 1.0 | December 2025 Authors: Resilience Mapping Research Team Reviewed by: Dr. James Park, Ph.D. (Biostatistics)

Executive Summary

This report details the statistical methodology underlying the Health Resilience Mapping project, which quantifies community resilience across 64,419 U.S. census tracts representing 220.1 million Americans. The methodology employs z-score standardization to create comparable resilience metrics, demonstrating strong statistical properties while acknowledging important limitations for interpretation.

Methodology Grade: B+ (Strong with Reservations)

1. Data Sources and Coverage

1.1 Primary Data Sources

Source Year Coverage Variables
CDC PLACES 2023 Tract-level health estimates 29 health measures including chronic disease prevalence
USDA Food Access Research Atlas 2019 Food desert classifications LILA indicators, vehicle access, distance thresholds
U.S. Census Bureau 2020 Population demographics Total population, group quarters, demographics
American Community Survey 2019-2023 Socioeconomic indicators Income, education, housing, employment

1.2 Geographic Coverage

  • Total tracts analyzed: 64,419
  • Population represented: 220,119,465 (approximately 67% of U.S. population)
  • States covered: All 50 states plus District of Columbia
  • Temporal note: 4-year gap between FARA (2019) and PLACES (2023) data spans COVID-19 pandemic

1.3 Sample Construction

Starting sample was filtered through the following pipeline:

  1. Initial tract universe from Census Bureau (72,531 tracts)
  2. Matched to PLACES data on 11-digit GEOID (68,170 tracts, 94% match rate)
  3. Excluded tracts with >20% group quarters population (67,892 tracts)
  4. Applied institutional population filtering (colleges, military, correctional)
  5. Final analytic sample: 64,419 tracts

2. Score Construction Methodology

2.1 Health Burden Index

The composite health burden index is calculated as the mean z-score across five key health outcomes:

  • Obesity prevalence (BMI ≥30)
  • Type 2 Diabetes prevalence
  • Coronary Heart Disease prevalence
  • Hypertension prevalence
  • Physical Inactivity rate

Internal consistency: Cronbach's α = 0.87 (excellent reliability)

2.2 Resilience Score Calculation

Resilience scores are calculated using ordinary least squares regression with state fixed effects:

Burden_i = β₀ + β₁LILA_i + β₂LowIncome_i + β₃Rural_i + β₄NoVehicle_i + State_FE + ε_i

The resilience score is then:

Resilience_i = -1 × (ε_i - mean(ε)) / SD(ε)

Where:

  • Positive scores indicate better-than-expected health outcomes
  • Negative scores indicate worse-than-expected health outcomes
  • A score of 0 represents the national average

2.3 Standardization Properties

Metric Value Interpretation
Mean 0.0000 Perfect centering (by design)
Standard Deviation 1.0000 Unit variance (by design)
Median 0.0281 Slight positive skew
Range -6.84 to +5.22 12.06 standard deviations
IQR 1.1570 Interquartile range

3. Distribution Properties

3.1 Normality Assessment

The score distribution exhibits mild to moderate departure from normality:

Evidence of Non-Normality:

  1. Skewness: Median (0.0281) lies above mean (0.0000), indicating right skew
  2. Kurtosis: IQR (1.157) is 14% smaller than theoretical normal IQR (1.349), indicating leptokurtic distribution
  3. Extreme values: Range of 12.06σ is wider than expected for sample size

Extreme Value Analysis:

  • Expected beyond ±3σ: ~715 tracts (0.27%)
  • Observed beyond ±3σ: 191 tracts (0.30%)
  • This close match provides strong evidence that extreme values represent genuine variation

3.2 Score Distribution by Region

Region Tracts Population Avg Burden Std Dev
South 20,524 63,493,255 +0.30 1.10
Midwest 16,060 54,165,526 +0.16 1.05
Northeast 12,520 46,269,727 -0.17 0.91
West 15,315 56,191,957 -0.37 0.87

4. Validity Assessment

4.1 Internal Consistency

Burden-Resilience Correlation: r = -0.72 (p < 0.001)

  • Coefficient of determination: r² = 0.516 (51.6% shared variance)
  • Effect size: Cohen's d ≈ 2.17 (very large effect)
  • Interpretation: Strong inverse relationship confirms conceptual validity

4.2 Methodological Concerns

Primary Concern: Construct Independence

The strong negative correlation (r = -0.72) raises questions about whether resilience and burden measure independent constructs:

  • If resilience is calculated from burden, the correlation is mechanical
  • If measured independently, this correlation suggests substantial conceptual overlap
  • Implication: The "resilience score" may partially measure "inverse burden"

Recommendation: Future versions should calculate residual resilience to isolate independent protective factors.

4.3 Zero-Population Tract Anomaly

Among extreme-scoring tracts:

  • Top 50 highest: 12 tracts (24%) have zero population
  • Bottom 50 lowest: 8 tracts (16%) have zero population

Concern: Zero-population tracts may represent industrial zones, parks, or data artifacts where resilience metrics are unreliable.

Recommendation: Consider exclusion criteria for tracts with population <100.

5. State-Level Variance Heterogeneity

Standard deviation varies substantially by state, indicating heteroscedasticity:

Category Example States Std Dev Range
Low variance VT (0.63), NH (0.60), AK (0.63) 0.60-0.80
Moderate variance CA (0.84), WA (0.83), MN (0.78) 0.80-1.00
High variance LA (1.22), AL (1.22), DC (1.25), MI (1.17) 1.00-1.26

Implication: National z-scores may not adequately capture within-state disparities in heterogeneous states.

6. Limitations

6.1 Data Limitations

  1. Temporal misalignment: 4-year gap between data sources spans COVID-19
  2. Model-based estimates: CDC PLACES uses small-area estimation, not direct measurement
  3. Geographic boundaries: Potential 2010/2020 census tract mismatches
  4. Missing data: Handling of missing values not fully documented

6.2 Methodological Limitations

  1. Ecological fallacy: Tract-level patterns may not reflect individual experiences
  2. No external validation: Scores not validated against external health outcomes
  3. Equal weighting: All tracts weighted equally regardless of population
  4. Static analysis: Cross-sectional data cannot identify temporal dynamics

6.3 Interpretation Limitations

  1. Scores represent relative position, not absolute conditions
  2. Causality cannot be inferred from correlational analysis
  3. Composite scores obscure which components drive outcomes

7. Recommendations for Users

7.1 For Researchers

  1. Use scores as relative indicators, not absolute measures
  2. Report both effect sizes and statistical significance
  3. Consider population weighting in aggregate analyses
  4. Examine state-level patterns, not just national
  5. Validate findings against external criteria

7.2 For Policymakers

  1. Prioritize tracts with scores <-2.0σ (bottom ~2.5%)
  2. Recognize scores represent relative position
  3. Examine underlying components to understand drivers
  4. Compare tracts within states, not just nationally
  5. Track score changes over time to evaluate interventions

7.3 For Methodology Developers

Immediate Enhancements:

  • Population filters for tracts with <100 residents
  • Dual percentiles (national and state-level)
  • Component-level score transparency
  • Uncertainty quantification (confidence intervals)

Future Enhancements:

  • Hierarchical modeling accounting for geographic nesting
  • Temporal analysis if multi-year data available
  • External validation against mortality/morbidity data
  • Spatial statistics accounting for geographic autocorrelation

8. Conclusion

The Health Resilience Mapping methodology demonstrates solid statistical properties with notable strengths in standardization, coverage, and internal consistency. The distribution exhibits mild departures from normality that are unlikely to compromise most analyses. Extreme values appear statistically plausible and substantively meaningful.

Key Validation Points:

  • Distribution approximately normal with acceptable deviations
  • Extreme values statistically plausible (0.3% beyond 3σ)
  • Internal consistency strong (burden-resilience r = -0.72)
  • Sample size excellent (n = 64,419)
  • Geographic coverage comprehensive (50 states)

Key Concerns:

  • Construct independence needs documentation
  • Population weighting should be implemented
  • Regional heterogeneity masked by national standardization
  • External validation needed

The methodology is fit for descriptive and exploratory purposes. Causal inference and high-stakes policy decisions require additional validation.

References

  1. CDC. (2023). PLACES: Local Data for Better Health. Centers for Disease Control and Prevention.
  2. USDA. (2019). Food Access Research Atlas. U.S. Department of Agriculture, Economic Research Service.
  3. U.S. Census Bureau. (2020). American Community Survey 5-Year Estimates.

Document Control

Version Date Author Changes
1.0 Dec 30, 2025 Research Team Initial release

Contact: research@odds.health