Methodological Comparison: GeoLift (SparseSC) vs. Time-Based Regression (TBR)

This document provides a rigorous methodological comparison between the Sparse Synthetic Control (SparseSC) approach used in GeoLift and the Time-Based Regression (TBR) approach developed by Google (Kerman, Wang, and Vaver, 2017).

While both methods aim to estimate the causal impact of geo-experiments (e.g., measuring incremental return on ad spend), they rely on fundamentally different econometric foundations. This document outlines why GeoLift’s formulation is generally superior for modern marketing analytics, particularly in environments with complex, non-linear geographic trends.

1. Core Methodological Differences

Time-Based Regression (TBR)

TBR frames the problem as a time-series forecasting task. It aggregates the control geos into a single pool and regresses the aggregate treatment geo time series on the aggregate control time series during the pre-treatment period. The model is typically a linear regression:

TBR relies on Bayesian inference (assuming standard noninformative priors, yielding a shifted t-distribution for the posterior) to predict the counterfactual and compute credible intervals.

GeoLift (SparseSC)

GeoLift frames the problem as a Synthetic Control matching task. Instead of regressing the treatment on an aggregate control, it constructs a synthetic counterfactual by finding a highly regularized, sparse vector of weights $\mathbf{W}$ applied to individual donor geos:

The weights are constrained to be non-negative and sum to one (the simplex constraint), and are selected via L1/L2 regularisation (SparseSC) to minimise out-of-sample prediction error.

2. Why GeoLift (SparseSC) is Methodologically Superior

A. The Extrapolation vs. Interpolation Problem

TBR is an unconstrained linear regression model. If the post-treatment control aggregate falls outside the range observed during the pre-treatment period (e.g., due to seasonality, sudden macro shocks like COVID, or holiday peaks), the linear model will extrapolate. This unconstrained extrapolation is a well-documented source of severe bias in causal inference.

GeoLift, by enforcing the simplex constraint ($w_j \geq 0$, $\sum w_j = 1$), strictly forces interpolation. The synthetic counterfactual is guaranteed to lie within the convex hull of the donor units. The model physically cannot project a counterfactual that exceeds the bounds of the observed donor data, structurally protecting against extrapolation bias during macro shocks.

B. Handling Heterogeneous Geographic Shocks

TBR aggregates the control pool into a single univariate predictor. By doing so, it destroys the underlying covariance structure of the individual geographic markets. If a localized shock hits a subset of the control pool (e.g., a regional competitor promotion or regional weather event), the aggregate predictor is skewed, corrupting the regression parameters.

GeoLift preserves the individual identity of every control geo. Because the regularisation penalty (L1) forces sparsity, the algorithm explicitly selects only the 2 to 5 donor geos that structurally share the same Data Generating Process (DGP) as the treatment geo. It ignores irrelevant control geos, perfectly isolating the counterfactual from heterogeneous shocks occurring in non-selected geos.

C. Overfitting and Pre-Treatment Noise

TBR relies on in-sample regression fit. In environments where the pre-treatment period ($T_0$) is short or highly volatile, ordinary least squares (and its Bayesian equivalent with noninformative priors) will overfit to the pre-treatment noise.

GeoLift uses rolling-origin cross-validation over the pre-treatment period to tune its L1/L2 penalty parameters. It actively penalises complexity (non-sparsity) to minimise out-of-sample Mean Squared Prediction Error (MSPE). This guarantees that the selected donor weights reflect true structural correlation rather than transient noise.

D. The “Small Sample Size” Bayesian Argument

A common argument for TBR is that its Bayesian backend allows for better uncertainty quantification in small samples. While TBR does compute posterior predictive distributions analytically (via the shifted t-distribution), this inference is strictly parametric. If the linear functional form is misspecified, or if the errors are not strictly i.i.d. Normal, the Bayesian credible intervals will have incorrect coverage.

GeoLift completely abandons asymptotic/parametric assumptions for inference. Instead, it computes exact p-values via in-space placebo permutations. By iteratively applying the exact same SparseSC algorithm to every single donor geo as if it were the treatment, GeoLift generates an empirical null distribution. This non-parametric, design-based inference provides exact Type I error control regardless of the sample size or error distribution, making it vastly more robust than parametric Bayesian bounds.

3. Summary

TBR is a capable tool when aggregate control trends perfectly map to treatment trends linearly and no extreme out-of-sample shocks occur.

However, GeoLift (SparseSC) provides a structurally safer and more robust estimation environment. By strictly interpolating (via simplex constraints), isolating geographic heterogeneity (via L1 sparsity), preventing overfitting (via out-of-sample CV), and controlling Type I error non-parametrically (via placebo inference), GeoLift systematically closes the methodological vulnerabilities inherent in TBR.