Survival Analysis Fundamentals

This page covers the statistical theory behind the Survival Analysis tabs. See that page for usage instructions.

Time-to-Event Data and Censoring

Survival analysis is a set of methods for analyzing time until an event occurs. Despite the name "survival," the event need not be death — it can be machine failure, customer churn, time to recidivism, or any event of interest.

The defining feature of survival data is censoring. Subjects who did not experience the event during the observation period (e.g., patients still alive at the end of a clinical trial, or patients lost to follow-up) carry only incomplete information: "the event had not occurred by at least this time."

Simply excluding censored observations biases the analysis toward subjects who experienced the event sooner, underestimating survival times. Treating censored observations as "no event" overstates survival times since the true event time is unknown. Survival analysis methods are designed to handle censoring properly, provided that censoring is independent of event occurrence (non-informative censoring). When censoring is related to the likelihood of the event — for example, when patients drop out due to worsening side effects — the KM estimator and Cox model estimates become biased. MIDAS handles right censoring only (censoring due to end of observation or loss to follow-up). Left censoring and interval censoring are not supported.

Why Ordinary Regression Fails

Without censoring, survival times could be analyzed as a response variable in ordinary regression. But censored data provides inequality information — "the true value is at least as large as the observed value" — and the usual residual ( $y_i - \hat{y}_i$ ) cannot be defined. Survival analysis incorporates this inequality into the likelihood function, correctly accounting for censoring.

Survival Function and Hazard Function

The distribution of survival time $T$ is characterized by two functions.

The survival function $S(t) = P(T > t)$ is the probability of not having experienced the event by time $t$ . It starts at $S(0) = 1$ and decreases monotonically over time.

The hazard function $h(t)$ is the instantaneous rate of event occurrence at time $t$ , given survival up to that point:

h(t) = \lim_{\Delta t \to 0} \frac{P(t \le T < t + \Delta t \mid T \ge t)}{\Delta t}

The hazard is a rate (per unit time), not a probability, so it can exceed 1. The survival and hazard functions are related by $S(t) = \exp\left(-\int_0^t h(u)\,du\right)$ ; knowing one determines the other.

Kaplan-Meier Estimator

The Kaplan-Meier estimator is a nonparametric estimator of the survival function. It makes no distributional assumptions, estimating $S(t)$ directly from observed event times.

Let the distinct event times be $t_1 < t_2 < \cdots < t_k$ , with $n_i$ subjects at risk and $d_i$ events at each time $t_i$ :

\hat{S}(t) = \prod_{t_i \le t} \left(1 - \frac{d_i}{n_i}\right)

This cumulatively multiplies the "survival fraction" at each event time. Censoring is reflected through changes in the risk set: when a subject is censored, they leave the risk set but are not counted as an event.

Under non-informative censoring, the KM estimator is a consistent estimator of $S(t)$ . The variance is estimated using Greenwood's formula, derived via the delta method:

\widehat{\operatorname{Var}}[\hat{S}(t)] = \hat{S}(t)^2 \sum_{t_i \le t} \frac{d_i}{n_i(n_i - d_i)}

Confidence intervals are constructed from this variance. MIDAS uses the log transformation method by default, computing $\exp(\log \hat{S}(t) \pm z \cdot \text{SE} / \hat{S}(t))$ . The log transformation prevents the interval from falling outside $[0, 1]$ .

Log-rank Test

The log-rank test is a nonparametric test for whether two or more groups have the same survival curve.

At each event time $t_i$ , the observed event count $d_{ij}$ and expected event count $e_{ij} = n_{ij} \cdot d_i / n_i$ are computed for group $j$ , where $n_{ij}$ is the size of group $j$ 's risk set at time $t_i$ . Under the null hypothesis, events at each time point are expected to be distributed according to each group's share of the risk set. This allocation follows a hypergeometric distribution, from which the variance is also derived. For two groups, since $O_1 + O_2 = \sum d_i$ and $E_1 + E_2 = \sum d_i$ , it follows that $O_1 - E_1 = -(O_2 - E_2)$ , so only one group's information is needed. The test statistic is:

\chi^2 = \frac{(O_1 - E_1)^2}{V_1}

where $O_1 = \sum_i d_{1i}$ (total observed events in group 1), $E_1 = \sum_i e_{1i}$ (total expected events), and $V_1 = \sum_i \frac{n_{1i} \cdot n_{2i} \cdot d_i (n_i - d_i)}{n_i^2(n_i - 1)}$ (variance). The denominator is the variance derived from the hypergeometric distribution, not the expected value $E_1$ . For three or more groups, this extends to the quadratic form $\chi^2 = \mathbf{U}' \mathbf{V}^{-1} \mathbf{U}$ using the variance-covariance matrix. Under the null hypothesis, this statistic approximately follows a chi-squared distribution with $g - 1$ degrees of freedom ( $g$ = number of groups).

The log-rank test is a member of the weighted log-rank test family. It has maximum power compared to other weighted variants (such as Wilcoxon-type tests) when the hazard ratio is constant over time (under the proportional hazards assumption). Power decreases when survival curves cross.

Cox Proportional Hazards Model

Model Formulation

The Cox (1972) proportional hazards model is a semiparametric model that estimates the effect of covariates on hazard:

h(t \mid X) = h_0(t) \exp(\beta_1 X_1 + \beta_2 X_2 + \cdots + \beta_p X_p)

$h_0(t)$ is the baseline hazard (the hazard when all covariates are zero), and $\exp(\beta_j)$ is the hazard ratio for a one-unit increase in covariate $X_j$ .

It is called "semiparametric" because $\beta$ is estimated parametrically, but no functional form is specified for $h_0(t)$ . This removes the need to assume a distribution for the baseline hazard. After estimating $\beta$ , the baseline hazard $h_0(t)$ can be estimated nonparametrically using the Breslow estimator, which in turn allows computing the survival function $S(t|X)$ for specific covariate values. MIDAS does not currently output baseline hazard estimates.

The covariates $X$ in this model are fixed values for each subject throughout the observation period. Handling covariates that change over time (time-varying covariates) requires extensions that MIDAS does not currently support.

Proportional Hazards Assumption

The core assumption is that covariate effects are constant over time. That is, the hazard ratio $h(t \mid X_1) / h(t \mid X_2) = \exp(\beta'(X_1 - X_2))$ does not depend on $t$ .

When this assumption is violated (e.g., a treatment effect that fades over time), the estimated $\beta$ represents a weighted average of the time-varying effect, with weights that depend on the risk set composition and baseline hazard, making interpretation difficult (Struthers & Kalbfleisch, 1986). Methods for checking the proportional hazards assumption include Schoenfeld residual plots and log-log plots, but MIDAS does not currently include these diagnostic tools.

Partial Likelihood

Cox model parameters are estimated using partial likelihood. For subject $i$ who experienced an event at time $t_{(i)}$ , consider the conditional probability that subject $i$ — among all subjects still at risk at that time $\mathcal{R}(t_{(i)})$ — is the one who experiences the event:

L(\beta) = \prod_{i:\text{event}} \frac{\exp(X_i'\beta)}{\sum_{j \in \mathcal{R}(t_{(i)})} \exp(X_j'\beta)}

Each factor corresponds to the conditional probability $h(t_{(i)}|X_i) / \sum_j h(t_{(i)}|X_j)$ within the risk set at time $t_{(i)}$ . Substituting $h(t|X) = h_0(t)\exp(X'\beta)$ , the $h_0(t_{(i)})$ terms cancel between numerator and denominator, so estimating $\beta$ does not require knowing $h_0(t)$ . Although the partial likelihood is not a full likelihood, it has been shown to yield estimators with the same asymptotic properties as maximum likelihood — consistency and asymptotic normality (Cox, 1975).

Interpreting Hazard Ratios

$\exp(\beta_j)$ is interpreted as the hazard ratio (HR):

HR > 1: A one-unit increase in $X_j$ increases the hazard by $(\text{HR} - 1) \times 100\%$
HR < 1: The hazard decreases by $(1 - \text{HR}) \times 100\%$
HR = 1: $X_j$ has no effect on the hazard

When the confidence interval for the hazard ratio does not include 1, the data are inconsistent with the null hypothesis of no effect. The width of the confidence interval reflects estimation precision: a narrow interval indicates a more precise estimate, while a wide interval indicates limited information from the data. Hazard ratios directly convey the direction and magnitude of the effect, making them more informative than the p-value alone.

Tied Events

When multiple subjects experience events at the same time, exact partial likelihood computation becomes combinatorially difficult. MIDAS uses the Efron (1977) approximation, which progressively removes tied subjects from the risk set denominator, providing a more accurate approximation to the exact partial likelihood than the Breslow method.

References

Cox, D. R. (1972). Regression models and life-tables. Journal of the Royal Statistical Society: Series B, 34(2), 187-220.
Kaplan, E. L., & Meier, P. (1958). Nonparametric estimation from incomplete observations. Journal of the American Statistical Association, 53(282), 457-481.
Cox, D. R. (1975). Partial likelihood. Biometrika, 62(2), 269-276.
Struthers, C. A., & Kalbfleisch, J. D. (1986). Misspecified proportional hazard models. Biometrika, 73(2), 363-369.
Efron, B. (1977). The efficiency of Cox's likelihood function for censored data. Journal of the American Statistical Association, 72(359), 557-565.