The ANOVA Tab

The ANOVA tab analyzes whether the means of a response variable differ across groups defined by categorical variables. Both one-way and two-way designs are supported. It assumes independent, between-subjects observations and does not support repeated measures or paired data—for example, the same patient measured before and after treatment or multiple samples from the same unit. For those designs, use a GLMM with the subject as the random-effect group.

Basic Usage

Open the Tab

Select Analysis > ANOVA... from the menu bar.

Run an Analysis

Configure the following in the settings panel:

1. Select a dataset from Dataset
2. Choose One-Way or Two-Way under Analysis Type
3. Select a categorical variable for Factor A (Categorical)
4. Select a numeric variable for Response Variable (Numeric)
5. Click Run Analysis

Changing the variable selection or the analysis settings after a run makes the displayed results no longer belong to the current settings. MIDAS removes those results from the view and shows a message asking you to run the analysis again. Restoring the previous settings brings the results back. Changing Dataset also resets the variable selection, so the results are discarded instead. Confidence Level is one of the analysis settings, because the Tukey HSD simultaneous confidence intervals are computed at the confidence level in effect at run time and there is no way to recompute them for a different level alone.

The displayed results also stop belonging to the current data when the dataset used for the analysis changes after a run. Reloading the dataset, editing cells, and changing an upstream dataset all count as such changes. MIDAS removes the results from the view and shows a message saying the dataset was modified. In this case there is no setting to restore, so run the analysis again to get results for the current data.

Data Format

Data must be in long format with one row per observation. Each row contains the factor value and the response variable value. Use Reshape to convert wide-format data.

One-Way ANOVA

Analyzes differences in the response variable means across groups defined by a single categorical factor. Use this when you have one grouping factor.

Statistical Model

$y_{ij} = \mu + \alpha_i + \varepsilon_{ij}$

$y_{ij}$ is the $j$-th observation in group $i$, $\mu$ is the overall mean, $\alpha_i$ is the effect of group $i$, and $\varepsilon_{ij}$ is the error term.

Assessing Group Differences

ANOVA decomposes the total variation in the response variable into between-group variation and residual variation. The decomposition appears as SS, df, and MS in the ANOVA table. Of the effect sizes, η² shows the proportion of the total variation attributable to the factor in the sample, and ω² is an estimate of that proportion in the population. The magnitude and precision of group differences can be read from the simultaneous confidence intervals for pairwise differences from Tukey HSD. The group mean confidence intervals in Group Statistics show the precision of each group mean estimate.

Variable Selection

Which columns can be selected as factors and as the response variable follows the column's measurement scale. Columns that do not meet the requirement still appear in the list, but are grayed out and cannot be selected. A short reason is appended after the column name.

Factor A (Categorical) and Factor B (Categorical) take a categorical variable that defines the groups. Columns with a nominal or ordinal scale can be selected. When an ordinal column is selected, ANOVA estimates no ordered contrast such as a linear trend and groups the levels the same way it groups a nominal column. For an enum column, the order of the enum definition sets the row order in Group Statistics and the direction of the Tukey HSD differences. That order comes from the enum definition, not from the measurement scale.

A numeric column that encodes categories, such as a dose code or a batch number, can also be used as a factor. Right-click the column header in Data Table and change its scale to nominal or ordinal from Edit Scale of Measurement. Each distinct value of the column then forms one group, and the groups are ordered as text, so 10 comes before 2. Do not do this with a measured quantity. Nearly every value would form a group of its own, leaving no degrees of freedom for the residuals.

Response Variable (Numeric) takes the numeric variable to analyze. Numeric columns with an interval or ratio scale can be selected. String and enum columns cannot be used as the response variable because their values are stored as text, and their scale cannot be changed to interval or ratio. When numbers are stored as text, convert the column to a numeric type in the Convert Column Types tab first, then select it.

A boolean column has a nominal scale by default and therefore cannot be selected as it is. Changing its scale to ratio makes it selectable, and its values are read as 0 and 1. In a 0/1 response the variance within a group is determined by that group's mean, so groups with different proportions do not have equal variances. Intervals computed from the pooled MSE can fall outside 0 and 1. To work with proportions, use a GLM with the binomial family.

Date and datetime columns cannot be used as the response variable even though their scale is interval. To analyze an elapsed duration, build a numeric column of days from a baseline date in the SQL Editor and select that column.

The measurement scale belongs to the column itself, so changing it here also affects graph axis types and the options offered by the other analysis tabs.

When a selected column stops meeting the requirement, the settings panel shows the reason and Run Analysis becomes unavailable. Reloading the dataset or changing the column's scale can produce this state, as can pointing Factor A and Factor B at the same column in a two-way design. Select another column, or restore the scale of that column.

Example

To analyze whether sepal length differs among the three species Iris-setosa, Iris-versicolor, and Iris-virginica in the Iris sample data:

1. Dataset: Iris
2. Analysis Type: One-Way
3. Factor A: species
4. Response Variable: sepal_length
5. Click Run Analysis

Confidence Level

Set the confidence level for Group Statistics confidence intervals and Tukey HSD post-hoc comparisons. Choose from 90%, 95% (default), or 99%. The Tukey HSD confidence interval width matches the confidence level. This setting also appears in two-way ANOVA, where it applies to the Group Statistics confidence intervals. Changing the confidence level after a run removes the results from the view until you run the analysis again (see Run an Analysis).

Two-Way ANOVA

Analyzes the effects of two categorical factors and their interaction on the response variable. Use this when you have two grouping factors.

Statistical Model

With interaction:

$y_{ijk} = \mu + \alpha_i + \beta_j + (\alpha\beta)_{ij} + \varepsilon_{ijk}$

$\alpha_i$ is the effect of factor A, $\beta_j$ is the effect of factor B, and $(\alpha\beta)_{ij}$ is the interaction effect.

Additional Settings

Factor B (Categorical): Select a second categorical variable, different from Factor A.

Include interaction term (A x B): Whether to include the interaction term in the model. Enabled by default. Include the interaction when the effect of one factor may depend on the level of the other. If the interaction is known to be absent, excluding it increases the residual degrees of freedom, providing a more precise estimate of the error variance.

Sum of Squares Type: Choose the method for computing sums of squares.

Sum of Squares Types

Type I computes sums of squares sequentially based on the order factors enter the model. Each factor's contribution depends on which factors are already in the model. MIDAS enters Factor A first, then Factor B, then the interaction term. The SS for Factor A is the reduction in residual sum of squares when Factor A is added to the intercept-only model, and the SS for Factor B is the further reduction when Factor B is added. Swapping the Factor A and Factor B assignments changes the results.

Type III computes sums of squares for each factor as if it were the last one entered. Each factor's contribution is adjusted for all other factors.

For balanced designs (equal sample sizes in all cells), Type I and Type III produce identical results. For unbalanced designs, Type III is generally preferred because results do not depend on factor ordering. The default is Type III. Type I is useful when the order of factor entry is meaningful—for example, when adjusting for a covariate before assessing a treatment factor.

Type III Interpretation with Interaction

MIDAS uses treatment coding. Treatment coding designates one level of each factor as the reference category (baseline level) and expresses the effects of all other levels as differences from it. The reference category is the first level when the level names are sorted as text in Unicode code point order. Enum definition order and numeric magnitude are not used for this ordering. Uppercase letters sort before lowercase letters, so with levels control and Treatment, the reference category is Treatment. When the interaction term is included, the Type III main effect for each factor depends on the treatment coding reference category. For example, if Factor A has levels A, B, C and Factor B has levels X, Y, the reference categories are A and X (the first level in code point order). The Type III sum of squares for Factor A represents the additional explanatory power of Factor A's levels compared to a model without them, evaluated when Factor B is at reference level X. With balanced data, this coincides with the effect averaged across all levels of Factor B. With unbalanced data, the two may differ.

Reading the Results

Observations

The total number of observations used in the analysis appears at the top. If rows were excluded, the count of excluded rows is also shown.

Group Statistics

A summary table of descriptive statistics for each group.

Some excluded rows do not appear in the Excluded column: rows with a missing factor value belong to no group, and rows of a level or cell removed entirely by exclusion have no row in the table. When such rows exist, their counts appear as notes below the table.

ANOVA Table

The main results table. Decomposes the total variance of the response variable into contributions from each factor and residual error. This table evaluates the magnitude of group differences using effect sizes (η² and ω²). It does not display test statistics (F-values) or p-values. The magnitude and precision of each pairwise difference can be assessed from the simultaneous confidence intervals in Tukey HSD.

When the variation in the response is below floating-point precision relative to its magnitude, the sum-of-squares decomposition is meaningless. This covers responses whose values are all identical, as well as responses with a tiny relative spread on top of a large offset. SS, MS, η², ω², and the Total sum of squares are then undefined and shown as "-", Tukey HSD and the residual Q-Q plots are not computed, and a warning explains why.

When the response does vary but the model explains it completely, leaving a residual sum of squares below floating-point precision relative to the response variation (for example, when the values within each group are identical), the error variance cannot be estimated. The residual MS, ω², and the confidence intervals of the group means in Group Statistics require an error estimate and are shown as "-", η² is shown as "-" because it is fixed at 1 when the residual sum of squares is zero, Tukey HSD is not computed, the residual Q-Q plots in Assumption Diagnostics are replaced by a note because residuals at rounding-noise level carry no information about the distribution of errors, and a warning explains why. The sums of squares and the group means are still shown, but without an error estimate the data gives no basis for judging whether the differences between groups exceed sampling variation.

Tukey HSD Post-Hoc Comparisons

The ANOVA table decomposes overall variation among groups, but does not show how large the difference is for each pair. Tukey HSD post-hoc comparisons estimate the mean difference and its simultaneous confidence interval for every pair of groups, allowing you to assess the magnitude and precision of each pairwise difference.

For one-way ANOVA, Tukey HSD is computed automatically. The Tukey-Kramer method is used, which handles unequal group sizes. Tukey HSD is not available for two-way ANOVA.

Tukey HSD constructs simultaneous confidence intervals for all pairwise mean differences. Constructed individually, the pairwise intervals would jointly cover all the true differences with probability below the nominal level; Tukey HSD widens the intervals so that this simultaneous coverage probability stays at or above the nominal level.

The interval center (the Diff value) is the point estimate of the difference, and the interval width reflects estimation uncertainty. Read the point estimate for the magnitude of the difference and the interval width for how precisely it is estimated.

Tukey HSD uses the pooled MSE, so it assumes equal population variances. When group variances differ substantially, the actual coverage of these intervals may deviate from the nominal level. Compare the group standard deviations in Group Statistics to check this assumption.

The critical value $q_{\text{critical}}$, MSE, residual degrees of freedom, and the interval formula are displayed below the table. The half-width of the simultaneous confidence interval is $q_{\text{critical}} \times \text{Std. Error} / \sqrt{2}$. The division by $\sqrt{2}$ arises because the studentized range quantile $q_{\text{critical}}$ is defined on the scale of the range statistic, not the mean difference.

Assumptions

ANOVA assumes the following. Verify that these are reasonable when interpreting results.

• Independence: Observations are independent of each other
• Normality: The response variable follows a normal distribution within each group
• Homogeneity of variance: The variance is equal across all groups

Assumption Diagnostics

ANOVA assumes normality within each group. Two types of Q-Q plots help assess this assumption.

The pooled residual Q-Q plot compares all residuals against a theoretical normal distribution. Points falling close to the diagonal line suggest approximate normality. Departures such as heavy tails or skewness are visible in the shape of the deviation from the line. Because this plot combines residuals from all groups, it can mask departures that affect only some groups.

The per-group Q-Q plots show the residual distribution within each group separately. These provide a visual check of the normality assumption, since ANOVA assumes normality within each group. When there are more than 12 groups, only the first 12 (in the same order as the Group Statistics table) are displayed.

When homogeneity of variance holds, the estimates of group mean differences and their confidence intervals become robust to departures from normality as sample sizes increase. If Q-Q plots show departures from normality, check homogeneity of variance first.

Homogeneity of variance can be assessed by comparing the SD values in the Group Statistics table. When group SDs are similar, minor departures from normality are tolerable. When SDs differ substantially across groups, this robustness does not hold. Consider using GLM, which can explicitly model the variance structure.

In two-way ANOVA, residuals are computed from the fitted values of the selected model. When the interaction term is included, the model fits a separate mean for each cell, so residuals equal deviations from cell means. When the interaction term is excluded, the main-effects-only model produces different fitted values, and the residuals reflect deviations from those predictions. The choice of model affects both the pooled and per-group Q-Q plots.

Add to Report

Each result section (Group Statistics, ANOVA Table, Assumption Diagnostics, and Pairwise Mean Differences (Tukey HSD)) has an Add to Report button in its header. Click it to add that section to a report. Sections are added independently, so you can send analysis results and diagnostic plots to separate reports.

Elements added to a report retain the original dataset and analysis settings. When the data changes, the results are recomputed.

Error Messages

If a referenced column does not meet the scale requirements described in Variable Selection, an error containing "measurement scale" is displayed. The message shows the column name, its current scale, and the required scales. If the response variable refers to a date, datetime, string, or enum column, an error containing "cannot use as a numeric variable" is displayed instead. Because the ANOVA tab does not let you select ineligible columns, these errors mainly appear when re-estimating a saved model or recomputing a report element after the column's scale was changed. Restore the column's scale, or rerun the analysis with a column that meets the requirements.

If fewer than 3 valid observations remain, an error beginning with "Not enough valid observations" is displayed. The message shows how many rows remained after excluding missing and invalid values. When exclusion removed entire levels, the message also names those levels.

If a factor has only one level among the valid observations, an error containing "must have at least 2 levels with valid observations" is displayed. This means all values in the factor column are identical, or only one level remained after exclusion. When exclusion removed entire levels, the message also names those levels. Comparing groups requires at least two levels.

If there are too few observations relative to the number of factor levels, no degrees of freedom remain for estimating the error variance, and an error beginning with "Too few observations" is displayed. The message shows the minimum number of observations required. When exclusion removed entire levels, the message also names those levels. In two-way ANOVA with the interaction term, turning it off reduces this requirement.

When the residual degrees of freedom is 2 or fewer, MIDAS shows a warning with the results rather than an error. Confidence intervals and effect size estimates computed from MSE cannot be trusted at this level. See Residual Degrees of Freedom and the Variability of the Error Variance Estimate for why, and how to address it.

In two-way ANOVA with the interaction term, every combination of factor levels must have at least one observation. If any cell is empty, an error beginning with "Empty cell detected" is displayed, indicating which combination is missing. When the cell became empty because its rows were excluded for missing or invalid values, the error says so. Turn off the interaction term or check whether your data has empty cells.

If factor levels have perfect collinearity, the error "The design matrix is rank deficient. This may occur when factor levels have perfect collinearity or insufficient observations." is displayed.

If the design matrix is ill-conditioned (a high estimated condition number), MIDAS shows a warning with the results rather than an error. This is more likely when the cell counts across factor-level combinations are highly unbalanced or some cells are nearly empty, and it lowers the numerical precision of the effect estimates. See Condition Number for what it means and how to address it.

If a sum of squares comes out negative from numerical error during the variance decomposition, MIDAS replaces it with 0. A warning appears only when the negative value exceeds the range of rounding error. The warning identifies which source it occurred for. This happens when the factor-level combinations are nearly collinear or the response contains extreme values, and the sum of squares, mean square, and effect sizes for that source are not numerically reliable. A negative value within the range of rounding error arises when the source's sum of squares is exactly 0. In that case MIDAS replaces it with 0 and shows no warning.

Missing and Invalid Values

Rows containing missing values (null), empty or blank factor cells, or Infinity or NaN response values are automatically excluded. The number of excluded rows is displayed in the results panel. For two-way ANOVA, rows with any such value in either factor or the response variable are excluded. This is listwise deletion (complete-case analysis). See Missing Data Mechanisms for conditions under which it yields valid estimates.

The per-group breakdown of exclusions appears in the Excluded column of Group Statistics. One-way ANOVA counts exclusions per group; two-way ANOVA counts them per combination of factor levels (cell). When exclusions concentrate in a particular group, that group's number of valid observations shrinks, its mean's confidence interval widens, and the design becomes more unbalanced. Concentration is also a hint that missingness is related to the factor. Whether the estimates after exclusion are biased depends on the missingness mechanism and cannot be determined from the counts; see Missing Data Mechanisms for how to reason about it.

If all rows of a factor level are excluded, that level is not included in the analysis, and a warning names the level and the number of excluded rows. In two-way ANOVA, when both factor levels survive but all rows of a particular cell are excluded, a warning appears for models without the interaction term. Models with the interaction term cannot handle empty cells, so instead of a warning this produces an error stating that exclusion caused the empty cell.

Related Pages

• Linear Regression -- the ANOVA table in the regression tab tests the overall model fit, while this tab uses categorical factors

Also available as a Markdown file.