Fig. 1

Evaluation of type I errors of \(W_{d}^{*}\) and PERMANOVA permutation tests. Simulation under the null hypothesis results for comparison of \(W_{d}^{*}\) (Wstar), PERMANOVA (Permanova), and distribution-based Welch ANOVA F (WelchF) tests are presented. In panel a, we evaluate the fraction of null hypotheses that have been rejected by each test at α=0.05. The subpanels of a correspond to simulated datasets with corresponding number of samples in the non-reference groups, with columns corresponding to the least dispersed and rows corresponding to the most dispersed sample. In panel b, the raw p values from \(W_{d}^{*}\) test are plotted against those for the same data with Welch ANOVA F test. In panel c, we do the same for PERMANOVA p values and color the points by respective degree of heteroscedasticity in the simulated dataset