Statistical Test Selection Tree — Free Decision Questionnaire

This tool recommends a statistical test from four answers: whether your outcome is continuous or categorical, how many groups you compare, whether samples are paired or independent, and whether the distribution is approximately normal. Picking the test is the step students most often get wrong, and the wrong test can produce a p-value that looks fine while answering a different question. The tree covers the standard first-line tests: t-tests, Mann-Whitney, Wilcoxon, ANOVA, Kruskal-Wallis, Friedman, chi-square, and McNemar. It recommends; it does not run the test on your data, and it cannot replace checking assumptions.

How to use it

Choose continuous or categorical outcome.
State how many groups you compare.
Say whether the samples are paired or independent.
Judge the distribution: approximately normal, or not normal / ordinal.
Click Recommend a test and note the assumptions to check.

Worked example

You compare exam scores for two independent classes and the scores look roughly bell-shaped. The tree recommends the independent two-sample t-test. If the same students were tested before and after a course, the answers change to paired, and the recommendation becomes the paired t-test — a different test, because the pairing carries information the independent test would waste. If instead the outcome is pass or fail counts for two classes, the tree recommends the chi-square test of independence. Three different designs, three different tests, all from four clicks.

Honest limits

The tree covers common designs only. It does not handle regression, mixed models, survival analysis, clustered data, or multiple outcomes, and it cannot judge normality for you — with small samples, normality tests themselves are unreliable, so plot the data and judge the shape. Equal variance matters for t-tests and ANOVA; if variances differ wildly, use Welch’s version or a robust alternative. If your design is not in the tree, that is a signal to consult a statistician, not to force the closest match.

Who it is for

This tool is for students choosing a test for an assignment or thesis, researchers sanity-checking a planned analysis, and supervisors reviewing a methods section. Use it at the design stage, before data collection, when switching tests is still free. Choosing the test after seeing the results is a form of p-hacking, however innocent it feels.

Going further

Checking assumptions is a habit, not a checkbox. For a t-test, plot both groups and look at shape and spread: strong skew, heavy tails, or wildly unequal variances push you toward Welch’s t-test or a rank-based alternative, and with very small samples no plot can rescue the decision, so a rank test is often the safer default. For ANOVA, the same logic scales up: check residuals, not raw scores, and follow a significant result with planned comparisons or a correction such as Tukey, because running every pairwise t-test afterward inflates false positives. For chi-square, check expected cell counts — the rule of thumb is at least five per cell — and use Fisher’s exact test for small tables. Write the test, its assumptions, and the check you performed into your methods in one sentence each. A reviewer who can see the assumption check trusts the p-value that follows; one who cannot will ask for it anyway. Keep a one-page decision record for your analysis: the design, the tree answers, the chosen test, the assumption checks, and the result. If a reviewer or supervisor questions the test later, that page answers in seconds, and if you revisit the project in a year, it restores the reasoning you would otherwise have to reconstruct from memory. Analysis decisions made under deadline pressure are the first to be forgotten and the hardest to defend, so writing them down while they are fresh is a small habit with an outsized payoff across a whole research career.

Embed this tool on your website — free

Copy the code below to add this working tool to your own page. A small credit link stays with it — that’s all we ask.