Statistical Hypothesis Tests

"Strategy A's backtest looks better than Strategy B's" is an eyeball judgement, and eyeballs are bad at telling a genuine edge apart from a lucky sample. These four tests give a vocabulary for the question that actually matters: is the difference between two samples of returns - or one sample's distribution shape - larger than plain noise would plausibly produce. HypothesisWelchTTest compares two sample means without assuming equal population variances. Plain Student's t-test makes that assumption, and two return series routinely violate it - a volatile strategy and a conservative one rarely have matching variance, and using Student's t anyway understates the standard error and overstates significance. Welch's version uses the Welch-Satterthwaite approximation for degrees of freedom and is the safer default whenever variances are not independently known to match. HypothesisKsTwoSample and HypothesisMannWhitneyU both compare whole distributions rather than only means, which matters when two return series have a similar average but a very different shape - one fat-tailed, one not; one skewed, one symmetric. Kolmogorov-Smirnov measures the largest gap between the two samples' empirical CDFs and uses the standard asymptotic p-value; Mann-Whitney ranks the combined samples and compares rank sums, with a tie-correction term in its variance so repeated identical values (common in price data quantised to a broker's point size) do not bias the result. HypothesisJarqueBera checks whether a single sample looks normally distributed at all, by comparing its skewness and excess kurtosis against the values a normal distribution would have. Most parametric statistics - including the classical (non-Welch) t-test and any calculation built on an assumed Gaussian - quietly assume this, and returns are usually not normal; Jarque-Bera makes that assumption checkable instead of implicit. p-values throughout come from a genuine Student-t CDF, built from the regularized incomplete beta function evaluated via a continued fraction (the standard Numerical Recipes betacf/betai construction, itself built on a Lanczos log-gamma approximation) - not a crude large-sample normal approximation substituted in its place. That keeps results meaningful at the sample sizes a strategy comparison typically has (tens to a few hundred trades), where a normal approximation to the t-distribution is still visibly wrong. Notes and limitations - All four tests assume the observations within each sample are roughly independent. Consecutive one-bar returns on the same instrument are autocorrelated to some degree, which understates true variance and can make a p-value look more significant than it should - use returns sampled far enough apart (or per-trade returns rather than per-bar) when the conclusion matters. - Mann-Whitney's p-value here is the standard large-sample normal approximation with a tie correction, appropriate once each sample has roughly 20 or more observations; it is not an exact permutation p-value for very small samples. - A p-value answers "how surprising would this data be if there were truly no difference", not "how large or how tradeable the difference is". Report the effect size (mean difference, D-stat) alongside the p-value, not the p-value alone. - This is a statistics toolkit, not a trading signal generator. It places no trades and issues no buy/sell decision on its own.

Script · MT5 · v1.0 · Free

Tags: MT5 Script · MQL5 Source Code

MQL4 & MQL5 scripts · PolyMaria