Bayesian Strategy Comparison
"58% win rate beats 54% win rate" is a comparison that ignores sample size completely. Twelve trades at 58% is barely any evidence at all; eight hundred trades at 58% is strong evidence. Naive win-rate comparison treats both cases identically and routinely leads to picking the wrong strategy from small samples, or dismissing a genuinely better one that just has not accumulated enough of a track record yet. This header models each strategy's true (unknown) win rate as a Beta distribution, updated from an uninformative Beta(1,1) prior by the observed wins and losses - the standard Beta-Bernoulli conjugate update, exact in closed form, no numerical fitting needed. Comparing the two strategies then means comparing two Beta posteriors, which this header does with the same Monte Carlo approach used for Bayesian A/B testing: draw many samples from each posterior, and report the fraction of draws where strategy A's sampled rate exceeds strategy B's. That fraction is P(true rate of A > true rate of B) - a single number that already accounts for how much each track record actually supports its win-rate estimate. A Monte Carlo estimate is used deliberately rather than a closed-form calculation, because the exact closed form for P(Beta_A > Beta_B) is a nested sum over the Beta-Binomial distribution that is easy to get subtly wrong, while twenty thousand cheap posterior draws converge to the same answer with a transparent, auditable method. The credible interval reported for the difference in win rates is read directly off the same Monte Carlo draws by the percentile method, so it needs no extra distributional assumption either. Notes and limitations - This compares win RATE only - it says nothing about payoff size, so a strategy with a lower win rate but a much larger average win can still be more profitable overall. Feed it win/loss counts, not P&L. - Assumes each trade's outcome is an independent Bernoulli draw with a fixed underlying probability. A strategy whose edge is drifting over time (regime change, parameter decay) violates this and the posterior will lag reality. - The Beta(1,1) default prior is uninformative; if genuine prior knowledge exists (e.g. a live strategy that performed close to breakeven walk-forward) a more informative prior can be passed instead. - Monte Carlo draws are cheap but not free - 20000 draws is fast on any modern machine but the function is not intended to run on every tick of an EA; call it periodically or on demand.
Script · MT5 · v1.0 · Free
About Bayesian Strategy Comparison
Bayesian Strategy Comparison is a script for MetaTrader 5. It has 9 settings, such as Wins a, Losses a and Wins b. MQL source code included.
Files in the download: BayesianStrategyComparison.mq5.mqh, BayesianStrategyComparison.txt, BayesianStrategyComparisonDemo.ex5, BayesianStrategyComparisonDemo.mq5.
Settings
| Setting | Default | What it does |
|---|---|---|
| Wins a | 58 | Strategy A - winning trades |
| Losses a | 42 | Strategy A - losing trades |
| Wins b | 22 | Strategy B - winning trades |
| Losses b | 14 | Strategy B - losing trades |
| Prior alpha | 1.0 | Prior Beta alpha (1.0 = uninformative) |
| Prior beta | 1.0 | Prior Beta beta (1.0 = uninformative) |
| Monte carlo draws | 20000 | Number of posterior samples |
| Confidence level | 0.95 | Credible interval level, e.g. 0.95 |
| Seed | 11 | Random seed, 0 = non-reproducible |
How to install
- Download the ZIP from your PolyMaria library and unzip it.
- In MetaTrader 5 open File → Open Data Folder, then go to MQL5 → Scripts.
- Copy the .mq5 file(s) there. Put any .mqh files in MQL5 → Include.
- Open the .mq5 file in MetaEditor and press Compile (F7) if MetaTrader does not show it.
- Drag the script onto a chart to run it once.
Tags: MT5 Script · MQL5 Source Code