Classification Metrics And Purged Splits

Anyone who trains a filter, a regime detector or a machine-learned entry signal on historical bars eventually needs to answer two separate questions: how good is the model on data it has already seen scored, and was the way it was evaluated even honest about what "already seen" means for time series. This library answers both. The metrics half is the standard confusion-matrix toolkit - accuracy, precision, recall, specificity, F1 - plus Matthews correlation coefficient, which is the one number that keeps meaning something when the classes are imbalanced. A model that always predicts "no trade" on a dataset where losing trades outnumber winners nine to one can still show 90% accuracy; MCC does not reward that. RocAucScore is computed with the Mann-Whitney rank-sum identity rather than by sweeping a grid of thresholds and integrating the curve numerically - sort once, sum the ranks of the positive class, done. It returns a sentinel -1.0, not 0.0, when the sample contains only one class, because 0.0 is itself a real and meaningful AUC reading (a model that ranks every positive below every negative) and must not be confused with "AUC could not be computed here". The walk-forward half exists because the metrics above are worthless if the split that produced train and test sets leaked. A plain k-fold split on chronological bar data lets a model train on bars that sit after the bars it is tested on. Worse, even a split that keeps train strictly before test can still leak if the label was built from a forward-looking horizon - a label that reads "close 20 bars ahead" computed for a bar near the train/test boundary was partly built from bars inside the test fold. BuildPurgedWalkForwardFolds produces anchored, expanding-window folds and removes two gaps around every test fold: a purge gap immediately before it (so no training label's forward horizon reaches in) and an embargo gap immediately after it (so the boundary itself is never reused as training data for the next fold either). Both gaps are plain bar counts the caller chooses to match their own label horizon. Notes and limitations - All functions treat any non-zero actualLabels[] entry as positive; use plain 0/1 for clarity. - RocAucScore's tie-handling averages ranks across equal scores, so a model that outputs the same score for many samples still gets a correct AUC, not an inflated or deflated one. - The rank sort is an O(n^2) insertion sort, the same trade-off Correlation.mqh makes for Spearman ranking - fine for evaluation sets of the size a bar-based trading model normally produces (tens to low thousands), not intended for streaming tick-by-tick scoring. - BuildPurgedWalkForwardFolds reserves the same embargo allowance for every fold including conceptually the last, which is intentionally conservative rather than maximally bar-efficient; it can therefore build fewer bars of final test fold than a hand-tuned split would, never more. - Purge and embargo sizes are the caller's responsibility to set correctly - they must be at least as large as the forward-looking horizon used to build the labels, or leakage still happens even though a split was produced. - This library scores predictions; it does not train anything. Pair it with DecisionTreeClassifier.mqh or your own model, and score the result here.

Script · MT5 · v1.0 · Free

Tags: Buy/Sell Arrows · Binary Options · MT5 Script · MQL5 Source Code

MQL4 & MQL5 scripts · PolyMaria