Point In Time Data Store

Many data series get revised after they are first published - a preliminary economic figure, a recalculated custom indicator, a sentiment score that gets refined as more data arrives. A naive backtest that reads "the current value of series X at bar N" is reading a value that may include revisions the trader could not possibly have known about at that historical moment. This is lookahead bias, and it does not announce itself as a bug - it shows up months later as a strategy that mysteriously performs worse live than it ever did in testing, because testing was quietly allowed to see the future. This header is the missing infrastructure that makes an honest point-in-time query possible. Every stored record carries two timestamps, not one: the effective time the value is ABOUT (what date or period it describes), and the known time the value BECAME visible (when this particular revision was actually published). AddRevision() keeps every version of a value ever recorded - a preliminary figure, a revision, a final figure - none are overwritten, because keeping the full revision history is the entire point. AsOf(effectiveTime, knownTime) then answers the honest question: the latest revision for that effective time whose own known time was at or before the queried known time. It is structurally impossible for AsOf() to return a revision that had not been published yet as of the time being queried, because the query only ever considers revisions whose knownTime already qualifies. Revisions are kept in a sorted array (sorted lazily - AddRevision() appends and marks the store dirty, and a query call sorts once before running if needed) so that AsOf() can binary-search for the matching effective time rather than scanning the whole revision history on every call. This is a deliberate performance decision: AsOf() is meant to be callable once per bar inside a backtest loop, where a linear scan per call would turn an O(n) backtest into an O(n^2) one as the revision history grows. LatestAsOf(knownTime) answers a different, related question: not "what was known about THIS specific effective time" but "what is the newest effective- time data point knowable at all, given this knowledge-time" - useful for driving a backtest's own simulated clock forward using only data that would genuinely have been available at each simulated moment. Notes and limitations - Sorting is lazy and insertion-sort based: AddRevision() marks the store dirty rather than re-sorting immediately, and any query call sorts first if needed. This is close to O(n) in practice because real feeds mostly arrive in roughly chronological order, but is O(n^2) worst case for badly out-of-order insertion - acceptable at the hundreds-to-low-thousands of revisions one MQL4 program typically keeps in memory, not intended for a very large or adversarially-ordered dataset. - AsOf() is a binary-search lookup for ONE effective time and is fast even in a tight per-bar loop. LatestAsOf() is a full linear scan by design - it answers a genuinely different question (a maximum over every effective time, not a lookup for one) that the effectiveTime-sorted index does not help narrow. Call LatestAsOf() sparingly (e.g. once per new bar), not per-tick, in a performance-sensitive loop. - This store holds one double value per record. A series with several fields (e.g. a figure plus its own revision confidence) needs one CPointInTimeStore instance per field, or a caller-side struct/index scheme layered on top - kept out of scope here to keep the core query semantics simple and unambiguous. - No persistence. Everything lives in memory for the lifetime of the EA/ Script instance; save/load to a file is the caller's responsibility if the revision history needs to survive a restart. - AddRevision() does not enforce knownTime >= effectiveTime. A caller feeding inverted timestamps (a revision "known" before its own effective moment) gets exactly what it asked for rather than a runtime error - this is deliberately permissive so the store can also model data that is genuinely knowable in advance (e.g. a scheduled economic calendar entry), but it means garbage-in-garbage-out for a caller that gets the two timestamps backwards.

Script · MT4 · v1.0 · Free

Tags: MT4 Script · MQL4 Source Code

MQL4 & MQL5 scripts · PolyMaria