File Backed Key Value Store With Compaction

A key-value store that only keeps its data in memory loses everything on a terminal restart or crash. This one appends every Set() and Delete() to a binary log file, flushed to disk immediately, and keeps an in-memory index in sync as it goes so lookups stay fast. Open() on a previously-used file replays every record from the start, rebuilding the exact same index a crash would otherwise have thrown away - overwritten keys resolve to their last-written value and deleted keys disappear, purely from the log's own contents. An append-only log that is never trimmed grows without bound even though most of it is dead weight - every overwritten key leaves its old value on disk forever, and every delete adds a record without removing anything. Compact() rewrites the log to hold exactly one SET record per currently-live key, using the in-memory index (not a second read of the old log) as the source of truth, so a corrupted or inconsistent stretch of old records cannot leak into the compacted file. Compaction on MQL4 cannot be made crash-atomic, because the language has no rename()-style call to swap a temp file over the original in one step. This library is explicit about the resulting window rather than pretending it does not exist - see NOTES AND LIMITATIONS. Notes and limitations - Lookups are O(n) over an in-memory array, not a hash table - MQL4 has no built-in hash map. Fine for the hundreds-to-low-thousands of keys a typical EA/indicator settings or state store needs; not intended for a large dataset. - Keys are capped at 256 bytes and values at 65536 bytes (FKV_MAX_KEY / FKV_MAX_VALUE in the header) - a sanity bound against a corrupted length field being trusted, raise them if genuinely needed. - Compact() is NOT crash-atomic. It writes a temp file, closes the live handle, deletes the original, then FileCopy()'s the temp file over the original name (MQL4 has no rename()). A crash between the delete and the copy would leave the store empty. This is a narrow window relative to the time Compact() actually needs to run, but it is real - call Compact() at a low-risk moment (e.g. EA startup, not mid-trade), and keep a backup copy of anything critical. - Not for concurrent writers. Two terminals/EAs opening the same file for writing at once are not coordinated by this library.

Script · MT5 · v1.0 · Free

Tags: Binary Options · MT5 Script · MQL5 Source Code

MQL4 & MQL5 scripts · PolyMaria