Transport Payload Compressor

WebRequest payloads sent from an EA - a webhook body, a batch of status-report log lines, a repeated telemetry record - are usually small, highly repetitive text: the same JSON key names, the same punctuation, the same handful of status words, over and over. A real general-purpose compressor (LZ77/Deflate) is built to adapt to arbitrary input via a sliding-window match search and a bit-level output stream - a substantial project on its own, and MQL4 has no zlib to validate a from-scratch implementation against. What is actually useful here is much narrower and much simpler: substitute each occurrence of a KNOWN recurring substring with a fixed 2-byte escape code. TpcCompress walks the input once, and at every position tries each dictionary entry in order; the first one that matches is replaced by [0x01, index] and the match's length is skipped. Everything that matches no dictionary entry is copied through unchanged, one byte at a time. TpcDecompress reverses this exactly, since the escape byte followed by an index is unambiguous against the same dictionary. A built-in default dictionary (TPC_DEFAULT_DICT) covers the JSON key names and punctuation this catalogue's own webhook-style Libraries produce; TpcBuildDictionary lets a caller prepend project-specific tokens (a custom field name, a repeated broker comment string) ahead of the defaults, so a longer project-specific match is tried before the generic ones. This genuinely shrinks payloads dominated by dictionary-entry repetition - a batch of near-identical JSON log lines compresses well because "timestamp":"..." and "status":"ok" collapse from a dozen-plus characters each to two bytes every time they recur. It does nothing for genuinely novel text (a free-text comment field with no repeated structure) and can very slightly EXCEED the original size on a very short string with no dictionary hits, since a match check is attempted per character - TpcCompressionRatioPct will report a negative percentage in that case rather than hide it. Notes and limitations - This is dictionary substitution, not a general-purpose compressor. It only shrinks input containing the dictionary's known substrings; arbitrary or novel text passes through close to unchanged (each unmatched character costs one output byte, same as the input). - TpcCompress REFUSES (-1) any input containing a literal 0x01 byte, since that value is the escape marker and a literal occurrence would be indistinguishable from one it inserted itself. 0x01 is a control character that essentially never appears in JSON or plain-text payloads sent over WebRequest - if your payload can genuinely contain arbitrary binary bytes, this compressor is the wrong tool for it. - The dictionary used to decompress MUST exactly match (same entries, same order) the one used to compress - only the numeric index is stored, there is no self-describing header naming the dictionary. If you change TPC_DEFAULT_DICT or a custom dictionary between compressing and decompressing, decompression will silently produce wrong text. - No framing, checksum or length prefix - the caller is responsible for transmitting the byte count alongside the compressed bytes (or transmitting them as a length-prefixed blob) so the receiver knows where the compressed data ends. - Matching is greedy left-to-right in dictionary order, not longest-match-first by length - list longer, more specific custom entries before shorter ones when building a custom dictionary, the same way TPC_DEFAULT_DICT itself is ordered.

Script · MT5 · v1.0 · Free

Tags: MT5 Script · MQL5 Source Code

MQL4 & MQL5 scripts · PolyMaria