Merge Two CSV Exports

Combining two CSV exports - two days of trade history, two symbols' worth of logged data, an old export and a new one - is a common enough chore that it is worth automating, and a bad enough place to cut corners that it is worth doing carefully. The trap in the naive version of this script is the header check, or rather the absence of one. Two CSV files that look alike at a glance can have columns in a different order, an extra column one export gained later, or a column renamed between runs. Merge them blind and every row after that point lands in the wrong field - a price ends up under a symbol column, a timestamp ends up under a lot size - and nothing in the output file complains, because CSV has no schema to violate. The damage is silent and it is discovered downstream, usually by whatever process reads the merged file next. This script checks first. It reads the header row of both files, splits each on the chosen delimiter, and compares them column by column. Column count and column text must match exactly, in the same order. If they do not, nothing is merged and no output file is written - the script prints both headers to the Experts log and shows them in a popup, side by side, so the mismatch is obvious rather than something you have to go digging for. Once the headers pass, the script reads every data row from both files and looks for duplicates using one column as the key - typically an ID, a timestamp, or whatever uniquely identifies a row in your data. How a collision is resolved is a choice, not a default: keep the version from the first file, keep the version from the second, or keep both and let the repeated key stand. The surviving rows are then written out either as a straight append (all of file A, then all of file B) or interleaved row by row, and the result goes to a new file named with a timestamp so running the script twice never overwrites the previous merge. Notes and limitations - Both files are read fully into memory, and duplicate detection compares every row of file A against every row of file B, so cost grows with the product of the two row counts. This is comfortable for exports up to a few thousand rows each; a script running for a long time on much larger files is spending it there. - This is a plain-split parser, not a CSV parser. Fields are split on the delimiter character with no support for quoted fields. A field that contains the delimiter itself - a comma inside a quoted text value, for instance - will be split into extra columns and corrupt that row. Export data without embedded delimiters, or use a delimiter such as a semicolon or tab that will not appear inside the values. - The header check is exact: same number of columns, same text, same order. A trailing space or a renamed column counts as a mismatch, by design. - Blank lines in either input file are skipped rather than treated as empty rows. - A data row with fewer columns than the header has an empty string as its key rather than being rejected, so it will not collide with any other row unless the other row's key is also empty. - The output filename is generated from the current time down to the second, so two merges started in the same second would collide; in practice this only matters when scripting repeated automated runs.

Script · MT5 · v1.0 · Free

Tags: Alerts · MT5 Script · MQL5 Source Code

MQL4 & MQL5 scripts · PolyMaria