Small tool to purge dupes from dictionary files. Look at testfiles/dictionnary_1.txt
to see if you need it.
- Windows (tested with vs2019 toolchain)
- Linux
Go to headers/const.h
and change MAX_RAM_ALLOCATION
and LINE_MAX_LEN
to fit with your config.
mkdir build
cd build
cmake ..
make
./purge_dupes dic_to_purge output_file