This file and the script were written by Mahmoud Dondeti. The script will remove duplicate reads from fasta or fastq file formats. It will output a sorted fasta or fastq file without any duplicate reads, with the count of the reads before and after.
Usage: python remove_duplicate_reads.py input.fasta output.fasta
or
Usage: python remove_duplicate_reads.py input.fastq output.fastq