A python script which can remove redundant lines.
A replacement to the convential bash sequence sort -u -f <filename>
. I will try to implement this following the techniques of functional programming, while ensuring type safety.
To do so, I am using the typing library.
pip install -U typing
The goal was to create a script capable of removing redundant lines of text without compromising on order of text.
I also intended to try achive doing in in a functional programming manner.
My script achives this in O(n) time.
Take the example of a text file with contents:
bee
apple
apple
A textfile, conventionally, needed to be sorted by bash command:
sort -u -f <filename>
Or
sort -f <filename> | uniq
The results would be:
apple
bee
We can see that the order of entries is not respected.
To overcome this I wrote a script.
Use command:
python3 removeRedundant.py
You will be prompted to enter the path to the file.
The script preserves whitespace. This feature is implemented in line 128-131.
if arr[st].val == '':
res.extend(arr[st:pt])
else:
res.append(reduceArray(arr, st, pt))
you can replace it with the following code to remove this feature.
res.append(reduceArray(arr, st, pt))