/preserved_uniq

a python script which can remove redundant lines.

Primary LanguagePythonGNU General Public License v3.0GPL-3.0

preserved_sort

A python script which can remove redundant lines.

Philosophy

A replacement to the convential bash sequence sort -u -f <filename>. I will try to implement this following the techniques of functional programming, while ensuring type safety.
To do so, I am using the typing library.

Prerequisites

pip install -U typing

Goal

The goal was to create a script capable of removing redundant lines of text without compromising on order of text.

I also intended to try achive doing in in a functional programming manner.

My script achives this in O(n) time.

Example

Take the example of a text file with contents:

bee
apple
apple

A textfile, conventionally, needed to be sorted by bash command:

sort -u -f <filename>

Or

sort -f <filename> | uniq

The results would be:

apple
bee

We can see that the order of entries is not respected.

To overcome this I wrote a script.

Usage

Use command:

python3 removeRedundant.py

You will be prompted to enter the path to the file.

Note

The script preserves whitespace. This feature is implemented in line 128-131.

            if arr[st].val == '':
                res.extend(arr[st:pt])
            else:
                res.append(reduceArray(arr, st, pt))

you can replace it with the following code to remove this feature.

            res.append(reduceArray(arr, st, pt))