OSError: [Errno 28] No space left on device

[hermangalio]

Hi, thank you for the nice module!

My reason for using in_place would be to edit very large files without needing to make copies (which would exceed the amount of space on my system). However, when I use in_place to edit a file line by line, after a while the amount of space on my computer runs out. Here's my code:

def strip_each_line(dataset_path):
    with in_place.InPlace(dataset_path) as dataset_file:
        for i, line in enumerate(dataset_file):
            if i % 10000 == 0:
                print(i)
            stripped_line = line.strip()
            dataset_file.write(stripped_line)


strip_each_line('main_data/main_dataset.txt')

Would there be a way to edit the file in place without taking up extra memory? Thanks in advance!

[jwodder]

in_place works by writing all output to a temporary file and then replacing the original file with the temporary one once the context manager exits (though the move_first option can be used to change the order in which the various bits happen). There are no attempts at operating on only the original file, which would involve sophisticated juggling of on-disk bytes. As a result, as in_place gets closer to the end of a file, you essentially have both the original and a modified copy on disk at the same time, which does not work with your use-case. I'm afraid that in_place just doesn't do what you want, and I'm not aware of any package that does.