HigherOrderCO/Bend

"Failed to parse result from HVM" from HVM returning Segmentation Fault when using lage array

ArthurPassosAcolita opened this issue · 2 comments

Reproducing the behavior

When running "bend run-c" using "large" arrays in the quicksort example in the repo with theese changes on the main function:

(GetDataset) = [1,2,3...2^19]

# Sorts and sums n random numbers
(Main) =
  let var = (Sort GetDataset )
  (1)

I'm getting this error output:

Errors:
Failed to parse result from HVM.

When trying with smaller arrays, it works fine (prints the "Result: 1").

When I run using "gen-hvm" and "hvm run-c" I get a "Segmentation fault" message.

System Settings

  • HVM: 2.0.21
  • Bend: 0.2.36
  • OS: Windows 11 WSL 2 with Ubuntu 22.04.4 LTS
  • CPU: Intel i5-1235U
  • RAM: 32 GB

Additional context

I tried increasing the memory of WSL to 26GB and makes no difference.

I attached the program I used (be careful, big lines like that may crash your editor with some manipulation):
quick_sort_random.txt

I dont know if its exactly relevant, but this bitonic sort using the same sequence in a tree seems to work fine:
bitonic_sort_random.txt

I think the problem is either in the hvm parser or in the compilation from the hvm ast to the internal structures, but we need to investigate.

Bend could give more useful error messages when hvm crashes

I dont know if will help the debugging, but I made these python scripts to generate the input data needed to reproduce.

import numpy as np

path = "../datasets/"

def generate_uniform_dataset(size, low, high):
    """
    Generate a uniformly distributed dataset.
    """
    return np.random.uniform(low, high, size).astype(int)

def generate_random_dataset(size, low, high):
    """
    Generate a randomly distributed dataset.
    """
    return np.random.randint(low, high, size)

def generate_skewed_dataset(size, low1, high1, low2, high2, skew_ratio=0.8):
    """
    Generate a skewed dataset with the specified ratio.
    """
    size_majority = int(size * skew_ratio)
    size_minority = size - size_majority

    majority_part = np.random.randint(low1, high1, size_majority)
    minority_part = np.random.randint(low2, high2, size_minority)

    return np.concatenate([majority_part, minority_part])

def save_dataset(filename, dataset):
    """
    Save dataset to a file in the format [1, 2, 3, 4].
    """
    with open(filename, 'w') as f:
        f.write(str(len(dataset)) +'\n' + '\n'.join(map(str, dataset)))

if __name__ == "__main__":
    ARRAY_SIZE = 2**15

    # Large random dataset
    large_random_dataset = generate_random_dataset(ARRAY_SIZE , 0, 2**19) # 524288
    save_dataset(path + "lines_random.txt", large_random_dataset)

    # Large skewed dataset
    large_skewed_dataset = generate_skewed_dataset(ARRAY_SIZE , 0, 1000, 1001, 1000000)
    save_dataset(path + "lines_skewed.txt", large_skewed_dataset)

    # Generate and save sorted datasets
    sorted_dataset = np.sort(large_random_dataset)
    save_dataset(path + "lines_sorted.txt", sorted_dataset)

    # Generate and save reverse sorted datasets
    reverse_sorted_dataset = sorted_dataset[::-1]
    save_dataset(path + "lines_reverse.txt", reverse_sorted_dataset)

    print("Datasets generated and saved to files.")