tazz4843/whisper-rs

Fails on "isnan()" check

Closed this issue · 4 comments

dozerman-ms commented

What is this issue? I'm not even sure where to start looking. Is the file that I'm reading mis-encoded or is there an issue with my implementation?

whsp: ggml.c:3860: ggml_compute_forward_gelu_f32: Assertion `!isnan(x)' failed.

Total dump:'

[name@machine whsp]$ RUSTFLAGS="-Clink-args=-lstdc++" cargo run
Compiling whsp v0.1.0 (/home/andrew/DEV/whisp/rs/whsp)
Finished dev [unoptimized + debuginfo] target(s) in 1.11s
Running target/debug/whsp
whisper_model_load: loading model from '/home/andrew/.whisper/mods/medium.bin'
whisper_model_load: n_vocab = 51865
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 1024
whisper_model_load: n_audio_head = 16
whisper_model_load: n_audio_layer = 24
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 1024
whisper_model_load: n_text_head = 16
whisper_model_load: n_text_layer = 24
whisper_model_load: n_mels = 80
whisper_model_load: f16 = 1
whisper_model_load: type = 4
whisper_model_load: mem_required = 2610.00 MB
whisper_model_load: adding 1608 extra tokens
whisper_model_load: ggml ctx size = 1644.98 MB
whisper_model_load: memory size = 182.62 MB
whisper_model_load: model size = 1462.12 MB
whsp: ggml.c:3860: ggml_compute_forward_gelu_f32: Assertion !isnan(x)' failed. whsp: ggml.c:3860: ggml_compute_forward_gelu_f32: Assertion !isnan(x)' failed.
Aborted (core dumped)

code:
use whisper_rs;
use whisper_rs::{FullParams, SamplingStrategy, WhisperContext};
use std::fs::File;
use std::io::{BufReader, Read};

/* RUSTFLAGS="-Clink-args=-lstdc++" cargo run */

fn main(){
let mut ctx = WhisperContext::new("/home/name/.whisper/mods/medium.bin").expect("Woopsie");
let params = FullParams::new(/DecodeStrategy/SamplingStrategy::Greedy { n_past: 0 });
let auddat = gen_audio();

ctx.full(params,&auddat).expect("Failed to run mod");

let numb_segs = ctx.full_n_segments();
for i in 0..numb_segs{
    let segment = ctx.full_get_segment_text(i).expect("Woopsiedoodle");
    println!("{}", segment);
}

}
/*
fn gen_audio() -> Vec{
std::fs::read("./sn14ep12.wav").expect("Failed audio read.")

}
*/

fn gen_audio() -> Vec{

let mut dat = BufReader::new(
    File::open("./sn14ep12.wav").expect("Didn't read right???")
);

let mut floats = Vec::new();
loop{
    use std::io::ErrorKind;
    let mut buffer = [0u8; std::mem::size_of::<f32>()];

    let res = dat.read_exact(&mut buffer);
    match res{
        // We detect if we read until the end.
        // If there were some excess bytes after last read, they are lost.
        Err(error) if error.kind() == ErrorKind::UnexpectedEof => break,
        // Add more cases of errors you want to handle.
        _ => {}
    }
    res.expect("Unexpected error during read");

    let f = f32::from_le_bytes(buffer);
    floats.push(f);

}

return floats;

}

tazz4843 commented

This sounds like either a corrupt model file or funky audio. I can't dig more into this issue for a few more hours, but it appears in your code, you haven't accounted for WAV files having interleaved audio and a header. Check out a crate like hound to read WAV files: https://crates.io/crates/hound

dozerman-ms commented

Thank you for the quick response. I'll give hound a look next chance I get.

I was in a hurry when I originally wrote the OP, and so left out some details that I should have included. The C++ example build for whisper.cpp (./main) works perfectly fine with the .bin model that I'm using, and the audio file was converted using the ffmpeg instructions in the readme. I've gotten this issue on both Arch and Fedora; haven't tried on any other distros.

tazz4843 commented

I'm not sure if this has been resolved since, but there's been lots of versions since so I figure one of those might've solved it. If it hasn't, feel free to reopen.

dozerman-ms commented