Only Run whisper

[AhmedSawx]

Hi @AnubhabB,
I'm making an app where the person hit's a button and it transcribe it and then sends an output of the text.
I tried to detach llama functionality from whisper but it panics!

How can i remove llama and run whisper only?

[AnubhabB]

Can you share the error you are getting? The process of transcribing and then llama inference is independent and only merged in the audio() method of struct Instruct.

[AhmedSawx]

Ok. i will show how i "removed" it (I'm very new to rust, so please tell me if anything i did are wrong).

in commands.rs
here's what i changed

    let res = match command {
        Mode::Audio => app.audio(),
    };

pub enum Mode {
    Audio,
}

impl Command {
    pub fn mode(&self) -> Result<Mode> {
        if self.audio.map_or(false, |d| d) {
            Ok(Mode::Audio)
        } else {
            anyhow::bail!("not a valid command")
        }
    }
}

that's for commands.rs .

on instruct.rs

    /// Public API to trigger audio inference
    pub fn audio(&self) -> Result<Response> {
        let (transcript, n_tokens, elapsed) = self.whisper.infer()?;

        Ok(Response::new(
            &transcript,
            &transcript,
            n_tokens.try_into().unwrap(),
            0,
        ))
    }

i also removed text function.

        let app = Arc::new(Self {
            whisper,
            send: send.clone(),
        });

this is on new function

pub struct Instruct {
    /// holds the `distil-whisper` model and the associated methods
    whisper: WhisperWrap,
    /// a channel for triggering Instruct methods through events
    send: Sender<Vec<f32>>,
}

use crate::{commands::Response, whisper::WhisperWrap};

i removed the 'llama' file.

i also removed let llama = LlamaWrap::new(datadir.as_path())?; on the new function

after all that. the app got compiled, and after it finishes downloading the whisper model (i choose tiny). i got this

thread 'main' panicked at src/whisper.rs:71:55:
called `Option::unwrap()` on a `None` value
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
error: script "tauri" exited with code 101

i'm using tauri v2.0.4. any help are appreciated!
(sorry for my english)

[AnubhabB]

Ok so as far as I can tell, the problem in your src/whisper.rs:71 seems to be unrelated. If its same as the repo code then something is wrong with the tokenizer initialization. So the whisper tokenizer should have a id for the token <|nospeech|>, it's panicking there because apparently the tokenizer.json file you are initializing doesn't have that. Can you double check if the tokenizer is reading from the correct file?