You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This error occurs when using Kernel Memory and can be reproduced when running the
Kernel Memory: Save and Load example and appears to be reproducible (at least by me) regardless of which model I try.
I have tried on two different systems with two distinct hardware configurations and the error persists between both setups. One is CPU bound and the other has a Nvidia 4080.
Reproduction Steps
Execute the Example Program. Allow the program to execute. When the question "What formats does KM support" is asked. The application will fail.
Please note:
The data from the files appears to be successfully tokenized and saved in the storage folder.
Once a question is and the ApplyPenalty method executes there is a line:
var nl_logit = logits[(int?)nl_token ?? 0];
This code consistently generates an Object Reference Not Set To An Instance of An Object. The variable appears to be set a few lines earlier:
var logits = NativeHandle.GetLogitsIth(logits_i);
I can't seem to debug this as any attempt to see what is contained within logits results in a Visual Studio error indicating an error while evaluating the SPAN.
Environment & Configuration
Operating system: Windows 11
.NET runtime version: .NET 8
LLamaSharp version: latest for GIT - 0.14.X
CUDA version (if you are using cuda backend): Compiling llamasharp directly and running examples
CPU & GPU device: Error occurs on various PCs of different configurations.
Known Workarounds
No response
The text was updated successfully, but these errors were encountered:
If that's throwing it must mean the Span<float> logits is pointing to an invalid memory location (that would also be why you can't inspect it in the debugger, trying to do so will attempt to read the invalid memory).
To help debug this, what is the value of logits_i when it happens?
Description
This error occurs when using Kernel Memory and can be reproduced when running the
Kernel Memory: Save and Load example and appears to be reproducible (at least by me) regardless of which model I try.
I have tried on two different systems with two distinct hardware configurations and the error persists between both setups. One is CPU bound and the other has a Nvidia 4080.
Reproduction Steps
Execute the Example Program. Allow the program to execute. When the question "What formats does KM support" is asked. The application will fail.
Please note:
The data from the files appears to be successfully tokenized and saved in the storage folder.
Once a question is and the ApplyPenalty method executes there is a line:
var nl_logit = logits[(int?)nl_token ?? 0];
This code consistently generates an Object Reference Not Set To An Instance of An Object. The variable appears to be set a few lines earlier:
var logits = NativeHandle.GetLogitsIth(logits_i);
I can't seem to debug this as any attempt to see what is contained within logits results in a Visual Studio error indicating an error while evaluating the SPAN.
Environment & Configuration
Known Workarounds
No response
The text was updated successfully, but these errors were encountered: