-
Notifications
You must be signed in to change notification settings - Fork 18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
convert Qwen/Qwen2.5-3B-Instruct tokens failed #301
Comments
I have the same problem: |
After analyzing this issue in depth, I found that the root cause is that in
However, in openvino_tokenizers, the In
|
Context
Hello, Here is my venv package version.
Here is my history.
What needs to be done?
Try to support Qwen/Qwen2.5-3B-Instuct mode
Example Pull Requests
I change
/qwen-agent-app/venv/lib/python3.12/site-packages/openvino_tokenizers/tokenizer_pipeline.py
75 line
With this patch, can success conver to openvino tokenizer. But I load in openvino ovms, is report error as follow.
Resources
Contact points
Thanks.
Ticket
No response
The text was updated successfully, but these errors were encountered: