You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Because of the lack of embeddings extraction to create a vector space database, I need to increase the context length up to 32k or more if possible.
I see that some models such as Qwen2 support up to 128k context. How can I increase it.
Should I have to update the rknn_api.h:
#define RKNN_MAX_DIMS 32 /* increased maximum dimension of tensor. /
#define RKNN_MAX_NUM_CHANNEL 20 / increased maximum channel number of input tensor. /
#define RKNN_MAX_NAME_LEN 512 / increased maximum name length of tensor. /
#define RKNN_MAX_DYNAMIC_SHAPE_NUM 32768 / increased maximum number of dynamic shape for each input. */
or is it enough to set param.max_context_len = 32768; ?
Because of the lack of embeddings extraction to create a vector space database, I need to increase the context length up to 32k or more if possible.
I see that some models such as Qwen2 support up to 128k context. How can I increase it.
Should I have to update the rknn_api.h:
#define RKNN_MAX_DIMS 32 /* increased maximum dimension of tensor. /
#define RKNN_MAX_NUM_CHANNEL 20 / increased maximum channel number of input tensor. /
#define RKNN_MAX_NAME_LEN 512 / increased maximum name length of tensor. /
#define RKNN_MAX_DYNAMIC_SHAPE_NUM 32768 / increased maximum number of dynamic shape for each input. */
or is it enough to set param.max_context_len = 32768; ?
@airockchip @waydong
The text was updated successfully, but these errors were encountered: