[Question] how to change system_message at runtime

[phgcha]

❓ General Questions

Hi,
I can't seem to find a way to update system_message at runtime. I've read this doc. I've tried modifying the mlc-chat-config.json before making the completion call but it doesn't seem like the effect is taking place.

I know I can add a message {"role": "system", "content": system_prompt} in messages but it adds additional tokens to the prompt.

Please shed some light!

[phgcha]

Is it a good idea to make the system_message in mlc-chat-config.json empty and add {"role": "system", "content": system_prompt} however I need?

[Mawriyo]

When you say

"I've tried modifying the mlc-chat-config.json before making the completion call but it doesn't seem like the effect is taking place."

• Have you re-packaged/recompiling the project?

"I know I can add a message {"role": "system", "content": system_prompt} in messages but it adds additional tokens to the prompt."

• My understanding is that additional tokens should only happen for the first run. If you tried sending another message, that system prompt should persist from my experience.

I shall preface that I work on the android side so our process may be different. I apologize if this is not helpful.