Have a parameter to backup or move chats from chat_cache

[tan-wei-xin-alez]

I'm getting the following error after using the --chat parameter for a while and I think it's because of the token limit as mentioned at https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/<username>/.local/lib/python3.10/site-packages/sgpt/app.py:100 in main                     │
│                                                                                                  │
│    97 │   )                                                                                      │
│    98 │                                                                                          │
│    99 │   full_completion = ""                                                                   │
│ ❱ 100 │   for word in completion:                                                                │
│   101 │   │   typer.secho(word, fg="magenta", bold=True, nl=False)                               │
│   102 │   │   full_completion += word                                                            │
│   103 │   typer.secho()                                                                          │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │           cache = True                                                                       │ │
│ │            chat = 'test'                                                                     │ │
│ │    chat_history = [... +14]                                                                  │ │
│ │            code = False                                                                      │ │
│ │      completion = <generator object ChatCache.__call__.<locals>.wrapper at 0x7f14c937a8f0>   │ │
│ │          editor = False                                                                      │ │
│ │ full_completion = ''                                                                         │ │
│ │    is_code_chat = False                                                                      │ │
│ │   is_shell_chat = False                                                                      │ │
│ │       list_chat = False                                                                      │ │
│ │          prompt = '...'+277                                                                  │ │
│ │           shell = False                                                                      │ │
│ │       show_chat = None                                                                       │ │
│ │     temperature = 1.0                                                                        │ │
│ │ top_probability = 1.0                                                                        │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                  │
│ /home/<username>/.local/lib/python3.10/site-packages/sgpt/cache.py:101 in wrapper                │
│                                                                                                  │
│    98 │   │   │   │   old_messages.append(message)                                               │
│    99 │   │   │   kwargs["messages"] = old_messages                                              │
│   100 │   │   │   response_text = ""                                                             │
│ ❱ 101 │   │   │   for word in func(*args, **kwargs):                                             │
│   102 │   │   │   │   response_text += word                                                      │
│   103 │   │   │   │   yield word                                                                 │
│   104 │   │   │   old_messages.append({"role": "assistant", "content": response_text})           │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │          args = (<sgpt.client.OpenAIClient object at 0x7f14c936bc70>,)                       │ │
│ │       chat_id = 'test'                                                                       │ │
│ │          func = <function OpenAIClient.get_completion at 0x7f14c9ac0af0>                     │ │
│ │        kwargs = {                                                                            │ │
│ │                 │   'messages': [...],                                                       │ │
│ │                 │   'model': 'gpt-3.5-turbo',                                                │ │
│ │                 │   'temperature': 1.0,                                                      │ │
│ │                 │   'top_probability': 1.0,                                                  │ │
│ │                 │   'caching': True                                                          │ │
│ │                 }                                                                            │ │
│ │       message = {...}                                                                        │ │
│ │      messages = [...]                                                                        │ │
│ │  old_messages = [... +15]                                                                    │ │
│ │ response_text = ''                                                                           │ │
│ │          self = <sgpt.cache.ChatCache object at 0x7f14ca588b20>                              │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                  │
│ /home/<username>/.local/lib/python3.10/site-packages/sgpt/client.py:91 in get_completion         │
│                                                                                                  │
│   88 │   │   :param caching: Boolean value to enable/disable caching.                            │
│   89 │   │   :return: String generated completion.                                               │
│   90 │   │   """                                                                                 │
│ ❱ 91 │   │   yield from self._request(                                                           │
│   92 │   │   │   messages,                                                                       │
│   93 │   │   │   model,                                                                          │
│   94 │   │   │   temperature,                                                                    │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │         caching = True                                                                       │ │
│ │        messages = [... +15]                                                                  │ │
│ │           model = 'gpt-3.5-turbo'                                                            │ │
│ │            self = <sgpt.client.OpenAIClient object at 0x7f14c936bc70>                        │ │
│ │     temperature = 1.0                                                                        │ │
│ │ top_probability = 1.0                                                                        │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                  │
│ /home/<username>/.local/lib/python3.10/site-packages/sgpt/cache.py:39 in wrapper                 │
│                                                                                                  │
│    36 │   │   │   │   yield cache_file.read_text()                                               │
│    37 │   │   │   │   return                                                                     │
│    38 │   │   │   result = ""                                                                    │
│ ❱  39 │   │   │   for i in func(*args, **kwargs):                                                │
│    40 │   │   │   │   result += i                                                                │
│    41 │   │   │   │   yield i                                                                    │
│    42 │   │   │   cache_file.write_text(result)                                                  │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │       args = (                                                                               │ │
│ │              │   <sgpt.client.OpenAIClient object at 0x7f14c936bc70>,                        │ │
│ │              │   [... +15],                                                                  │ │
│ │              │   'gpt-3.5-turbo',                                                            │ │
│ │              │   1.0,                                                                        │ │
│ │              │   1.0                                                                         │ │
│ │              )                                                                               │ │
│ │ cache_file = PosixPath('/tmp/shell_gpt/cache/67439a6e2d3d2bc706f96358237e0afc')              │ │
│ │  cache_key = '67439a6e2d3d2bc706f96358237e0afc'                                              │ │
│ │       func = <function OpenAIClient._request at 0x7f14c9c30ca0>                              │ │
│ │     kwargs = {}                                                                              │ │
│ │     result = ''                                                                              │ │
│ │       self = <sgpt.cache.Cache object at 0x7f14ca55fd90>                                     │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                  │
│ /home/<username>/.local/lib/python3.10/site-packages/sgpt/client.py:57 in _request               │
│                                                                                                  │
│   54 │   │   response = requests.post(                                                           │
│   55 │   │   │   endpoint, headers=headers, json=data, timeout=REQUEST_TIMEOUT, stream=True      │
│   56 │   │   )                                                                                   │
│ ❱ 57 │   │   response.raise_for_status()                                                         │
│   58 │   │   # TODO: Optimise.                                                                   │
│   59 │   │   # https://github.com/openai/openai-python/blob/237448dc072a2c062698da3f9f512fae3    │
│   60 │   │   for line in response.iter_lines():                                                  │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │            data = {                                                                          │ │
│ │                   │   'messages': [... +15],                                                 │ │
│ │                   │   'model': 'gpt-3.5-turbo',                                              │ │
│ │                   │   'temperature': 1.0,                                                    │ │
│ │                   │   'top_p': 1.0,                                                          │ │
│ │                   │   'stream': True                                                         │ │
│ │                   }                                                                          │ │
│ │        endpoint = 'https://api.openai.com/v1/chat/completions'                               │ │
│ │         headers = {                                                                          │ │
│ │                   │   'Content-Type': 'application/json',                                    │ │
│ │                   │   'Authorization': 'Bearer                                               │ │
│ │                   sk-************************************************'                       │ │
│ │                   }                                                                          │ │
│ │        messages = [... +15]                                                                  │ │
│ │           model = 'gpt-3.5-turbo'                                                            │ │
│ │        response = <Response [400]>                                                           │ │
│ │            self = <sgpt.client.OpenAIClient object at 0x7f14c936bc70>                        │ │
│ │     temperature = 1.0                                                                        │ │
│ │ top_probability = 1.0                                                                        │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                  │
│ /home/<username>/.local/lib/python3.10/site-packages/requests/models.py:1021 in raise_for_status │
│                                                                                                  │
│   1018 │   │   │   )                                                                             │
│   1019 │   │                                                                                     │
│   1020 │   │   if http_error_msg:                                                                │
│ ❱ 1021 │   │   │   raise HTTPError(http_error_msg, response=self)                                │
│   1022 │                                                                                         │
│   1023 │   def close(self):                                                                      │
│   1024 │   │   """Releases the connection back to the pool. Once this method has been            │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │ http_error_msg = '400 Client Error: Bad Request for url:                                     │ │
│ │                  https://api.openai.com/v1/chat/completion'+1                                │ │
│ │         reason = 'Bad Request'                                                               │ │
│ │           self = <Response [400]>                                                            │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
HTTPError: 400 Client Error: Bad Request for url: https://api.openai.com/v1/chat/completions

Correct me if I'm wrong, but I think the entire history is being sent every time --chat is used to preserve context? Because the error goes away when I remove all of the previous messages (except the 1st prompt and response) with

cp /tmp/shell_gpt/chat_cache/test /tmp/shell_gpt/chat_cache/test-$(date +'%Y%m%d_%H%M%S')
jq '.[0:2]' /tmp/shell_gpt/chat_cache/test-20230329_165325 > /tmp/shell_gpt/chat_cache/test

Would it make sense to have like a --backup-chat or --move-chat parameter to automatically perform the above actions? Or is something similar being worked on to combat the above issue?

[TheR1D]

Currently OpenAI has several GPT models with increased token limit that would partially solve the issue. Other than that, it seems this issue might be uniquely personal. Consequently, I'm closing this issue to maintain focus on broader user concerns. Please feel free to reopen it if additional relevant information arises.