AI proxy and service strip path conflict?

[brandonbiggs]

Hi,

i have kong 3.8.0 installed. I have the following configuration in a DB-less kong instance:

services:
- name: llm_test_service
  url: http://test.example.com:8102
  routes:
  - name: llm_test_route
    paths:
    - /test
  
plugins:
- name: ai-proxy
  service: llm_test_service
  config:
    route_type: preserve
    model:
      provider: llama2
      options:
        upstream_url: http://test.example.com:8102
        llama2_format: openai

this correctly routes to my API, but it's including /test in the path:

POST /test/v1/chat/completions HTTP/1.1" 404 Not Found

I want it to strip /test/ like it does with normal endpoints without the ai proxy.

If I don't use the AI proxy, the service correctly gets me to my endpoint and my call works because strip_path defaults to true. If in my plugin config I set route_type to anything except preserve, it seems the whole path gets stripped out.

I'm running a basic vLLM endpoint and it works great. I'm just trying to utilize the Kong API gateway with AI plugin to route to my vLLM endpoints.

This is my first experience with the AI plugin, so I may be doing something wrong, but I can't seem to figure out what it is. Any thoughts?