Problem with HTTP in CORS handling

[haenf]

Please specify the following versions when submitting a bug report:

• Julia 1.10.5
• HTTP.jl 1.10.8
• MbedTLS.jl 1.X.X not used

When trying to find the reason for malfunctioning of Oxygen/HTTP CORS handling , we finally came to a minimal example using HTTP.jl only:

using HTTP

function simple_handler(req::HTTP.Request)
    println("Simple handler")
    response_body = "{\"message\": \"Hello, world!\"}"
    headers = Dict(
        "Content-Type" => "application/json",
        "Access-Control-Allow-Origin" => "*"
    )
    return HTTP.Response(200, response_body, headers)
end

HTTP.serve(simple_handler, "0.0.0.0", 4091)

The output:

Simple handler
┌ Error: handle_connection handler error. 
│ 
│ ===========================
│ HTTP Error message:
│ 
│ ERROR: ArgumentError: invalid header key-value pair: {
│ Stacktrace:
│   [1] mkheaders(h::String, headers::Vector{Pair{SubString{String}, SubString{String}}})
│     @ HTTP.Messages C:\Users\Administrator\.julia\packages\HTTP\sJD5V\src\Messages.jl:211
│   [2] mkheaders
│     @ C:\Users\Administrator\.julia\packages\HTTP\sJD5V\src\Messages.jl:209 [inlined]
│   [3] #Response#1
│     @ C:\Users\Administrator\.julia\packages\HTTP\sJD5V\src\Messages.jl:108 [inlined]
│   [4] Response
│     @ C:\Users\Administrator\.julia\packages\HTTP\sJD5V\src\Messages.jl:105 [inlined]
│   [5] simple_handler
│     @ C:\inetpub\wwwroot\Tx_Viewer_api\src\AsmApi.jl:44 [inlined]
│   [6] (::HTTP.Handlers.var"#1#2"{typeof(Main.AsmApi.simple_handler)})(stream::HTTP.Streams.Stream{HTTP.Messages.Request, HTTP.Connections.Connection{Sockets.TCPSocket}})
│     @ HTTP.Handlers C:\Users\Administrator\.julia\packages\HTTP\sJD5V\src\Handlers.jl:58
│   [7] #invokelatest#2
│     @ .\essentials.jl:892 [inlined]
│   [8] invokelatest
│     @ .\essentials.jl:889 [inlined]
│   [9] handle_connection(f::Function, c::HTTP.Connections.Connection{Sockets.TCPSocket}, listener::HTTP.Servers.Listener{Nothing, Sockets.TCPServer}, readtimeout::Int64, access_log::Nothing)
│     @ HTTP.Servers C:\Users\Administrator\.julia\packages\HTTP\sJD5V\src\Servers.jl:469
│  [10] (::HTTP.Servers.var"#16#17"{HTTP.Handlers.var"#1#2"{typeof(Main.AsmApi.simple_handler)}, HTTP.Servers.Listener{Nothing, Sockets.TCPServer}, Set{HTTP.Connections.Connection}, Int64, Nothing, ReentrantLock, Base.Semaphore, HTTP.Connections.Connection{Sockets.TCPSocket}})()
│     @ HTTP.Servers C:\Users\Administrator\.julia\packages\HTTP\sJD5V\src\Servers.jl:401
│   request =
│    HTTP.Messages.Request:
│    """
│    POST /get_networks HTTP/1.1
│    Host: localhost:4091
│    User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:131.0) Gecko/20100101 Firefox/131.0
│    Accept: */*
│    Accept-Language: en-US,en;q=0.5
│    Accept-Encoding: gzip, deflate, br, zstd
│    Content-Type: text/plain
│    Content-Length: 22
│    Origin: https://viewer.trexyz.net
│    Connection: keep-alive
│    Sec-Fetch-Dest: empty
│    Sec-Fetch-Mode: cors
│    Sec-Fetch-Site: cross-site
│    Priority: u=4
│ 
│    {"database":"Mirna_2"}"""

The stack trace points to an issue during the creation of headers in the HTTP.Response call. Specifically, the error arises in mkheaders, indicating that there's something unexpected in how the headers are passed.

A next test:

using HTTP

function simple_handler(req::HTTP.Request)
    println("Simple handler")
    
    # Log the request method, headers, and body
    println("Request Method: ", HTTP.method(req))
    println("Request Body: ", HTTP.payload(req))

    # Simple response with no headers at all
    response_body = "{\"message\": \"No Headers, Simple Response\"}"
    return HTTP.Response(200, response_body)
end

had the following output:

POST
	http://localhost:4091/get_networks
Status
200
OK
VersionHTTP/1.1
Transferred89 B (42 B size)
Referrer Policystrict-origin-when-cross-origin
DNS ResolutionSystem \n response: message	"No Headers, Simple Response" \n output in terminal: Simple handler
Request Method: POST
Request Body: UInt8[0x7b, 0x22, 0x64, 0x61, 0x74, 0x61, 0x62, 0x61, 0x73, 0x65, 0x22, 0x3a, 0x22, 0x4d, 0x69, 0x72, 0x6e, 0x61, 0x5f, 0x32, 0x22, 0x7d]

The fact that now we were getting a 200 OK status with the message "No Headers, Simple Response" seems to confirm that the issue was indeed related to the headers. By removing the headers, the error is no longer occurring, and the request is processed successfully. By introducing a header back in the error came also back.

The issue appears to be how HTTP.jl is interpreting or constructing headers, specifically related to its internal handling of key-value pairs for headers. The error invalid header key-value pair: { implies that HTTP.jl is encountering malformed data while building or sending the response headers.

Trying different versions didn't help. What can be done to overcome this error?

[benelsen]

The method signature for HTTP.Response with body and headers is Response(status, headers, body) not Response(status, body, headers).
Switching around response_body and headers in the first example works as expected for me.

[haenf]

Ok, simple enough... Thank you for your quick reply!