[Feat] Add streaming support to Codestral Mamba

[xNul]

Python -VV

N/A

Pip Freeze

N/A

Reproduction Steps

N/A

Expected Behavior

N/A

Additional Context

N/A

Suggested Solutions

mamba-ssm has added streaming support for the Mamba architecture Codestral Mamba uses via the streamer parameter which can be passed as an argument in this function call:

mistral-inference/src/mistral_inference/generate.py

Line 21 in 337e3db

output = model.model.generate(

So now it should be pretty easy to implement streaming support for Codestral Mamba in mistral-inference.