Skip to content

v1.8.1: Add AudioURLChunk

Choose a tag to compare

@juliendenize juliendenize released this 16 Jul 11:20
· 173 commits to main since this release
a153ac8

What's Changed

Now you can use http(s) URLs, file paths and base64 string (without specifying format) in your content chunks thanks to AudioURLChunk !

from mistral_common.protocol.instruct.messages import AudioURL, AudioURLChunk, TextChunk, UserMessage
from mistral_common.protocol.instruct.request import ChatCompletionRequest
from mistral_common.tokens.tokenizers.mistral import MistralTokenizer

repo_id = "mistralai/Voxtral-Mini-3B-2507"
tokenizer = MistralTokenizer.from_hf_hub(repo_id)

text_chunk = TextChunk(
    text="Wat do you think about this audio?"
)
user_msg = UserMessage(
    content=[
        AudioURLChunk(audio_url=AudioURL(url="https://freewavesamples.com/files/Ouch-6.wav")),
        text_chunk,
    ]
)

request = ChatCompletionRequest(messages=[user_msg])
tokenized = tokenizer.encode_chat_completion(request)

# pass tokenized.tokens to your favorite audio model
print(tokenized.tokens)
print(tokenized.audios)

# print text to visually see tokens
print(tokenized.text)

Full Changelog: v1.8.0...v1.8.1