Expand description
Core traits that every provider and model implementation must satisfy.
Structs§
- Audio
Output - Audio output produced by a speech model.
- Generated
Image - An image produced by a generation call (e.g. from a diffusion model).
- Generation
Options - Sampling and length parameters for text generation.
- Generation
Result - The output of a generation call.
- Message
- A single message in a conversation, containing one or more content blocks.
- Provider
Capabilities - Advertised capabilities of a
ModelProvider. - Scored
Doc - A single scored document returned by a
RerankerModel. - Token
Usage - Token counts for a generation request.
Enums§
- Content
Block - A single block of content within a
Message. - Image
Input - Image data that can be passed as part of a
ContentBlock. - Message
Role - The role of a message in a conversation.
- Provider
Health - Health status reported by a provider.
Traits§
- Embedding
Model - A model that produces dense vector embeddings from text.
- Generator
Model - A model that generates text, images, or audio from a conversational message history.
- Model
Provider - A pluggable backend that knows how to load models for one or more
ModelTasktypes. - Reranker
Model - A model that re-scores documents against a query for relevance ranking.
Type Aliases§
- Loaded
Model Handle - A type-erased, reference-counted handle to a loaded model instance.