Expand description
Core traits that every provider and model implementation must satisfy.
Structs§
- Generation
Options - Sampling and length parameters for text generation.
- Generation
Result - The output of a text generation call.
- Provider
Capabilities - Advertised capabilities of a
ModelProvider. - Scored
Doc - A single scored document returned by a
RerankerModel. - Token
Usage - Token counts for a generation request.
Enums§
- Provider
Health - Health status reported by a provider.
Traits§
- Embedding
Model - A model that produces dense vector embeddings from text.
- Generator
Model - A model that generates text from a conversational message history.
- Model
Provider - A pluggable backend that knows how to load models for one or more
ModelTasktypes. - Reranker
Model - A model that re-scores documents against a query for relevance ranking.
Type Aliases§
- Loaded
Model Handle - A type-erased, reference-counted handle to a loaded model instance.