Expand description
Reliability primitives: circuit breaker, instrumented model wrappers with timeout and retry support, and metrics emission.
Structsยง
- Circuit
Breaker Config - Tunable parameters for the circuit breaker.
- Circuit
Breaker Wrapper - Thread-safe circuit breaker that tracks failures and short-circuits calls when a provider is unhealthy.
- Instrumented
Audio Embedding Model - Wrapper around an
AudioEmbeddingModeladding timeout / retry / metrics. - Instrumented
Document Extraction Model - Wrapper around a
DocumentExtractionModeladding timeout / retry / metrics. - Instrumented
Embedding Model - Wrapper around an
EmbeddingModelthat adds per-call timeout enforcement, exponential-backoff retries for transient errors, and metrics emission (model_inference.duration_seconds,model_inference.total). - Instrumented
Generator Model - Wrapper around a
GeneratorModelthat adds timeout, retry, and metrics. - Instrumented
Hybrid Embedding Model - Wrapper around a
HybridEmbeddingModeladding timeout / retry / metrics. - Instrumented
Image Embedding Model - Wrapper around an
ImageEmbeddingModeladding timeout / retry / metrics. - Instrumented
Multi Vector Embedding Model - Wrapper around a
MultiVectorEmbeddingModeladding timeout / retry / metrics. - Instrumented
Multimodal Embedding Model - Wrapper around a
MultimodalEmbeddingModeladding timeout / retry / metrics. - Instrumented
NlpModel - Wrapper around an
NlpModeladding timeout / retry / metrics. - Instrumented
OcrModel - Wrapper around an
OcrModeladding timeout / retry / metrics. - Instrumented
RawTensor Model - Wrapper around an
RawTensorModelthat adds timeout, retry, and metrics. - Instrumented
Reranker Model - Wrapper around a
RerankerModelthat adds timeout, retry, and metrics. - Instrumented
Sparse Embedding Model - Wrapper around a
SparseEmbeddingModeladding timeout / retry / metrics. - Instrumented
Transcription Model - Wrapper around a
TranscriptionModeladding timeout / retry / metrics to the batchtranscribecall.