Expand description
Reliability primitives: circuit breaker, instrumented model wrappers with timeout and retry support, and metrics emission.
Structsยง
- Circuit
Breaker Config - Tunable parameters for the circuit breaker.
- Circuit
Breaker Wrapper - Thread-safe circuit breaker that tracks failures and short-circuits calls when a provider is unhealthy.
- Instrumented
Embedding Model - Wrapper around an
EmbeddingModelthat adds per-call timeout enforcement, exponential-backoff retries for transient errors, and metrics emission (model_inference.duration_seconds,model_inference.total). - Instrumented
Generator Model - Wrapper around a
GeneratorModelthat adds timeout, retry, and metrics. - Instrumented
Reranker Model - Wrapper around a
RerankerModelthat adds timeout, retry, and metrics.