Struct InstrumentedGeneratorModel 

pub struct InstrumentedGeneratorModel {
    pub inner: Arc<dyn GeneratorModel>,
    pub alias: String,
    pub provider_id: String,
    pub timeout: Option<Duration>,
    pub retry: Option<RetryConfig>,
}

Wrapper around a GeneratorModel that adds timeout, retry, and metrics.

See InstrumentedEmbeddingModel for details on the instrumentation behavior.

Fields

inner: Arc<dyn GeneratorModel>

alias: String

provider_id: String

timeout: Option<Duration>

retry: Option<RetryConfig>

Trait Implementations

impl GeneratorModel for InstrumentedGeneratorModel

fn generate<'life0, 'life1, 'async_trait>( &'life0 self, messages: &'life1 [String], options: GenerationOptions, ) -> Pin<Box<dyn Future<Output = Result<GenerationResult>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Generate a response given a conversation history and sampling options.

fn warmup<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Optional warmup hook. The default is a no-op.