Module runtime

Module runtime 

Source
Expand description

The core runtime that manages providers, catalogs, and loaded model instances.

Structsยง

ModelRegistry
Internal registry that caches loaded model instances and coordinates concurrent load requests to prevent duplicate work.
ModelRuntime
The central runtime that owns registered providers and a catalog of model aliases.
ModelRuntimeBuilder
Builder for constructing a ModelRuntime with registered providers, a model catalog, and a warmup policy.