Feature Flags¶

Uni-Xervo's feature surface is small and orthogonal. Three independent axes:

Providers — which model backends to compile in.
ORT linking — bundled or BYO. Only matters if you use ONNX.
GPU acceleration — additive; off by default.

Defaults¶

uni-xervo = "0.19" enables both local backends and all nine remote providers on CPU:

provider-candle, provider-onnx,
provider-openai, provider-gemini, provider-vertexai, provider-mistral,
provider-anthropic, provider-voyageai, provider-cohere, provider-azure-openai,
provider-llamacpp

provider-whisper-cpp is not in defaults: it compiles whisper.cpp's C/C++ source via CMake and needs a working C/C++ toolchain on the build host, so it stays opt-in. Most users don't need to think about features. Pass default-features = false when you want a leaner build.

Providers¶

Local¶

Feature	Provider ID	Tasks
`provider-candle`	`local/candle`	embed
`provider-onnx`	`local/onnx`	raw, rerank, embed, embed_sparse, embed_multi_vector, embed_image, nlp, ocr, document_extract
`provider-onnx-dynamic`	`local/onnx`	same as `provider-onnx` (BYO ORT linking)
`provider-whisper-cpp`	`local/whisper-cpp`	transcribe (opt-in; needs CMake + C/C++ toolchain at build time)

provider-onnx and provider-onnx-dynamic are mutually exclusive — pick one.

For local/onnx, document_extract ships as a scaffold today (catalog wiring, options, and a reusable greedy autoreg decoder are production-ready; the inference path returns Unavailable until an upstream canonical ONNX export of a target VLM ships). All other tasks on this provider are fully wired — including ocr, which gained an optional DBNet detection stage for full-page detect→recognize. The live document_extract path today is provider-llamacpp, which runs a GGUF document VLM on a llama-server started with --mmproj.

Remote¶

Feature	Provider ID	Tasks
`provider-openai`	`remote/openai`	embed, generate
`provider-gemini`	`remote/gemini`	embed, generate, embed_multimodal
`provider-vertexai`	`remote/vertexai`	embed, generate
`provider-mistral`	`remote/mistral`	embed, generate
`provider-anthropic`	`remote/anthropic`	generate
`provider-voyageai`	`remote/voyageai`	embed, rerank
`provider-cohere`	`remote/cohere`	embed, rerank, generate, embed_multimodal
`provider-azure-openai`	`remote/azure-openai`	embed, generate
`provider-llamacpp`	`remote/llamacpp`	embed (llama.cpp `llama-server`, tokenizer-aware truncation), generate (text and vision), rerank

All remote providers share a single reqwest dependency, so enabling all of them costs roughly the same as enabling one. provider-cohere and provider-gemini additionally pull base64 for encoding raw image / audio bytes into the multimodal embed request payload.

ORT linking¶

Only matters if you use provider-onnx*.

Feature	Behavior
`provider-onnx`	ONNX Runtime is statically linked into your binary at build time (pyke fetches the prebuilt). Self-contained, zero runtime configuration. Default.
`provider-onnx-dynamic`	ONNX Runtime is loaded at runtime via `dlopen`. You supply the shared library at deploy time and set `ORT_DYLIB_PATH`. Use for sandboxed CIs that can't reach the network at build time, custom ORT builds, or vendor-supplied builds (AMD ROCm, Intel OpenVINO, Microsoft DirectML, Qualcomm QNN).

build.rs panics if both are enabled.

GPU acceleration¶

CPU is always available — there is no "CPU feature." GPU features are purely additive: at runtime, ORT registers the GPU EP first and silently falls back to CPU per-op if the GPU path isn't available.

Feature	Targets	What it activates
`gpu-cuda`	Linux + Windows, NVIDIA	ORT CUDA EP (when ORT is in the build) + CUDA kernels for candle. Pyke fetches the CUDA-flavored ORT bundle automatically; sidecar EP libs are staged into `target/<profile>/` via `ort/copy-dylibs`.
`gpu-metal`	macOS + iOS	ORT CoreML EP (when ORT is in the build, hits Apple GPU + Neural Engine) + Metal kernels for candle. The CoreML EP is in pyke's standard macOS bundle — no extra fetch.

Build host requirements:

gpu-cuda needs nvcc and CUDA_COMPUTE_CAP set for candle PTX generation. ORT side is handled by pyke.
gpu-metal needs an Apple build target — build.rs panics on non-Apple targets.

Don't enable both — they target different platforms. For other vendors (AMD ROCm, Intel OpenVINO, Microsoft DirectML, Qualcomm QNN, TensorRT, WebGPU), use provider-onnx-dynamic with a vendor-supplied ORT build at runtime via ORT_DYLIB_PATH, and request the EP per alias by setting execution_providers to e.g. ["rocm", "cpu"].

For NVIDIA / Linux runtime library setup (cuDNN, LD_LIBRARY_PATH) and how to verify the GPU EP actually loaded, see the GPU setup guide.

Runtime EP selection (ORT only)¶

When a catalog spec doesn't set execution_providers, the default list is feature-aware:

Compiled with…	Default EP list
`gpu-cuda`	`[Cuda, Cpu]` — CUDA preferred, CPU fallback
`gpu-metal` (no `gpu-cuda`)	`[CoreMl, Cpu]` — CoreML preferred, CPU fallback
neither	`[Cpu]`

User-supplied execution_providers strings are checked in two stages:

Catalog options validation (at runtime build/register time) accepts only "cpu", "cuda", "coreml", "directml". Any other string — including "rocm", "openvino", "qnn", "tensorrt", "webgpu" — is rejected here with a RuntimeError::Config.

Session build (when the ORT session is created) additionally requires the right feature/runtime backing for the EP you asked for. The vendor EPs (directml, rocm, openvino, qnn, tensorrt, webgpu) require the provider-onnx-dynamic feature plus a vendor-supplied ONNX Runtime library via ORT_DYLIB_PATH; cuda requires gpu-cuda and coreml requires gpu-metal.

What happens when the backing is missing depends on whether the list you supplied leaves a way out:

Requested list	Result
Has a viable entry — e.g. `["cuda", "cpu"]` without `gpu-cuda`	`cuda` is dropped with a `WARN` log and the session is built from what remains (`["cpu"]`). An explicit fallback entry is read as asking for exactly this.
Nothing viable — e.g. `["cuda"]` or `["rocm"]`	`RuntimeError::Config` naming the missing feature. Asking for one specific accelerator and silently getting CPU would defeat the point.

Since cpu is always available, the error case is exactly "a list with no cpu entry whose every accelerator is unbacked". If you want a missing feature to be a hard failure — as a build-misconfiguration canary — request the accelerator on its own, without a cpu entry.

Common build recipes¶

# Default — everything except GPU.
uni-xervo = "0.19"

# Add NVIDIA GPU (Linux / Windows).
uni-xervo = { version = "0.19", features = ["gpu-cuda"] }

# Add Apple GPU + Neural Engine (macOS / iOS).
uni-xervo = { version = "0.19", features = ["gpu-metal"] }

# Lean — only candle.
uni-xervo = { version = "0.19", default-features = false, features = ["provider-candle"] }

# Remote-only — no native deps at all.
uni-xervo = { version = "0.19", default-features = false, features = [
  "provider-openai",
  "provider-anthropic",
] }

# Local stack with ONNX Runtime.
uni-xervo = { version = "0.19", default-features = false, features = [
  "provider-candle",
  "provider-onnx",
] }

# BYO ONNX Runtime (ROCm, OpenVINO, custom builds, sandboxed CI).
uni-xervo = { version = "0.19", default-features = false, features = [
  "provider-candle",
  "provider-onnx-dynamic",
] }
# Then at runtime: ORT_DYLIB_PATH=/path/to/libonnxruntime.so ./your-binary

# Add local speech-to-text via whisper.cpp (opt-in).
uni-xervo = { version = "0.19", features = ["provider-whisper-cpp"] }
# Build host needs cmake + a C/C++ toolchain.

Runtime registration reminder¶

Enabling features compiles provider code; it does not auto-register providers.

Register each provider in ModelRuntime::builder() before build().