Skip to content

Provider Selection

Choose providers based on task coverage, latency profile, data governance, and operational constraints.

Capability-first matrix

Provider ID Embed Rerank Generate Raw Typical use
local/candle Yes No No No Low-latency local embedding with simple deploys
local/onnx Yes Yes No Yes ONNX Runtime-backed dense embeddings (subsumes the retired local/fastembed provider; FastEmbed alias strings still resolve), cross-encoder reranking, and raw tensor execution. Also serves the extended tasks embed_sparse, embed_multi_vector, embed_image, nlp, and ocr — see the task trait surface
local/mistralrs Yes No Yes No Self-hosted local embedding + multimodal generation (text, vision, diffusion, speech)
remote/openai Yes No Yes No Hosted general-purpose embeddings and chat
remote/gemini Yes No Yes No Hosted Google model family
remote/vertexai Yes No Yes No GCP-native hosted models
remote/mistral Yes No Yes No Hosted Mistral models
remote/anthropic No No Yes No Hosted generation/chat only
remote/voyageai Yes Yes No No Hosted embedding + reranking focus
remote/cohere Yes Yes Yes No Hosted unified embedding/rerank/generate
remote/azure-openai Yes No Yes No Azure-governed OpenAI deployments

The columns above are the four foundational tasks. uni-xervo defines 13 ModelTask variants in total — the multimodal / structured-output tasks (embed_image, embed_audio, embed_multimodal, embed_sparse, embed_multi_vector, nlp, document_extract, transcribe, ocr) and the providers that implement them are covered in the task trait surface guide.

Decision framework

  1. Task coverage: ensure provider supports required task.
  2. Data policy: local providers for stricter data residency/control.
  3. Latency and throughput: local can reduce network latency; remote can simplify scaling.
  4. Reliability posture: tune timeout, retry, and warmup strategy per alias.
  5. Change management: keep alias names stable while swapping providers in catalog.

Common patterns

  • Local embed + remote generate:
  • embed/default -> local/candle
  • generate/default -> remote/openai
  • Multi-provider remote fallback strategy in app layer:
  • generate/primary -> remote/anthropic
  • generate/backup -> remote/gemini
  • RAG pipeline split:
  • embed via remote/voyageai
  • rerank via remote/cohere
  • generate via remote/azure-openai
  • Multimodal local pipelines:
  • generate/local -> local/mistralrs (text pipeline, default)
  • vision/qwen -> local/mistralrs (vision pipeline for image understanding)
  • image/flux -> local/mistralrs (diffusion pipeline for image generation)
  • tts/dia -> local/mistralrs (speech pipeline for audio synthesis)
  • ONNX-backed pipelines (local/onnx serves several tasks beyond dense embed — see the task trait surface for the full list):
  • embed/local -> local/onnx with task: "embed" and a preset alias like BGESmallENV15 (replaces the retired local/fastembed provider; same alias strings resolve)
  • rerank/cross -> local/onnx with task: "rerank" for cross-encoder rerankers
  • embed_sparse/splade -> local/onnx with task: "embed_sparse" for learned-sparse SPLADE / BGE-M3 term-weight vectors
  • embed_mv/colbert -> local/onnx with task: "embed_multi_vector" for ColBERT late-interaction per-token vectors
  • ocr/ppocr -> local/onnx with task: "ocr" for PP-OCR text recognition
  • raw/classifier -> local/onnx with task: "raw" for Hugging Face ONNX classifier exports
  • raw/tabular -> local/onnx with task: "raw" for custom numeric or regression graphs

Developer notes

  • Enable only required provider feature flags.
  • Register providers explicitly in runtime builder.
  • Validate catalogs in CI before deployment.
  • Use the provider reference pages for official model/config links: Provider Reference.