Skip to content

ONNX Configuration

Required catalog fields

Every ONNX alias needs:

  • alias
  • task — one of raw, rerank, embed, embed_sparse, embed_multi_vector, embed_hybrid, embed_image, nlp, ocr, document_extract
  • provider_id: "local/onnx"
  • model_id

Example:

{
  "alias": "raw/minilm",
  "task": "raw",
  "provider_id": "local/onnx",
  "model_id": "nixiesearch/all-MiniLM-L6-v2-onnx",
  "options": {
    "artifact": "model.onnx"
  }
}

model_id

Supported forms:

  • absolute local path
  • relative local path
  • Hugging Face repo ID

HF repos are snapshotted into cache before ONNX Runtime loads the artifact.

Provider options

local/onnx supports a common set of options across every task:

  • artifact
  • max_batch_size
  • execution_providers
  • graph_optimization_level
  • inter_op_num_threads
  • intra_op_num_threads
  • cache_dir

Most tasks also accept task-specific keys (validated per task):

  • embed: tokenizer_path, pooling (cls/mean/max/last-token), normalize, dimensions, max_seq_len, token_type_ids, output_name
  • rerank: max_seq_len, style (cross-encoder/generative), instruction
  • embed_sparse: tokenizer_path, sparse_method (mlm/lexical), output_name, output_index, max_seq_len, token_type_ids, top_k
  • embed_multi_vector: tokenizer_path, dimensions, normalize, drop_special_tokens, output_name, output_index, max_seq_len, token_type_ids
  • embed_image: onnx_path, image_size, dimensions, normalization (siglip/imagenet), pool (none/mean), normalize, output_name
  • nlp: onnx_path, tokenizer_path, label_maps_path, max_seq_len
  • ocr: onnx_path, char_dict_path, image_height, image_width, normalization, blank_class, output_name, plus an optional DBNet detection stage (det_onnx_path, det_model_id, det_limit_side, det_bin_threshold, det_box_score_threshold, det_unclip_ratio, det_min_box_size, det_input_name, det_output_name)
  • document_extract: style (granite-docling/mineru/olmocr), onnx_path, tokenizer_path, max_seq_len

artifact

Use artifact when the HF repo contains more than one .onnx file.

{
  "artifact": "model.onnx"
}

If a repo has exactly one .onnx file, artifact can be omitted.

execution_providers

execution_providers is an ordered list.

Examples:

{
  "execution_providers": ["cpu"]
}

{
  "execution_providers": ["cuda", "cpu"]
}

Supported names:

  • cpu
  • cuda
  • coreml
  • directml

Defaults:

  • CPU-only builds: ["cpu"]
  • gpu-cuda builds: ORT-backed providers prefer ["cuda", "cpu"]

Batch tuning

max_batch_size controls the ceiling for dynamic-batch models.

{
  "max_batch_size": 32
}

For non-batch models, Uni-Xervo falls back to sequential execution in run_batch().

ORT tuning

Example:

{
  "graph_optimization_level": "extended",
  "inter_op_num_threads": 2,
  "intra_op_num_threads": 4
}

Accepted optimization levels:

  • disable
  • basic
  • extended
  • all

Validation behavior

Uni-Xervo validates ONNX options at runtime build/register time.

It rejects:

  • unknown option keys,
  • wrong value types,
  • unsupported execution provider names,
  • invalid batch/thread counts.

Schema files:

  • schemas/model-catalog.schema.json
  • schemas/provider-options/onnx.schema.json