ONNX Usage Patterns

This page maps common ONNX use cases to the Uni-Xervo developer flow.

Tabular regression or classification

Best fit for local/onnx.

Developer flow:

1. Build numeric feature tensors in app code.
2. Put them into TensorBatch.
3. Call run().
4. Read scalar or logits outputs.

Why it works well:

• no tokenizer,
• usually one or two tensors,
• minimal postprocessing.

Sequence classification

Examples:

• sentiment
• topic classification
• moderation

Developer flow:

1. Tokenize text in app code.
2. Build input_ids and attention_mask.
3. Call run().
4. Read logits and map labels.

This is one of the best HF ONNX patterns for Uni-Xervo today.

Token classification / NER

Examples:

• NER
• PII tagging
• slot extraction

Developer flow:

1. Tokenize with offsets.
2. Build transformer input tensors.
3. Call run().
4. Read token logits.
5. Reconstruct spans in app code.

Uni-Xervo handles execution cleanly here, but the token-to-span logic still belongs in your application.

Dense text embedding (Embed task)

As of 0.8.0, local/onnx serves the Embed task directly — this replaces the retired local/fastembed provider. The popular text-embedding aliases (BGESmallENV15, AllMiniLML6V2, NomicEmbedTextV15, MultilingualE5Base, …) ship as built-in presets that resolve the HF repo, ONNX path, pooling kind, dimensions, and token_type_ids automatically. For multilingual dense embeddings, BGEM3 (official BAAI/bge-m3) and BGEM3Dense (the dense head of the multi-output aapot/bge-m3-onnx export) are both available.

Catalog config (preset alias):

{
  "alias": "embed/local",
  "task": "embed",
  "provider_id": "local/onnx",
  "model_id": "BGESmallENV15"
}

Catalog config (custom HF model, pass-through):

{
  "alias": "embed/custom",
  "task": "embed",
  "provider_id": "local/onnx",
  "model_id": "Snowflake/snowflake-arctic-embed-m",
  "options": {
    "artifact": "onnx/model.onnx",
    "pooling": "cls",
    "dimensions": 768,
    "token_type_ids": true
  }
}

Developer flow:

1. Resolve the typed handle: let embedder = runtime.embedding("embed/local").await?;
2. Call embedder.embed(&["hello world", "second doc"]).await? — Uni-Xervo handles tokenization, ORT session execution, pooling, and L2 normalization.
3. Each row of result.vectors is Vec<f32> of length embedder.dimensions().

If you have a custom export that returns hidden states and you want to handle pooling yourself, use the Raw task instead and pool in app code.

Learned-sparse embedding (EmbedSparse task)

Produces term-weight (lexical) vectors for SPLADE-style or BGE-M3 sparse retrieval. sparse_method selects the post-processing recipe: "mlm" reads MLM logits over the full vocabulary (SPLADE term expansion), "lexical" re-weights the input token ids (BGE-M3 lexical weights).

Catalog config:

{
  "alias": "sparse/splade",
  "task": "embed_sparse",
  "provider_id": "local/onnx",
  "model_id": "naver/splade-v3",
  "options": {
    "sparse_method": "mlm",
    "max_seq_len": 512
  }
}

Developer flow:

1. Resolve the typed handle: let embedder = runtime.sparse_embedder("sparse/splade").await?;
2. Call embedder.embed(&["hello world", "second doc"]).await?.
3. Each row of result.vectors is a SparseVector — a Vec<(u32, f32)> of (term_id, weight) pairs; use options.top_k to cap the number of retained terms.

Multi-vector / late-interaction embedding (EmbedMultiVector task)

Produces per-token ColBERT-style embeddings for late-interaction (MaxSim) retrieval.

Catalog config:

{
  "alias": "colbert/v2",
  "task": "embed_multi_vector",
  "provider_id": "local/onnx",
  "model_id": "colbert-ir/colbertv2.0",
  "options": {
    "dimensions": 128,
    "normalize": true,
    "drop_special_tokens": true
  }
}

Developer flow:

1. Resolve the typed handle: let embedder = runtime.multi_vector_embedder("colbert/v2").await?;
2. Call embedder.embed(&["hello world", "second doc"]).await?.
3. Each row of result.vectors is a Vec<Vec<f32>> — one dense vector per surviving token.

Single-pass hybrid embedding (EmbedHybrid task)

For multi-output graphs that fuse several heads — notably BGE-M3 (aapot/bge-m3-onnx, which emits dense_vecs / sparse_vecs / colbert_vecs) — the hybrid task runs one forward pass and post-processes every requested head from it, instead of loading three sessions for the per-task Embed / EmbedSparse / EmbedMultiVector resolvers.

Catalog config (preset alias):

{
  "alias": "embed_hybrid/bgem3",
  "task": "embed_hybrid",
  "provider_id": "local/onnx",
  "model_id": "BGEM3Hybrid"
}

Developer flow:

1. Resolve the handle: let model = runtime.hybrid_embedder("embed_hybrid/bgem3").await?;
2. Call model.embed(&["query", "doc"], HeadSet::ALL).await? — a single pass.
3. result.dense / result.sparse / result.multi_vector are each Some for the heads you requested via the HeadSet (and the graph exposes), None otherwise. model.available_heads() reports what the graph supports.

This is preset-driven: only models with a hybrid preset (declaring each head's output and recipe) resolve here. Single-head models use the per-task resolvers.

Cross-encoder reranking (Rerank task)

Catalog config:

{
  "alias": "rerank/cross",
  "task": "rerank",
  "provider_id": "local/onnx",
  "model_id": "cross-encoder/ms-marco-MiniLM-L6-v2"
}

Developer flow:

1. Resolve the handle: let reranker = runtime.reranker("rerank/cross").await?;
2. Call reranker.rerank("query", &["doc a", "doc b"]).await? to get scored documents back.

Image classification

Developer flow:

1. Resize/normalize image in app code.
2. Build the expected image tensor layout.
3. Call run().
4. Decode logits.

The runtime is a good fit; preprocessing remains model-specific.

Object detection or complex multimodal graphs

These are possible, but more manual.

Developer flow:

1. Inspect signatures carefully.
2. Build custom tensors.
3. Run inference.
4. Decode boxes, masks, spans, or custom outputs outside Uni-Xervo.

That is the point where local/onnx intentionally stays low-level.

Which ONNX cases feel easiest today

Best current DX:

• tabular models
• regression
• sequence classification
• NER
• custom numeric graphs

More manual but still supported:

• image models
• object detection
• audio models
• unusual multi-input graphs