Architecture & Backend Compatibility
This is the single reference for which model architectures ColabHive can serve and on which
inference backends. Architecture support is 100% database-driven: when you register a model, the
platform reads its config.json architecture and picks a compatible backend automatically.
Counts and per-architecture details are a snapshot. To know whether a specific HuggingFace repo will run — and on which backend — ask the live compatibility check, which reads the repo and returns an authoritative answer:
curl "https://api.colabhive.com/api/builder/v1/models/hf/{owner}/{repo}/info" \
-H "X-API-Key: hive_..."
The compatibility object reports status (compatible · requires_review · incompatible), the
runtime_profile_id (backend that will be used), and checks; estimated_requirements reports VRAM
and a recommended config.
How a backend is chosen
ColabHive selects an inference backend from the model's architecture:
- vLLM (native). Architectures with first-class vLLM support get PagedAttention, continuous batching, and CUDA graphs — the fast path for production LLM serving.
- Transformers (universal fallback). Any HuggingFace
*ForCausalLM(and custommodeling_*.pyarchitectures) that vLLM does not natively accelerate falls back to the transformers backend. Slower, but 100% compatible. - Specialized backends. Diffusion/generative models, time-series specialists, SSM/hybrid models, and CPU GGUF (llama.cpp) each route to their own backend (see the backend table below).
You do not choose the backend manually — registration resolves it from the architecture and the model's runtime profile.
Supported architectures (snapshot 2026-08-15)
As of this snapshot the registry recognizes 393 active architectures across the families below. This is a moving target — it more than quadrupled between the 2026-07-23 and 2026-08-15 snapshots; verify a specific repo with the compatibility check above.
Representative families (not exhaustive):
| Family | Example architectures | Example models |
|---|---|---|
| Llama | LlamaForCausalLM, Llama4ForCausalLM, CodeLlamaForCausalLM | Llama 2/3/4, CodeLlama, Vicuna |
| Mistral | MistralForCausalLM, MixtralForCausalLM | Mistral 7B, Mixtral MoE |
| Qwen | Qwen2ForCausalLM, Qwen3ForCausalLM, Qwen2MoeForCausalLM, QWenLMHeadModel | Qwen 1/2.5/3, Qwen MoE |
| Gemma | GemmaForCausalLM, Gemma2ForCausalLM, Gemma3ForCausalLM | Gemma, Gemma 2, Gemma 3 |
| Phi | PhiForCausalLM, Phi3ForCausalLM, PhiMoEForCausalLM | Phi-2, Phi-3/3.5, Phi MoE |
| DeepSeek | DeepseekForCausalLM, DeepseekV2ForCausalLM, DeepseekV3ForCausalLM | DeepSeek, DeepSeek Coder, v2/v3 |
| GPT / classic | GPT2LMHeadModel, GPTNeoXForCausalLM, GPTJForCausalLM, BloomForCausalLM, OPTForCausalLM, FalconForCausalLM, MPTForCausalLM | GPT-2, Pythia, GPT-J, BLOOM, OPT, Falcon, MPT |
| GLM / ChatGLM | ChatGLMModel, Glm4ForCausalLM | ChatGLM, GLM-4 |
| MoE (incl. MLA) | Glm4MoeForCausalLM, Qwen3MoeForCausalLM, GptOssForCausalLM, NemotronHForCausalLM | GLM-4.7 Flash, Qwen3-Coder 30B-A3B, gpt-oss, Nemotron-3 Super |
| Multimodal / vision-language | Qwen3VLForConditionalGeneration, Qwen3VLMoeForConditionalGeneration, Mistral3ForConditionalGeneration, Glm4vForConditionalGeneration, InternVLForConditionalGeneration, KimiVLForConditionalGeneration, NemotronH_Nano_VL_V2 | Qwen3-VL, Ministral 3, GLM-4V, InternVL, Kimi-VL, Nemotron Nano VL |
| Other LLMs | BaichuanForCausalLM, InternLMForCausalLM, InternLM2ForCausalLM, YiForCausalLM, XverseForCausalLM, CohereForCausalLM, Cohere2ForCausalLM, DbrxForCausalLM, ArcticForCausalLM, NemotronForCausalLM, GraniteForCausalLM, OlmoForCausalLM, Olmo2ForCausalLM, StableLmForCausalLM, Starcoder2ForCausalLM, ExaoneForCausalLM, SolarForCausalLM, … | InternLM, Yi, Command, DBRX, Arctic, Nemotron, Granite, OLMo, StableLM, StarCoder 2, EXAONE, SOLAR |
| SSM / hybrid | MambaForCausalLM, Mamba2ForCausalLM, FalconMambaForCausalLM, JambaForCausalLM, ZambaForCausalLM | Mamba, Falcon-Mamba, Jamba, Zamba — fine-tuning templates only (see below) |
| Diffusion / generative | diffusers pipelines | SDXL, SD3, FLUX, MusicGen, AudioLDM2 (see Generative) |
| Time series | PatchTSTForPrediction, PatchTSTForClassification, BiTCN, TimesFM | PatchTST, BiTCN, TimesFM 2.5 |
These architectures are present for fine-tuning (mamba-ssm-finetune, jamba-hybrid-finetune).
They are not benchmarked and there are no public SSM inference endpoints. In the compatibility data
their rows are untested (throughput unknown). Do not treat them as ready-to-call chat models.
Benchmarked throughput (tested)
Only a small set of architecture/backend pairs is marked tested (14 of them today), and only the
ten below carry a real throughput number — all on the vllm-cu121 (NVIDIA) backend. All other rows in
the compatibility table are untested with no throughput figure — treat those as "should work", not
"measured".
| Architecture | Backend | Throughput (tok/s) | Benchmark model |
|---|---|---|---|
GPT-2 (GPT2LMHeadModel) | vllm-cu121 | 150 | gpt2 |
Mixtral MoE (MixtralForCausalLM) | vllm-cu121 | 120 | mistralai/Mixtral-8x7B-Instruct-v0.1 |
Phi-2 (PhiForCausalLM) | vllm-cu121 | 110 | microsoft/phi-2 |
Phi-3 (Phi3ForCausalLM) | vllm-cu121 | 105 | microsoft/Phi-3-mini-4k-instruct |
Llama (LlamaForCausalLM) | vllm-cu121 | 100 | meta-llama/Llama-2-7b-chat-hf |
Mistral (MistralForCausalLM) | vllm-cu121 | 95 | mistralai/Mistral-7B-Instruct-v0.3 |
Gemma (GemmaForCausalLM) | vllm-cu121 | 90 | google/gemma-7b-it |
Qwen 2 (Qwen2ForCausalLM) | vllm-cu121 | 90 | Qwen/Qwen2.5-7B-Instruct |
Qwen 1 (QWenLMHeadModel) | vllm-cu121 | 85 | Qwen/Qwen-7B-Chat |
GPT-NeoX (GPTNeoXForCausalLM) | vllm-cu121 | 80 | EleutherAI/pythia-6.9b |
Numbers are single-GPU order-of-magnitude figures for the benchmark model, not a guarantee for every model in the family. Transformers-fallback serving is typically several times slower than native vLLM.
The four remaining tested pairs are the GLM-4 MoE architectures (Glm4MoeForCausalLM,
Glm4MoeLiteForCausalLM and their MTP variants) on vllm-xpu — validated on Intel Arc on
2026-08-03 during the GLM-4.7 Flash bring-up, recorded as tested but without a throughput figure.
Inference backends
ColabHive runs 16 active inference backends across NVIDIA, AMD, Intel, and CPU — not just vLLM.
Thirteen of them appear in the current compatibility matrix. Images are referenced by repo; the
platform resolves the running version from system_versions at dispatch (never :latest), with two
legacy rows still carrying an explicit v1.0.0 pin.
backend_id | Vendor | Framework | Image (repo) |
|---|---|---|---|
vllm-cu121 | NVIDIA | vLLM | registry.colabhive.com/inference-vllm |
transformers-cu121 | NVIDIA | transformers | registry.colabhive.com/inference-base |
tgi-cu121 | NVIDIA | TGI | registry.colabhive.com/inference-tgi:v1.0.0 |
mamba-vllm-cu121 | NVIDIA | vLLM (+mamba-ssm) | registry.colabhive.com/inference-vllm |
diffusers-cu121 | NVIDIA | diffusers | registry.colabhive.com/inference-generative |
generative-cu121 | NVIDIA | transformers | registry.colabhive.com/inference-generative |
specialists-ts-cu121 | NVIDIA | specialists | registry.colabhive.com/inference-specialists |
vllm-rocm | AMD | vLLM | registry.colabhive.com/inference-rocm |
transformers-rocm | AMD | transformers | registry.colabhive.com/inference-rocm |
specialists-rocm | AMD | specialists | registry.colabhive.com/inference-rocm |
vllm-xpu | Intel | vLLM | registry.colabhive.com/inference-ipex |
ipex-xe | Intel | transformers (IPEX) | registry.colabhive.com/inference-ipex |
specialists-ipex-xpu | Intel | specialists | registry.colabhive.com/inference-ipex |
transformers-cpu | CPU | transformers | registry.colabhive.com/inference-base |
specialist-cpu-optimized | CPU | specialists | registry.colabhive.com/inference-base-cpu:v1.0.0 |
llm-llamacpp-cpu | CPU | llama.cpp | registry.colabhive.com/inference-llamacpp-cpu |
Intel XPU (vllm-xpu / ipex-xe) and AMD ROCm backends are in production alongside NVIDIA — each
vendor now has a matching specialists backend as well, so embeddings/rerank/OCR/STT/moderation run
on whatever silicon the node has. The CPU GGUF backend (llm-llamacpp-cpu) powers the
CPU LLM tier.
vllm-rocm60 and transformers-rocm60 were the earlier AMD entries pointing at a local
inference-base-rocm image. They are deactivated — the live AMD backends are vllm-rocm,
transformers-rocm and specialists-rocm on registry.colabhive.com/inference-rocm.
Requesting or adding architecture support
- Try the compatibility check first — most models resolve to vLLM-native or the transformers fallback automatically.
- If a model reports
incompatibleor you need native vLLM acceleration for a new architecture, open a GitHub issue with the repo ID, the architecture name fromconfig.json, and your use case.
Related
- Model Catalog — curated base + HuggingFace import
- LLM Inference · CPU LLM (GGUF)
- Models API — registration, compatibility, lifecycle
- Choosing a Model
Authors: José Luis Minich, Maximiliano Lucius.