Skip to main content

Import a Model from HuggingFace

ColabHive is not a fixed list of models. You get two things at once:

  1. A curated base catalog maintained by ColabHive — LLMs, specialists, generative models, and trainable templates that are ready to call.
  2. The ability to import any compatible model from HuggingFace — search it, check that it will run, and register it. Registration creates a model_config and a public inference endpoint, and the model is usable immediately (the first request triggers a one-time cold start).

This guide covers the import path end to end: search → info → register → infer. Many models in the catalog got there exactly this way (their names carry an hf- prefix).

HuggingFace imports are public, free, and base — always

When you register a model through the HuggingFace path, the platform forces it to visibility=public, lifecycle_status=candidate, is_base_model=true, and price_per_request=0. Only those public/candidate values are accepted during import; alternatives return HTTP 422. If you need a private, priced endpoint, that is the trained-model path instead — see Register a Trained Model for Inference.

Prerequisites​

  • An API key (prefix hive_) and account ID — see Authentication.
  • Optionally the Python SDK: pip install colabhive.

Your account is derived from the API key, so X-Account-ID is optional on the REST calls below and must match that key's account when supplied. The SDK sends it when account_id is configured.

1. Search HuggingFace​

Find candidate repos. Results are pre-checked for compatibility so you can see up front what will run.

from colabhive import ColabHive

client = ColabHive(api_key="hive_...", account_id="...")

results = client.models.hf.search(
    query="multilingual embeddings",
    task_type="embeddings",     # ColabHive task vocabulary (see below)
    max_size_gb=5,
    min_downloads=1000,
    limit=20,
)

for m in results:               # results are HFModelInfo dataclasses — use attributes, not dict keys
    print(m.repo_id, m.compatibility.status, m.downloads)

REST equivalent:

curl -X POST "https://api.colabhive.com/api/builder/v1/models/hf/search" \
  -H "X-API-Key: $COLABHIVE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query": "multilingual embeddings", "task_type": "embeddings", "max_size_gb": 5, "limit": 20}'

The response carries a results array. Each entry has repo_id, task_type, downloads, likes, size_mb, license, architectures, framework, a compatibility object (status, runtime_profile_id, checks), and metadata.

task_type is the ColabHive task vocabulary, not the HuggingFace pipeline tag. Valid values: embeddings, rerank, translation, ocr, stt, moderation, text-generation. A HuggingFace-style tag such as feature-extraction does not map and the filter is silently ignored.

2. Inspect a candidate​

Before registering, pull the full compatibility report and resource estimate for one repo.

details = client.models.hf.info("sentence-transformers/all-MiniLM-L6-v2")

print(details.compatibility.status)          # compatible | requires_review | incompatible
print(details.estimated_requirements)        # VRAM / params estimate for placement

REST equivalent (URL-encode the repo id in the path):

curl "https://api.colabhive.com/api/builder/v1/models/hf/sentence-transformers/all-MiniLM-L6-v2/info" \
  -H "X-API-Key: $COLABHIVE_API_KEY"

The info response returns repo_id, model_card, config, size_mb, files, a compatibility object, and estimated_requirements. Use the live estimate here rather than assuming VRAM — it is derived per repo.

Reading the compatibility status​

compatibility.statusMeaningWhat to do
compatibleNative support for the architectureRegister directly
requires_reviewRuns, but has warnings (license, size, config)Read the checks, usually fine
incompatibleThe architecture/backend won't serve itPick another repo

3. Register the model​

Registration creates the model_config and a public inference endpoint in one call. task_type is required.

result = client.models.hf.register(
    repo_id="sentence-transformers/all-MiniLM-L6-v2",
    task_type="embeddings",             # REQUIRED
    display_name="MiniLM Embeddings",
    tags=["embeddings", "multilingual"],
)

print(result.endpoint_id)               # attribute access — HFRegistrationResult
print(result.model_config_id)           # a UUID (the hf-... string is the model NAME, not the id)
print(result.inference_url)             # ready-to-call endpoint path
print(result.test_request)              # a sample request body to try

REST equivalent:

curl -X POST "https://api.colabhive.com/api/builder/v1/models/hf/register" \
  -H "X-API-Key: $COLABHIVE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "repo_id": "sentence-transformers/all-MiniLM-L6-v2",
    "task_type": "embeddings",
    "display_name": "MiniLM Embeddings",
    "tags": ["embeddings", "multilingual"]
  }'

The response contains model_config_id (a UUID), endpoint_id (a UUID), repo_id, status, message, inference_url, and test_request.

What registration sets, and what it ignores:

  • The new model must enter at lifecycle_status="candidate"; promote it later through the lifecycle endpoint.
  • visibility must be public; the endpoint is base and free. Non-public values return HTTP 422.
  • The model_config's model_name is generated with an hf- prefix (e.g. hf-sentence-transformers-all-MiniLM-L6-v2). That prefix is how you tell an import apart from a base-curated model in the catalog.

4. Run inference​

Call the endpoint straight away. The first request may cold-start the model (it has to download and load), so the initial call is slower; subsequent calls hit the warm replica.

out = client.endpoints.infer(
    result.endpoint_id,
    {"text": "Hello world"},
)
print(out["result"])

For chat/LLM imports (task_type="text-generation"), send {"messages": [...]} and read out["result"]. See Inference Lifecycle for warm/cold behavior and how sync vs. polling works.

Managing what you imported​

List and re-stage models in your registry, and move a model through its lifecycle.

# List imported (HuggingFace-sourced) models — offset pagination
imported = client.models.hf.list_registry(source="huggingface", limit=50, offset=0)
for m in imported:
    print(m.repo_id, m.lifecycle_status)

# Promote a validated model, or disable one that misbehaves.
# lifecycle values: candidate | ready | preferred | disabled  (disabled REQUIRES a reason)
client.models.hf.update_lifecycle(result.model_config_id, "ready")
client.models.hf.update_lifecycle(result.model_config_id, "disabled", reason="superseded by v2")

GGUF (CPU) imports​

A GGUF repository can be imported to run on the CPU tier (llama.cpp) by passing the gguf_file field on register. The CPU-LLM tier is wired in the runtime, but note there is no live CPU-LLM catalog today — see CPU LLM Inference (llama.cpp / GGUF) for the current state.

See also​