Skip to main content

Mamba (SSM) Fine-Tuning

Fine-tune a Mamba state-space language model on your own data

Overview​

  • Model ID: mamba-ssm-finetune
  • Type: Training template (fine-tuning), not a hosted inference model
  • Architecture: Mamba — a selective state-space model (SSM) for sequences
  • Framework: HuggingFace Transformers
  • Hardware: GPU (~8 GB VRAM class)
This is a training template

mamba-ssm-finetune is a fine-tuning template. ColabHive does not ship a ready-to-call public Mamba inference endpoint — you train an adapter/model on your data, then register the result for inference. (If you just want to run a Mamba checkpoint, import one from Hugging Face via POST /api/builder/v1/models/hf/register, subject to backend support.)

Mamba replaces attention with a selective state-space mechanism, giving linear-time sequence processing and a large effective context without the quadratic cost of attention. Fine-tune it when you want SSM behavior adapted to your domain.

When to Use​

✅ Perfect for:

  • Adapting a Mamba SSM base to a domain corpus or instruction set
  • Long-sequence workloads where linear-time processing is attractive
  • Research and benchmarking against attention-based fine-tunes

❌ Not ideal for:

Quick Start​

from colabhive import ColabHive

client = ColabHive(api_key="...", account_id="...")

dataset = client.datasets.upload(name="my-corpus", file="./train.jsonl")

job = client.training.create(
    model="mamba-ssm-finetune",
    dataset_id=dataset.id,
    hyperparameters={
        "epochs": 3,
        "batch_size": 4,
        "learning_rate": 2e-4,
        "max_length": 1024,
    },
)
job.wait()
print(job.get_metrics())

# After training, publish it as an inference endpoint
endpoint = client.training.register_for_inference(
    run_id=job.id,
    name="my-mamba-finetune",
    description="Fine-tuned Mamba SSM",
    visibility="account",
)

The exact accepted hyperparameters come from the template's schema: GET /api/builder/v1/training/model-configs/{model_config_id}/schema.

Dataset Format​

Text data in JSONL (instruction or chat) or plain-text records — the same shapes accepted by the LLM QLoRA template:

{"instruction": "Summarize:", "input": "Long text...", "output": "Summary..."}
{"messages": [{"role": "user", "content": "Hi"}, {"role": "assistant", "content": "Hello!"}]}

Canonical dataset formats: jsonl, parquet, hf_dataset, csv.