LLM Fine-Tuning (QLoRA)

Custom chat and instruction models via QLoRA / PEFT

---

Overview

• Active template ID: llm-qlora-finetune (QLoRA, lifecycle=ready)
• Type: Large Language Model fine-tuning
• Method: QLoRA — 4-bit quantized base + trainable LoRA adapters
• Hardware: GPU
• Best for: Custom chatbots, instruction following, domain assistants

QLoRA fine-tunes a quantized base model by training small LoRA adapters, so you can adapt a multi-billion-parameter model on a single GPU. llm-qlora-finetune is the tested, active template; llm-qlora-peft (PEFT) and llm-merge-adapter (adapter merging) are companion templates.

About llama-3b-finetune

A dedicated llama-3b-finetune preset exists in the catalog but is currently not active. For Llama-family (and other) LLM fine-tuning today, use the active llm-qlora-finetune template and select your base model there. Always confirm availability against the live catalog: GET /api/builder/v1/inference/models?category=deep_learning.

---

When to Use

✅ Perfect for:

• Custom chatbots and domain-specific assistants
• Instruction fine-tuning and Q&A systems
• Adapting tone/format to your data

❌ Not ideal for:

• Simple text classification (use BERT)
• Very small datasets (a few hundred examples or fewer)

---

Quick Start

from colabhive import ColabHive

client = ColabHive(api_key="...", account_id="...")

# Upload an instruction/chat dataset (JSONL)
dataset = client.datasets.upload(name="instructions", file="./train.jsonl")

# Fine-tune with QLoRA
job = client.training.create(
    model="llm-qlora-finetune",
    dataset_id=dataset.id,
    hyperparameters={
        "epochs": 3,
        "batch_size": 4,
        "learning_rate": 2e-4,
        "max_length": 512,
        "lora_r": 8,
        "lora_alpha": 16,
    },
)
job.wait()
print(job.get_metrics())

# Register the fine-tuned model as an inference endpoint
endpoint = client.training.register_for_inference(
    run_id=job.id,
    name="my-llm-custom",
    description="Fine-tuned instruction model (QLoRA)",
    visibility="account",
)

# Run inference
result = client.endpoints.infer(
    endpoint_id=endpoint.endpoint_id,
    input_data={
        "messages": [
            {"role": "user", "content": "Explain quantum computing in simple terms."}
        ]
    },
)
print(result)

---

Dataset Format

JSONL (instruction)

{"instruction": "Summarize this text:", "input": "Long text...", "output": "Summary..."}
{"instruction": "Translate to Spanish:", "input": "Hello world", "output": "Hola mundo"}

JSONL (chat)

{"messages": [{"role": "user", "content": "Hi"}, {"role": "assistant", "content": "Hello!"}]}

Requirements

• A few hundred high-quality examples minimum; a few thousand is better.
• Diverse instructions and clean, consistent responses.

---

Hyperparameters

Parameter	Default	Range	Description
epochs	3	1-10	Training epochs
batch_size	4	1-8	Micro-batch size
learning_rate	2e-4	1e-5 to 5e-4	LoRA learning rate
max_length	512	128-2048	Max tokens (input + output)
lora_r	8	4-64	LoRA rank (higher = more capacity)
lora_alpha	16	8-32	LoRA scaling factor

---

QLoRA Explained

QLoRA = Quantized Low-Rank Adaptation.

• 4-bit quantization of the frozen base model cuts memory dramatically.
• LoRA trains only a tiny fraction of parameters (the adapters).
• Result: fine-tune large models on a single GPU instead of a multi-GPU cluster.

---

Tips

1. Quality over quantity: a small set of high-quality examples beats a large noisy one.
2. Diverse formats: train on varied instruction/chat shapes for robustness.
3. System prompts: include them in training for consistent behavior.
4. Longer context: raise max_length, lower batch_size, and add gradient_accumulation_steps to compensate.
5. Evaluate on held-out instructions before publishing.

---

Related Templates

• SSM / hybrid fine-tuning: Mamba (SSM) · Jamba (hybrid)
• BERT Classification — text classification
• Preparing Instruction Datasets
• All Models