BERT Classification (GPU)

Text classification with transfer learning (fine-tune BERT on your labels)

---

Overview

• Model ID: bert-classification-gpu
• Type: Text classification (fine-tuning template)
• Framework: HuggingFace Transformers
• Hardware: GPU (small footprint, ~1 GB VRAM)
• Base: BERT-base (110M parameters)
• Best for: Sentiment, topic, intent, and spam classification

Lifecycle

bert-classification-gpu is currently a candidate template (functional, still completing full validation). Check the live catalog (GET /api/builder/v1/inference/models?category=deep_learning) for its current lifecycle status.

---

When to Use

✅ Perfect for:

• Sentiment analysis
• Topic classification
• Intent detection
• Spam detection
• Any short-text classification task

❌ Not ideal for:

• Text generation (fine-tune an LLM instead — see LLM Fine-Tuning)
• Very long documents (BERT-base caps around 512 tokens)
• Tiny datasets (a few hundred labeled rows or fewer)

---

Quick Start

from colabhive import ColabHive

client = ColabHive(api_key="...", account_id="...")

# Upload a labeled text dataset (CSV with text + label columns)
dataset = client.datasets.upload(name="reviews", file="./reviews.csv")

# Fine-tune BERT
job = client.training.create(
    model="bert-classification-gpu",
    dataset_id=dataset.id,
    hyperparameters={
        "epochs": 3,
        "batch_size": 16,
        "learning_rate": 2e-5,
        "max_length": 128,
        "text_column": "review_text",
        "label_column": "sentiment",
    },
)
job.wait()
print(job.get_metrics())

# Register the fine-tuned model for inference
endpoint = client.training.register_for_inference(
    run_id=job.id,
    name="sentiment-classifier",
    description="Fine-tuned BERT sentiment classifier",
    visibility="account",
)

# Predict
predictions = client.endpoints.infer(
    endpoint_id=endpoint.endpoint_id,
    input_data={
        "instances": [
            {"text": "This product is amazing!"},
            {"text": "Not worth the money."},
        ]
    },
)
print(predictions)

---

Hyperparameters

Parameter	Default	Range	Description
epochs	3	1-10	Training epochs (3 is usually enough)
batch_size	16	8-32	Samples per batch
learning_rate	2e-5	1e-5 to 5e-5	Fine-tuning LR
max_length	128	32-512	Max tokens per text
text_column	required	—	Name of the text column
label_column	required	—	Name of the label column

---

Dataset Format

CSV Format

text,label
"This movie was great!",positive
"Terrible experience.",negative
"Just okay, nothing special.",neutral

Requirements

• At least ~1,000 labeled examples (a few thousand recommended for stable accuracy).
• Balanced classes preferred.
• English text works out of the box; other languages benefit from a matching base model.

---

Tips

1. Pre-trained power: BERT is pre-trained on a large corpus, so fine-tuning converges fast.
2. Keep max_length low: shorter sequences train faster; raise it only for longer texts.
3. 3 epochs is usually enough: more epochs rarely help and can overfit.
4. Batch size 16 is a good default on most GPUs.
5. Data quality > quantity: clean, correctly-labeled data beats a larger noisy set.

---

Advanced

Multi-label Classification

hyperparameters = {
    "task_type": "multi_label",  # multiple labels per text
    "num_labels": 5,
}

Custom BERT Variant

hyperparameters = {
    "model_name": "bert-large-uncased",  # larger base
    "batch_size": 8,                      # smaller batch for the larger model
}

---

Next Steps

• Text Data Preparation
• LLM Fine-Tuning (QLoRA) — for generation, not classification
• All Models