Skip to main content

LightGBM Regression

Leaf-wise gradient boosting for large, wide tabular datasets

Overview​

  • Model ID: lightgbm-regression-gpu
  • Framework: LightGBM
  • Hardware: GPU (histogram training; small footprint)
  • Multi-output: Yes (predicts multiple target columns simultaneously)
  • Specialist type: tabular

LightGBM grows trees leaf-wise using a histogram-based algorithm, which makes it fast and memory-efficient on large, high-dimensional datasets. Use the exact ID lightgbm-regression-gpu as the model= argument.

For classification, the companion ID is lightgbm-classification (CPU). See the ML Classical index for the full model table.

When to Use​

✅ Perfect for:

  • Large datasets (100k+ rows)
  • High-dimensional data (many features)
  • Fast training iteration
  • Multi-output regression (predict multiple targets at once)

❌ Not ideal for:

  • Very small datasets — try XGBoost or Random Forest
  • Problems better suited to deep learning (very large, unstructured)

Quick Start​

Single-output​

from colabhive import ColabHive

client = ColabHive(api_key="...", account_id="...")

dataset = client.datasets.upload(name="sales_data", file="./train.csv")

job = client.training.create(
    model="lightgbm-regression-gpu",
    dataset_id=dataset.id,
    hyperparameters={
        "n_estimators": 200,
        "num_leaves": 63,
        "learning_rate": 0.05,
        "target_column": "price",
    },
)
job.wait()
print(job.get_metrics())

# Register as an inference endpoint
endpoint = client.training.register_for_inference(
    run_id=job.id,
    name="lightgbm-sales-regression",
    description="LightGBM regression model",
    visibility="account",
)

# Run inference
result = client.endpoints.infer(
    endpoint_id=endpoint.endpoint_id,
    input_data={"instances": [{"feature1": 1.0, "feature2": 2.5}]},
)
print(result)

Multi-output​

job = client.training.create(
    model="lightgbm-regression-gpu",
    dataset_id=dataset.id,
    hyperparameters={
        "n_estimators": 200,
        "num_leaves": 63,
        "learning_rate": 0.05,
        "target_columns": ["price", "quantity", "margin"],
    },
)
Multi-output

When using target_columns (plural), LightGBM trains one model per target (via MultiOutputRegressor). Metrics are reported per target and aggregated.

Hyperparameters​

ParameterDefaultRangeDescription
n_estimators20010-1000Number of boosting iterations
num_leaves632-255Max leaves per tree
learning_rate0.050.001-0.3Step size for weight updates
max_depth-1-1 or 3-15Max tree depth (-1 = unlimited)
min_data_in_leaf201-100Min samples per leaf
target_column——Single target column
target_columns——Multiple target columns (multi-output)

Expected Results​

Regression metrics reported after training:

  • MAE (lower is better)
  • RMSE (lower is better)
  • R² (higher is better; 1.0 is perfect)

Tips​

  1. LightGBM vs XGBoost: LightGBM tends to be faster on large/wide data; XGBoost is a robust default on medium data.
  2. Leaf-wise growth: grows the leaf with the largest loss reduction — powerful, but tune num_leaves/min_data_in_leaf to avoid overfitting small data.
  3. Memory efficient: histogram-based training keeps memory low.
  4. Multi-output: use target_columns to predict related targets in one job.

Next Steps​