A better 4-bit base, a better fine-tune: QLoRA on CQ vs. Unsloth NF4

CQ 4-bit vs. unsloth-bnb-4bit (NF4), Qwen3.8-27B, QLoRA on Alpaca, 500 steps. These are reported losses, not a downstream-task quality comparison.

Why the base matters for QLoRA

QLoRA keeps the base model frozen in 4-bit and trains small LoRA adapters on top. That is what makes fine-tuning a 27B model possible on one GPU, and what lets enterprises fine-tune in-house instead of sending private data to an outside service.

But the adapter learns a correction on top of whatever the quantized base still knows. If quantization has damaged the base, the adapter spends some of its capacity compensating for that damage. A more faithful base should leave more room to learn your task. We wanted to test whether that shows up in the training numbers.

Step 1: start from a more faithful base

First we compared the two bases against the full-precision model on WikiText-2, using KL divergence between output distributions (lower is better) and top-1 token agreement (higher is better). unsloth-bnb-4bit is the variant Unsloth uses for QLoRA.

The CQ base is about 20% smaller on disk and closer to full precision on every metric. CQ does this without calibration data. That gave us a better starting point. The next question was whether it leads to a better fine-tune.

Step 2: same recipe, two bases

Both runs used the same dataset files and the same nominal QLoRA hyperparameters:

Model

Qwen3.8-27B

Dataset

Alpaca (train / validation split)

LoRA

r = 32, α = 32, dropout 0

Batch

2 × grad accumulation 4

Sequence length

512

Learning rate

2e-5

Steps

500 optimizer steps, eval every 50

Seed

42

Hardware

Single NVIDIA A40, 48 GB

CQ: thousandml/Qwen3.8-27B-CQ-4bit. Unsloth: unsloth/Qwen3.8-27B-unsloth-bnb-4bit.

Results

On loss, the gap is large and consistent. CQ ended at 1.0315 validation loss vs. 1.2484 for NF4, which is 17.4% lower. Its best checkpoint reached 0.8385. Unsloth's best validation loss was its final value.

The loss curves

One detail is worth calling out. CQ's validation loss bottoms out at step 100 and then drifts upward, while its training loss keeps falling. That pattern usually means the model is starting to overfit Alpaca. In other words, CQ reached its best validation loss in about a fifth of the run. With early stopping, a CQ run could stop around step 100 and still beat NF4's 500-step result by a wide margin. That matters because of the speed gap below.

Where CQ is behind

CQ does not win everything. Unsloth's kernels are heavily optimized, and that shows:

Faster convergence helps offset the speed gap. If CQ reaches its best validation loss in about 100 steps, the time to a usable adapter is much closer than the per-step numbers suggest. We still plan to close the per-step gap directly.

Caveats

• —These are reported losses from training logs, not a downstream-task evaluation. Lower loss is a strong signal, but it does not prove the model is better at your task.
• —Both runs used the same dataset files and nominal QLoRA settings, but the scripts are different. Tokenization and loss-mask equivalence have not been independently verified. If one pipeline masks prompt tokens differently, that can shift absolute loss values.
• —This is one model, one dataset, one seed, and one GPU type. We will publish more runs as we add models.

We would rather show the caveats than oversell the result. The best check is to run both scripts on your own data and compare.

Reproduce it

Install Training Factory, then run both configurations from the repository root.

CQ 4-bit

hf download thousandml/Qwen3.8-27B-CQ-4bit \
  --local-dir ./models/Qwen3.8-27B-CQ-4bit

python training/train.py \
  --model ./models/Qwen3.8-27B-CQ-4bit \
  --data training/examples/alpaca_train.jsonl \
  --eval-data training/examples/alpaca_val.jsonl \
  --output ./outputs/qwen38-27b-cq-4bit \
  --kernel-module cq.cq_kernels \
  --targets q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj \
  --rank 32 \
  --alpha 32 \
  --lora-dropout 0 \
  --batch-size 2 \
  --grad-accum 4 \
  --max-length 512 \
  --max-steps 500 \
  --learning-rate 2e-5 \
  --seed 42 \
  --eval-steps 50 \
  --comparison-logs \
  --benchmark \
  --benchmark-warmup-steps 10

Unsloth NF4

python training/train_unsloth_nf4.py \
  --model unsloth/Qwen3.8-27B-unsloth-bnb-4bit \
  --train-data training/examples/alpaca_train.jsonl \
  --eval-data training/examples/alpaca_val.jsonl \
  --output ./outputs/qwen38-27b-unsloth-bnb-4bit \
  --max-steps 500 \
  --max-length 512 \
  --eval-steps 50 \
  --save-steps 100 \
  --seed 42 \
  --benchmark-warmup-steps 10

CQ 4-bit bases currently available: Qwen3.8-27B, Llama-3.1-8B-Instruct, Llama-3.3-70B-Instruct, and Qwen3-Coder-Next, with more on the way.