GCMP on BF16, CQ, or NF4: choosing a base

The short answer

• —Choose CQ 4-bit if you want to fine-tune and serve on a single 48 GB GPU, keep downstream quality at BF16 level, and serve many adapters with vLLM.
• —Choose BF16 if you already have 80 GB GPUs, want the fastest GCMP training step, and are happy to evaluate or serve with Transformers for now.
• —NF4 trains fast on 48 GB, but its base loses accuracy on reasoning tasks. In our runs it was 12.0 points lower than BF16 on GSM8K.

Qwen3.8-27B, Alpaca, 500 steps, batch 2 × gradient accumulation 4, sequence length 512. Training speed and memory are for the GCMP adapter. CQ runs use the Training Factory trainer; BF16 and NF4 runs use Unsloth with GCMP.

Quality

Your adapter can only build on what the base still knows. Across nine downstream tasks, CQ 4-bit showed no significant difference from BF16 on any of them. NF4 was 12.0 points lower than BF16 on GSM8K (p = 3e-6), and CQ beat NF4 by 8.7 points on GSM8K and 2.8 points on BoolQ.

GCMP itself holds up on every base we tried. On BF16 it matched LoRA r=32 on every benchmark, and its validation loss was lower than LoRA at all 10 checkpoints on both BF16 and NF4 bases. Benchmarks for GCMP on CQ and NF4 bases are still running.

Memory and speed

A 27B model in BF16 is 55.6 GB of weights alone, so training it with GCMP needs an 80 GB GPU. On a CQ base the weights are 18 GB, and the full GCMP run peaked at 22.0 GiB, well within a 48 GB card. Smaller models need proportionally less on either base.

On speed, the 4-bit trainers are slower per step than BF16, and the Training Factory trainer is slower than Unsloth today. GCMP currently costs about 2× the step time of LoRA on every base. Kernel work to close that gap is in progress. If you are memory-bound, CQ is the better trade; if you have spare 80 GB GPUs and want faster iterations, BF16 is.

Serving

This is the biggest practical difference today. vLLM serving of GCMP adapters, including multi-adapter serving and pipeline parallelism, supports CQ bases only. Adapters trained on a full-precision base load in Transformers, which is fine for evaluation and lower-traffic use.

Commands for each base

CQ 4-bit (48 GB GPU):

python training/train.py \
  --model ./models/Qwen3.8-27B-CQ-4bit \
  --data training/examples/train.jsonl \
  --output ./outputs/my-gcmp \
  --kernel-module cq.cq_kernels \
  --adapter gcmp

Full precision, no quantization (80 GB GPU for 27B):

python training/train_unsloth_16bit_gcmp.py \
  --model Qwen/Qwen3.8-27B --dtype bfloat16 \
  --train-data training/examples/train.jsonl \
  --output ./outputs/my-gcmp-bf16

Evaluate the BF16 adapter in Transformers:

python training/downstream_eval.py --backend unsloth \
  --model Qwen/Qwen3.8-27B \
  --adapter ./outputs/my-gcmp-bf16/final_adapter \
  --output-dir eval/my-gcmp-bf16

NF4 (48 GB GPU):

python training/train_unsloth_nf4.py \
  --model unsloth/Qwen3.8-27B-unsloth-bnb-4bit \
  --train-data training/examples/train.jsonl \
  --adapter gcmp --output ./outputs/my-gcmp-nf4

For FP16 instead of BF16, change --dtype.

Caveats

• —Results so far cover one model (Qwen3.8-27B) and one dataset (Alpaca), using a quick benchmark subset that detects differences of about 5 points rather than 1.
• —Training runs on a single GPU on every base.
• —Measure quality, speed, and memory on your own data before relying on these numbers.