ThousandML / Journal

Notes from Training Factory

Benchmarks, methods, and lessons from fine-tuning open models on CQ 4-bit bases.

• Benchmark7 min read

  A better 4-bit base, a better fine-tune: QLoRA on CQ vs. Unsloth NF4

  Qwen3.8-27B, QLoRA on Alpaca, 500 steps. CQ reported 17.4% lower final validation loss and lower validation loss at all 10 checkpoints. We also cover where it is behind.

  Read the post
• Fine-tuning8 min read

  Fine-tune on a quantized base. Just make sure the base is good.

  Why training adapters on a 4-bit base lowers cost enough to fine-tune in-house, and why a calibration-free CQ base is a strong place to start.

  Read the post