Cut fine-tuning costs. Not model quality.
Fine-tune powerful models on a high-quality 4-bit base using a fraction of the GPU memory. ThousandML's Coded Quantization (CQ) preserves near-full-precision quality without calibration data, giving you the cost savings of quantization without starting from a weaker model. In our Qwen3.8-27B QLoRA benchmark, fine-tuning on CQ reached lower validation loss than Unsloth's NF4 base. Because training fits on hardware you already own, enterprises can fine-tune in-house and keep their data private.
Runs on your own GPU · Your data never leaves your environment · Free during early access
Qwen 3.8 27B · base weights
A smaller base to train on. Quality close to full precision.
~4x
smaller base weights
92.4%
top-1 agreement vs. BF16
0
calibration data needed
17.4% lower validation loss
QLoRA fine-tune on Alpaca vs. Unsloth NF4 base
Model weight storage only. Activations, adapters, optimizer state, and buffers still use additional memory.
Cheaper to train on. Good enough to build on.
The quantized checkpoint is your starting point, not the last step. You keep the base frozen at 4 bits and train only small adapters on top. The savings are real only if the base holds up, and that is what CQ is built for.
CQ 4-bit · Qwen 3.8 27B · WikiText
0.0282
mean KL divergence
92.4%
top-1 agreement vs. full precision
Calibration-free, so quality is not tuned to a single dataset. Quantization is not lossless, so check results on your own evaluation sets.
QLoRA fine-tune · Qwen3.8-27B · Alpaca · 500 steps
1.03 vs 1.25
final validation loss, CQ vs. NF4
10 / 10
checkpoints lower on CQ
A better base leads to a better fine-tune: 17.4% lower reported validation loss than Unsloth NF4. Reported loss, not a downstream-task evaluation. Read the benchmark
A smaller frozen base
CQ 4-bit storage cuts the base model weights to roughly a quarter of 16-bit. That frees GPU memory for longer sequences and bigger batches, or lets you train on smaller, cheaper GPUs.
One base, many adapters
Reuse one CQ checkpoint across every task and train a small LoRA adapter for each. You no longer need to store and serve a full fine-tuned copy per use case.
A base worth building on
Training on a quantized model only makes sense if that model is still good. Coded Quantization (CQ) is built to keep models useful at 4 bits, without calibration data that can overfit to one benchmark. In QLoRA tests, adapters trained on CQ reached lower validation loss than adapters trained on an NF4 base.
Small enough to train in-house
When fine-tuning fits on hardware you already own, there is no reason to send sensitive data to a third-party training service. Enterprises can adapt open models on-premises or in their own cloud, and keep data, adapters, and model weights under their control.
One workflow, end to end
Quantize a supported model, fine-tune its CQ checkpoint, and serve it with CQ-enabled vLLM. The base stays in the same format the whole way through.
From a CQ checkpoint to a trained adapter.
Training Factory puts quantization, fine-tuning, and inference in one open-source repository. Everything runs on your machine, so your training data stays there. Training currently supports compatible CQ-quantized architectures on a single NVIDIA GPU (Linux x86-64 or WSL2). Verify Qwen support with your installed release before treating these commands as production-tested.
01
Install
Install Git LFS and the Training Factory toolkit in one setup.
sudo apt-get update sudo apt-get install -y git-lfs git lfs install git clone https://github.com/thousandml/training-factory.git cd training-factory chmod +x install.sh ./install.sh02
Download a CQ 4-bit base
Start from a ready-made checkpoint.
hf download thousandml/Qwen3.8-27B-CQ-4bit \ --local-dir ./models/Qwen3.8-27B-CQ-4bit03
Train a LoRA adapter
The CQ base stays frozen. Only the adapter is trained.
python training/train.py \ --model ./models/Qwen3.8-27B-CQ-4bit \ --data training/examples/train.jsonl \ --output ./outputs/my-lora \ --targets q_proj,v_proj \ --kernel-module cq.cq_kernels \ --max-steps 2 --max-length 128 --batch-size 1 --grad-accum 104
Serve base + adapter
Start the server with the fine-tuned adapter enabled.
vllm serve thousandml/Qwen3.8-27B-CQ-4bit \ --quantization cq05
Call the fine-tuned adapter
In another terminal, verify the adapter and request a response from it.
# After the server is ready: # Check that the adapter is listed as "my-lora" curl http://localhost:8000/v1/models # Request a response from the fine-tuned model, not just the base checkpoint curl http://localhost:8000/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "my-lora", "messages": [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Explain how LoRA fine-tuning works in one sentence."} ], "max_tokens": 128, "temperature": 0.2 }'
Free preview access
License configuration will be part of the general Training Factory setup. For now, CQ fine-tuning is available to try for free, so you can run this quick start without a license key or any additional configuration.
Benchmark
With the same QLoRA recipe on Qwen3.8-27B, CQ reached 17.4% lower final validation loss than Unsloth NF4 (1.0315 vs. 1.2484). It was lower at all 10 checkpoints.
For a quick smoke test, add --max-steps 2 --max-length 128 --grad-accum 1. A smoke test checks that the pipeline runs. It does not measure task accuracy.
Your data
Plain text or conversations
Provide train.jsonl with text records or chat messages. Messages use the tokenizer's chat template.
{"text":"Question: When does the warranty expire? Answer: Check the purchase date and warranty terms."}
{"messages":[{"role":"user","content":"Summarize LoRA in one sentence."},{"role":"assistant","content":"LoRA fine-tunes small adapter matrices while the base stays frozen."}]}The output is a PEFT-style LoRA adapter stored separately from the CQ base. Do not merge it back into the frozen base.
Optional
Quantize your own base model
Already have a ThousandML CQ checkpoint? Skip this. Otherwise, CQ quantizes compatible models without calibration data.
./quantize-model YOUR_ORG/Qwen3.8-27B \
--output-dir ./quantized_models
python build_convert_upload.py \
--model YOUR_ORG/Qwen3.8-27B \
--repo-id YOUR_USERNAME/Qwen3.8-27B-CQ-4bit \
--no-uploadHow the pieces fit
Existing CQ 4-bit model ──────────────────┐
v
Hugging Face model -> CQ quantization -> Frozen CQ 4-bit base
| |
v v
LoRA training vLLM serving
|
v
LoRA adapter
|
v
CQ + LoRA serving (after validation)Memory, throughput, cost, and accuracy depend on your model, task, sequence length, hardware, and baseline. Check them on your real workload before relying on them in production.
Private fine-tuning, on hardware you control.
Many enterprises can't send sensitive data to an outside training service. Fine-tuning a full-precision model in-house has meant buying or renting large GPU clusters. A CQ base lowers that requirement, so you can adapt open models to private data inside your own environment. Try the open-source workflow today. We're building a managed platform you deploy in your own environment, so you keep control of the data.
Your data stays on your infrastructure
Run fine-tuning on-premises or in your own VPC. Training data, adapters, and weights never pass through a third-party service.
Adapter fleets on one base
Run dozens of task-specific adapters on one shared CQ base instead of a full model copy per team.
Managed end to end
Quantization, training, evaluation, and serving in one place, run by the team that built CQ.