Cut fine-tuning costs. Not model quality.
Fine-tune powerful models on a high-quality 4-bit base using a fraction of the GPU memory. ThousandML's Coded Quantization (CQ) preserves near-full-precision quality without calibration data, giving you the cost savings of quantization without starting from a weaker model. On top of it, our GCMP adapter matches LoRA with 55× fewer trainable parameters, on a CQ base or a full-precision one. Because training fits on hardware you already own, enterprises can fine-tune in-house and keep their data private.
Runs on your own GPU · Your data never leaves your environment · Free during early access
Qwen 3.8 27B · base weights
A smaller base to train on. Quality close to full precision.
~4x
smaller base weights
92.4%
top-1 agreement vs. BF16
0
calibration data needed
GCMP: 55× fewer parameters than LoRA
Matches LoRA r=32 quality · ~11 MB adapter vs. ~320 MB
Model weight storage only. Activations, adapters, optimizer state, and buffers still use additional memory.
Cheaper to train on. Good enough to build on.
The quantized checkpoint is your starting point, not the last step. You keep the base frozen at 4 bits and train only small adapters on top. The savings are real only if the base holds up, and that is what CQ is built for.
CQ 4-bit · Qwen 3.8 27B · WikiText
0.0282
mean KL divergence
92.4%
top-1 agreement vs. full precision
Calibration-free, so quality is not tuned to a single dataset. Quantization is not lossless, so check results on your own evaluation sets.
Downstream benchmarks · Qwen3.8-27B · 9 tasks
0.650 vs 0.563
GSM8K, CQ vs. NF4 (p = 0.0005)
0 / 9
tasks significantly below BF16
Same prompts and questions for every model, greedy decoding, paired significance tests. Quick benchmark subset, one model. Read the benchmark
A smaller frozen base
CQ 4-bit storage cuts the base model weights to roughly a quarter of 16-bit. That frees GPU memory for longer sequences and bigger batches, or lets you train on smaller, cheaper GPUs.
Tiny adapters with GCMP
GCMP is a LoRA alternative that matches LoRA r=32 with 55× fewer trainable parameters: 2.9M instead of ~159M, and an ~11 MB adapter instead of ~320 MB. Train one per task, then serve many of them from a single CQ base, with the adapter chosen per request. GCMP also trains on full-precision BF16/FP16 bases if you prefer not to quantize.
A base worth building on
Training on a quantized model only makes sense if that model is still good. Coded Quantization (CQ) is built to keep models useful at 4 bits, without calibration data that can overfit to one benchmark. On Qwen3.8-27B, CQ showed no significant difference from BF16 on any of 9 downstream tasks, while NF4 dropped 12 points on GSM8K.
Small enough to train in-house
When fine-tuning fits on hardware you already own, there is no reason to send sensitive data to a third-party training service. Enterprises can adapt open models on-premises or in their own cloud, and keep data, adapters, and model weights under their control.
One workflow, end to end
Quantize a supported model, fine-tune its CQ checkpoint, and serve it with CQ-enabled vLLM. The base stays in the same format the whole way through.
From a CQ checkpoint to a trained adapter.
Training Factory puts quantization, fine-tuning, and inference in one open-source repository. Everything runs on your machine, so your training data stays there. It needs Linux x86-64 or WSL2, an NVIDIA GPU, Conda, and Python 3.12. GCMP is the default adapter. Pass --adapter lora to train a standard PEFT LoRA instead. GCMP also works on a full-precision base if you prefer not to quantize.
01
Install
Install Git LFS and the Training Factory toolkit in one setup.
sudo apt-get update sudo apt-get install -y git-lfs git lfs install git clone https://github.com/thousandml/training-factory.git cd training-factory chmod +x install.sh ./install.sh02
Download a CQ 4-bit base
Start from a ready-made checkpoint.
hf download thousandml/Qwen3.8-27B-CQ-4bit \ --local-dir ./models/Qwen3.8-27B-CQ-4bit03
Train a GCMP adapter
The CQ base stays frozen. Only the small GCMP adapter is trained.
python training/train.py \ --model ./models/Qwen3.8-27B-CQ-4bit \ --data training/examples/train.jsonl \ --output ./outputs/my-gcmp \ --kernel-module cq.cq_kernels \ --adapter gcmp04
Serve base + adapter
Start vLLM with the GCMP adapter applied to the CQ base.
CQ_GCMP_ADAPTER=./outputs/my-gcmp \ vllm serve ./models/Qwen3.8-27B-CQ-4bit \ --quantization cq --max-model-len 819205
Call the fine-tuned model
In another terminal, send a request once the server is ready.
# After the server is ready: curl http://localhost:8000/v1/models curl http://localhost:8000/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "./models/Qwen3.8-27B-CQ-4bit", "messages": [ {"role": "user", "content": "Explain fine-tuning in one sentence."} ], "max_tokens": 128, "chat_template_kwargs": {"enable_thinking": false} }'
Free preview access
The native kernels include a ThousandML license check, which is free during early access. You can run this quick start without paying for a license.
Benchmark
On Qwen3.8-27B, GCMP matched LoRA r=32 with no significant difference on any benchmark, using 2.9M trainable parameters instead of ~159M. Its validation loss was lower at all 10 checkpoints.
For a quick smoke test, add --max-steps 2 --max-length 128 --grad-accum 1. A smoke test checks that the pipeline runs. It does not measure task accuracy. Qwen3.x models start in thinking mode, so pass enable_thinking: false or start vLLM with --reasoning-parser qwen3.
Your data
Plain text or conversations
Provide a JSONL file with text records or chat messages. Messages use the tokenizer's chat template.
{"text":"Question: When does the warranty expire? Answer: Check the purchase date."}
{"messages":[{"role":"user","content":"Summarize LoRA."},{"role":"assistant","content":"It trains small adapters on a frozen base."}]}Adapters are saved on their own (adapter_config.json and adapter_model.safetensors). A GCMP adapter for Qwen3.8-27B is about 11 MB. Do not merge it into the CQ base.
Many adapters
One server, one adapter per task
Load several GCMP adapters on one CQ base. Requests for different adapters run in the same batch, and /v1/models lists them. This feature is new, so please report issues.
CQ_GCMP_ADAPTERS="support=./outputs/support,sql=./outputs/sql" \
vllm serve ./models/Qwen3.8-27B-CQ-4bit --quantization cq
# Pick an adapter per request: "model": "sql"Optional
Train GCMP on a full-precision base
GCMP does not require quantization. If you have the GPU memory, train it on the original BF16/FP16 model instead of the CQ 4-bit base. For Qwen3.8-27B, this needs an 80 GB GPU rather than a 48 GB one. vLLM serving of GCMP adapters currently requires a CQ base.
python training/train_unsloth_16bit_gcmp.py \
--model Qwen/Qwen3.8-27B --dtype bfloat16 \
--train-data training/examples/train.jsonl \
--output ./outputs/my-gcmp-bf16Optional
Quantize your own base model
Already have a ThousandML CQ checkpoint? Skip this. Otherwise, CQ quantizes compatible models without calibration data. The output folder is ready for --model in training and for vllm serve --quantization cq.
# 1. Quantize (calibration-free)
./quantize-model YOUR_ORG/Qwen3.8-27B \
--output-dir ./models/Qwen3.8-27B-CQ-4bit
# 2. Build the CQ checkpoint. --no-upload keeps it local.
python build_convert_upload.py \
--model YOUR_ORG/Qwen3.8-27B \
--quantized-model-dir ./models/Qwen3.8-27B-CQ-4bit \
--repo-id YOUR_USERNAME/Qwen3.8-27B-CQ-4bit \
--no-uploadHow the pieces fit
Existing CQ 4-bit model ──────────────────┐
v
Hugging Face model -> CQ quantization -> Frozen CQ 4-bit base
| |
v v
GCMP training vLLM serving
| ^
v |
GCMP adapters (~11 MB) ───┘
one or many per serverTraining runs on a single GPU. CQ serving supports pipeline parallelism but not tensor parallelism. Memory, throughput, cost, and accuracy depend on your model, task, sequence length, and hardware. Check them on your real workload before relying on them in production.
Private fine-tuning, on hardware you control.
Many enterprises can't send sensitive data to an outside training service. Fine-tuning a full-precision model in-house has meant buying or renting large GPU clusters. A CQ base lowers that requirement, so you can adapt open models to private data inside your own environment. Try the open-source workflow today. We're building a managed platform you deploy in your own environment, so you keep control of the data.
Your data stays on your infrastructure
Run fine-tuning on-premises or in your own VPC. Training data, adapters, and weights never pass through a third-party service.
Adapter fleets on one base
Run dozens of task-specific adapters on one shared CQ base instead of a full model copy per team.
Managed end to end
Quantization, training, evaluation, and serving in one place, run by the team that built CQ.