ThousandML / Journal / Guide

One base, many adapters: serving GCMP at scale

Because GCMP adapters are about 11 MB each, you can keep one per team, customer, or task and serve all of them from a single CQ base in vLLM. Requests for different adapters run in the same batch.

ThousandML Training Factory5 min readGuide

Why one base, many adapters

A fine-tuned model is a frozen base plus a small adapter. With LoRA r=32 on Qwen3.8-27B, each adapter is around 320 MB. With GCMP it is about 11 MB, with no significant quality difference in our benchmarks. That makes it practical to specialize per use case instead of training one model that has to do everything.

Keeping adapters separate from the base also means one 18 GB CQ checkpoint sits in GPU memory, and each specialization only adds a few megabytes. For enterprises running on their own hardware, that is the difference between one GPU per model and one GPU for all of them.

Start the server

List your adapters in CQ_GCMP_ADAPTERS as comma-separated name=path pairs:

CQ_GCMP_ADAPTERS="support=./outputs/support,sql=./outputs/sql" \
vllm serve ./models/Qwen3.8-27B-CQ-4bit --quantization cq

Every adapter appears in the models list:

curl localhost:8000/v1/models

Multi-adapter serving is new. If something looks wrong, please open an issue on GitHub. For a single adapter, use CQ_GCMP_ADAPTER instead.

Pick an adapter per request

There are two ways to choose. The simplest is to use the adapter name as the model:

curl localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "sql",
  "messages": [{"role": "user", "content": "Write a query for monthly active users."}],
  "max_tokens": 256,
  "chat_template_kwargs": {"enable_thinking": false}
}'

Or keep the base model name and pass the adapter in vllm_xargs. This is useful when a client library always sends the same model name:

curl localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "./models/Qwen3.8-27B-CQ-4bit",
  "messages": [{"role": "user", "content": "How do I reset my password?"}],
  "max_tokens": 256,
  "vllm_xargs": {"gcmp_adapter": "support"},
  "chat_template_kwargs": {"enable_thinking": false}
}'

Multiple GPUs

CQ models support pipeline parallelism in vLLM, and GCMP adapters work across pipeline stages. Tensor parallelism is not supported. CQ MoE models use TP=1 with expert parallelism.

CQ_GCMP_ADAPTER=./outputs/my-gcmp \
vllm serve ./models/Qwen3.8-27B-CQ-4bit --quantization cq --pipeline-parallel-size 2

Serving LoRA instead

If you trained with --adapter lora, serve it with vLLM's standard LoRA flags and select it with "model": "my-lora":

vllm serve ./models/Qwen3.8-27B-CQ-4bit --quantization cq \
  --enable-lora --max-lora-rank 32 --lora-modules my-lora=./outputs/my-lora

What is supported today

Setup	1 GPU	Pipeline parallel	Tensor parallel
Training (any base)	Yes	–	–
vLLM, CQ base + GCMP or LoRA	Yes	Yes	–
vLLM, full-precision base + GCMP	Not yet	Not yet	Not yet
Transformers, full-precision base + GCMP	Yes	–	–

vLLM serving of GCMP adapters supports CQ bases only for now. Adapters trained on a full-precision base can be loaded in Transformers.
MoE expert weights and non-CQ dense layers are not adapted.
Qwen3.x models start in thinking mode: pass "enable_thinking": false or use --reasoning-parser qwen3.

Train your first adapter Which base should you use?GCMP benchmarks

Looking for the bigger picture? Read why a CQ base works.