qmd-query-expansion-1.7B-gguf
QMD Query Expansion is a 1.7B parameter fine-tuned language model designed to expand short search queries into multiple formats (lexical, vector, and hypothetical document) for hybrid retrieval systems. It uses a two-stage training approach (SFT then GRPO reinforcement learning) and is distributed in GGUF format for local deployment via Ollama or llama.cpp.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | tobil |
| Parameters | Unknown |
| Context window | Unknown |
| License | mit — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 46.3k |
| Likes | 25 |
| Last updated | 2026-01-29 |
| Source | tobil/qmd-query-expansion-1.7B-gguf |
What qmd-query-expansion-1.7B-gguf is
Quantized Qwen3-1.7B base model fine-tuned via LoRA (SFT: rank 16 all projections; GRPO: rank 4 q/v only with KL beta 0.04). Trained on ~2,290 examples to generate structured output with lex:/vec:/hyde: prefixes. Uses rule-based reward scoring (5 dimensions, 0-140 points) as RL signal. Context length and total parameters unknown. Deployed as merged GGUF for inference.
Run qmd-query-expansion-1.7B-gguf locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="tobil/qmd-query-expansion-1.7B-gguf")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE: 1.7B quantized (GGUF q4_k_m format ~0.8–1.2 GB VRAM). Training requires GPU (A10G shown in docs, ~45 min SFT + ~20 min GRPO). Inference on CPU possible but slow; GPU recommended for <100ms latency. No explicit precision/batch requirements stated; assume fp16 training, int4 inference.
Two-stage fine-tuning is fully open-sourced (train.py, sft.yaml, grpo.yaml). LoRA parameters provided (SFT: rank 16, alpha 32 all projections; GRPO: rank 4, alpha 8 q/v). Requires ~1,000 labeled examples (provided in qmd_expansion_v2.jsonl). Training can run on HuggingFace Jobs (A10G) or local GPU. Merged GGUF conversion tool provided. Well-documented reward function (reward.py) allows custom fine-tuning with modified scoring rules.
When to avoid it — and what to weigh
- Multi-turn Conversational AI — Model is optimized for single-query expansion, not interactive dialogue. No multi-turn context or conversation memory in architecture.
- High-Throughput Cloud Serving — 1.7B parameter model with GGUF quantization targets local/edge deployment. If you need cost-efficient large-scale API inference, consider cloud-hosted alternatives or larger models with better batch efficiency.
- General-Purpose Language Tasks — Model is narrowly trained for query expansion output format (lex:/vec:/hyde:). Do not use for translation, summarization, or open-ended generation without retraining.
- Uncertain Query Domains — Reward function assumes well-formed queries with extractable entities. Malformed, non-English, or domain-specific jargon queries may produce low-scoring or invalid expansions.
License & commercial use
MIT license. Permissive, OSI-approved open-source license allowing modification, distribution, and commercial use with attribution.
MIT is a permissive open-source license. Commercial use, modification, and redistribution are allowed provided original MIT license and copyright notice are included. Base model is Qwen/Qwen3-1.7B (verify its license separately for full compliance chain). No gating or commercial restrictions stated on the model card.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Strong |
| License clarity | Clear |
| Deployment complexity | Low |
| DEV.co fit | Strong |
| Assessment confidence | High |
Model is deployed as quantized GGUF for local inference, reducing supply-chain and API-call risks. GGUF format is a serialization standard without inherent encryption. For sensitive use: validate model provenance (tobil/qmd-query-expansion-1.7B-gguf on HuggingFace Hub), host on trusted infrastructure, and review base model (Qwen/Qwen3-1.7B) security posture separately. Reward function is rule-based (no LLM judge), reducing prompt-injection or jailbreak risk in the training loop. No explicit security audit or red-team testing mentioned.
Alternatives to consider
ColBERT / Dense Passage Retrieval (DPR)
Specialized dense retrieval models; require different training pipeline but may offer better vector representation without explicit query expansion. Trade-off: no lexical or HyDE outputs.
Proprietary Query Expansion APIs (e.g., Elasticsearch Query Suggestions, Azure Cognitive Search)
Managed solutions with pre-tuned expansion; no deployment overhead. Trade-off: cloud dependency, cost per query, less control over expansion format.
LLaMA 2 / Mistral 7B (Generic Fine-Tuned for Query Expansion)
Larger base models may generate higher-quality expansions but require more VRAM and inference latency. Use if accuracy outweighs deployment constraints.
Ship qmd-query-expansion-1.7B-gguf with senior software developers
QMD Query Expansion is fully open-source and documented. Download the GGUF quantized model, integrate with Ollama or llama.cpp, or fine-tune on your domain queries using the provided training pipeline. Start with the quick-start scripts in the repository.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
qmd-query-expansion-1.7B-gguf FAQ
Can I use this commercially in a SaaS product?
What hardware do I need to run inference locally?
How do I fine-tune this for my domain (e.g., legal documents)?
What if my queries are in languages other than English?
Custom software development services
DEV.co is a software development agency delivering custom software development services to companies building on open source. Our software developers and web developers design, integrate, and ship production systems — spanning web development, APIs, AI, data, and cloud. If qmd-query-expansion-1.7B-gguf is part of your open-source llms roadmap, our team can implement, customize, migrate, and maintain it.
Ready to Deploy Local Query Expansion?
QMD Query Expansion is fully open-source and documented. Download the GGUF quantized model, integrate with Ollama or llama.cpp, or fine-tune on your domain queries using the provided training pipeline. Start with the quick-start scripts in the repository.