gte-Qwen2-7B-instruct
gte-Qwen2-7B-instruct is a 7-billion-parameter text embedding model fine-tuned on Qwen2-7B for multilingual sentence similarity and semantic search tasks. It ranks first on MTEB benchmarks for English and Chinese as of June 2024, supports 32k token context, outputs 3584-dimensional embeddings, and is distributed under Apache 2.0 license without access gating.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | Alibaba-NLP |
| Parameters | 7.6B |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | sentence-similarity |
| Gated on HuggingFace | No |
| Downloads | 80.2k |
| Likes | 482 |
| Last updated | 2025-03-24 |
| Source | Alibaba-NLP/gte-Qwen2-7B-instruct |
What gte-Qwen2-7B-instruct is
Encoder-decoder embedding model based on Qwen2-7B LLM architecture with bidirectional attention and instruction tuning (queries only). Trained on multilingual, weakly-supervised and supervised data across diverse domains. Requires transformers>=4.39.2 and flash_attn>=2.5.6. Outputs normalized embeddings via last-token pooling. Max sequence: 32k tokens; embedding dim: 3584. Model card references custom_code and safetensors format.
Run gte-Qwen2-7B-instruct locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="Alibaba-NLP/gte-Qwen2-7B-instruct")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE: 16–32 GB GPU VRAM (fp32 ~30GB; bfloat16 ~15–16GB). Requires NVIDIA Compute Capability ≥ 8.0 (Ampere or newer). CPU inference possible but slow. flash_attn>=2.5.6 dependency suggests optimized inference on modern GPUs. Verify actual footprint on target hardware before production deployment.
Model card does not detail LoRA, QLoRA, or full fine-tuning feasibility. Instruction tuning applied only to queries in base training. Custom code flag suggests potential complexity in adapter integration. Fine-tuning for domain adaptation is Unknown; recommend testing on small dataset first or consulting Alibaba-NLP community.
When to avoid it — and what to weigh
- Sub-second latency requirements in single-GPU setups — 7B model with 3584-dim output requires significant VRAM and compute per inference. Batch processing and specialized inference servers (TensorRT) needed for low-latency production. Single queries may be slow on consumer hardware.
- Fine-tuning on highly specialized domain terminology — Model card states training used 'weakly supervised and supervised data' but does not detail domain adaptation or LoRA performance. Fine-tuning feasibility for niche domains is Unknown.
- Real-time embedding generation with strict memory constraints — Requires ~16-32GB VRAM (per infinity example). Not suitable for edge devices, mobile, or memory-constrained environments. Smaller alternatives (1.5B variant mentioned but not evaluated in detail).
- Guaranteed reproducibility across versions — Model uses custom_code (trust_remote_code=True required). No versioning strategy or reproducibility guarantees documented. Upstream Qwen2 or transformers library changes may affect behavior.
License & commercial use
Apache License 2.0: permissive OSI-approved license. Allows use, modification, and distribution under Apache 2.0 terms. No proprietary restrictions stated in model card.
Apache 2.0 is a permissive OSI license that explicitly allows commercial use, provided Apache 2.0 license terms and copyright notice are retained. However, Qwen2-7B base model license should be verified independently (not stated in provided data). No known commercial-use restrictions on gte-Qwen2-7B-instruct itself, but always review upstream dependencies and indemnification terms for production deployments.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Adequate |
| License clarity | Clear |
| Deployment complexity | Moderate |
| DEV.co fit | Strong |
| Assessment confidence | High |
Model uses custom_code and requires trust_remote_code=True, increasing attack surface if model source is compromised or malicious. Verify model provenance before use in sensitive environments. No security audit, vulnerability disclosure policy, or data provenance details provided. Qwen2 upstream security posture Unknown. Standard precautions: run inference in isolated environment, monitor for model tampering, and review custom code before enabling in production.
Alternatives to consider
e5-mistral-7b-instruct (Intfloat)
7B LLM-based embedding; MTEB 66.63 (English), 60.81 (Chinese). Slightly lower MTEB scores but may have different trade-offs in latency or resource usage.
gte-Qwen1.5-7B-instruct (Alibaba-NLP)
Previous iteration of gte-Qwen series; MTEB 67.34 (English), 69.52 (Chinese). Lower scores but potentially better documented and more stable. Lower resource footprint possible.
NV-Embed-v1 (NVIDIA)
MTEB 69.32 (English) but no multilingual benchmark data provided. NVIDIA hardware optimizations may offer faster inference; license and commercial use policy require separate review.
Ship gte-Qwen2-7B-instruct with senior software developers
gte-Qwen2-7B-instruct offers SOTA multilingual semantic search without licensing restrictions. Estimate your infrastructure needs, verify Qwen2 base model compliance, and start with our deployment guides for Infinity or Text-Embeddings-Inference.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
gte-Qwen2-7B-instruct FAQ
Can I use gte-Qwen2-7B-instruct commercially in a production SaaS product?
What GPU and VRAM do I need to deploy this in production?
Do I need to fine-tune this model for my domain?
Why does the model require trust_remote_code=True?
Software developers & web developers for hire
DEV.co helps companies turn open-source tools like gte-Qwen2-7B-instruct into production software. Our software development services cover the full lifecycle — architecture, web development, integration, and maintenance — delivered by software developers and web developers who ship. Engage our software development agency to implement or customize it for your open-source llms stack.
Ready to Deploy Multilingual Embeddings?
gte-Qwen2-7B-instruct offers SOTA multilingual semantic search without licensing restrictions. Estimate your infrastructure needs, verify Qwen2 base model compliance, and start with our deployment guides for Infinity or Text-Embeddings-Inference.