LLM.coPrivate, self-hosted LLMs & custom AI applications.
Automatic.coAI-powered workflow & business process automation.
RMA.aiAI-powered cybersecurity monitoring, threat detection & defense.
VB.coOperating AI™ — one intelligent layer for middle-market ops.
LAW.co
RFP.co
Search.co
VDR.aiRAG-Retrieval
RAG-Retrieval is a Python library for training and deploying retrieval-augmented generation (RAG) models, supporting embeddings, ColBERT, and rerankers. It provides end-to-end fine-tuning, distillation from larger to smaller models, and a lightweight inference library for unified model calling.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Repository | NovaSearch-Team/RAG-Retrieval |
| Owner | NovaSearch-Team |
| Primary language | Python |
| License | MIT — OSI-approved |
| Stars | 1.1k |
| Forks | 88 |
| Open issues | 2 |
| Latest release | Unknown |
| Last updated | 2026-05-24 |
| Source | https://github.com/NovaSearch-Team/RAG-Retrieval |
What RAG-Retrieval is
Unified training framework for RAG retrieval components (embedding, late interaction, reranking) with support for BERT-based and LLM-based architectures. Includes distillation pipelines (MRL, Stella), multi-GPU training (DeepSpeed, FSDP), and a PyPI-published inference package supporting cross-encoder and decoder-only LLM rerankers with configurable document handling.
Get the RAG-Retrieval source
Clone the repository and explore it locally.
git clone https://github.com/NovaSearch-Team/RAG-Retrieval.gitcd RAG-Retrieval# follow the project's README for install & configurationNeed it deployed, integrated, or customized instead? DEV.co ships production installs.
Best use cases
Implementation considerations
- CUDA/PyTorch version compatibility must be manual; README explicitly warns to install torch before requirements.txt to avoid conflicts.
- Training code is organized by model type (embedding, colbert, reranker); each subdirectory has separate README and bash scripts. Requires understanding of RAG architecture to select correct pipeline.
- Multi-GPU training (DeepSpeed, FSDP) is supported but configuration and debugging of distributed setups is operator responsibility; not abstracted away.
- Inference library (rag-retrieval PyPI package) is separate from training code; requires understanding of which models are compatible and how long-document truncation vs. splitting behaves.
- Distillation pipelines (Stella, MRL) require teacher models and careful hyperparameter tuning; no autoML or reference hyperparameter sets published beyond news posts.
When to avoid it — and what to weigh
- No Training Data or Benchmark Validation — If you lack domain training data or cannot validate improvements on domain-specific benchmarks, fine-tuning complexity may not justify the effort over using pre-trained models directly.
- Strict Real-Time Latency Requirements at Scale — No published benchmarks on inference latency (p99, throughput) or distributed serving. Verify the lightweight inference library meets your latency SLAs before production adoption.
- Non-Python or Legacy Technology Stack — RAG-Retrieval is Python-only (PyTorch, Hugging Face). Integration with non-Python services, Rust systems, or older ML frameworks will require custom wrappers.
- Closed-Source or Proprietary Model Support — Framework targets open-source models (BGE, BCE, GTE, Roberta). If you need to fine-tune proprietary models (OpenAI, Claude, Anthropic), this project cannot be used.
License & commercial use
MIT License. Permissive OSI-approved license allowing commercial use, modification, and redistribution with attribution and no warranty.
MIT permits commercial use. No indication of dual licensing, proprietary restrictions, or service-level agreements. Use of trained models and derivative works is permitted; review your own IP strategy and any third-party model licenses (BGE, BCE, etc.) used in fine-tuning.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Adequate |
| License clarity | Clear |
| Deployment complexity | Moderate |
| DEV.co fit | Good |
| Assessment confidence | High |
No published security audit, vulnerability disclosure policy, or hardening documentation. Uses standard PyTorch/Hugging Face dependencies; ensure supply chain and dependency versions are scanned. Training data provenance (BGE training data mentioned) should be verified. Inference models run locally; no external API calls or data transmission documented.
Alternatives to consider
Hugging Face Sentence-Transformers + Custom Fine-Tuning
Mature, well-documented library for embedding and reranking tasks. Supports similar models but requires manual training loop setup; less unified than RAG-Retrieval for end-to-end RAG pipelines.
LangChain + OpenAI/Cohere Rerankers
Higher-level orchestration for RAG pipelines with closed-source reranker APIs. No self-hosted fine-tuning; better for prototyping but higher cost and vendor lock-in.
Vespa (Open Source Search Engine) + Custom Rerankers
Full-stack IR solution with native ranking, relevance tuning, and serving. Steeper learning curve but provides distributed search + reranking in one system; not Python-centric.
Build on RAG-Retrieval with DEV.co software developers
RAG-Retrieval provides the code and tools to build domain-specific retrieval pipelines. Assess your training data, CUDA setup, and model choices—then contact our AI team to architect a production deployment.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
RAG-Retrieval FAQ
Can I fine-tune closed-source models like GPT-4 or Claude with RAG-Retrieval?
What is the difference between the training code and the rag-retrieval PyPI package?
Does RAG-Retrieval provide a REST API for inference?
How much training data do I need to see improvements from fine-tuning?
Custom software development services
Need help beyond evaluating RAG-Retrieval? DEV.co is a software development agency building custom software, web applications, APIs, and rag frameworks integrations — and maintaining them long-term.
Ready to Fine-Tune RAG Models for Your Domain?
RAG-Retrieval provides the code and tools to build domain-specific retrieval pipelines. Assess your training data, CUDA setup, and model choices—then contact our AI team to architect a production deployment.