surogate
Surogate is a C++/CUDA-based training framework optimized for high-speed LLM fine-tuning and pre-training using low-precision formats (FP8, FP4) and CPU offloading. It supports LoRA/QLoRA training, multi-GPU/multi-node setups, and popular models like Qwen, Llama, and Nemotron across diverse NVIDIA GPU architectures.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Repository | invergent-ai/surogate |
| Owner | invergent-ai |
| Primary language | C++ |
| License | Apache-2.0 — OSI-approved |
| Stars | 806 |
| Forks | 5 |
| Open issues | 7 |
| Latest release | v1.2.6 (2026-05-20) |
| Last updated | 2026-07-07 |
| Source | https://github.com/invergent-ai/surogate |
What surogate is
Native C++/CUDA engine with AOT auto-differentiation DSL, FP8/FP4/BF16 mixed-precision training recipes, smart CPU offloading for weights/gradients/activations, multi-threaded multi-GPU/Ray-based multi-node DDP, and adaptive training with automatic phase detection and early stopping. Targets sm80–sm121 NVIDIA GPUs with deterministic, reproducible configs.
Get the surogate source
Clone the repository and explore it locally.
git clone https://github.com/invergent-ai/surogate.gitcd surogate# follow the project's README for install & configurationNeed it deployed, integrated, or customized instead? DEV.co ships production installs.
Best use cases
Implementation considerations
- Requires CUDA 12.8, 12.9, or 13.x with matching NVIDIA driver (≥570 recommended for 12.8); pre-built wheels provided but build-from-source demands libnccl-dev and development environment setup.
- Docker deployment recommended for reproducibility; ensure GPU passthrough (--gpus=all) and volume mounts for configs/datasets are properly configured to avoid path/permission issues.
- FP8/FP4 recipes are hardware-dependent (FP4 NVFP4 targets Blackwell SM100+); validate precision choice against GPU compute capability before full-scale training runs.
- Config-driven training via YAML with explicit recipe selection (BF16/FP8/NVFP4/mixed-precision); test recipes on representative data subset to verify convergence and loss trajectories before production workloads.
- Adaptive training features (auto-LR, early stopping, phase detection) are opaque; if determinism and reproducibility are critical, disable auto features and lock hyperparameters explicitly.
When to avoid it — and what to weigh
- Requires immediate production-grade SLA support or commercial indemnity — Project is open-source (Apache 2.0) with 806 stars and small community. No explicit commercial support entity or warranty stated in available data. Requires organization's own reliability assessment.
- Need to train on AMD, Intel, or non-NVIDIA accelerators — Purpose-built for NVIDIA CUDA; no support for AMD ROCm, Intel GPUs, or TPUs. Docker images and install scripts are x86-64 only.
- Dependency on ecosystem integrations (MLOps, monitoring, deployment platforms) — No integrations with Weights & Biases, Neptune, or other MLOps platforms mentioned. Custom monitoring and logging would be needed for enterprise observability pipelines.
- Unfamiliar with low-precision training tradeoffs or unstable convergence tolerance — FP8/FP4 training introduces numerical stability considerations (per-tensor delayed scaling, stochastic rounding). Projects requiring strict numerical reproducibility or lacking domain expertise should prototype extensively.
License & commercial use
Apache License 2.0 (Apache-2.0). Permissive OSI-approved license permitting commercial use, modification, and redistribution under permissive terms (no copyleft). No explicit patent grants or indemnification clauses beyond Apache 2.0 standard terms.
Apache 2.0 permits commercial deployment. However, the project offers no stated commercial support, SLA, warranty, or indemnity from the maintainer (invergent-ai). Organizations deploying in production should: (1) conduct own security/liability review, (2) establish internal support capability or engage third-party support, (3) confirm no patent/IP conflicts within their use case. Use of pre-trained weights (Qwen, Llama) may have separate license obligations.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Adequate |
| License clarity | Clear |
| Deployment complexity | Moderate |
| DEV.co fit | Good |
| Assessment confidence | High |
No explicit security audit, threat model, or vulnerability disclosure policy provided in available data. C++/CUDA codebase is attack surface for memory safety issues (buffer overflows, UAF); however, modern C++ tooling and NVIDIA CUDA APIs provide some mitigations. Distributed training (Ray) and dataset loading from external sources (MLAbonne/HF) introduce supply-chain and data-injection risks. Requires: (1) review of C++ dependencies and CUDA version EOL status, (2) input validation for YAML configs and dataset paths, (3) network isolation for multi-node clusters.
Alternatives to consider
Hugging Face Transformers + bitsandbytes QLoRA
Mature, widely-used Python ecosystem with extensive integrations (Weights & Biases, HF Hub, vLLM). BnB QLoRA provides memory-efficient fine-tuning on consumer GPUs. Trade-off: slower training throughput than Surogate's claimed performance; steeper learning curve for low-precision tuning.
PyTorch FSDP (Fully Sharded Data Parallel) + DTensor
Native PyTorch distributed training with autograd support. Flexible for custom models and research. Trade-off: requires more manual optimization (quantization, offloading); less turnkey than Surogate's pre-built recipes; community support only.
NVIDIA NeMo Framework
Enterprise-focused training framework with built-in model zoo (Nemotron, Paxml), mixed-precision, and multi-node support. Overlaps with Surogate's feature set. Trade-off: heavier dependency stack; less emphasis on low-precision (FP4) training; requires NVIDIA ecosystem buy-in.
Build on surogate with DEV.co software developers
Explore Surogate's high-performance fine-tuning framework for on-premise or cloud LLM workloads. Start with Docker, review docs at docs.surogate.ai, and run your first training in minutes.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
surogate FAQ
Can I train Llama 3.1 405B on a single GPU?
How do I export or deploy a trained Surogate LoRA adapter?
Does Surogate work with older NVIDIA GPUs (e.g., V100, A100)?
What happens if my training diverges with FP8 quantization?
Work with a software development agency
Need help beyond evaluating surogate? DEV.co is a software development agency offering software development services and web development for teams of every size. Our software developers and web developers build custom software, web applications, APIs, and ai frameworks integrations — and maintain them long-term.
Accelerate Your LLM Training Today
Explore Surogate's high-performance fine-tuning framework for on-premise or cloud LLM workloads. Start with Docker, review docs at docs.surogate.ai, and run your first training in minutes.