AI Frameworks · hiyouga

EasyR1

EasyR1 is an open-source reinforcement learning framework for training vision-language and language models efficiently at scale. It supports multiple RL algorithms (GRPO, DAPO, ReMax, etc.) and popular model families (Qwen, Llama, DeepSeek-R1) with multi-GPU and multi-node training capabilities.

Talk to DEV.co Discuss Implementation

Source: GitHub — github.com/hiyouga/EasyR1

GitHub stars

373

Forks

Python

Primary language

Apache-2.0

License (OSI-approved)

Key facts

Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.

Field	Value
Repository	hiyouga/EasyR1
Owner	hiyouga
Primary language	Python
License	Apache-2.0 — OSI-approved
Stars	5k
Forks	373
Open issues	53
Latest release	v0.3.2 (2025-09-18)
Last updated	2026-04-06
Source	https://github.com/hiyouga/EasyR1

What EasyR1 is

Built as a fork of veRL, EasyR1 leverages HybridEngine and vLLM's SPMD mode for distributed RL training. It provides padding-free training, LoRA support, checkpoint resumption, and integrations with Weights & Biases, SwanLab, MLflow, and TensorBoard for experiment tracking.

Quickstart

Get the EasyR1 source

Clone the repository and explore it locally.

terminalbash

1git clone https://github.com/hiyouga/EasyR1.git2cd EasyR13# follow the project's README for install & configuration

Need it deployed, integrated, or customized instead? DEV.co ships production installs.

Best use cases

Vision-Language Model Fine-Tuning with RL

Train Qwen2-VL, Qwen2.5-VL, or Qwen3-VL models on custom vision-text datasets using GRPO or other RL algorithms. Well-suited for organizations building specialized multimodal reasoning capabilities.

Large-Scale Distributed RL Training

Train 70B+ parameter models across multiple nodes using Ray orchestration and multi-GPU configurations. Ideal for research teams and enterprises with access to GPU clusters and need for efficient scaling.

DeepSeek-R1 Distillation and Adaptation

Fine-tune DeepSeek-R1 distill models on domain-specific datasets using supported RL algorithms. Applicable for organizations wanting reasoning-enhanced models tailored to specific tasks.

Implementation considerations

Python 3.9+, transformers ≥4.54.0, flash-attn ≥2.4.3, and vllm ≥0.8.3 are hard requirements; validate your environment before large training runs.
Docker image (hiyouga/verl:ngc-th2.8.0-cu12.9-vllm0.11.0) is provided for reproducibility; consider containerized deployment to avoid environment drift.
Data must conform to specific text and vision-text formats documented in the repository; custom datasets require upfront formatting and validation.
Multi-node training with Ray requires head and worker node setup; network configuration and Ray dashboard monitoring should be part of operational planning.
LoRA fine-tuning reduces memory significantly (e.g., 4×80GB for 72B full-tune → 4×80GB for LoRA); choose method based on your hardware budget and adaptation needs.

When to avoid it — and what to weigh

Minimal GPU Resources or Single-GPU Constraints — Framework is designed for multi-GPU training. Hardware requirements start at 2×24GB for smallest models (1.5B) in AMP mode. Unsuitable for CPU-only or single consumer GPU deployments.
Need for Proprietary Model Support Outside Listed Families — Supported models are limited to Llama3, Qwen, and DeepSeek families. If your primary models are Claude, GPT, or other architectures, you will need custom integration work.
Production Inference without Research/Experimentation Mindset — EasyR1 is a training framework with recent releases (v0.3.2 Sept 2025) and ongoing feature additions. Stability for production serving is not guaranteed; intended for research and model development.
Strict Compliance or Auditing Requirements in Regulated Industries — No security audit, compliance certifications, or formal SLA documentation is mentioned. Requires careful review for healthcare, finance, or regulated sector deployments.

License & commercial use

Licensed under Apache License 2.0, a permissive OSI-approved open-source license allowing commercial use, modification, and distribution with attribution and liability disclaimers.

Apache-2.0 explicitly permits commercial use. However, ensure compliance with dependent libraries (transformers, vllm, flash-attn, PyTorch) which may have different licenses. Verify all transitive dependencies before enterprise deployment. No warranty or support SLA is included in the open-source license.

DEV.co evaluation signals

Editorial assessment — not user reviews. Directional, with an explicit confidence level.

Signal	Assessment
Maintenance	Active
Documentation	Adequate
License clarity	Clear
Deployment complexity	High
DEV.co fit	Good
Assessment confidence	High

Security considerations

No formal security audit, threat modeling, or vulnerability disclosure process mentioned. Input validation and data sanitization practices are not documented. When training on sensitive data, review handling of gradients, checkpoints, and logged artifacts. Ray multi-node communication should use network isolation in untrusted environments. Dependent libraries (vllm, PyTorch) should be kept updated for security patches.

Alternatives to consider

veRL (upstream project)

Original high-performance RL framework by ByteDance. EasyR1 is a fork adding vision-language support. Choose upstream if you need language-only training and prefer official maintenance.

TRL (Transformers Reinforcement Learning)

Hugging Face's RL library with GRPO trainer and broader model support. Simpler integration with Hugging Face ecosystem but less optimized for very large-scale distributed training.

vLLM + custom RL loop

Combine vLLM for inference and a custom PyTorch RL implementation. Offers maximum flexibility but requires significant engineering effort; suitable only for teams with strong ML infrastructure.

Software development agency

Build on EasyR1 with DEV.co software developers

EasyR1 provides production-grade distributed RL capabilities for multimodal AI. Start with the 3-step Qwen2.5-VL example, or contact our AI specialists to architect a training pipeline for your model and data.

Talk to DEV.co

Related open-source tools

Surfaced by semantic similarity across the DEV.co open-source index.

xtuner DeepSeek-R1 DeepSeek-R1-Distill-Llama-8B

Related on DEV.co

Explore the category and the services that help you build with it.

AI Frameworks AI Application Development Cloud Deployment & Infrastructure Talk to DEV.co

EasyR1 FAQ

Can I use EasyR1 with proprietary models like GPT or Claude?

No. EasyR1 is designed for open-source models in the Llama, Qwen, and DeepSeek families. Proprietary APIs do not support the local RL training pipeline required.

What is the minimum hardware needed to start?

For 1.5B models in AMP mode with GRPO LoRA: 1×12GB GPU. For 3B full fine-tuning: 1×24GB. See the hardware requirements table in README for other model sizes. CPU-only training is not practical.

How does EasyR1 differ from veRL?

EasyR1 adds vision-language model support (Qwen2-VL, Qwen2.5-VL, Qwen3-VL) and recent algorithms (DAPO, GSPO, CISPO). It is a maintained fork; veRL is the upstream project.

Is there production SLA or paid support?

No. EasyR1 is open-source community-driven software. AWS mentions using it internally, but no commercial support contract or SLA is documented. GitHub issues are the primary support channel.

Software developers & web developers for hire

DEV.co helps companies turn open-source tools like EasyR1 into production software. Our software development services cover the full lifecycle — architecture, web development, integration, and maintenance — delivered by software developers and web developers who ship. Engage our software development agency to implement or customize it for your ai frameworks stack.

Ready to Scale RL Training for Vision-Language Models?

Talk to DEV.co Schedule a Consultation