Ornith-1.0-35B-NVFP4-MTP-GGUF
Ornith-1.0-35B-NVFP4-MTP-GGUF is a 35-billion-parameter mixture-of-experts (MoE) language model optimized for NVIDIA Blackwell GPUs. It combines aggressive 4-bit quantization (NVFP4), multi-token prediction for speculative decoding, and is packaged as a single GGUF file for llama.cpp. Designed for agentic coding and reasoning tasks, it achieves ~1.5–2× decode speedup over non-speculative baselines on supported hardware while maintaining model quality.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | s-batman |
| Parameters | Unknown |
| Context window | Unknown |
| License | mit — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 37.9k |
| Likes | 28 |
| Last updated | 2026-06-29 |
| Source | s-batman/Ornith-1.0-35B-NVFP4-MTP-GGUF |
What Ornith-1.0-35B-NVFP4-MTP-GGUF is
Ornith-1.0-35B is a post-trained derivative of Qwen3.5-35B-A3B using reinforcement learning, inheriting the qwen3_5_moe architecture (40 layers, 256 experts, ~3B active parameters, 248K token vocab). This GGUF variant uses NVFP4 quantization (E2M1 elements, 16-element blocks, E4M3 scales) on the 80 trunk 3D expert tensors (4.50 bpw average), Q8_0 on 2D linears and sensitive tensors, and F32/BF16 for routing logic. An auxiliary MTP (multi-token prediction) head is grafted from Qwen3.6-35B-A3B, enabling draft-mtp speculative decoding. File size: 21.31 GB. Measured RTX PRO 6000 (Blackwell sm_120) performance: 310–320 tok/s decode (MTP+body) vs 240 tok/s (body-only), ~1.5–1.85× speedup depending on sampling temperature. Context length: Unknown.
Run Ornith-1.0-35B-NVFP4-MTP-GGUF locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="s-batman/Ornith-1.0-35B-NVFP4-MTP-GGUF")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
NVIDIA Blackwell GPU (sm_120 / sm_121): RTX PRO 6000 (97 GB GDDR7, ~1.79 TB/s) or DGX Spark GB10 (121 GB LPDDR5X, ~273 GB/s). File size 21.31 GB; with 200k context and KV cache q8_0, estimate ~28–35 GB total VRAM for single-instance inference (including MTP head KV overhead ~280 MiB). Multi-instance on DGX Spark: 3 parallel 200k-context instances fit within 121 GB. CPU dequantization possible but not recommended—NVFP4 requires native Blackwell FP4 support for stated performance.
Ornith-1.0-35B is a post-trained model; fine-tuning via LoRA or QLoRA would require unquantized weights. Obtaining base weights is Unknown (not stated in card). GGUF format is inference-only. If fine-tuning is needed, contact DeepReinforce or HuggingFace for unquantized Ornith-1.0-35B or upstream Qwen3.5-35B-A3B, then apply LoRA before quantizing to GGUF. No guidance given on adapter stability with MoE routing.
When to avoid it — and what to weigh
- AMD or older NVIDIA GPUs (pre-Blackwell) — NVFP4 is Blackwell-only. MXFP4 variant supports AMD MI355X but this NVFP4 file does not. Older GPUs (H100, A100) cannot natively dequantize either format; fallback to CPU dequantization will be slow.
- Require vision/multimodal capabilities — Card states 'vision tower not included in this GGUF.' Multimodal vocabulary is preserved but the model cannot process images or other modalities.
- Need guaranteed single-token latency or low-concurrency real-time inference — MTP speculative decoding introduces draft verification overhead. At low concurrency or with greedy sampling, MTP accept rates are highest (~100%) but per-token latency may not improve; MTP costs ~280 MiB of KV context. If <200 ms p95 latency on a single request is critical, measure first.
- Require fine-tuning or custom training — GGUF is a serialized inference format. To fine-tune, you must obtain the base Qwen3.5-35B-A3B or Ornith-1.0-35B unquantized weights (Unknown availability/licensing from HuggingFace or DeepReinforce). LoRA / QLoRA on GGUF is not standard practice.
License & commercial use
MIT License (inherited from Qwen3.5-35B-A3B upstream and Ornith-1.0-35B base model). MIT is a permissive OSI-approved open-source license allowing commercial use, modification, and redistribution with minimal restrictions (attribution required).
MIT license permits commercial deployment, SaaS, and derivative products provided attribution is given. No additional restrictions stated in card. However, verify that the base model Ornith-1.0-35B and Qwen3.5-35B-A3B (from DeepReinforce and Alibaba Qwen respectively) do not impose additional clauses. Requires review of upstream model cards and any commercial support agreements with DeepReinforce or Alibaba.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Strong |
| License clarity | Clear |
| Deployment complexity | Moderate |
| DEV.co fit | Strong |
| Assessment confidence | High |
No security posture, threat model, or adversarial evaluation stated in card. Standard considerations: (1) GGUF is a binary serialization format; verify checksums if distributed. (2) Model inherits safety from Qwen3.5-35B-A3B; no additional safety tuning mentioned. (3) Speculative decoding (MTP) does not change model outputs—same distribution as non-MTP. (4) Running untrusted code during inference (e.g., tool calls) carries normal LLM risks. (5) No input validation, jailbreak mitigations, or content filtering described. Recommend standard inference sandboxing and input sanitization.
Alternatives to consider
Qwen3.6-35B-A3B (unquantized or native GGUF)
Direct upstream. No post-training via RL, so reasoning capability is baseline Qwen3.6 vs Ornith's DeepReinforce RL tuning. Offers more flexibility (fine-tuning, custom quantization) but larger VRAM footprint without quantization.
Gemma 4-31B (Google, unquantized)
Comparable 31B size, code/reasoning competitive per card benchmarks. Permissive license (Apache 2.0 / Gemma terms). No MoE; simpler deployment. Requires different quantization pipeline (not GGUF-native).
Llama 3.1 70B (Meta, unquantized or community quantizations)
Larger, stronger general-purpose model. Many community GGUF variants exist. Wider ecosystem support (vLLM, TGI, etc.). Trade-off: 2× VRAM, longer latency, overkill for coding-specific agentic tasks unless reasoning depth matters more.
Ship Ornith-1.0-35B-NVFP4-MTP-GGUF with senior software developers
Verify your hardware (RTX PRO 6000 or DGX Spark), download the 21.31 GB GGUF, and run inference with llama.cpp. Check upstream Ornith-1.0-35B and Qwen3.5-35B-A3B licenses before commercial deployment. Contact DeepReinforce for support, fine-tuning guidance, or unquantized weights.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
Ornith-1.0-35B-NVFP4-MTP-GGUF FAQ
Can I use this model commercially, e.g., in a SaaS product or closed-source application?
What hardware do I need to run this model?
Does MTP (multi-token prediction) change the model's output quality?
Can I fine-tune this quantized GGUF?
Software developers & web developers for hire
Adopting Ornith-1.0-35B-NVFP4-MTP-GGUF is usually one piece of a larger software development effort. As a software development agency, DEV.co provides software development services and web development expertise — pairing senior software developers and web developers with your team to design, build, and operate open-source llms software in production.
Ready to Deploy Ornith-1.0-35B on Blackwell?
Verify your hardware (RTX PRO 6000 or DGX Spark), download the 21.31 GB GGUF, and run inference with llama.cpp. Check upstream Ornith-1.0-35B and Qwen3.5-35B-A3B licenses before commercial deployment. Contact DeepReinforce for support, fine-tuning guidance, or unquantized weights.