Jan-nano-128k
Jan-Nano-128k is a 4B-parameter open-source language model from Menlo Research with a native 128k token context window, designed for research and document analysis. It builds on the Jan-Nano base model and is distributed under Apache 2.0. The model is non-gated and can be self-hosted via vLLM, llama.cpp, or similar inference servers.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | Menlo |
| Parameters | 4B |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 59.6k |
| Likes | 223 |
| Last updated | 2025-07-01 |
| Source | Menlo/Jan-nano-128k |
What Jan-nano-128k is
A 4.02B-parameter transformer-based text-generation model fine-tuned from Qwen3 architecture with extended RoPE (rope_type: yarn, factor 3.2, original_max_position_embeddings 40960). Natively supports 128k context length. Last updated 2025-07-01. Inference tested with vLLM and llama.cpp; compatible with text-generation-inference endpoints. Chat template included in tokenizer (Qwen3 non-thinking format).
Run Jan-nano-128k locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="Menlo/Jan-nano-128k")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE: 8–16 GB GPU VRAM for inference at full 128k context (fp16/bfloat16). Exact requirements depend on batch size and serving framework (vLLM with auto-scaling may require upper bound). CPU-only inference via llama.cpp possible but slower. Verify empirically for your infrastructure before production deployment.
Unknown whether LoRA/QLoRA adapters have been published or tested on this model. Model card does not mention fine-tuning recipes or community fine-tuning efforts. Contact developer (GitHub issues or HF discussions) for guidance on custom domain adaptation. Standard transformer fine-tuning tools (transformers library, Axolotl) likely compatible but untested.
When to avoid it — and what to weigh
- Latency-Critical Real-Time Applications — Processing 128k tokens incurs non-trivial inference latency. Not suitable for sub-second response requirements or high-throughput streaming scenarios.
- Superior Reasoning or Creative Output — Model is explicitly non-thinking (no internal reasoning chain). May underperform on complex logic, math, or creative tasks compared to larger reasoning-enabled models.
- Hardware-Constrained Environments — While compact relative to larger LLMs, running full 128k context locally requires 8–16 GB+ VRAM. Not suitable for edge devices, mobile, or heavily resource-constrained setups.
- Tasks Requiring Domain-Specific Expertise Without Fine-tuning — Base model was not fine-tuned for specialized domains (medical, legal, financial). Domain tasks require additional fine-tuning or RAG augmentation.
License & commercial use
Licensed under Apache 2.0, a permissive OSI-approved license. Permits commercial use, distribution, modification, and private use. Requires retention of copyright notice and license text.
Apache 2.0 explicitly permits commercial use, commercial distribution, and private deployment. No usage restrictions for commercial products or services. However, verify your derivative work (if any) complies with the license. No commercial support, SLA, or indemnification from Menlo Research is stated; support is community-driven via HF discussions and GitHub issues.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Adequate |
| License clarity | Clear |
| Deployment complexity | Moderate |
| DEV.co fit | Strong |
| Assessment confidence | High |
No explicit security audit or vulnerability disclosure process documented. As an open-source model, security relies on community review and standard transformer inference safeguards. Self-hosted deployment avoids third-party API/data exposure. Review inference server (vLLM, llama.cpp) security practices separately. No information on prompt injection or jailbreak testing.
Alternatives to consider
Llama 2 / Llama 3 with RoPE extension
Larger parameter counts (7B–70B) with more established community and enterprise support. May offer better performance on complex tasks but higher hardware cost and no native 128k training.
Qwen (full size) models with native long context
Base architecture is Qwen3; larger Qwen variants may offer stronger performance and broader community adoption, though increased deployment complexity and cost.
Mistral 7B with context extension
Larger parameter count (7B vs. 4B) with well-established open-source community. Supports extended context via rope scaling but not natively trained on it.
Ship Jan-nano-128k with senior software developers
Jan-Nano-128k enables research-grade document processing on your infrastructure. Start with our deployment guides for vLLM or llama.cpp, or explore how Devco can help architect a private LLM pipeline for your team.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
Jan-nano-128k FAQ
Can I use Jan-Nano-128k in a commercial product?
What GPU do I need to run this locally?
How does Jan-Nano-128k compare to the original Jan-Nano?
Is this model suitable for reasoning and math?
Custom software development services
DEV.co is a software development agency delivering custom software development services to companies building on open source. Our software developers and web developers design, integrate, and ship production systems — spanning web development, APIs, AI, data, and cloud. If Jan-nano-128k is part of your open-source llms roadmap, our team can implement, customize, migrate, and maintain it.
Ready to Deploy Long-Context Intelligence Privately?
Jan-Nano-128k enables research-grade document processing on your infrastructure. Start with our deployment guides for vLLM or llama.cpp, or explore how Devco can help architect a private LLM pipeline for your team.