Qwen3.6-35B-A3B-Abliterated-Heretic-AWQ-4bit
Qwen3.6-35B-A3B-Abliterated-Heretic-AWQ-4bit is a 36.2B-parameter multimodal Mixture-of-Experts model quantized to 4-bit for efficient inference. It handles text, images, video, and audio inputs, supports tool calling, and is derived from an 'uncensored' variant. The model is unmodified after quantization, lightweight for deployment, and openly available under Apache-2.0.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | genevera |
| Parameters | 36.2B |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 52.9k |
| Likes | 8 |
| Last updated | 2026-04-26 |
| Source | genevera/Qwen3.6-35B-A3B-Abliterated-Heretic-AWQ-4bit |
What Qwen3.6-35B-A3B-Abliterated-Heretic-AWQ-4bit is
A quantized derivative of Qwen3.6-35B-A3B-Abliterated-Heretic-BF16. Architecture: Qwen3_5MoeForConditionalGeneration with 40 text layers (256 MoE experts per layer, 8 activated per token), 27-layer vision encoder, Grouped Query Attention (16 query / 2 KV heads), 262,144 token context window. Quantization: 4-bit AWQ via auto-round v0.10.2 with group size 128; critical components (lm_head, shared expert gates, vision encoder, vision merger) preserved in full precision. Vision: supports up to 768 video frames at 2 FPS; multimodal chat template includes reasoning blocks and function-calling. Transformers 5.6.0.dev0.
Run Qwen3.6-35B-A3B-Abliterated-Heretic-AWQ-4bit locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="genevera/Qwen3.6-35B-A3B-Abliterated-Heretic-AWQ-4bit")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE: 4-bit quantization + hybrid precision strategy suggests ~18–24 GB VRAM for single-instance inference on a single GPU (e.g., A40, RTX 6000, H100). Full-precision load would require ~72 GB+. Exact footprint depends on: (1) vision encoder batch size, (2) context length utilization, (3) KV cache retention. Transformers + auto-round will auto-allocate across multi-GPU if available. For batch serving, assume ≥40 GB VRAM per replica. CPU inference via llama.cpp is not feasible for this model size and architecture complexity. Requires NVIDIA/AMD with CUDA/ROCm support.
Model card provides no guidance on fine-tuning. LoRA/QLoRA on quantized weights is feasible but not documented. Risks: (1) QLoRA adapters may require careful learning-rate tuning due to hybrid precision (full-precision gates + quantized layers). (2) Vision encoder is full-precision and relatively large; fine-tuning it may exceed memory budgets without gradient checkpointing. (3) MoE expert load-balancing during fine-tuning is non-trivial; uneven expert utilization can degrade generalization. Recommendation: test on small dataset first; consider keeping vision and gate layers frozen unless domain shift is severe.
When to avoid it — and what to weigh
- Strict Content-Moderation Requirements — Model is explicitly labeled 'Abliterated' and 'Uncensored.' If your application requires safety guardrails, content filtering, or compliance with strict output policies, this model's training pedigree does not prioritize such constraints. Requires custom filtering downstream.
- Latency-Critical Single-Token Inference — MoE architecture with 256 experts per layer introduces routing overhead. On very low-latency endpoints (e.g., <50ms per token), traditional dense models may outperform even with 4-bit quantization. Benchmark against dense alternatives if tail latency is strict.
- Guaranteed Numeric Accuracy or Scientific Computing — 4-bit quantization and hybrid precision strategy preserve output quality for language tasks but do not guarantee bit-for-bit reproducibility or numerical precision required for symbolic math, formal verification, or scientific computation.
- Resource-Constrained Edge Deployment — Even quantized, 36.2B parameters demand substantial VRAM and compute. Consumer GPUs (e.g., RTX 3090 with 24GB) may struggle with batched inference. Requires GPU with ≥40GB VRAM for comfortable deployment; edge CPUs are infeasible.
License & commercial use
Apache License 2.0 (apache-2.0). Permissive OSI-approved license. Permits commercial use, modification, distribution, and private use subject to license and copyright notice retention.
Apache-2.0 is a permissive OSI license that explicitly allows commercial use. However, note: (1) Model is a quantized derivative of 'Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-BF16,' which itself is a derivative of Qwen3.6 (Alibaba). Verify Qwen3.6's license and terms (typically Qwen/Alibaba Model License) to ensure the derivative is lawfully offered under Apache-2.0. (2) The 'Abliterated-Heretic-Uncensored' variant may have training data or procedural lineage not fully documented; legal review recommended if model output will be used in high-stakes or regulated contexts. (3) No commercial support or SLA is implied; you assume all liability for model outputs. Recommendation: consult your legal team if embedding in commercial products.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Moderate |
| Documentation | Adequate |
| License clarity | Clear |
| Deployment complexity | Moderate |
| DEV.co fit | Good |
| Assessment confidence | High |
No explicit security audit, threat model, or adversarial robustness assessment provided. Considerations: (1) Uncensored model may be more susceptible to prompt injection or jailbreak attacks. (2) Multimodal input (image, video) parsing introduces surface area for adversarial image or EXIF-based attacks. (3) No mitigation against model inversion or membership inference noted. (4) Quantization does not introduce cryptographic or information-disclosure protections. (5) No built-in rate limiting, authentication, or audit logging in model weights. Recommendation: assume model outputs require downstream validation and sandboxing; do not deploy as-is in untrusted or high-security environments without additional layers (input filtering, output classification, rate limiting, monitoring).
Alternatives to consider
Llama-2-70B-AWQ (or Llama-3 equivalents)
Denser, single-expert architecture; wider adoption and better documented; moderation layers may be stronger. Trade-off: larger parameter count and less dynamic expert routing.
Phi-3.5-MoE-8x7B
Smaller MoE model with similar quantization options; faster inference on constrained hardware; Microsoft-backed with stronger maintenance signals. Trade-off: smaller expert count and narrower multimodal capabilities.
Qwen2.5-72B-Instruct (full precision or native quantization)
Earlier-generation Qwen model with clearer lineage, stronger Alibaba backing, and better safety training. If multimodal is not critical, dense variant may have lower operational overhead. Trade-off: no MoE routing; larger VRAM footprint.
Ship Qwen3.6-35B-A3B-Abliterated-Heretic-AWQ-4bit with senior software developers
Contact our AI engineering team to discuss architecture fit, quantization trade-offs, fine-tuning feasibility, and deployment strategy. We help you benchmark against alternatives and optimize for your hardware and latency constraints.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
Qwen3.6-35B-A3B-Abliterated-Heretic-AWQ-4bit FAQ
Can I use this model commercially?
What GPU do I need to run this model?
Does this model support fine-tuning?
Why is this model called 'Abliterated' and 'Uncensored'?
Software development & web development with DEV.co
Need help beyond evaluating Qwen3.6-35B-A3B-Abliterated-Heretic-AWQ-4bit? DEV.co is a software development agency offering software development services and web development for teams of every size. Our software developers and web developers build custom software, web applications, APIs, and open-source llms integrations — and maintain them long-term.
Ready to Evaluate This Model for Your Use Case?
Contact our AI engineering team to discuss architecture fit, quantization trade-offs, fine-tuning feasibility, and deployment strategy. We help you benchmark against alternatives and optimize for your hardware and latency constraints.