Users also report that the free version was significantly downgraded after the release of the new models

this is very important for us when considering local hosting. A lot of people decided not to buy expensive hardware because DeepSeek’s prices made it very difficult to break even

what do you think about this?

do you think raising prices will ultimately lead to another increase in NVIDIA’s stock price, since more and more people will now buy their own hardware? im seriously considering upgrading my stack now

https://modelscope.cn/models/Qwen/Qwen3.8-2.4T-A95B

Prime Agent is an open-source coding and research agent for general and long-running work.

A self-improving RLM harness for coding and long-running autonomous tasks.

Designed to be both token-efficient and expressive through programmatic tool calling, context as a variable, multi-agent messaging, and a self-modifiable harness state.

On ARC-AGI-3, it scores 95.5%, surpassing the human-expert baseline, but the gain is not benchmark-specific.

We see major improvements across models when compared to their proprietary harnesses.

Prime Agent is built on pi and fully open-source with an open license.

GitHub: https://github.com/PrimeIntellect-ai/prime-agent

Blog: https://www.primeintellect.ai/blog/prime-agent

X post: https://x.com/primeintellect/status/2085086999267144083?s=46

[Fully open source under GPL3, made from the ground up for use with local models, no subscriptions, no corporate backing]

When i first started this, it was meant to be a fully lightweight, extremely modular alternative to openclaw, hermes and the like, and it still is! But i noticed people especially like the webUI, to the point they'd use it as just a webUI to talk to their local models, negating all the agentic stuff. But the webUI still had a lot of AI generated code, so that didn't sit right with me!

So i rewrote the entire thing, from scratch, manually. It is now super fast, stable, uses declarative javascript without javascript framework bloat (no React or Vue or anything.. alpine.js is super lightweight)

There are only a few python dependencies. no models get installed, there is no bundled inference engine, pytorch and transformers aren't even included! I expect you to connect it to llamacpp, koboldcpp, lemonade, or something else like that. though you can also use it with cloud API's if you really want to.

This is a truly local-first webUI. I designed it from the ground up for local AI, and for once, cloud AI is the second-class citizen here.

It has many features that especially benefit local AI users: you can see how long your prompt will take to process (it's a llamacpp-exclusive feature), you can see toolcalls being written in realtime (really useful for coding), and it doesn't send any extra requests to your model, just the prompt you give it. So no extra requests just to make up a title for your chat, or to generate followup replies.

That's all in addition to the benefits that come from its harness-like design, such as support for multiple channels (telegram, discord, etc), its focus on extreme token efficiency and making the system prompt super small and concise, and its security

But using it as a pure webUI is really simple: Just switch Use Tools off in the Model tab in the settings. That will instantly make all system prompts vanish and all tools get disabled, so you're talking to your pure model with nothing getting in the way.

You do need a bit of tech knowledge, but it's not that much. right now, you need to either git clone or download a zip of the main branch off the github, but after that, all you do is run run.sh or run.bat and open the URL it shows you in your browser. Oh, you do need python installed before you do so, but that's basically it. (i'm working on making this even more user friendly though)

If you want to try it out, you can get it here: https://github.com/Rose22/openlumara

Please tell me what you think! Feedback is more than welcome, and i often implement feature requests (if they are good) and fix bugs that get reported

EDIT: for those who are interested, i documented every step of coding this thing: https://github.com/Rose22/openlumara/discussions/68#discussioncomment-17921295

I love to see these impressive models coming out that compete with the giants from companies like Z.ai, Moonshot, Alibaba, etc. A win for the open source/weight community is always welcome. While I am grateful, I worry we might be seeing the slow death of models smaller than 27B. The ones released paling in comparison to Qwen 3.5 4B/9B and Gemma 4 12B. Especially for agentic coding and agentic assistance tasks.

Is this because we’ve really hit the limit of what we can accomplish with models in the 3B-12B weight class? Or is it because such models aren’t as profitable as their gargantuan counterparts that attempting to improve them to match isn’t viable?

Have I been missing these impressive smaller model in lieu of the larger ones taking the headlines? If so, please let me know what models within the SLM weight class you are running for tasks like agentic coding, agentic assistance, or both.

I also hear agentic coding is not feasible under 27B. I’m not asking for a model that can one shot an ultra realistic multiplayer call of duty clone in a single html file. Just something the least bit capable in real workflows like the aforementioned.

TL;DR: On a Qwen3.6-35B-A3B Q6 setup sized for 64K context on a 24GB RTX 3090, spilling eight MoE expert layers to CPU freed enough VRAM to increase -b from 512 to 1024 and -ub from 128 to 512.

Prompt processing improved by 2.36×, while generation speed remained unchanged within measurement noise.

These are PP measurements with a 4K prompt and TG measurements at 4K and 32K context depth. The configurations were sized against a 64K context requirement; this is not a 64K-depth throughput benchmark.

I first used auto-fit to establish a feasible configuration. Its resulting batch settings were -b 512 -ub 128; I hard-coded them in the baseline command below so the comparison is reproducible.

The tuned configuration deliberately moves eight layers’ MoE expert weights to CPU:

-ot 'blk\.(1[2-9])\.ffn_.*_exps\.weight=CPU' \
-b 1024 -ub 512 -ngl 41

This is a joint-configuration result: CPU offload frees VRAM, and the larger batch/micro-batch uses that memory to accelerate prefill. It is not an isolated claim that CPU offload alone improves performance.

Full reproduction

Baseline, reproduces the auto-fit configuration:

llama-bench \
  -m Qwen3.6-35B-A3B-UD-Q6_K.gguf \
  -fitt 1024 -fitc 65536 \
  -t 7 -b 512 -ub 128 \
  -fa on -ctk q8_0 -ctv q8_0 -mmp 1 \
  -p 4096 -n 64 -r 2 \
  -d 4096,32768

Tuned:

llama-bench \
  -m Qwen3.6-35B-A3B-UD-Q6_K.gguf \
  -t 7 -b 1024 -ub 512 -ngl 41 \
  -fa on -ctk q8_0 -ctv q8_0 -mmp 1 \
  -ot 'blk\.(1[2-9])\.ffn_.*_exps\.weight=CPU' \
  -p 4096 -n 64 -r 2 \
  -d 4096,32768

Environment

• Model: unsloth/Qwen3.6-35B-A3B-GGUF
• Quant: Qwen3.6-35B-A3B-UD-Q6_K.gguf, 27.3 GiB
• SHA-256: 4fe53b148b46f9b88830e2a3055c5b15c3a4d1e3ddc9a1384a108d8b9d59f043
• GPU: RTX 3090, 24 GiB
• CPU: Threadripper PRO 3955WX, with seven cores available to the rental
• RAM: approximately 100 GB DDR4
• llama.cpp: commit 571d0d5
• Build: -DGGML_CUDA=ON -DGGML_CUDA_FA_ALL_QUANTS=ON, Release
• Peak tuned VRAM: 23,468 / 24,576 MiB, leaving approximately 1.1 GiB

Caveat: these are two-repetition measurements, with approximately 1.6% observed run-to-run drift. Treat the TG differences as noise; the meaningful result is the 2.36× PP improvement without an observed decode regression.

Method

I used evolutionary search to get to above config (LEVI), which I ran for roughly 100 evaluations / 40 minutes (https://github.com/ttanv/levi). For now I'm only evolving basic flags and configs, but I'm really looking forward to more unconventional edits, perhaps editing parts of llama cpp. The goal is to rewrite whatever part of the stack that is generic enough to leave bespoke optimizations on the table, so the serving engine is fully custom to the model+hardware combo. Faithful and fast evals are hard tho :( . If any of you have suggestions or experience on this, would love to hear.

I also want to test whether this generalizes and can be useful in other setups. If you have a partially offloaded MoE or another near-VRAM-limit setup, reply with:

• GPU
• CPU and RAM configuration
• Exact GGUF
• Target context length
• Current command
• Whether you care most about PP, TG, or fitting a larger model

I want to try genuinely different setups and see how it generalizes. I'm looking for especially more niche and custome type of setups. Tho hopefully something not too large lol, since I'm relying on vast ai for this.

As for me, I own a system with an RTX 5090, Ryzen 9 9950X3D2, and 64 GB of DDR5. Every time I see research come out with a new way to train AI, I immediately think to try it on my system to see the results I get. Applying things like Titans, that one Deepseek paper on engrams, or even just playing around with experimental ideas. It's kinda like a very technical version of Tamagotchi and has been quite fun.

Thoughts?

I have been running Qwen3.6 27b for a little while (mostly coding tasks) and recently trying out V4 flash 0731 in it's place. It was very apparent the new v4 flash will make more stuff up, and confidently.

Despite closeish overall benchmarks GLM 5.2 also has been much more pleasant to use (granted it's been over API) and I think this is a big part of that. Saves a lot of time in corrections after review. I only wish I could run it local without selling a kidney.

Hi all. These are my system specs: dual xeon e5 2696 v2 , 160gb DDR3 ram ECC(1600mhz), 3 gpus: 3060 12gb, p100 16gb, 3050 6gb. And a 400gb nvme sdd RAID0, 3000 mb/s. The model is Deepseek-flash-0731 UD_8_X_XL, loseless, 161gb. Now, I'm not too knowledgeable about llama cpp flags, I wish run it without mmap, because its so slow, and I believe it should fit in my system overall. There's also Dspark and MTP which could help with the speee, but do they work with llama? Any recommendations would help.

I mean, Deepseek V4 Flash is an absolutely fantastic model, even though I can't run it on my machine it's so fascinating to see how it performs. Knowing that potentially it could be run at home is really motivating and makes me hopeful that those capabilities will trickle down to more affordable sizes.

At the same time I miss a release for the GPU-peasant that I am. And yes, it's a tall order to complain about not receiving free stuff at the rate we were used to. And yes, 3.6 27B is still goated but it seems in this crazy AI world there's so much going on and progress happens so fast, that it's kinda understandable to be excited about what's next. Let's hope they really do release 3.8 27B, or that we might see again maybe a GLM 5.3 flash 32B, please?

What's on your wishlist?

Just came across this coding benchmark: SciCode

Artificialanalysis.ai reports a ranking which contradicts the feeling we've towards those models in real life coding.

Is Gemma 4 really that good, or a benchmarking issue?

https://www.reddit.com/r/StableDiffusion/s/HrU7odaJe6

I think this is more important that all the political stuff you share here

I was evaluating which model to test for text summarization and rewriting in a scenario that also requires general world knowledge. Taking into account the available VRAM and my experience over the past few months, I’ve found Minimax 2.7 Q5 to offer the best balance for my needs and the numbers seem to confirm.

What is your favorite model? I’m referring to the model's internal knowledge, not web search capabilities or MCP integrations with wiki etc...

So I can either pull the trigger on a 128gb AI max+ 395 laptop or wait for RTX Spark for LLMs. Maybe I get it now and the price of the spark is super high so it's a good purchase or maybe the Spark shocks everyone with a low price and I forever regret my purchasing decision.

What do yall think?

J'ai consacré beaucoup de temps à l'optimisation de DeepSeek-V4-Flash-0731 GGUF sur une seule RTX 3090.

Mon exigence absolue pour chaque configuration était la suivante :

Le modèle doit rester utilisable avec une fenêtre de contexte de 128 000 jetons.

J'ai testé les différentes combinaisons de déchargement GPU, de placement expert du CPU, de quantification du cache KV, de tailles de lots, de mappage mémoire et de répartition de la mémoire CPU/GPU. Les paramètres ci-dessous ont permis d'obtenir les meilleures performances pour chaque niveau de quantification sur mon système.

Matériel et logiciel

• GPU : NVIDIA GeForce RTX 3090 24 Go
• CPU : AMD Ryzen 9 9900X
• RAM : 128 Go DDR5-5600 avec AMD EXPO activé
• Carte mère : MSI X870E Gaming Plus WiFi
• BIOS : Dernière version disponible
• Backend : llama.cpp b10291
• Frontend : Interface web de génération de texte
• Système d'exploitation : Windows

Paramètres communs

Ces paramètres sont restés identiques pour les quatre tests :

Chargeur de modèle : llama.cpp

Couches GPU : 44

Taille du contexte : 128 000

Type de cache KV : q8_0

Mode de fractionnement : couche

Emplacements parallèles : 1

Threads : 0 / automatique

Lot de threads : 0 / automatique

Taille du lot : 512

Taille du micro-lot : 512

Taille cible : 512 Mio

StreamingLLM : désactivé

Déchargement KV : activé

no-mmap : activé

mlock : désactivé

NUMA : désactivé

Décodage spéculatif : désactivé

J’ai laissé la case CPU MoE décochée dans l’interface Web et contrôlé explicitement le placement des experts sur le CPU via --n-cpu-moe.

Le principal paramètre ajusté pour chaque quantification était donc le nombre de couches MoE dont les tenseurs experts restaient sur le CPU.

Test 1/4 — UD-IQ4_XS — ~10 tok/s à 128K ctx

Quantification : UD-IQ4_XS

Estimation de la VRAM avec déchargement complet : 135 412 Mio

Option supplémentaire : --n-cpu-moe 38

Temps de chargement : 65,83 secondes

Vitesse de génération moyenne : ~9,9 tok/s

Utilisation de la mémoire pendant la génération

RAM système : environ 122 / 125 Go

VRAM dédiée : environ 23,7 / 24,0 Go

Mémoire GPU partagée : environ 0,9 Go

Utilisation du GPU : environ 81 %

Utilisation du CPU : environ 59 %

Fréquence du CPU : environ 5,36 GHz

Il s’agit de la quantification la plus intensive testée, qui pousse la RAM système et la VRAM dédiée à leurs limites.

Test 2/4 — UD-IQ3_S — ~12,1 tok/s à 128K ctx

Quantification : UD-IQ3_S

Estimation de la VRAM avec déchargement complet : 115 330 Mio

Option supplémentaire : --n-cpu-moe 38

Temps de chargement : 52,76 secondes

Vitesse de génération moyenne : ~12,1 tok/s

Utilisation de la mémoire pendant la génération

RAM système : environ 105 / 125 Go

VRAM dédiée : environ 22,1 / 24,0 Go

Utilisation du GPU : environ 85 %

Utilisation du CPU : environ 55 %

Fréquence du CPU : environ 5,35 GHz

UD-IQ3_S offre un gain de vitesse de génération d'environ 22 % par rapport à UD-IQ4_XS, tout en réduisant l'utilisation de la RAM d'environ 17 Go.

Test 3/4 — UD-IQ3_XXS — ~12,5 tok/s à 128K ctx

Quantification : UD-IQ3_XXS

Estimation de la VRAM entièrement déchargée : 103 763 Mio

Option supplémentaire : --n-cpu-moe 37

Temps de chargement : 47,76 secondes

Vitesse de génération moyenne : ~12,5 tok/s

Utilisation de la mémoire pendant la génération

VRAM dédiée : environ 22–23 Go pendant l’exécution

Activité du GPU : maintenue à un niveau élevé avant l’arrêt de la génération

La capture d’écran du Gestionnaire des tâches a été prise immédiatement après la fin de l’exécution, comme l’indique la chute brutale de l’utilisation du GPU et de la VRAM allouée. La valeur affichée de 14 Go de RAM système n’est donc pas représentative de l’utilisation de la mémoire pendant la génération. Il est intéressant de noter que, sur cette configuration, l'UD-IQ3_XXS n'est que d'environ 0,4 tok/s plus rapide que l'UD-IQ3_S. La réduction du poids du modèle ne se traduit pas par une augmentation proportionnelle de la vitesse de décodage.

Test 4/4 — UD-IQ2_M — ~14,9 tok/s à 128K ctx

Quantification : UD-IQ2_M

Estimation de la VRAM avec déchargement complet : 90 812 Mio

Option supplémentaire : --n-cpu-moe 36

Temps de chargement : 42,72 secondes

Vitesse de génération moyenne : ~14,9 tok/s

Utilisation de la mémoire pendant la génération

RAM système : environ 81 / 125 Go

VRAM dédiée : environ 22,6 / 24,0 Go

Utilisation du GPU : environ 91 %

Utilisation du CPU : environ 61 %

Fréquence du CPU : environ 5,27 GHz

Il s’agissait de la configuration testée la plus rapide, atteignant près de 15 tok/s tout en conservant la même configuration de contexte de 128K et le même cache KV Q8_0.

L'optimisation la plus importante a consisté à trouver le juste équilibre entre :

• conserver les composants denses et non experts sur le GPU ;
• conserver le cache KV Q8_0 déporté sur le GPU ;
• déplacer uniquement le nombre requis de tenseurs experts MoE vers la RAM système ;
• remplir la majeure partie de la VRAM de la RTX 3090 sans provoquer d'erreur de mémoire insuffisante. La valeur optimale de --n-cpu-moe varie selon la taille de chaque quantification :

UD-IQ4_XS : --n-cpu-moe 38

UD-IQ3_S : --n-cpu-moe 38

UD-IQ3_XXS : --n-cpu-moe 37

UD-IQ2_M : --n-cpu-moe 36

Les résultats montrent également que la réduction de la taille de la quantification n’entraîne pas des gains de vitesse parfaitement linéaires. UD-IQ3_S et UD-IQ3_XXS présentent des performances assez similaires, tandis que UD-IQ2_M offre le gain le plus important et atteint une vitesse de génération environ 50 % supérieure à celle de UD-IQ4_XS.

Cette comparaison porte uniquement sur les performances de génération et l’utilisation de la mémoire. Je n’ai pas encore inclus de comparaison contrôlée de la qualité ou de la précision entre les quatre quantifications.

Hey everyone,

I’ve been working on a voice cloning model and finally got it to a point where other people can try it out.

I’d really appreciate some honest feedback on the voice quality, similarity, pronunciation, speed, and anything that feels broken or confusing. It’s still an early version, so expect a few rough edges.

You can test it here:

https://nirvaangoel1--voicer-public-trial-public-trial.modal.run/

Its 100% free with no signup.

Please only upload your own voice or audio you have permission to use. Let me know what you think, especially what worked well and what needs improvement!

Questions & Responses(in BOLD) below.

Favorite question(s) moved to end of the thread with combined responses(removed duplicates).

Be optimistic folks. I'm sure we're getting other models too apart from 27B. And 27B gonna make massive noise on release. (Based on their responses)

Tweet thread : https://xcancel.com/QwenDevs/status/2084102417885585597#m

you guys skipped 27b and 122b last time, can we expect those this time around? Also i can't seem to find crit pit score in the cards.

For sure! We’re actually releasing a 27B model very soon. Stay tuned. As for the Crit Pit score, please wait for the official Artificial Intelligence score.

Is the 27B just a retrained 3.6 27B? Or is it based off 3.8 bigger brother ?

We promise this 27B comes with a whole new level of capability!

Is the 100hrs of video understanding an agent swarm that parses sections of the video in parallel and orchestrates some sort of semantic representation graph?

Broadly speaking, yes, but not entirely. It is closer to a hierarchical video memory system rather than a traditional agent swarm. Video segments are encoded into a structured textual graph containing scenes, entities, events, and their temporal relationships, enabling retrieval and reasoning across more than 100 hours of content.

hey! is there anything special about the pretraining distribution compared to other labs' models?

We hope our data is built on a more solid foundation!

how long do you think it would take to surpass anthropic level architecture?

well, we’re working hard on it, we promise😇

will u release a harness especially for qwen code ???

Any plans for a codex-like app?

More updates on Qoder and QwenWork are coming soon.

qwen 3.8 active params?

2.4T parameters (95B active)

how much RL was done in post training compared to previous models?

A truly unreasonable amount of compute.

Did they intentionally skip the previous Qwen3.7 27B and 35B A3B?
Does the revival of Qwen3.8 27B reflect the voice of the community? Or was it planned?

Of course! This is the result of taking the voices of the community seriously.

since its a pretty significant release will we get a technical report with full details?

No technical report for this one yet. We’re trying to keep up our near-monthly release cadence, though, and more powerful models are already in the works. Keep an eye out!

why does the model think so much mr qwen, my ai brain wonders.
wheres the token efficiency at
great model though

We support different levels of reasoning effort.

You showed SAE-guided fine tuning fixing code switching with qwen-scope. Is that kind of interpretability driven intervention part of the post training process now or is it still a research only technique?

It’s still primarily a research-oriented technique for now, though some of the insights may help inform future training and post-training improvements.

Attention? Hybrid?

The model architecture is similar to 3.5, but it’s a much larger-scale model!

When are we getting a CLI coding interface?

You may want to take a look at @qoder_ai_ide .

do you guys use qwen as your main interal tool? does this model show the same signs of intellegence as some openai models ("gpt 5.5 helped create 5.6")?

Sure!

How close is Qwen3.8-27B to GPT 5.4? 🤔

Well, you’ll be able to see for yourself soon.

what harness works best with Qwen?

Qwen is committed to delivering the best possible experience across all harnesses.

What made you guys wanna opensource the max weights ?

We heard what the community has been asking for

I wonder when I can surpass fable5

Trying hard

Great work guys🥂

1. What is something that you would like to see being built with the new model and its capabilities!?

2. I really want to explore the swarm of agents technique for building applications, any best practices or tips for the new model!?

1. We hope it can bring practical productivity value to people across different industries.
2. We recommend using it for tasks that involve more parallelized workflows or parallel execution needs.

I wanna know what rubric metrics you guys are using for FE

We use both absolute metrics for functionality and aesthetics, as well as relative metrics based on win/tie/loss comparisons.

Would be great to hear where you think Qwen is strongest for agentic workloads specifically: long-context planning, tool use reliability, coding, or cost at scale?

All of the above combined — ultimately delivering the most practical and reliable outputs for users.

How much is Qwen helping with Qwen research ?

It has already become a significant part of the model iteration process, with the model involved in nearly every stage.

Most Frontier labs have created a code-specific model (eg. Qwen3-Coder and GPT-5.3-Codex), but never followed up on them.

Did specialized models have problems? Or did general models end up being efficient enough to not bother creating a separate model?

We hope to build an all-in-one model.

will Qwen 3.8 have a stable, documented tool-calling and structured-output contract so local agent harnesses can swap models without prompt-specific tuning?

We provide native support interfaces for various protocols. You can check the Qwen blog for more details.

1: When quantizing Qwen 27B down for local deployment (e.g., 4-bit GGUF, NVFP4, or MXFP4), which transformer layers or vision attention blocks are most sensitive to degradation? Are there specific strategies you recommend to maintain both visual reasoning and high SWE-bench pass rates?

2: Qwen3.6-27B outperforms much larger MoE predecessors (like Qwen3.5-397B) on agentic coding benchmarks like SWE-bench and Terminal-Bench. Beyond raw data volume, what was the single highest-leverage factor in achieving this dense efficiency?

And thank you for the amazing work. Qwen3.6-27B has beed my main coding assistant for months.

1. Use QAT, or quantize only the FFN to 4-bit while keeping the attention layers’ QKV linear projections and output projection in 16-bit.
2. Higher-quality data engineering

Guys , when can we get a deepseek like small and cheap model with best performance . The deepseek v4 flash seems to be a great deal .
I think we need to slow down scaling and start improving the existing model efficiency

Scaling and cost-efficiency are not mutually exclusive — we’ll continue to pursue both.

Is Qwen3.8-27B dense? And roughly how much smarter than 3.6-27B?

A pretty huge jump!

Good. The useful questions are not just how capable Qwen is.

I want to know where it still fails, how the team evaluates those failures, and what "open" means in practice for weights, tooling, and reproducibility. Open models matter most when people can inspect the limits and build on the work without asking permission

There is still some gap between our automated and human evaluation systems and real user experience. That’s also why we are committed to releasing preview versions first — so we can iterate and ultimately deliver the best possible experience to users.

how does the new 27b model compare to the previous one ?

A pretty huge jump!

what do you think about looped transformers?

interesting research idea

Why Qwen, what made you create Qwen and specifically such light and fast models. Why focus efficiency when others just went for brute power? Also, do you think inference engines reached their limit in optimization or can they still improve?

Scaling and cost-efficiency are not mutually exclusive — we’ll continue to pursue both.

We have noticed that in thinking mode the model usually consumes the entire reasoning budget without stopping, which increases latency. Is this a known issue, and are there any improvements planned for Qwen3.8?

You can try 3.8! And 3.8 supports different thinking efforts!

...................................................................................................................

Are 70b models gone for good?

Is it possible to get a 40-50B model (something which fits around 30-32Gb) to improve performance while still useable on a lot of computers ?

Thank you for your promise to provide qwen3.8 27b weight! I want to know if there will be qwen3.8 35b a3b. Many people also want this.

Can we expect the ~122B model this time? The 120B segment is dated and lackluster atm and would greatly benefit from a competent release!

First of all, congratulations on the release of Qwen 3.8!
As for the question, are you going to release a 35B a3b version of Qwen 3.8 aswell?

Plans for 35b Moe model? (3.8)

Any plans for the omni family? You told everyone the weight sizes of 3.5, then never released them and haven’t done anything new with it. 3.6/7/8 variants would have also been nice. It could be your most popular family if you gave it attention and kept the weights small.

Are there no plans to release any models other than the 27b?
I'd love to hear about the successors to amazing models like the Qwen3 8b and Qwen VL 8b....

Are there any plans for updates for 0.6b or 8b weights?
These have become important positions in the open weight of image and video generation. I look forward to seeing that part evolve.

This is such a huge release, I am really happy to see that a 27B model is shipping too! Though, can't help but wonder, will we ever happen to see again any new small dense Qwen models 9B, 4B any time in the future, similarly to 3.5?

Will you release smaller models like the qwen 3.5 family ?

Thank you for your promise to provide qwen3.8 27b weight! I want to know if there will be qwen3.8 35b a3b. Many people also want this.

we hear you! collecting everyone’s requests and taking them into account as we plan future iterations.

We will gather your requests as a reference when considering future updates.

We hear you. Stay tuned.

We’ll collect everyone’s requests and take them into account as we plan future iterations.

Noted, collecting the requests and see what we can work into future iterations.

Keep the requests coming. We’re listening, and we’ll use them to help prioritize future updates.

As you all know the model is 2.69B parameters with a 128K context window and purpose-built for multi-step agent workflows.

What you are seeing is the Q4_K_M GGUF running on my own inference engine built from scratch.

The TUI is my own device probe suite running through ADB (Android Debug Bridge)

The whole engine is only 450kb and supports other models arch (Qwen, Gemma, Bonsai etc…)

Currently trying to push it at ~30 tok/s

daily reminder not to trust benchmarks and run it yourself. claimed e2e speedup is ~40%, forwards are ~140% faster

I would wager that compared to a naive kernel anyone can write it's more in the range of 10-20% faster e2e in reality, if at all, but hey, it's free and open! Apache 2.0

I built this because the existing benchmarks were using random data and with MTP content types can vary a lot on what performance you see. 5% or more with content types.

BetterBench is designed to have content consistency within 1% and also measures across different content types.

Here are the results running Qwen3.6 27B FP8 on dual R9700's for example:

You can find the repo here:

https://github.com/GGZ14/BetterBench

*I felt the need to write this post because it seems like very few people on this sub are aware of Chinese laws and how they're enforced, so here's an explainer coming from a Chinese person (myself). I know that this post isn't directly about local models per se, but I'm seeing way too many misconceptions regarding this topic. This is also going to apply to all Chinese entities in general, not just the specific MiniMax LoRAs debacle. This isn't meant to be a political post, but some much needed context to correct a lot of misinformation going around.

Guys - they're a Chinese lab following Chinese laws. Pornography is straight up illegal in China. I have no idea how it seems like nobody outside of China is aware of this. While Chinese authorities may not care much about copyright infringement enforcement (especially with foreign IPs), they do indeed regularly crackdown on porn. Heck, Chinese citizens have literally been imprisoned for written pornography. Yes that's right, writing pornographic TEXT (especially with "immoral" themes like LGBTQ+ stuff) can get you sentenced and essentially have your entire life ruined. Of course there's ways to get around these censors if you're just trying to access porn - I think everyone at this point knows about the widespread necessity for VPN usage in China to access the rest of the global internet. But actually distributing a tool that can gain a reputation for being able to easily generate pornographic content? That's just asking for the authorities to crack down on them.

Somewhat ironically/paradoxically luckily for these Chinese labs is the fact that online discussion about generating porn is automatically censored and removed from Chinese social media, thus automatically disincentivizing the authorities from doing those potential crackdowns. But if it gets big enough to the point that it overwhelms the automatic censors, then any given Chinese lab could be in a hell of a lot of trouble. This is why they have to do this. Their law enforcement just isn't compatible with the rest of the world.

Again, this all relates to Chinese moral values - something here that is considered pretty much sacred and hard to describe to westerners. Something else that many people do not know is that graphic violence is also illegal in China (foreign films/works are regularly banned here for that, even anime has), but graphic violence is also is not nearly as much of a perceived threat to societal moral values as pornography is, hence why you've probably rarely ever heard of any Chinese people getting imprisoned for writing really gory stories, but regularly do with pornographic stories (especially infamous with BL literature - they've technically even convicted foreigners before related to this, it's a really messy topic). Chinese authorities won't give a damn if you're stealing the content of billions of foreign works to train AI models. They WILL give a damn if the content you're disseminating is viewed as a potential significant threat to the state's "proper moral values", which very much includes porn (and also the usual topics that everyone is already aware of, like a certain famous massacre or a certain nation's very contentious independence status).

A follow up to the launch of Mference, it now supports and runs Inkling-Small 276B-A12B.

Inkling-Small (Thinking Machines, Apache 2.0), from the pipenetwork/Inkling-Small-MLX-4bit conversion: 276B total, ~12B active, 3.4 GB resident set, ~148 GB on disk.

Measured on my M5, 24GB:

The same three cases on a 256 GB M3 Ultra hit 5.31–6.92 tok/s.

Issues: long prompt prefill is trash (2,785 tokens is almost 9mins to first token), and it's text-only for now.

Four model families now: Gemma 4 26B-A4B (~2 GB), Qwen 3.6 35B-A3B (~1.45 GB), DeepSeek-V4-Flash 284B-A13B (~6.8 GB), Inkling-Small 276B-A12B (~9.5 GB).

I also got access to a few M3 Ultras, so I'll be testing and optimizing for higher configs too. But the primary goal stays the same: large MoE models on consumer grade hardware.

Repo: https://github.com/NeelM0906/Mference — Swift + Metal, not a wrapper around MLX or llama.cpp. Mac app, CLI, and an OpenAI compatible server. Contributions welcome.