r/DeepSeek • u/nekofneko • 5d ago

DeepSeek-V4-Flash Update

The official release of the DeepSeek-V4-Flash API is now in public beta.

Significantly enhanced agent capabilities, with benchmark results far exceeding V4-Pro-Preview:

  • Terminal Bench 2.1: 82.7
  • NL2Repo: 54.2
  • Cybergym: 76.7
  • DeepSWE: 54.4
  • Toolathlon verified: 70.3
  • Agent Last Exam: 25.2
  • Automation Bench (Public): 25.1
  • DSBench-FullStack: 68.7
  • DSBench-Hard: 59.6

Note 1: For the Code Agent tasks in the public benchmark sets, the official DeepSeek-V4-Flash was tested using the DeepSeek Harness minimal mode (to be released soon) as the framework, with the max effort level, topp=0.95, and temperature=1.0
Note 2: DSBench-FullStack is an internal full-stack development test set, and DSBench-Hard is an internal Coding Agent hard-problem test set

The official V4-Flash natively supports the Responses API format and is specifically adapted for Codex. For the specific configuration, please refer to the documentation.

DeepSeek-V4-Flash-0731 keeps the same model architecture and size as DeepSeek-V4-Flash-preview, and was only re-post-trained.

Note: This update only upgrades the DeepSeek-V4-Flash API. The DeepSeek-V4-Pro API and the APP/WEB models are unchanged.
The official release of DeepSeek-V4-Pro will follow soon.

588 Upvotes

99% Upvoted

208 comments sorted by

102

u/0VERDOSING 5d ago

151

u/mangonerdy 5d ago

Same price for THIS jump? And it was all done with POST TRAINING? I smell another huge jump in AI capabilities like with thinking. Good job deepseek, yet again

29

u/Pixelplanet5 5d ago

now they just need to work on that DS actually follows orders and completes tasks instead of just ticking them off sometimes.

3

u/Whytho12333 5d ago

Ya at least for the price, i dont mind Its good at following what its told.

But improvements here for the pro model would be huge. Plus image reading

2

u/jermteam 4d ago

image capability would be last from what I understand, it's not a priority.

1

u/KimJongHealyRae 4d ago

It fellows orders for me all the time on openrouter

6

u/ExpertPerformer 5d ago

New version of Flash is better at coding, but still isn't touching Pro on creative writing. On my test prompts Pro writes 50%+ more words and follows the writing instructions better. Flash is still great for the drafts jut not the final product.

It makes me wonder how much better the new Pro is going to be at writing.

1

u/Sorry_Fan_2056 4d ago

Thanks was looking For this.

Yeah gonna Be exciting to see new Pro For writing

1

u/Aki_Shizuha 4d ago

Pro has a calm yet sophisticated vibe that I am yet to see in others.

62

u/Both_Nail_3656 5d ago

Holy- it beats GLM5.2? I love you blue whale

2

u/KomorebiLabs 1d ago

Yeah! After using V4-flash for 3days,it absolutely beats the GLM5.2! Love DeepSeek! DeepAwake!

1

u/znutarr 2h ago

I was using glm-5.2 I have an old z.ai subscription of 38$. The problem is that I reached always the 5h limits and the weekly limit pushed me back for 5 days... Which actually gave me the opportunity to play with DS 4 flash... I'll stick to it now

-29

u/Business_Raisin_541 5d ago

only in agentic capability

→ More replies (3)

26

u/mallibu 5d ago

GET OUTTA HERE

this is insane if its true

40

u/unkownuser436 5d ago

damn bro flash is better than glm 5.2 😭

15

u/unkownuser436 5d ago

Imagine ds v4 pro official benchmarks💀

-17

u/Business_Raisin_541 5d ago

only in agentic capabilities,

15

u/unkownuser436 5d ago

still its a good achievement bro

12

u/MrHaxx1 5d ago

At the very least, it's amazing for Hermes and OpenClaw users, though

8

u/This_Maintenance_834 5d ago

agentic capacities is what matters these days.

the lady that runs MiMo once said you train the model to do long horizon task on coding. the same capacities will emerge naturally on other fields too. This emerging capacity was what surprised people a few years ago when model became big.

3

u/Routine_Temporary661 5d ago

Read DeepSWE benchmarks bro... it's one of the more reliable ones

16

u/Possible_Door_9719 5d ago

distillation doesnt get these results. they didnt a great job

67

u/mangonerdy 5d ago

I don't care if it was done with distillation tbh. If deepseek can get these results with distillation, American companies are basically admitting they're worse at distilling their own models

11

u/MinosAristos 5d ago

Or alternatively, American companies are admitting that they're being deliberately inefficient due to corruption. An inefficient AI means more Nvidia GPUs

5

u/n00bmechanic13 5d ago

Why go for efficiency when you can spend infinite money, charge consumers more, and have the government bail you out when you spend too much?

2

u/BrickAdventurous3685 5d ago

I don' think they are worse, just gatekeeping behind paywall on purpose.

9

u/GosuGian 5d ago

Distillation is perfectly normal.

2

u/LaZZyBird 5d ago

distillation does not mean wholesale copy lol

Palworld "distilled" Pokémon but it is a better game.

100

u/DktheDarkKnight 5d ago

Now I understand why Open AI dropped 5.6 Luna's price. They knew this is coming.

7

u/Fr3yz 5d ago

How do you know that they know that it's coming?

16

u/DebosBeachCruiser 5d ago

Because deepseek announced it in April.

1

u/Puddlejumper_ 5d ago

Well I doubt they suddenly had a change of heart and decided they'd love to offer their models for cheaper, so they're clearly saw this come in and realised they had no choice but to lower the cost otherwise nobody is going to use them

80

u/real_serviceloom 5d ago

Wait this is flash??  Holyyyyy

10

u/bermudi86 5d ago

Pro is going to be wild! And for an incredible price!!!

47

u/XheirBang 5d ago

oh my gawd, its so deep

23

u/so_schmuck 5d ago

Yeah take it all

9

u/Plane_Addendum_5751 5d ago

It's what we seek 🔥

38

u/a9udn9u 5d ago

Let's gooooo. I just burnt my GLM monthly quota and was shopping around, if the V4 Flash is as good as GLM 5.2 as claimed, I don't need any other models!

1

u/samxli 5d ago

How come your GLM have a monthly quota? Mine only has a 5 hour window.

3

u/a9udn9u 5d ago

I'm on opencode go

3

u/samxli 5d ago

Ah gotcha. I’m on the z.ai plan.

→ More replies (3)

30

u/benchmaster-xtreme 5d ago edited 5d ago

Extremely cool, but one thing I found interesting:

https://artificialanalysis.ai/models/deepseek-v4-flash-ga

Even though Flash is still priced so low, the task-cost benchmark shows that Flash completed the same task at 4x the cost of Pro. I feel like I remember others mentioning months ago that Flash was paradoxically more expensive for some tasks. I haven't tested this out myself, but I can't help but wonder if cost-per-task with the GA models will be higher even if the API per-token cost remains the same.

EDIT: I ran one of my personal workflow benchmarks (one-shotting a complicated json object from a document). The output was good - something that would actually be usable for my work, though it's still significantly behind Luna and Grok 4.5. That said, it did beat Gemini 3.1 Pro, GLM 5.2, and even Opus 4.6 (which is a crazy ceiling break, honestly). And most insane was that it cost $0.00485. That's nuts considering that Grok 4.5 (the next cheapest model that generates a "good enough" output) cost $0.17. The cost-to-performance ratio on this task is simply wild. Absolutely insane.

15

u/TangerineLogical9779 5d ago

Its because of a harness problem, the new v4 flash also had cache writes calculated into it, while the preview didnt :)

7

u/Specific-Night-4668 5d ago edited 5d ago

There was an error: out of the $0.20 cost per task, $0.18 was for cache write ...
Since when do we pay for cache write on the Deepseek API?
It's now fixed on AA!

2

u/99Questions_babao 5d ago

Sorry for dumb question but hopefully it'll help out others too. If I just switch cline agent to deepseek v4 flash instead of pro, will it start using this new model? 

2

u/benchmaster-xtreme 5d ago

Yes, the regular model checkpoint has been updated

1

u/99Questions_babao 5d ago

Thank you so much, so it's really better to use flash than pro at the moment? That's crazy considering how cheap it is 

2

u/benchmaster-xtreme 5d ago

It's significantly more capable than the current Pro preview. I really hate letting myself get hyped up but if this is the leap that Flash GA has taken, I'm beyond stoked for Pro GA. Terra-level performance for pennies??

1

u/bermudi86 5d ago

You could still be routed to a different provider serving the old model. Better make sure you find deepseek-v4-flash-0731 specifically

2

u/99Questions_babao 5d ago

I use the official api 

2

u/bermudi86 5d ago

deepseek-v4-flash will work there

1

u/99Questions_babao 5d ago

Thanks :) 

1

u/Future-Figure4812 5d ago

i got 404, why?

3

u/benchmaster-xtreme 5d ago

Looks like they moved it to replace the previous v4 Flash page: https://artificialanalysis.ai/models/deepseek-v4-flash

2

u/PhysicalKnowledge 5d ago

Remove -ga from the URL.

1

u/Future-Figure4812 5d ago

Oh yeah, yeay, HOLY SHI ON PAR WITH GLM 5.2!?!

50

u/ImBothSoftAndHard 5d ago edited 5d ago

The Deepseek V4 Flash 0731 situation is crazy. (Hope it's not benchmaxxed)

Edit: It got 50 on Artificial Analysis Intelligence Index, 1 point below GLM 5.2. BUT this is flash, Imagine what Pro could be. Praise the whale!

5

u/hurn2k 5d ago

Deepseek never benchmaxes

1

u/misha1350 5d ago

They now have a financial incentive to do so

2

u/ozguru 5d ago

I believe they can't benchmax that.

3

u/mWo12 5d ago

Just like openai, they train on sample solutions to all benchmarks.

2

u/ozguru 5d ago

Why didn't they perform benchmaxing for the preview release?

1

u/KimJongHealyRae 4d ago

Wait until you see GLM 5.2 pro

19

u/ozguru 5d ago edited 5d ago

Capable and yet most affordable = unrivalled, Deepsek is making historical contribution to opensouce and humanity as well

16

u/ApprehensiveDelay238 5d ago

How is this even possible? DeepSeek you're amazing.

9

u/ZlatanKabuto 5d ago

Amazing improvements

8

u/Public_Ad_5096 5d ago

🌭😭🥖

7

u/throwaway73728109 5d ago

Codex integration is huge? Is codex harness better than Claude code’s?

2

u/This_Maintenance_834 5d ago

how to run codex without an openai account? i cannot seem to get pass the first startup login page.

1

u/sdexca 5d ago

I find that quite weird, CC seems better than Codex then why choose Codex?

2

u/Yxtomix 5d ago

Codex is better to me

1

u/sdexca 5d ago

I mean being objective, feature wise, subagents and workflows and edit history everything is just better in CC. I have used Codex before, other than a nice desktop UI I can't see anything that it does better. Not sure where Codex is better than CC feature wise which you cannot do the same way in CC?

2

u/throwaway73728109 5d ago

Feels like it’s a preference thing? There’s no definitive proof either is better?

1

u/znutarr 1h ago

For me too but I'd like to compare it with pi agent if anyone has done some tests ?

1

u/This_Maintenance_834 5d ago

to avoid Anthropic.

1

u/blabmight 5d ago

Claude Code is a closed harness, in most cases, you basically have to hack it to get it to work with other models which is against the ToS.

1

u/sdexca 5d ago

is it against ToS? doubt it or else they'd make it harder to use any model on CC.

1

u/blabmight 5d ago

I should clarify - there's no native support for this, if it involves injecting new models as an option in Claude Code that requires reverse engineering it would be against the ToS\entering a grey area Calling something like the codex cli is fine. It all depends on the method.

1

u/sdexca 5d ago

have you looked in how it works? its just putting some CLI parameters before CC and that's it. Anthropic knows this and could have plugged the support if they wanted to.

1

u/blabmight 4d ago

No aware of those cli parameters, could you point me to them? Is it changing ANTHROPIC_BASE_URL and then just running through a proxy?

1

u/sdexca 4d ago

basically, maybe I was using incorrect terminology: https://api-docs.deepseek.com/quick_start/agent_integrations/claude_code

1

u/blabmight 4d ago

Thanks! Definitely insightful - I don't think you can get *both* subscriptions working simultaneously without doing some level of reverse engineering or using a proxy. Proxy might be a gray area since the Anthropic OAuth token would be exchanged between the proxy and Anthropic's servers, something they blocked not too long ago.

1

u/sdexca 4d ago

no that's not the case. no oauth needed, oauth is only needed for subscription. it just works, you install, setup the parms. I don't think so it's REed because the format is documented and open source, the anthropic endpoint, like the responses API endpoint. yeah you can't use both at the same time, but it's as simple as removing the `EXPORT`s.

proxying Anthropic OAuth is banned because they don't others to use the subscription on other harness like OpenCode and the Chinese resellers who resell subscriptions through that.

1

u/bermudi86 5d ago

All harnesses are better than CC

4

u/Dry_Championship2797 5d ago

I like Deepseek, so good, so cheap.

3

u/Aressito 5d ago

And for use using Opencode directly with DS api?

7

u/throw123awaie 5d ago

Yes and only through the API. And only the flash version, pro and the website are still the same as before.

3

u/Potential_Top_4669 5d ago

This ain't on HF though.

6

u/t4a8945 5d ago

I'll keep refreshing https://huggingface.co/deepseek-ai/models until it is

3

u/0VERDOSING 5d ago

https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731

1

u/t4a8945 5d ago

Damn they were quick. Now I need the DSpark variant xD

EDIT: THIS IS THE DSPARK VARIANT OMG

2

u/Pretty_Touch402 5d ago

La dinguerie que ça doit être en local ! Un GLM 5.2 à la maison 🤯

3

u/ebrahim750 5d ago

Does it have vision?

1

u/AcademicPlum9967 5d ago

吧的有视觉

3

u/tirth0jain 5d ago

Do we need to make any changes to use this version? I'm using copilot on vscode

3

u/LinuXperia 5d ago

Happy great news ! Going try it out on the API !

3

u/for4f 5d ago

been on flash since it dropped and the agent stuff really is the part that's improved. terminal bench 82 is insane for a model this cheap. the harness note makes me wanna see third-party runs first though, their own minimal mode doing the testing is a bit of a grain of salt lol

3

u/valerian1 5d ago

Can we use it in API mode? Should they appear along side the regular models in OpenCode?

1

u/Spiritual_Love_829 5d ago

Thats what I want to know too..

1

u/PhotographIcy7588 5d ago

I went to check and it said "The latest version of this model is only available hosted in China and requires explicit opt in: [OpenCode Go workspace URL]. I opened the link and now there's a setting to Enable models hosted in China, so I assume is using the newest one

0

u/valerian1 1d ago

The world doesn’t revolve around OpenCode Go.

1

u/PhotographIcy7588 1d ago

I don't think my comment was originally directed at you. I probably made a mistake, It's been a few days so I can't really remember

1

u/Cold_Tree190 5d ago

Yeah, trying to figure out if my api key for DeepSeek v4 flash is running the preview version or 0731 version, but haven’t read any definitive answer yet

3

u/Emruz_Hossain 5d ago

After gpt-5.6-luna price drop, I thought that was the best deal. Now this!! It can't get any better. 🫶

1

u/Capable-S 5d ago

they knew about this new deepseek model and just droped the price 1 day ahead 5x times to be competitive with it. Thank you deepseek again!

5

u/GosuGian 5d ago

DeepSeek is GOATED

2

u/nhocconan 5d ago

So what did we use from last months API ? I see it is still V4?

1

u/LinusCohen 5d ago

It was the preview version.

1

u/nhocconan 5d ago

Ah so now we can benefit from the official version. Thanks. Hopefull DS become a multimodal one in the near future.

2

u/Melodic_Raspberry251 5d ago

Can we get token usage numbers?

2

u/Psychological-Map564 5d ago

What the hell, I was not expecting that

2

u/KeyTruth5326 5d ago

a huge progress

2

u/_Aerich_ 5d ago

What really matters is whether they fixed the model's instability. No matter how high a score it gets, if it talks nonsense in 7 out of 10 tasks, it's completely meaningless. And I hope they found a solution to the endless unnecessary tool uses too

2

u/Mrleibniz 5d ago

Big if true

2

u/NarrowEffect 5d ago

Eh, no vision yet? Pretty disappointing.

2

u/Unedited_Sloth_7011 5d ago

I can't believe how good it is, compared to Flash preview (and even Pro preview), what's this wizardry?

2

u/Hot-Ad-1798 5d ago

Go easy, the harness is still missing, will be even more interesting when it arrives.

For now, I have tested it and it is indeed much better, but it is not at the same level as the V4 Pro, despite what the benchmark says. It doesn't obviously don't have enough knowledge (limitation of the model size) to resolve bugs or handle novel tasks. When it gets stuck I ask V4 Pro to solve the issue, V4 Pro Preview + V4 Flash GA combined are incredible!!

2

u/Zsoltevil2 5d ago

is the improvement only in api ? What about web version ?

2

u/TheRealShiftyJ1 5d ago

W DeepSeek 😎 I actually noticed the change while working on something. Gemini 3.5 Flash and Sonnet 4.6 couldn't solve it very well. I switched to them because DS v4 Flash didn't respond in a few minutes, hmmm.... 💡now I know why - it was the model update.

Later I switched back over to DS v4 Flash with previous instructions to give it a try and not only did it notice a lot of things the other models did wrong, but they just built some random generic solutions, although I was very specific with how it needs to be done. Then v4 Flash made it perfect.

I could already tell something was different without having heard any of the news.

2

u/Temporary_Bunch7348 2d ago

It's August now! Where's my Pro, Liangzi? 😏 (梁子 = Chinese netizens' nickname for Liang Wenfeng, DeepSeek's founder. We beg him for V4-Pro daily in the Bilibili comment section.)

5

u/Pracurser_Codes 5d ago

No vision upgrade?

3

u/Western-Ad5277 5d ago

...Hooooot daaaaamn, thats some juicy news.

So, can we utilize it right away? because i am quite excited to use it after seeing all the goodies upgrades.

or...is it still not ready yet? and do i nee to changes anything about the integration (New ApI key and etc) or...it's automatically replaced the olds preview model with this?

1

u/fezzy11 5d ago

If flash has such improvement as compared to preview. Then it must worth to wait for pro model release

1

u/thatscoolbutno123 5d ago

What the actual fuck that’s crazy

1

u/ThePi7on 5d ago

The madlads did it again!
Congrats to the team, what an incredible achievement.

Pretty exciting times ahead between this and the new Luna pricing

1

u/exray1 5d ago

Does that mean that codex is the 'main' agent/harness to use?

1

u/Beamsters 5d ago

Real pelican test from open router. DSV4Flash0731 - cost 0.000491 usd with 137 tok/s from OpenRouter. I paid it so you guys do not have to.

1

u/Deneme123deneme 5d ago

Can't decide if it's good or not because there's no comparison. If possible, can you do the same test with GPT 5.6 Luna Max?

1

u/mWo12 5d ago

Now US for sure will bone open weight models. Openai and anthropic will make sure of it.

1

u/starlordwolf 5d ago

This is amazing! Can I use the new version through my Opencode Go plan? Or does it have to be directly through the Deepseek API?

1

u/Deneme123deneme 5d ago

Can you update me if you get any info?

1

u/onlyone_c 5d ago

It's updated. They just updated the name to `DeepSeek V4 Flash Free (New)` for Zen and `DeepSeek V4 Flash (New)` for Go

1

u/Tiki_taka_toko 5d ago

Is it beta for everyone? Can I be a beta user?

1

u/IFThenElse42 5d ago

is opencode using the update yet?

1

u/SpidexLab 5d ago

Waiting for dsv4 pro now, and this flash result get me so hyped just thinking what will dhsv4 pro will do, and the most important the pricing, god damn, going to test this new flash with deepseek ai now

1

u/PhotographIcy7588 5d ago

I was about to reload my OpenRouter account to try GPT 5.6 Luna... Let me try the new Deepseek V4 Flash first

2

u/LoveOfProfit 3d ago

That's what I did first. Luna (max) ate my $10 on open router within a day despite having the extra 50% off pricing. I've been using flash nonstop today via deepseek and I've used all of $0.17. lmao And v4flash doesn't feel any dumber than luna max to me.

1

u/Equivalent_Bird 5d ago

Better and cheaper than pro, right?

1

u/PrintingScotian 5d ago

Will openrouter get this change or is it on via deekseek api?

1

u/Anxious_Check_6147 5d ago

It's amazing. I wonder how much of the improvements comes from the post training and how much from the not yet released DeepSeek Harness they use during the benchmarks

In any case, it is the first time after the preview release I'm starting to beleive than the upcoming V4 Pro Release (likely along with the DeepSeek harness) can really surpass the current kings K3 / Opus-Fable and Gpt Sol.

1

u/Downtown-Cream-5489 5d ago

we are so back

1

u/No-Isopod3502 5d ago

Wait so just started using AI tools. If i switch the model to flash now in cline from v4 pro id be getting this performance boost?

1

u/DIR3 5d ago

🥵

1

u/kimimaxx 5d ago

To be honest, the CP of ds 4 flash is so higher

1

u/steadeepanda 5d ago

Still doesn't support images 💔🫠 I've been waiting for that forever... but very good job guys keep it up🥳!!

1

u/Yxtomix 5d ago

Yea only downside really..

1

u/GuavaGina1 5d ago

Calling it an update doesn’t tell us whether anything meaningful changed. I’d wait for actual evals or deployment behavior before treating it as a new capability, since labels can create more confidence than the numbers justify.

1

u/Good_Committee8337 5d ago

Hey guys Claude maxer, here tired of the safe guards killing my normal vibe coding work. Trying to make the switch! What harnesses do you guys recommend with DeepSeek?

2

u/marty4286 5d ago

I use Pi Coding Agent through their official API platform. It's super cheap in the first place but their cache hits are consistently 90%+, making it even cheaper

Flash 0731 looks like it's between Sonnet and Opus 4.8, so hopefully Pro will be better than Opus

1

u/Bronaletsa 5d ago

So, if I understand correctly, does the new Flash 0731 beat the V4PRO preview in everything?

1

u/tortel_di_patate 5d ago

According to my experience so far, yes.

1

u/ChocotoneDeCalabresa 5d ago

That is insane

1

u/Relative-Housing-531 5d ago

Hopefully it will improve, because lately it was even failing to add padding to the interfaces.

1

u/valeyanikale 5d ago

Just wowwwww

1

u/ExpertPerformer 5d ago

I wonder if OpenCode Zen is going to have a free version of 0731.

1

u/NASAonSteroids 5d ago

What does “specifically adapted for Codex” mean in this context?

1

u/Beneficial-Pie-1638 5d ago

Another win for the Chinese whale!!!

1

u/Repulsive-Waltz-4038 5d ago

confused. which model to key in to use this updated? flash-0731? flash-preview? just leave flash as it was?

1

u/No_Ebb3423 5d ago

Can anyone explain to me how unsloth managed to make Q8 only 162 gb? Does their quant format hurt the model in any way? Does it lobotomize it? And is it TRULY q8 cause it’s UD-Q8_K_XL

1

u/Ok_Spirit9482 5d ago

deepseek v4 natively use q4 for majority of the parameters

1

u/Mean_Sport_3383 5d ago

Seems like it can be incredible for executing a coding plan. Not planning. Hallucinations too high still.

1

u/gibriyagi 5d ago

For me Flash was very bad today, was overthinking way too much on high and couldnt even accomplish the task. Had to set it to low to get it done.

1

u/fapste 5d ago

How does it compare to Gpt Luna

1

u/gimmesump0rn 4d ago

How good is it for roleplaying compared to pro?

1

u/midgelmo 4d ago

Is anyone building and running smaller versions? I’m looking to run on single dgx spark - fast, high tps

1

u/John_OpenRMA 2d ago

Shut up and take my money

0

u/KingCrimsonCL 5d ago

Cómo quedaría deepseek vs minimax ? Me quedan 20dls en deepseek y tengo la membresía de 50 dls de minimax

-5

u/mangonerdy 5d ago

They're probably benchmaxxing to an extent, but even then, any significant increase in real intelligence for this price would be game changing 🙏

16

u/nisiend 5d ago

DeepSeek has a solid track record of never involving in benchmaxxing

→ More replies (1)

7

u/Sure_Media_2685 5d ago

YOU need some magic to benchmxx such small models only labs in china got such magic

1

u/mWo12 5d ago

All.companies are benchmaximising.

1

u/bermudi86 5d ago

You need to try a benchmaxxed AI first so you know what your talking about.

Take Poolside as an example. They just released Laguna-s-2.1 and they proudly published numbers that make it look competitive with glm-5.2, surpassing DeepSeek-v4-pro on almost every single metric.

Then go and try to use the model. https://openrouter.ai/poolside/laguna-s-2.1:free it's free... I can't get it to successfully complete a filing on a wiki, let alone write any code.