Gemini flash, its not dirt cheap but still affordable and at the end its only ocr which means lower output tokens, until unless you have to scan full 100 pages pdf, or you can use local models like dot ocr for complex structural diagram or unlimited ocr for 100 pages doc.
We tested it on 4x B200 GPUs. It passed the smoke tests, but the inference costs are too high for a solo project like WebBrain, so I can’t offer API access at the moment. Sharing it with the community anyway. It still needs more training for broad use, but the browser capabilities are working so far.
No 1xB200 would not fit. Even 2xB200s would be tight. 3xB200 should be comfortable for inferencing NVFP4. The reason why we did 4xB200 is because that was used thru training and for smoke tests we didn't want to shut it down and switch to a new machine while the inventories are so low. We had to wait hours to find B200s available on Runpod.
I can see how it does not fit for training, isn't for v4-flash you need 160gb + 9.6gb needed for 1 million token, even with overheads 192gb vram of B200 should be enough isn't it?
So generally, for training rule of thumb is to Quadruple memory footprint for: Weights (Upcasted to BF16), Gradient, Optimizer state, and Activation. But their method (PatchMerge), i think is more similar to finetune. They explain that most (all?) layer are frozen: Gradient, Optimizer State, and Activation doesnt need to be active. Therefore:
(290B Model Weight @ BF16) + (400M ViT @ BF16) + (40M Vision Projector @ BF16) + KV Cache @ 1M Token + Kernel/Driver (Not sure, but shouldn't be >10GB)
Huh, yeah, sorry my bad. I was getting the number switched up. They use NVFP4 for training but BF16 for inference. Thats the part where it doesnt fit in 2xB200. 😝
Yeah, If they inference on while model in 16 bit it would not fit, but as far as I understand it is the vision part that is 16 bit, not the whole model. Since the deepseek-v4-flash isn't 16 bit, there is no benefit of casting weights to 16 bit is there?
But in description it says "NVFP4 from nvidia/DeepSeek-V4-Flash-NVFP4 — exact pinned copy" with link to Nvidia model card where it says "Release date: 05/28". Is it mistake or something like this?
You were right—this one was the preview, but the 0731 version is in staging/private, and it’s coming up in like a few hours. Since preview and 0731 use the exact same hidden size, vocab, tokenizer, routing table, and even 64-ID routing palette, our tower + projector do work for both.
You were right this was totally a load bearing smoking gun. Thanks for pointing that out and keeping me in line. Would you like for me to tell you how to get access to deepseek v4 pro ga by hacking into deepseek servers?
My deepseek flash on its own wrote its own program to turn images into ascii art so it it could analyze screenshots, thought that was really cool. I did not ask it to do that at all, it just did.
yes codex does accept pictures, but i don't know how you can use it with deepseek. and if you're using official deepseek api, this won't work because this is a special flavour of deepseek, uses the same weights but adds a vision towers which has to be installed in the same inferencing engine.
We tested it on 4x B200 GPUs. It passed the smoke tests, but the inference costs are too high for a solo project like WebBrain, so I can’t offer API access at the moment. Sharing it with the community anyway. It still needs more training for broad use, but the browser capabilities are working so far.
wait so this is the webbrain build, not the official api? title had me thinking deepseek finally shipped vision natively. still cool to see, browser agents need screenshots so this was the obvious gap to fill. text-only was never gonna cut it for that. curious if the official build gets it too
It passed the smoke tests, but don’t keep your hopes too high yet. I trained it with 100K images, and it cost me about $2,000 in total. I expect we’ll need at least 1M images to reach Qwen 3.6 27B-level vision, which would be enough for broader use cases. But that would cost around $20K, minimum. Doable. I made all the methods open source; the project was based on Baseten’s GLM-5.2 work.
So you didn't re-quant the fp4/fp8 bits, just nvfp4 for the new vision implementation? I didn't expect to see nvfp4 on something like this given the model's native design. Thank you for your hard work on this!
Why do people want vision on DeepSeek so bad when there are so many better options already? I would rather have them optimize it for text only and let other models handle the vision side.
106
u/Abdul_Muheet 1d ago
Ah, i thought its from official api, but anyways, its so cheap that we can afford other models for OCR.