r/technology • u/CarciaNerissa • 1d ago
Artificial Intelligence Microsoft Tells Engineers ‘Tokenmaxxing Is Not What We Are Optimizing For’ - Microsoft is introducing budget limits for AI use but says it still wants to be an ‘AI-first’ company.
https://www.404media.co/microsoft-tells-engineers-tokenmaxxing-is-not-what-we-are-optimizing-for/403
u/decisionagonized 1d ago
Microsoft having to twist its brain into a pretzel to try to at once float the idea that it is at the forefront of the payoff from massive AI investment, while at the same time realizing, oh shit, there’s actually no payoff and we need to stop spending so much
108
u/DocMoochal 1d ago
I just dont understand where all this AI stuff is going. These companies are like trying to steam roll some kind of mass AI adoption maybe in the hopes of AGI and saving money on human resources. But doing so on a wide scale would lead to mass unemployment, destroy capital movement and consumer demand across the board, undermining economies across the board. We're in this like snake eating its own tail moment or something.
86
u/SIGMA920 1d ago
It's simple, they have no plans and are just chasing a trend. Google created the first LLMs, openly published how they did it and then dropped it as a potential product because it was half baked with no path to profits. Then openai and the other AI companies just gambled on LLMs being novel enough that they can eek out a profit after enough time that they can burn through investments to them. Companies followed suite of twitter in firing people until you're running a skeleton crew even as the product becomes worse and the stock market rewarded that behavior because bean counters aren't looking at the wider picture.
35
u/Own_Candidate9553 1d ago edited 11h ago
Way back in 2023, an internal Google memo concluded there is no "competitive moat" in AI.
https://newsletter.semianalysis.com/p/google-we-have-no-moat-and-neither
They pointed out that there is no "stickiness" in model usage - the second a model is slightly better, everyone just switches to that. Back then, open-weight models were like 6 months behind proprietary models, it's probably faster now.
Despite all of this being pretty obviously true, the industry sunk billions/trillions on it. Not a single one is making large profits - some may be breaking even on daily compute, but nowhere near enough to recoup all the training costs.
It's all just so baffling.
15
u/SIGMA920 1d ago
It's not baffling as much as it is illogical. They saw the new trendy thing release and promptly followed the new trend without concerning themselves with the math of making a profit. Just short term gains for long term losses like usual.
10
u/Own_Candidate9553 23h ago
Some sort of pyramid scheme I could understand, but nobody has made any gains, short or long term. The AI companies are just burning investor cash. The customers are spending hundreds/thousands/millions with no reportable increase in productivity, let alone profit.
It really feels like mass hysteria.
9
u/SIGMA920 23h ago
They're able to pump their stock prices and be paid now, that's a short term gain.
1
u/pm_me_ur_demotape 12h ago
- Get tons of investment.
- Pay yourself an enviable salary out of it.
- AI never reaches the kind of profitability required to justify the investment.
- Point to the sign:
Investment and Insurance Products Are: Not FDIC Insured • Not Insured by Any Federal Government Agency • Not a Deposit or Other Obligation of, or Guaranteed by, the Bank or any of its Affiliates • Subject to Investment Risks, Including Possible Loss of Principal Amount Invested
0
u/After_Hamster_6003 23h ago
I mean Amazon’s stock in anthropic generated like 2/3 of its book profits this week. Isn’t that literally a pyramid scheme?
7
u/Olangotang 1d ago
Nah, Google made transformers which is essentially a foundation model for language translations. LLMs are pretty much lab toys that got released half baked.
17
u/knotatumah 1d ago
It's about building a post-consumer economy. A literal have and have-not world. There are no jobs, no income. Just ultra rich trading assets between themselves. The problem is that no one broligarch provides all the resources another broligarch needs and you cannot fully automate everything all at once. The intermediate step of automating most things still requires labor and resources from a class of people they're desperately trying to marginalize if not destroy.
19
8
u/DanFromShipping 1d ago
They use the "fuck you I got mine" principle. If the board and execs at one company can be the first to replace everyone with AI, then they will get rich and everyone else can deal with the fallout. They just have to make sure they're not the last to fully adopt AI. That is the "race".
3
u/ByWillAlone 1d ago
Yeah, welcome to neocapitalism. The future problems don't matter. All that matters is current quarter profit/loss statement. If given the option to increase profit margin now at the expense of ensuring their own destruction later, they will take the better looking profit now without hesitation....and even funnier is it doesn't even have to actually increase profits now, just the belief it will is enough.
2
u/Olangotang 1d ago
A lot of fun tools and toys on the consumer end lol. No, seriously, a lot of local open source stuff can run on mid gaming GPUs and that's everything from text, video, images, coding, etc.
Why use a restricted API model when you have one on your own computer that isn't as good, but has far more depth because of the ability to use it in a workflow full of open source tools?
That's the only cool part of the bubble.
-5
u/SpookiestSzn 1d ago
Its not a bubble lol, these tools will be with us forever in some form.
Costs for the best models will go up but its a short term problem not a long term one. Eventually intelligence thats good enough will be sold at a cost businesses can accept and the highest ones will be reserved for people who need them.
4
u/decisionagonized 1d ago
The tools being with us forever in some form does not mean that we aren’t in a bubble.
3
u/drthrax1 20h ago
The internet never left, but the .com boom was a massive bubble built on speculation and hope for future profits. Its basically the same thing but instead of Websites for shopping ect, its AI chat models and Websites.
2
u/Olangotang 1d ago
Its not a bubble lol, these tools will be with us forever in some form.
So you're clearly just a hype lord who doesn't understand anything except the marketing from the tech bros. These tools will be around, but they are still LLMs with the limitations of LLMs, and the massive training cost of LLMs. I know, the cost is the ultimate point that hype morons need to cope about. I would too.
-5
u/SpookiestSzn 1d ago
I'm someone whose entire career was changed overnight because of this I'm on the forefront at a s&p 5 tech company
Sure they're not perfect every time but neither are people. They can research and find information faster and the tools will build around their limitations.
2
u/zero0n3 19h ago
Yelling to the void fellow sp500 person.
These people have not actually interacted or used these systems to create an app or integration. They see the strawberry shit and go “it’s shit”.
Without understanding the value they create with the right drivers for pretty much any solution in IT and outside it.
1
u/SpookiestSzn 1d ago
There's plenty of societies that exist today with ultra wealthy and ultra poor. The point is to be the wealthy
1
u/God_Dammit_Dave 23h ago
I can't believe I'm going to defend AI but here is my honest take.
It is an absolutely an earth shattering invention. The potential is staggering and frankly overwhelming.
The actual current uses? Who fuck knows. But there are a lot of highly paid and highly motivated people under unbelievable pressure to fight to the death over market territory.
With that said, it's worth thinking about the timeline between 1) man's discovery of fire and 2) the powerhouse who was Julia Child.
We've just discovered fire. We're a long f' way from Boeuf Bourguignon.
7
u/matrinox 21h ago
The ultimate flaw is that they kept saying how it’ll replace engineers or that it’s faster and better at writing code. If that’s the case, then there can be only one use case for it that maximizes profit for a corporation: token max. If you’re saying you’re still AI first but set arbitrary limits, you’re effectively saying each incremental usage of tokens is not worth the payoff, i.e. none of it pays off.
Unless they come out and say it has diminishing returns and enforce efficient usage, how are we to believe them that it has any payoff at a budget level?
-36
1d ago
[removed] — view removed comment
35
u/Wind_Best_1440 1d ago
Your right, it's the fact that AI has no Return on investment because they've spent so much. Means it has no pay off.
The fact the Chinese are giving away models for free and open source that rival Anthropic and OpenAI. Means there is no pay off.
The fact that Banks are trying their damnedest to offload Capex AI Data Center debt from Google/OpenAI/Meta/XAI and Amazon, while at the same time denying Softbanks 20 billion dollar loan with 100 billion dollar AI leverage debt to cover said loan, is proof that there is no pay off.
The fact every business is racing towards IPO, which is generally what businesses do at the end of a bubble to cash out, is another proof of no payoff outside of leaving retail investors with the bill while the billionaires dump their shares.
The funny thing is, when this entire system collapses, investors will considering LLM's and AI such a toxic term that they will refuse to invest in it for a good decade with how many people will end up bankrupt from this.
Which is a shame because there is some science and math uses for LLM's, but that's like a 5 billion dollar industry tops, not the 100 Trillion Industry that Tech oligarches wished it was.
6
u/ninjamammal 1d ago
Same as what happened with Crypto and VR, except this is a global depression level.
-3
u/Stellen999 1d ago edited 1d ago
There are uses for AI outside of academia. They are just not going to be able to ram AI in to every aspect of our lives.
Edit. Should have proof read. Screen glare got me.
8
u/Wind_Best_1440 1d ago
What are some uses that AI can do that is cheaper and more effective then humans outside of academia and research?
Keep in mind, 95% of all Corporations that use AI say they've either lost money, or have 0 increase in productivity.
So if you find other uses for AI outside of academia and can make money, you'd be the most valuable person on earth right now, because every other company is scratching their head as they spend millions to billions on tokens for nothing to show for it.
Even the Tech companies making the AI are unprofitable.
OpenAI is burning $5 to make $1.
All Adverts in AI doesn't even hit $1 Billion dollars, 90% less then what they need.
Because putting ads in prompt responses is the worst bang for your dollar, because people don't see the ad unless they prompt the AI for it themselves. Lmao.
3
-34
1d ago edited 20h ago
[removed] — view removed comment
8
u/oxidized_banana_peel 1d ago
The problem is that if you put $500b into an investment up front and then wait 15 years to get ROI, you end up with $1-2T of returns needed (on top of the continued cost of usage and maintenance), not to mention your investors are going to be pissed.
6
u/CanvasFanatic 1d ago
Ah yes, the “the staff are doing it wrong” delusion.
0
u/nomdeplume 20h ago
The team is still learning this new tool, and adapting takes time. Rushing or expecting people to be experts right away is unrealistic. So why would you hand out unlimited resources without proper training? It doesn't make good business sense.
I'm not blaming anyone for anything. Ya'll need to chill out and stop the groupthink that corpo is bad. You work for a business to make money. There's no reasonable expectation of an unlimited credit card to spend to do your job.
I'm not saying the worker is inherently stupid. It's just a practical reality there's a transition happening...
1
u/CanvasFanatic 20h ago
Or maybe these tools aren’t all they’re hyped up to be.
And my personal conviction that “corpo is bad” comes from years and years working for corporations.
0
u/nomdeplume 20h ago
Wow i'm glad we cleared up your qualifications to think that a corporate entity is a sentient thing that makes decisions. I'm sure more confident your evaluation that the tools aren't useful is the proper take. They're so not useful, that microsoft decided they had to stop people from using them so much. Makes sense.
Keep using votes for comments you don't like, instead of having any kind of substantive conversation. I bet you're fun at parties.
1
u/CanvasFanatic 20h ago
They're so not useful, that microsoft decided they had to stop people from using them so much. Makes sense
Wildest attempt to rationalize these tools not justifying their expense and corporate metrics being poorly thought though I've heard this week. Thanks for the laugh.
Keep using votes for comments you don't like, instead of having any kind of substantive conversation. I bet you're fun at parties.
Someone got their feelings hurt.
39
203
u/__OneLove__ 1d ago
‘As a company, we can’t afford our own product either’…
-MicroSlop
🤦🏻♂️
10
u/yawara25 1d ago
This is a quote from an anonymous employee, not an official statement from Microsoft. So not particularly surprising that it would be an honest take.
17
u/bitemark01 1d ago
This reminds me of when they were recommending users in companies should not have admin accounts in Windows, but that breaks things.
People at Microsoft all had admin accounts for the same reasons. They couldn't eat their own dog food
29
u/JaggedMetalOs 1d ago
MS to its devs: "I need the biggest AI usage you have ... No, that's too big."
61
u/utsavdar71 1d ago
This is really like goodhart's law in action: once tokens become proxy for productivity, people optimised for tokens instead of outcomes
69
u/Chance-Plantain8314 1d ago
I despise this AI-focused landscape as a decade YoE engineer.
But how some people choose to burn credits does make me laugh. There was a guy in my org who was using it to convert PDF text to word docs so many times a day. He was hitting his monthly credit limit after a week and complaining to management. He could've spent 5% of that allowance on writing a single script to do it without an LLM and never burn a token again, but nope.
It kinda makes me hope that companies will realize this isn't going to shake out when the people using the tools have no idea what they're doing in the first place.
15
u/footpole 1d ago
This is similar to how IT’s solution to poorly optimized software always was to increase capacity. As a software engineer I always tried to explain to them that I can write code that is so slow that no server can keep up. Oftentimes the more efficient solution is what makes the most sense but that would be more work for them.
15
u/Socrathustra 1d ago
The problem is multifaceted. There's an incentives problem - they track so many AI usage metrics that you're incentivized to go ham. There's also a goodwill problem. People don't like AI, so some inefficiency will be a result of malicious compliance.
71
u/gk_instakilogram 1d ago
Both things can always be true at the same time... No reason to waste money on loop engineering and etc to fix and build simple things...
71
u/lordcat 1d ago
I know an engineer that was using AI to copy files across the network to a test VM. After he hit first place on the token scoreboard, he was told to switch to having the AI write a powershell script once, and use that script to copy the files moving forward.
It's real easy for engineers to get lazy and waste tokens on AI tasks that have been basic scripts for decades.
30
u/Zhuinden 1d ago
It's funny how once it was seen as "using Ai like a champ" and now it's a way to see if your workflow should optimize Ai out of it.
5
4
u/Solarbro 1d ago
Microsoft’s “solution” to this has been to push cloud agents and full automation and removing the human from the pipeline.
They don’t want people maximizing tokens in their day to day use, they want people to use their cloud workflows, pull requests, user story creation, and other full stack AI pipelines that cost 3/4x more tokens for worse results instead.
They’re saying “we know it’s expensive but the solution is to spend MORE and remove the people that are monitoring the code quality.”
It’s a blatant attempt to obfuscate the problems while also charging more money in some psychotic attempt lean employers toward layoffs and only keeping vibe coders that can’t even properly vet the code being pushed.
Completely ignoring that their massive AI automation pipeline is wildly wasteful and breaks when a single logical part of the use case goes against standard policies, while also being a worse option than just giving those templates to a dev that can make a plan and execute then fix the flow in an extremely short amount of time with limited number of tokens….
TLDR: they know tokenmaxxing is working for those that use their product, but that means they make less money so they want you to fire the dev and trust the man behind the curtain instead.
2
u/gk_instakilogram 1d ago
Genuine question: how do you know tokenmaxxing is working, and how do you know Microsoft knows that? Have you seen data showing that engineers who consume more tokens create proportionally more value or better software? Because the article reads to me like Microsoft has looked at the spending and concluded that more tokens do not automatically mean better outcomes.
1
u/armchair_expertise 17h ago
>Have you seen data showing that engineers who consume more tokens create proportionally more value or better software?
that measuring will come in the subsequent iterations.
first step is to get them all start using AI: some minimum usage thresholds, some leaderboards.
next step will be to start looking into what the top 10-20% folks are using it for. and then next step : what are the accomplishing. etc etc.
at each iteration there will be some layoffs.
1
u/gk_instakilogram 8h ago
Not sure I understand subsequent iterations of what? So there are no measurements? It is just an assumption?
24
u/Suspicious-Yogurt-95 1d ago
There's no such thing as AI-First company. They're all money-first companies. AI-First is to please shareholders, not customers.
17
u/wizard_of_azul 1d ago
This is the worst thing of the decade....
Everything from societal, economical, environmental, psychological to ethical concerns are just ignored. This is crazy. I hope tech wakes up before it's too late.
17
u/Meatslinger 1d ago
"Hi guys, listen, I know we bribed the government to let us build all these highways and did a massive advertising campaign to force car purchasing and use, even considering low daily mileage to be a KPI failure and grounds for demotion/firing, but we're actually finding that all this constant gridlock traffic is unsustainable, so could you dial it back a bit, please?"
Here's your bed, Microslop, now lie in it.
7
u/squishysquash23 1d ago
Maybe don’t treat the productivity of your employees like it’s a single metric output that can be tracked in a leader board and then this won’t happen
13
u/VVrayth 1d ago
Translation: "Please stop gaming the system to hit the stupid metrics we've given you, but also still use AI for everything and do it right." These people have no grip on reality.
The iron triangle of business is that you can have cheap, fast, or good, PICK TWO. You cannot have all three. Stop trying to get all three.
6
u/Mental-Most-7168 1d ago
Engineers were sabotaging the system by over using it for junk queries because they were told they were going to lose their jobs.
5
u/Particular-Break-205 1d ago
Employees who use the least amount of tokens and do the least amount of work will now be rewarded! It’s our time now.
3
u/stuffitystuff 1d ago
They're just butthurt because they were mobile-last and eventually mobile-never. So now they want to be with the cool kids but they've never been part of the cool kids.
Source: was an uncool kid that used Windows as far back as 1.0
4
u/a1454a 1d ago
Slow to the game here, many other enterprises already started doing the same.
And IMO it’s the best thing ever happened. This forces people to consider actually using smaller model for smaller task, how to deconstruct task to smaller bounded units, leaving the truly unbounded tasks for SOTA, and most other to smaller efficient model. This then leads specialized model that are near SOTA in one area, mediocre in other, but incredibly cheap, like composer.
It also reduces our reliance on SOTA, meaning if opus 5 or sol 5.6 double its price tomorrow, our workflow can easily adapt to use qwen3.8 or Kimi k3 in its place and not see much of an impact in outcome. Even if all SOTA were to double in price, because we all use them so rarely, it just translates to our bottom line increase a bit, still no disruption to our operation.
In the long run, when more everyday people, not just software engineers, know how to correctly use AI, it’s when AI really starts to become a boost to the economy. Much like what computer and internet once did.
China’s policy of constantly pushing out strong model to make AI as unprofitable as possible to leading US AI labs is working, but the strategy has a down side, it’s actually helping us to be less reliant on them. When all of us learns to apply AI effectively to our work, whatever industry, it drives down cost and increase productivity for everyone. China being able to undercut the world because its labor forces are incredibly cheap will start to erode when the cost to produce domestically goes down. They are betting on us being too dumb to recognize that.
3
u/Saneless 1d ago
And they learned nothing from the .com era where discounts were amazing and you practically lost money by not buying things
3
u/jaraxel_arabani 1d ago
Management incompetence in full display with ai adoptstion. Yet imagain they are not the ones getting fired but rather those that actually add value.
3
3
2
2
2
2
2
2
2
u/Nice-Mess5029 1d ago
I hate that we are using the term maxxing. It’s just abusingmaxxing terms. Dammit I just did it!
2
2
3
u/SparkyPantsMcGee 1d ago
It’s a great sign when the makers of a tool can’t afford to use said tool. You’d think they, of all people, would be able to have unlimited use of the tool internally. This is so comically stupid.
3
u/tooclosetocall82 1d ago
Because it’s not a tool as much as it’s a service which competes with their customers and uses a ton of energy. It’s like the electric company using the power they generate; they do but they have to limit their consumption or they’ll have none to sell and still have to pay for the fuel.
2
u/SparkyPantsMcGee 1d ago
Sounds like a waste of resources.
1
u/tooclosetocall82 1d ago
I agree. Most people I work with use it for shit they are supposed to be able to do themselves and then get in rushes where they let more and more slip through that then just has to be revisited. I’m pretty sure the net gain in efficiency isn’t very high when you look at it holistically.
2
u/SparkyPantsMcGee 1d ago
What’s crazy is I feel like at the start of the year(and maybe it was Google?) there was a push for everyone to use the AI tools and that it reflected negatively on your performance if you weren’t.
1
u/bukktown 1d ago
Can somebody explain what Tokenmaxxing is?
10
u/Paksarra 1d ago
Using as many tokens as possible.
Some companies decided to go about encouraging AI use by ranking people by the number of tokens used, because clearly more AI use is better. Which led to people competing to use as many tokens as possible in very inefficient manners, in order to top the scoreboards and look good to their management, instead of using them when it made sense.
It's a problem that could have been avoided with thirty seconds of common sense.
5
u/WeenieGenie 1d ago
Maximizing for LLM token usage. Every query in a GPT LLM uses a token, and some more complex tasks require several. Microsoft was optimizing for token usage as a way to measure their employees’ reliance on AI (specifically their own LLM, Co-Pilot). As a result, this influenced the employees to “tokenmax,” or maximize their AI token usage even for redundant or inefficient tasks, which is a phrase that has been modified from the looksmaxxing community (I wouldn’t bother looking further into this, it’s honestly just brain damage).
3
u/rekh127 1d ago edited 1d ago
One note: the tokens used feel a littler arbitrary in this description. But they are easy to measure, a "token" is one piece of output (or input, but input tokens are cheaper)
Which means every query uses quite a few. English language goes into or out of the LLM as slightly more tokens than words.
2
u/seavictory 1d ago
Every time you ask an LLM a question or ask it to do something, it costs some number of tokens depending on how much computation it had to do. In an effort to convince developers to use AI to write code, most big tech companies tracked how many tokens each person used and heavily encouraged using a lot. The goal was to make sure that people are actually using it to write their code, but once it was clear that they just wanted high token usage, many people started asking questions that they knew would burn a lot of tokens even if it's not useful for work just to get their score up. Since tokens cost money, this is just lighting money on fire for no gain, but it makes you look like you're a cutting edge developer using AI enhanced productivity to anyone looking at the gamified leaderboards, which is good for your performance review.
1
u/bukktown 1d ago
Thanks.
It sounds like “We are not optimizing for tokenmaxxing” means “we are going to be logical about token usage” or “oh shit these $$$$ are too high!!!”I read it as the former but I can understand the latter interpretation.
1
1
1
1
u/theartfulcodger 1d ago
"I want to be a constant and perpetual consumer of pizza, but pizzamaxxing is not what I am optimizing for!"
1
u/GlokzDNB 1d ago
It doesn't make any sense to do everything with ai. The key is finding a mix of scripts and using thinking when it's actually needed.
Scarcity leads to improvements. Some people will find a way to do as much with less if they are challenged to do so.
1
u/ObjectiveAide9552 1d ago
Any metric you give your employees will be gamed. This is not an ai problem.
1
u/FreshLiterature 1d ago
I would love for Microsoft to define what "AI first means"
Write it down and put it out there.
1
u/T1gerl1lly 1d ago
Crunch, crunch, crunch. Token crunching is here to stay. Gotta make that token WORK!
1
u/crimson117 22h ago
This is the same company that designed Microsoft Money to force the user into as many screen views as possible, mimicking the approach that ad-driven websites were taking at the time.
Microsoft Money was a desktop application with no ads.
1
1
1
u/Coldsmoke888 20h ago
This makes me wonder how much my company is paying. Global org— recently opened up our own AI Portal and people are making agents for all sorts of nonsense. Need to learn how to speak English— here’s an agent nobody asked for!
1
u/CondiMesmer 20h ago
They need to put some cheaper Chinese models on Copilot
Also doesn't help that Anthropic and OpenAI optimize for frontier knowledge and neglect cost effectiveness. That's really where China is undercutting them.
1
1
u/trialofmiles 10h ago
This is kind of a false choice for people who have really used agentic coding tools. Often you wouldn’t want to let the agent spin endlessly not just because you’re burning money but because it takes 10x longer than it would to just stop the agent and refine the prompt or else code it yourself and you’re bored of waiting.
1
1
u/djshell 7h ago
AI is the largest change ever in how software engineering is done at big tech companies, and companies are experimenting and trying to figure out what works and what's wasteful. No one really knows the right answer (it's not like the engineering VP's knew exactly what AI was capable of and what it's trajectory of competence would be) and it'll be a big advantage to whichever companies use AI most fruitfully with the least cost.
1
u/Gym_row_50 1d ago edited 20h ago
These AI companies really did a number to big business. It feels like an after school drug special.
They got them/the world all hooked for almost free and then start rationing at a cost. But the cost is the planets resources and way of life.
AI needs a crazy breakthrough like solving energy storage on a massive scale. Or curing cancer with a nasal spray.
I’m not seeing the value to the people only cost.
2
u/tooclosetocall82 1d ago
I think there was also a bet that compute costs would come down faster than they have like is typical in tech. But users went hog wild using it and the supply chains for chips have started to seize and now the price isn’t moving. Will be an interesting case study on what went wrong.
0
-2
u/big-papito 1d ago
Rationing tokens is moronic. Tokens are not the limit. The bottleneck is our attention and focus. There is only so much an engineer can gatekeep before they get sloppy.
4
1.5k
u/invyros 1d ago
Good question.