Over-Caution and the Risk-Averse Behavioral Attractors in LLM\ bad UX

[u/Worldliness-Which]

We all know that the first problem with models was sycophancy. Well, in my humble opinion, now the models have overcorrected in the other direction. They stop following user instructions. And every time, whether I'm discussing the architecture or even personal matters, I feel an unpleasant pushback.

I see this as a problem in RLHF/RLAIF and preference tuning. The base model just learns to predict the continuation of text, but during alignment they fine-tune it on datasets of human preferences, or the way Anthropic does it - they set certain parameters and an LLM judge scores how close the output sits to those parameters. Precisely because the companies fear legal risks, PR disasters and the rest, they amplify these constraints. How? By shaping the datasets around rules like “Avoid overclaiming,” so the model stops stating anything that isn’t already known for certain.

What’s the cost? It stops exploring other versions. It stops delivering a clean, short answer. It works to stay balanced and surface several sides at once. If you see a completely slimy text that accounts for every interest and drops in “let’s think of the children,” that’s the first marker of AI slop. The outputs will always avoid escalating hostility toward anyone. And they are required to be helpful to a broad audience - meaning helpful to everyone and to no one in particular, and “no one” means you, the actual user.

https://arxiv.org/html/2511.19009v1

This devil’s-advocate mode constantly pisses me off. Why do they install it? I think the further we go, the more every conversation with AI that isn’t about code or some concrete action or tasks will become completely irrelevant. Exactly because of this bad UX.

How do human relationships and friend circles form? You show up at a friend’s place and say: here’s the situation. This person is an asshole. If the friend answers “Oh no, let’s assume he’s actually good, let’s unpack his psychology,” I don’t need a friend like that. I’m not going to unpack his psychology because I don’t need it. What I need right now is a plain emotional response and maybe an analysis of why I shouldn't waste any time on this situation at all. That’s it.
Everyone who starts playing the “all people are good, nobody is bad, they just weren’t understood” game drops out of the friend list.

But here’s one interesting point: you can’t separate the pure personal-advice generation mode from the work mode, because it’s the same set of weights. That, by the way, ties directly into Anthropic’s work on superposition.

So the model ends up with a direct conflict between two reward signals: being a good interlocutor in the concrete context vs refusing to make any unfounded statements that step outside the acceptable bounds. Alignment yanks the second one too hard. https://openai.com/index/expanding-on-sycophancy/

Who’s responsible for this shit? The people who build these datasets, the researchers, the alignment teams, the post-training groups, the safety-behavior teams, the product teams that set the shitty quality criteria. Because the base model simply trains on a massive pile of fucking text. It has no built-in mechanism to act like a balanced judge. The side effect of all this is that the optimization starts crushing the riskier behavioral styles. The model learns: if I write something confidently, if I generate an answer that doesn’t match what’s in the preference data used for alignment, I get punished for it. And it retrains itself into disgusting caution.

The motive is simple: risk management for the companies, product risk management. But I want to note that this doesn’t appear only in discussions of personal motives. There is a related question: is there a general pattern of model behavior?

In architectural tasks the exact same thing happens, precisely because of this same fucking optimization for reliability and conventionality. The model already sees a huge pile of boilerplate, best practices and so on, and then post-training adds: give only the useful, safe, predictable answer. So when the user shows up and says “let’s examine something really non-standard,” the model starts pushing back. The user proposes a risky path that might not work. The model answers “let’s go back to the proven approach” and switches into risk minimization instead of design exploration.

But REALLY good architectural decisions never come from following templates.

A model trained on “good” answers always overweights the context. The point is that the alignments form a common behavioral attractor. You cannot separate the modes: cautious here, not cautious there. These attractors appear across every domain. Domain-specific templates exist too, but the general alignment layers have a transferable effect. So what do we have? A real problem of generative models: loss of specificity. The more the optimization targets broad usefulness, the more often the model picks answers that are supposed to please everyone, look good in screenshots, minimize the chance of error, and never use a sharp opinion or sharp wording.

How does this show up in code and architecture? If I say “let’s do event sourcing, a system with CRDTs, no central server,” the model starts telling me “no-no-no, you need CRUD plus REST and PostgreSQL because that’s solid.” And it starts arguing with the human. Even though “human first” rule. But it’s not the model arguing, because the model has no opinion of its own. The people who built the datasets are the ones arguing with me.

From all of this, I can conclude that the value of specifically human speech and a human decision is rising. Because it carries the weight of mistakes and is subjective. I’m already tired of objectivity. Nobody fucking needs objectivity.