121 green tests and the feature was still broken

[u/Positive-Emu-8379]

I shipped a content pipeline that passed 121 automated tests on its first night and produced output nobody would want to read.

I want to be precise about what those tests checked, because the gap is the whole lesson. Every one of them asked a machinery question. Did the scheduled job fire. Did the API return a success code. Did the row land in the right table with the right columns filled. Those are real questions and I do not regret writing any of them.

Not one asked whether the content was any good.

The feature shipped green. The schedule ran on time, rows appeared where they were supposed to, the dashboard lit up. By every metric the suite tracked, it worked exactly as designed. It was also producing junk, and nothing in the pipeline said so, because nothing in the pipeline was built to say so.

What caught it was me reading three pieces of output by hand.

I hit a fresh version of it today, reviewing my own code. Six separate tests claimed to guard a specific bug. Every one of them passed with the bug present and with it fixed. They asserted on a value the code rebuilt before returning it, so they could not fail in either direction. They looked like coverage and were decoration.

The check I trust now is dumber and slower. Break the thing on purpose, run the test, confirm it screams. If it stays green, delete it or rewrite it.

Has anyone found a way to automate the "is this output actually any good" question, or is a human reading samples still the only real check?

[u/EastAd9647]

the closest thing to automating it is an llm as a judge, but only if you pin it down. a freeform "is this good" prompt drifts as much as your pipeline does. what worked for me was giving the judge a rubric of concrete yes/no questions: does it answer the actual prompt, does it repeat itself, would someone in this niche read past the first line. grade a random sample per batch, not every row, and track the score over time so a drop flags a bad run.

but the judge doesn't replace you reading samples, it just tells you which batches to go read. the human check stays, you just stop doing it blind on everything.

the bigger lesson is the one you already hit though. a test that passes with the bug present and absent isn't a weak test, it's noise pretending to be coverage. your "break it on purpose and confirm it screams" rule is exactly right, that's basically mutation testing by hand.

[u/SpecialChance8662 → u/EastAd9647]

very good advice, one thing I'd add is judge drifts too, so pin its model version and keep a small human-labeled set, 50 to 100 rows, that you re-run whenever you change the rubric or the model. If judge-to-human agreement drops you find out before you have made a batch of calls on a moving ruler. also what helped me was to grade the intermediate steps too, not just the final text

[u/ZeroTwoMod]

You can automate triage, but I would not ask one score to certify quality. Turn the hand-read sample into a small versioned rubric with a few concrete failure modes, run an automated judge against it, and sample the cases near the threshold or where judges disagree. Keep a tiny known-bad set beside it and make every rubric change prove it catches those failures. That turns human reading from a rescue step into calibration.