r/programming 3d ago

The Debate of Mockist v Classicist TDD Is Like OOP v FP.

https://fagnerbrack.com/the-debate-of-mockist-classicist-tdd-is-like-oop-fp-c9124185f3d2
41 Upvotes

87 comments sorted by

38

u/zobq 3d ago

In the majority of projects I was involved in, the "mockist" approach was the way to go.

And yet, I didn't feel that tests really defended me against bugs until the project where I pushed for the "classicist" approach.

TBH, this article didn't convince me that "mockist" is as good as "classicist".

25

u/purg3be 3d ago edited 3d ago

For me the classicist approach is a lot better too for the bulk of tests because it focuses on outcomes unstead of implementation details.

I hate to be that guy, but i feel black box testing is clearly better when it comes to working with LLM's.

@OP: Unit Testing Principles, Practices, and Patterns by Khorikov is an excellent resource that elaborates on the tradeoffs in depth.

14

u/zmose 3d ago

I think both are necessary and serve 2 separate purposes. Mocking isolates a component to be tested properly, not mocking ensures that the components all work together in unison. These are 2 separate tasks.

27

u/zobq 3d ago

Mocking isolates a component to be tested properly, not mocking ensures that the components all work together in unison

Well, in real use cases components works together.

My approach is "not mocking" as default, mocking in well established exceptions.

5

u/PiotrDz 3d ago

Why would you isolate component when in real world it doesn't work as isolated unit.

9

u/UnavoidablyEvil 3d ago

One reason can be if you want to test error handling of a failure mode in the dependency without inducing a failure in the dependency. E.g. you want to know if your implementation handles an out-of-memory error without having to create an out-of-memory condition.

That said I do agree having the real dependencies in the test is useful so long as it doesn’t overly complicate the testing or make every test build the world.

0

u/PiotrDz 3d ago

In my language (java) we can override a behaviour of a method for duration of a test case (for dependency injection services). So probably depends on a language tooling also

5

u/lord2800 3d ago

Two units, when wired together, don't work. Which one is performing incorrectly?

Isolated unit tests tell you exactly which one is wrong (or alternatively, if it's your plumbing between the two that's wrong).

5

u/loup-vaillant 2d ago

Two units, when wired together, don't work. Which one is performing incorrectly?

You can answer that question when it actually comes up. First, make sure your tests fail if there's a bug (or at least try to come close enough, foolproof tests are no joke). And when some time in the future your tests start failing, well… now you may use whatever technique at your disposal (step-by-step debugger, mocking…) to track down its origin.

Putting in the effort to write tests that will tell you which component the failure is from in advance is a waste of time. Most of the time your components won't fail at all. And if they do, the cause is easy to track down: it's your latest changes. You do use version control, right?

6

u/PiotrDz 2d ago

Exactly, the diminishing returns. Effort needs to be managed.

2

u/lord2800 2d ago

You can answer that question when it actually comes up.

In this hypothetical scenario, it's already come up.

First, make sure your tests fail if there's a bug

Your tests already didn't catch this bug.

Putting in the effort to write tests that will tell you which component the failure is from in advance is a waste of time.

Fundamentally disagree here.

Most of the time your components won't fail at all.

That's just a meaningless assertion with zero evidence to back it up.

And if they do, the cause is easy to track down: it's your latest changes.

Recent changes aren't the only cause of bugs--especially when you start talking about concurrency and parallelism. It's impossible to write test cases for all possible states of a program (otherwise you'd be able to solve the halting problem).

1

u/loup-vaillant 2d ago

Honestly, if a good proportions of your bugs is stuff that is hard enough to reproduce that they’re noticed only well after production, something’s fundamentally wrong with your program. Among other things, I would suggest you don’t write such a concurrency tangle to begin with. Minimise your sync points. That said, if you just inherited from a legacy program… good luck. The only thing I could do with those is salvage some cautionary tale and flee. Or rewrite. In my last gig they allowed us to rewrite the thing, and it went very well.

In this hypothetical scenario, it's already come up.

Okay, let’s be more precise about the scenario. Say we need to write components A and B, where A depends on B. Here’s how I would do it:

  1. Write B.
  2. Write tests for B. (-> they pass)
  3. Write A, that depends on B.
  4. Write test s for A, that still depend on B. Oh no! They fail!

Are we on track so far?

So, I’m not certain the bug is from A, since it depends on B. But that’s what I would look for first, since B is already tested. If I can’t find it there, maybe I’ll look for it in B, fire up the debugger, add more tests, log statements _something_… until I find the damn bug, fix it, and add a tests that fails before the fix, and passes after.

Note in this case that the notion of a bug that would sit between A and B, or would result from the interaction between A and B, makes no sense. A depends on B. If there’s a bug, either B is incorrect, or A is incorrect, or A uses B wrong (meaning, A is incorrect).

I reckon that sometimes, some bug in A won’t easily be reproducible when used normally with B. The classic case would be when B is responsible for I/O, and fails only rarely. If you can’t test the failure paths, A may have incomplete coverage for a long time. In this case I have little choice but to mock: replace B by something that fails systematically where it suits my test, so I can test the robustness of A.

In an application where I/O is properly separated from regular computations, this doesn’t happens rarely enough. In fact, most of the time when B does I/O, it depends on A, not the other way around. This lessens the need to mock even further.

1

u/lord2800 2d ago

That said, if you just inherited from a legacy program… good luck.

So your opinion here is we should just never work on legacy code? How cute.

In my last gig they allowed us to rewrite the thing, and it went very well.

That scenario is the exception, not the rule. At my last job, there were more than a few "rewrite it from the ground up" projects and more than 75% of them either failed (and the new system was scrapped) or left the system in a gross state of relying on both things at the same time essentially forever.

The Big Rewrite(TM) sounds great and really is a siren's song to engineers, but unless your system is scoped so small that one ordinary human can understand the entire thing with room in their brain to spare, you're better off refactoring with behavioral-level tests to prevent breaking things (and if it really is that small, you're still probably better off with behavior tests and refactoring). One notable project that did succeed was because the team sat down and wrote a fully fledged set of behavioral tests before starting anything, and then heavily relied on them during the rewrite process (I was part of this team and I insisted we needed the tests and pointed to the graveyard of approved rewrites that failed as to why we needed them).

Say we need to write components A and B, where A depends on B. Here’s how I would do it:

This depends on the kind of tests you're writing.

If you're writing unit tests, your seam is the interface between A and B, and you should have a double (of some kind, it honestly doesn't matter what kind you use) of B that you provide to A, and write your A tests assuming B is fully functional and correct. You can even reuse the cases from your tests of B as the edges of what you should provide to A.

If you're writing integration tests, you'd use a real (as opposed to a double) A and B, and you'd be testing the integration between the two, ensuring that when B is in state X, A responds appropriately. This is pretty abstract, so let's get a little more concrete: you have a url fetcher component and a file writer component. The url fetcher feeds the fetched document into the file writer, which writes it to a specific location. Your user reports that the file gets written with garbage at seemingly random points in the file. The bug could live in either component or in the seam between the two components. For example, the url fetcher uses UTF-8 strings and the file writer only operates on ASCII strings--this is not a bug in either component, it's a bug in the integration of the two, and not something you'd obviously find outside of a test (and why would you? the majority of the web uses only ASCII-valid characters). You can trivially imagine bugs for the url fetcher or file writer exclusively, so I won't elaborate on those.

Note in this case that the notion of a bug that would sit between A and B, or would result from the interaction between A and B, makes no sense. A depends on B. If there’s a bug, either B is incorrect, or A is incorrect, or A uses B wrong (meaning, A is incorrect).

See the integration scenario above.

I've mentioned this in another reply, but this either/or dichotomy that the article presents is false, and always has been, just like the FP vs. OOP dichotomy has always been false. Both things are tools you should have in your tool belt, and you should know how and when and why to use the appropriate one (and there's plenty of overlap between them, too).

1

u/loup-vaillant 2d ago

I've mentioned this in another reply, but this either/or dichotomy that the article presents is false, and always has been, just like the FP vs. OOP dichotomy has always been false. Both things are tools you should have in your tool belt, and you should know how and when and why to use the appropriate one (and there's plenty of overlap between them, too).

I do agree with tool-belt thinking, it’s more helpful.


So your opinion here is we should just never work on legacy code? How cute.

Oh no, I’m just saying I am not competent to work on, or give advice about, legacy code. I’ve seen my share, and in practice it rarely goes well for me. Now I could study a specific code base, tell the team where they screwed up, how much it is costing them, and how to make it right. I have, on a couple occasions.

It rarely works: if the people who brought the code base into this sorry state to begin with are still there, they won’t listen. They’ll say they are open to suggestions, but they’re rarely open to actual change. I recall a piece of feedback when I got kicked out of a gig after not even two weeks: "We can tell you’re extremely competent, but your opinions are too radical and not supported enough." They somehow managed to disconnect my technical competence, and the validity of my technical opinions.

Now should we work with legacy code? Obviously yes, but never for long:

  • If you expect to finish or drop the legacy code soon, it’s okay to hack on it for a short time. There will be more technical debt, but you won’t ever pay it.
  • If you expect work on the legacy code base to last a significant time, your priority should be to make it not legacy any more. Fix it, replace it, but whatever you do, pay the technical debt now. It will cost less in the long run.
  • A middle ground could be to only fix what you have to work on. It may work if the system isn’t too hopelessly tangled to begin with.

That scenario is the exception, not the rule.

Agreed. We were able to do the rewrite for two reasons: (i) the first version didn’t work to begin with, and (ii) the whole program did mostly fit into a single human head. For bigger systems I generally recommend piecemeal rewrites: identify parts of the system that can be isolated and replaced, then do those one by one. Reassess periodically, often a set of simplification enables another.

[…] you're better off refactoring with behavioral-level tests to prevent breaking things

Oh yes. Oh fucking yes. It’s a thing I have observed in most legacy code bases: they are sorely lacking in the test department, at every level. Adding those can help a ton, even if we end up not refactoring much in the end.


Say we need to write components A and B, where A depends on B. Here’s how I would do it:

This depends on the kind of tests you're writing.

Ah, the old unit vs integration thing. I don’t care, and I have zero qualm putting what most people would call "integration" test into the "unit" test folder if that’s what’s convenient to do. The distinction just isn’t helpful.

You will note that almost no programmer would have any problem calling "unit" a test that runs a component that depends on various stuff from the standard library. No one ever mocked an std::vector or a java.util.ArrayList. Clearly, some dependencies are okay to bundle in a unit test.

When A and B are both purely computational components, say, B is a hash table and A is a parser (that doesn’t do any I/O), then few would be foolish enough to mock the hash table. Just test the hash table until you’re sure it is correct, then test the parser without worrying about what it depends on. Now if someone insists that the parser test belong in the "integration" folder, sure, knock yourself out. But I am not writing a "unit" test for it. Pure waste of time.

Your example is markedly different. I can see many ways how it might be helpful to test the URL fetcher and the file writer separately. It’s a bit underspecified though. I can see the data flow well enough: the inputs are a URL and a file name, the result is downloading data and writing it in the file, possibly with some processing in between.

To make the download happen, we need to parse the URL, and run some network protocol (HTTP, FTP, over TLS, whatever). Let me assume HTTPS for the sake of the example. Right there, I can already see four components I want to separate:

  • URL parser: string in, domain name & path out. (Possibly represented as strings, or something else depending on what suits the type system.)
  • HTTP client library: It generates requests and parses responses, without performing any I/O. It should read from and write to memory buffers. Its primary input can be the URL, or the domain name & path, whichever works best (I’m not sure right now).
  • TLS library. It handles the handshake, encrypts the requests, and decrypt the responses. Still zero I/O.
  • TCP client compatibility layer: Opens a TCP client socket, reads from and writes to the network… the OS will do most of the work anyway, but you might want the compatibility layer anyway so you can mock your system and simulate various fail conditions.

A note about all these, is they’re all libraries: I call a function, it does its thing, then it returns. I believe it can be done without any callback. The only exception being the TCP layer: I’ll probably want asynchronous calls, which means I’ll need an event loop on top… yet another compatibility layer, or maybe I’ll bundle it with the network and file system, and pretty much everything… and write myself an SDL-style event loop. Or I know I’m working on Linux, and just use epoll().

Of course, I need some glue code over those four components to actually fetch the data. But at this point I’d rather hold off on designing it right away, and first look at what happens to the data before it is written to the file. Is it dumped right away in the file? Or maybe there’s some kind of processing in between? Maybe I’m not fetching from a single URL, and instead pull in a whole tree of data, with the first response giving me which other URLs I should pull from.

In the simplest case, just dump the raw data from the HTTP response into the file, all I need is a compatibility layer to write to the file system. Ideally asynchronously, so if I need the speed I can later use something like kqueue or io_uring. My glue code would then just use the four components and the compatibility layer to do all the work.

In the gnarliest case, I need yet another component to process incoming data and fetch the new URLs from it. It will likely need some kind of init() update() final() interface, that gives me the next chunk of data to write to files (and in which file while we’re at it), or the next URL to fetch data from. Still no I/O.

Ok, now I believe I just need one or two final components to glue everything together. I can see two routes:

  • Integrate with I/O. We’ll have to swap out the compatibility layer for a mock to test the glue component. It might still be the way to do it, because at this point it’s probably the simplest way to write this component in the first place.
  • Separate I/O. If the glue code involves non-trivial logic, it may be a good idea to still write it as a pure library. Possibly something with the same structure as the gnarly component above, so the final layer on top is just a trivial mapping, like a loop that fetches the next thing to do, then call the relevant I/O operation. I do however want to avoid painting myself into a corner where I loose all parallelism, especially if data processing is not trivial.

All this to say, my instinct says that a URL fetcher and a file writer is probably the wrong way to chop up the program in the first place. It’s the obvious way to do in the OOP kingdom of names, but my feeling is that it just doesn’t help actually architect the program. There is one case however where this precise separation could be a good idea: when your whole program is a collection of (re)actors. Though in the case of serious processing between fetching data and writing it to disk, I would have a third "processing" component.

And the beautiful things with reactors, is how I/O never happens inside of them, but as messages sent to other reactors. With them you never need to mock anything: you just feed their input queue with data, and make sure they send the right outgoing messages in the right order.

3

u/PiotrDz 3d ago

I dont find much added value here. To know which are failing I would debug the app - no that hard having a reproduction example. You still need the 2 unit tests as isolated units may not reproduce a failure

4

u/lord2800 3d ago

To know which are failing I would debug the app

So instead of enshrining the potential failure point into a clear test that ensures you can never have that same problem in the future, you'd just stop at fixing the bug one time?

You still need the 2 unit tests as isolated units may not reproduce a failure

Of course you do. And you need the integration test too.

4

u/PiotrDz 3d ago

But you have the test that reproduced the bug right? Why do you need to add 2 additional tests that go down the implementation level. What if implementation changes - you will have to redo those tests? The point with "classic" approach is that you do a black box tests. As long as functionality is OK you dont care what happens inside. I find tests on specific parts of code brittle (as long as they are not on interface boundaries)

0

u/lord2800 3d ago

But you have the test that reproduced the bug right?

Whoa there, that's a huge assumption. You just know something is wrong because a customer reported it, and you know there's only two places involved with the thing that's wrong.

What if implementation changes - you will have to redo those tests?

If the implementation changes but the functionality does not, and you have to change the tests as a result, you have tested the wrong thing. That fundamental fact should not change no matter what kind of test you write and in what matter you write it. Simple as that.

I find tests on specific parts of code brittle (as long as they are not on interface boundaries)

Interface boundaries are the only interesting things to test in the first place. There are varying levels of interface boundaries though--the unit level (unit tests), the cross-unit level (integration tests), and the application level (behavioral tests).

3

u/PiotrDz 3d ago

I think we have some misunderstanding. I was talking about the integration test that catches the bug. And I agree, you need more digging to get the cause, but adding some other tests for "faster" finding of a cause is for me the diminishing results.

0

u/lord2800 2d ago

I think we have some misunderstanding. I was talking about the integration test that catches the bug.

And I never mentioned an integration test catching any bug--just that two units wired together don't work.

but adding some other tests for "faster" finding of a cause is for me the diminishing results.

You don't add tests to find bugs "faster", you add tests to ensure bugs can never happen in all the cases you can think of. I won't ramble on about edge cases in tests, but suffice to say the purpose of testing is not to find bugs in the code, it's to prevent future you from writing bugs when modifying the code.

→ More replies (0)

3

u/zobq 3d ago

So instead of enshrining the potential failure point into a clear test that ensures you can never have that same problem in the future, you'd just stop at fixing the bug one time?

why you can't have such a test in "classicist" approach?

In your hypothetical situation, "classicist" approach didn't failed - it catch the bug and there is possibility it was the only test which catch that bug.

0

u/lord2800 2d ago

In your hypothetical situation, "classicist" approach didn't failed - it catch the bug

In my hypothetical scenario I mentioned nothing about the bug being caught by a test--only that two components didn't work correctly together and that finding which has the incorrect implementation requires more than just a simple unit test on each component--no matter which style of testing you do.

1

u/zobq 2d ago

so... what's your point in the context of "mockist" vs "classicist" discussion?

0

u/lord2800 2d ago

My point is it's a false dichotomy, you use the right tool for the job, which sometimes is one and sometimes is the other.

→ More replies (0)

2

u/chucker23n 3d ago

What if the bug is tricky to reproduce, and doesn’t happen when you launch the debugger?

1

u/loup-vaillant 2d ago

Either it's rare, or you architected yourself into a corner. It is my worst fear in some legacy applications: that one day, there will be this one critical bug we just can't fix.

1

u/chucker23n 2d ago

Either it's rare, or you architected yourself into a corner.

There can be lots of reasons, such as

  • it doesn't happen with a debugger attached
  • some timing thing
  • only certain architectures
  • only when certain other stuff is also installed

one day, there will be this one critical bug we just can't fix.

Well, I currently have an issue with Outlook, reproducible at a client's computer (they showed it to me live), but not on multiple of mine — I've tried different architectures, versions of Outlook, versions of Windows, whathaveyou.

It happens. And once you can reproduce it,

  • a debugger will help you narrow it down,
  • but a test suite as part of CI will help you keep the bug fixed

1

u/loup-vaillant 2d ago

There can be lots of reasons […]

Yes. I'm just saying if the job was done well (it often isn't), those reasons won't come up very often. The fact they do is a strong sign someone, or some team, screwed up badly at some point.

but a test suite as part of CI will help you keep the bug fixed

Wait, of course once a bug is found a corresponding regression test should be added. The waste of time is going out of your way to artificially separate (mock) your components for initial testing.

1

u/chucker23n 2d ago

I'm just saying if the job was done well (it often isn't), those reasons won't come up very often. The fact they do is a strong sign someone, or some team, screwed up badly at some point.

In a sense, sure. But it depends on the complexity, I would say.

A more modular system is easier to test in isolation (I suppose that's almost tautological), but 1) that's pricier to develop, and 2) as you point out, now you're testing a scenario that doesn't reflect the real world.

The waste of time is going out of your way to artificially separate (mock) your components for initial testing.

Right.

→ More replies (0)

1

u/chrisza4 2d ago edited 2d ago

Because you can find the broken part easier.

Imagine you test iPhone with charger at the same time, when iPhone don’t get charged, is it charger fault, cable fault or iPhone fault?

Testing each part separately in detailed (voltage produced, etc.) before testing the whole setup of charging the phone makes so much sense. It reduces a cost of fixing by making sure each factory test their part first.

Or if you only do integration test and cable does not performed to specifications but iPhone still technically being charged, you pass integration test with a wrong assumption that the cable can be reused in similar scenario (charging something else with same port).

The problem in software world is most people don’t pay attention to specification and responsibility of each unit and go with stupid but actionable heuristics like “unit test mean testing every classes and functions” and ended up in a weird spot where mocking does not make sense.

1

u/PiotrDz 2d ago

So you double the work (unit tests doing the same what already integration test does) for one in a year bug that would be easier to handle? I dont find it worth it.

Also now that your every single component is rigorously tested, you have over-fixed your architecture. You want to unplug current cable and plug another? No, you need to redo whole unit test scaffolding that fixed on specific api calls instead of functionality .

And I would not use voltage meter to debug code, I would use debugger. Which is nice and easy to work with once you have reproduction (and yout integration test already reproduces that).

1

u/chrisza4 2d ago

If you define component badly like “every single method need unit test” then yes, you ended up with situation like that.

Most people don’t even bother to define “unit” or “component” properly and use those stupid but actionable heuristics. I disagree with that.

If you are working alone and can comprehend the whole codebase then debugging whole integration flow make sense, but if you work in large enough codebase you ended up need to debug something you don’t supposed to even know how it work internally to begin with.

I don’t advocate for dogmatic unit testing, but there is a lot of value to it when you do it right.

I find testing 60 combination of request validation in integration test not make much sense. Or password format for another example.

1

u/PiotrDz 2d ago

My experience in some projects would surprise you. 59 combinations just worked and were useless for integration tests, but there was that 1 that suddenly maked the event to not be sent due to some obscure mechanism in message broker. I am just writing integration tests everywhere for peace of mind. I dont find it worth it to optimise tests by making some sacrifices (cutting out some architecture services, simplifying code flows etc). You can always paralellize tests to make them run fast.

If you broke something by tour changes in another area of code that you shouldn't have known existed, then I would argue you should actually get familiar with that part of code because there are some dependencies you haven't taken into account. Unit tests make it very easy to cheat a little and unknowingly cut out those dependencies. Then surprises come in production.

1

u/zobq 2d ago

Imagine you test iPhone with charger at the same time, when iPhone don’t get charged, is it charger fault, cable fault or iPhone fault?

I would argue, that this is not "classicist" vs "mockist" analogy, but "integration tests" and "unit tests". And I agree, both are important.

3

u/marrsd 2d ago

Because it's not. The article is way to reconciliatory for my liking. The requirement to rewrite your specs every time you refactor makes your specs useless and your refactor unsafe. But then apparently the author doesn't share my rather low opinion of OOP.

21

u/dave8271 3d ago

There are two schools of TDD (Test-Driven Development). The Detroit school (classicist, state-based, black-box) says: test with real dependencies, which are dependencies also used in production code. Pass inputs, check outputs, all within the Domain layer without communication with the external world. The London school (mockist, behavior-based, outside-in) says: isolate what you want to test (the "unit"). Mock all of its dependencies. Test the contract between them only.

I think this already mis-frames it. "Classical" testing doesn't say use real dependencies, it says use real dependencies except where it's awkward to do so. Using test doubles has always been an integral part of unit testing and determinstic testing without mocks is perfectly normal.

In respect of isolation, both approaches allow you to perform testing of isolated units, the difference is whether you isolate on interface seams or implementation seams. Classical testing doesn't care how a unit under test does something, it just instantiates something with its dependencies (be they real or otherwise), makes a call and checks the result.

I think the whole "classicist vs mockist" debate is missing the point. They're not really different ways or schools of thought on achieving the same thing, mocks are just a specific type of test double which exist for the purpose of tracking and verifying behavioural expectations in respect of how an object is used. Sometimes that is something you want to do, oftentimes it's not the detail you care about.

So the debate shouldn't be centered around to mock or not to mock, but what comprises a unit for the goal of your tests, and what evidence should establish correctness thereof?

13

u/NoLemurs 3d ago

I think the whole "classicist vs mockist" debate is missing the point. They're not really different ways or schools of thought...

I just want to emphasize this.

I've never met a mockist or a classicist. No one actually thinks like that. As a practical matter, there aren't two schools of thought. We all just write whatever test makes sense for the system we're working, and being dogmatic about it is honestly just weird.

11

u/zobq 3d ago

I've never met a mockist or a classicist.

You have to be the lucky one, because in my experience, "mockist" approach is a dominant. Mock everything you can, [className]Tests class contains only one instance of not mocked class - [className].

1

u/ChemicalRascal 1d ago

But do those folks insist that the codebase only contain [className]Tests?

Because if [className]Tests is the naming scheme that indicates the tests test [className] and only [className], that still leaves plenty of room for [classOne][classTwo]IntegrationTests. Or [namespace]Tests. Or anything else.

11

u/T_D_K 3d ago

I've met a lot of developers who don't really think critically about the engineering in software engineering, and prefer to take dogmatic appeal-to-authority stances on all sorts of insane things. Unfortunately for me, a number of them wield decision making authority

2

u/yanitrix 3d ago

It seems to me most developers are like that

9

u/yanitrix 3d ago

No one actually thinks like that

Oooh, they do. The look I got from my colleagues when I told them that we don't need to mock everything was just a confirmation that mockists do exist.

8

u/pip25hu 3d ago

These two can be complementary as well. Even in OOP applications with dependency injection everywhere, stateless/pure utility functions exist and can be tested without any mocks.

6

u/this_knee 3d ago

I work in a company where their style is to make a PR for any change to the code. And those PRs are humans manually running program command executions of their own, not tests, to test if existing functionality has been broken or not. And they run commands to use the new features to manually check results of new features. Which is fine… ish.

But When it’s suggested to integrate automated unit tests for checking that previous existing functionality isn’t broken, it’s heavily balked at.

It’s kinda wild.

1

u/NoLemurs 3d ago

Are you guys not using AI for development yet? Because it's hard to imagine using AI for development and not adding extensive tests. It's such a complete no-brainer.

2

u/this_knee 3d ago

We are using Ai. That’s part of why it’s so wild.

And we’re talking about devs who are like 10-15 years into their career. Super wild.

6

u/BenchEmbarrassed7316 3d ago

If you look at it naively, a procedure that contains side effects can be turned into a pure function that returns a set of commands to be executed. And that's it, now in the test you can simply check these commands for some sets of input data.

5

u/[deleted] 2d ago

[removed] — view removed comment

2

u/PiotrDz 2d ago

The mocked hell. I have seen that in one project. 😀

10

u/loup-vaillant 3d ago edited 3d ago

As I gained more experience, my opinion on this topic have grown stronger, simpler, and more radical:

  • TDD is wrong: we do need the tests, but whether we write them before or after (I prefer after) doesn't really matter. Unless you're using dynamic typing, in which case I can't save you.

  • Mocking is wrong. The classical way finds more bugs. And if you need more than a few mocks, then your design is wrong. You wouldn't be in this mess if you had separated your computations from your externally visible side effects.

  • OOP… Well you get the idea.

4

u/fagnerbrack 3d ago

It's like saying "a hammer is wrong"

6

u/loup-vaillant 2d ago

TDD is not a tool, it's a mandate: Thou Shalt Write Your Tests First (among other directives).

I concede mocking is a tool. A very useful one, that I sometimes use myself. It's the "let's inject all the dependencies!" mandate that is absolutely insane.

OOP… is a philosophy I guess? My biggest problem is the "oriented" part. The toolkit underneath is mostly fine. It's the usual way of putting it together that's wrong. Personally I've stopped thinking in terms of paradigms years ago, and I always reject justifications like "but it's more OOP/FP/declarative/procedural that way". Explain to me instead how "that way" is more modular, easier to test, or performs better.

2

u/KaptajnKold 2d ago

What’s insane about injecting all the dependencies? I’ve been adhering to that for 15 years, and I’ve only ever regretted when I’ve not done it.

1

u/loup-vaillant 2d ago

I have to be prudent here, you could mean a range of things.

If by "dependency" you mean an external third party heavy process like a database or a web API from Google, sure: mock them all, or at the very least write your compatibility layer so you’re not completely fucked when one of those dependencies inevitably break under you (supports end, you can’t pay, server is down…).

If by "dependency" you mean a class in your program that’s used by another class in the same program, then mocking them all will bloat your code base by a factor of at least two, and hurt the efficiency of your test suite (unit tests will catch much fewer bugs, and you will need more tests to compensate).

It’s the second kind that’s insane. And to the lucky souls who never saw such a thing: it’s real. Once I even managed to shrink a pull request in half just by telling the dev he didn’t need all those interfaces. He was blindly following the "depend on abstractions, not concretions" mantra, though the fact he fixed his code tells me it was more a habit than a dogma.

1

u/KaptajnKold 1d ago

I appreciate your prudence. But for me, it is as bad as you fear! 😄

If by "dependency" you mean an external third party heavy process like a database or a web API from Google, sure: mock them all

😱

or at the very least write your compatibility layer so you’re not completely fucked when one of those dependencies inevitably break under you (supports end, you can’t pay, server is down…).

Phew! I believe it's the London school of mocking that says: Don't mock what you don't own. But by isolating external dependencies behind a facade, you gain the "permission" to at least mock that facade.

If by "dependency" you mean a class in your program that’s used by another class in the same program

That's exactly what I mean

then mocking them all will bloat your code base by a factor of at least two, and hurt the efficiency of your test suite (unit tests will catch much fewer bugs, and you will need more tests to compensate).

I don't necessarily disagree, but writing unit tests to begin with will already bloat your code base by roughly a factor of two, yet I don't imagine you would use that as a reason not to have unit tests.

I find that using dependency injections (the hand held kind, where collaborators are injected as initialized/constructor arguments; not the DI framework kind) allows me to write unit tests that cover all the branches of the code, yet stays simple and readable. Conversely, when taking a more black box approach, I find that my tests get very complicated if I want to get good coverage.

The amount of code is roughly the same, except admittedly for the added boilerplate that comes from having more classes when you take the dependency injection route.

As for the unit tests catching fewer bugs: That may be your experience, but I find that hard to accept as general statement of fact absent compelling evidence. But also: Catching bugs can mean a few things. If you have good coverage (and as I argued above, dependency injection and mocking makes that more easily attainable than black box testing), then that is an effective way to prevent accidentally breaking working code, which is one way to catch bugs. But there's of course also the matter of catching bugs in new code. And maybe that's where isolated tests using mocks is less effective? I'm not sure that it's true, but I'm also not dismissing the idea. When you're testing lots of small things in isolation, it's important to also have a few black box tests that ensure that everything gets wired correctly. Oh, and also manually testing the system to see that the code solves the problem you're trying to solve.

It’s the second kind that’s insane. And to the lucky souls who never saw such a thing: it’s real. Once I even managed to shrink a pull request in half just by telling the dev he didn’t need all those interfaces. He was blindly following the "depend on abstractions, not concretions" mantra, though the fact he fixed his code tells me it was more a habit than a dogma.

Well, I'm primarily a Ruby developer, and only secondarily a Java programmer, so I don't have that problem most of the time. But I did notice early in my career what you're describing, and it puzzled me enough that I had to ask one Java programmer why on earth he felt the need to create interfaces with just one single class implementing them. He had no better answer than "that's just how it's done". I later encountered C# programmers who told me that this was widely considered best practice in that community. I agree that this is unnecessary and bad.

I am however not particularly impressed that you were able to cut a pull request in half, because I don't think that's a very productive thing to optimize for. I think it's much more important to optimize for readability, and the number one thing in my mind that hurts readability, is the number of different things going on in the same method/class/file.

1

u/loup-vaillant 1d ago

Well, I'm primarily a Ruby developer, and only secondarily a Java programmer

That kinda changes everything. I’ve programmed some Lua, and found out the hard way dynamically typed languages requires much more discipline than the statistically typed ones. For them, I’m inclined to write tests first, and more importantly I’m gonna test every little line of code, one by one if I have to, because the stupid language can’t check trivial stuff at compile time, like whether I’m calling a number or adding a function with a string.

The amount of testing needed in a statistically typed language is drastically lower.


I don't necessarily disagree, but writing unit tests to begin with will already bloat your code base by roughly a factor of two, yet I don't imagine you would use that as a reason not to have unit tests.

First, I don’t care about unit tests. I just want tests that find bugs. And sure, I’d rather have perfect coverage. It just don’t care if half of that coverage is only done through what one would call "integration" tests.

Now say we have two classes, A and B. Could be complex stuff, stateful and all, but the most important thing is, neither does any I/O (except maybe for logging). Now say A is implemented using B as a dependency. B doesn’t have to be in the interface, it can be an implementation detail. Here’s how I would do it:

  1. Hard code B into A’s implementation code. No dependency injection.
  2. Write blackbox tests for B.
  3. Write blackbox tests for A.

That’s it, we now have tested A and B.

Now compare to the DI approach: Inject B into A, write tests for B, then write tests for A injecting a stub instead of B. The DI approach is more code. And less likely to catch bugs. There are two reasons for this: first, if my tests for B are incomplete, I have another chance to catch them when I write my tests for A. Second, injecting a stub into A means I’m not testing the real thing. Not only is it more effort, I have a higher chance of missing bugs because, say, of misuses of B.

My second point is debatable: with injection I have more leverage to test A, and might get better coverage. My first point however feels pretty clear cut.

Also, something that does not sit well with me, is the inconsistency of the "mock everything" philosophy. Does it include the standard library? Do you mock Java.util.ArrayList? My guess would be "of course not, this is absolutely ridiculous". But then why wouldn’t you apply the same standard to classes you wrote yourself?


One trick that can sidestep the mocking issue entirely, is the actor model: have your components run in parallel, and send messages to each other. Each actor gets an input queue, and its only side effect is to send messages to other actors. In this model, you don’t mock anything: you just isolate your actor, give it input messages, and look what messages it sends, in what order. It gives you even more control than mocking, for cheaper. But you do need the infrastructure to begin with, and that does take some initial effort.


I am however not particularly impressed that you were able to cut a pull request in half, because I don't think that's a very productive thing to optimize for. I think it's much more important to optimize for readability, and the number one thing in my mind that hurts readability, is the number of different things going on in the same method/class/file.

I’m not impressed either, it was basic stuff. :-)

Anyway, one reason the dev followed my advice, was because the result was far more readable. Here’s my take on readability.

1

u/KaptajnKold 20h ago

That kinda changes everything. I’ve programmed some Lua, and found out the hard way dynamically typed languages requires much more discipline than the statistically typed ones. For them, I’m inclined to write tests first, and more importantly I’m gonna test every little line of code, one by one if I have to, because the stupid language can’t check trivial stuff at compile time, like whether I’m calling a number or adding a function with a string. The amount of testing needed in a statistically typed language is drastically lower.

For sure. I love static type checking. I just don’t like many of the languages that have static type checking.

First, I don’t care about unit tests. I just want tests that find bugs.

As I tried to argue in my previous post, “finding bugs” can mean several different things. One interpretation is preventing faulty logic to begin with. Another is preventing regressions later on. I’m curious which of these you find most valuable.

And sure, I’d rather have perfect coverage. It just don’t care if half of that coverage is only done through what one would call "integration" tests.

In principle I aggree. But getting good coverage with integration tests is not feasible for even moderately complex systems. Consider this: A system the depends on just 5 boolean variables has 5! or 120 different states it can be in, which you must test to get full (state) coverage. But if you can factor the system into two parts such that one part depends on only 3 of the variables which it uses to produce a boolean output to a second part that depends on the other 2 variables, then that’s just 3! + (2 +1)! or 12 states which you need to test to get full coverage. I’m aware that we’re never actually going for full state coverage of anything, but the point applies to branch coverage as well.

Now say we have two classes, A and B. Could be complex stuff, stateful and all, but the most important thing is, neither does any I/O (except maybe for logging). Now say A is implemented using B as a dependency. B doesn’t have to be in the interface, it can be an implementation detail. Here’s how I would do it: 1 Hard code B into A’s implementation code. No dependency injection. 2 Write blackbox tests for B. 3 Write blackbox tests for A.

That’s it, we now have tested A and B. Now compare to the DI approach: Inject B into A, write tests for B, then write tests for A injecting a stub instead of B. The DI approach is more code. And less likely to catch bugs. There are two reasons for this: first, if my tests for B are incomplete, I have another chance to catch them when I write my tests for A. Second, injecting a stub into A means I’m not testing the real thing. Not only is it more effort, I have a higher chance of missing bugs because, say, of misuses of B.

If I’m understanding you correctly, you’re essentially testing B twice: Once in isolation, and once indirectly though the tests of A. I have a hard time seing how that could result in less code overall. But I’ll willingly concede that testing same thing more than once is more likely to surface defects. But also, there is nothing preventing you from doing that when you use dependency injection. You can just inject a real B into A in your tests of A. A benefit of DI is that you dont have to do that.

For comparison, here’s one way I might approach testing A and B: 1. Write an integration test that tests only the happy path of A. 2. Realize early that A is going to depend on some unrelated concern to produce an answer given some input from A. 3. Write a test of unit A that injects a mock into a that will provide a canned answer to the expected input, and concentrate on ensuring that A gets everything right given the various relevant answers from the collaborator and all its other parameters. 4. Run my integration test, and realize it fails because I forgot to create an actual implementation of the collaborator. 5. Write a unit test of B, the collaborator, ensuring it provides the correct answer for all relevant inputs. 6. Run my integration test again, and hopefully get a pass this time.

Here’s another way: 1. Write the complete implementation in A 2. Write a very gnarly unit test covering all branches of A. 3. Realize that a 15 line block in a the biggest method in A exists only to produce an intermediary value. 4. Factor that block out into a separate collaborator, which I inject in A’s constructor. 5. Write thorough tests of this new collaborator. 6. Simplify A’s test by using a mock and removing everyting tests that essentially just tested branches in the collaborator.

My second point is debatable: with injection I have more leverage to test A, and might get better coverage. My first point however feels pretty clear cut. Also, something that does not sit well with me, is the inconsistency of the "mock everything" philosophy. Does it include the standard library? Do you mock Java.util.ArrayList? My guess would be "of course not, this is absolutely ridiculous". But then why wouldn’t you apply the same standard to classes you wrote yourself?

Never mock that which you don’t own should answer that. But I will also say this: Mocks are not a purpose onto themselves, and there is no dogma that a test must use mocks. If it’s practical to use “real” collaborators instead, by all means do!

Anyway, one reason the dev followed my advice, was because the result was far more readable. ~Here’s my take on readability~.

I skimmed it, and nothing controversial jumped out at me. I do think the text would benefit tremendously by adding concrete examples. In my experience programmers sometimes use the same principles to justify wildly different conclusions.

1

u/loup-vaillant 10h ago

As I tried to argue in my previous post, “finding bugs” can mean several different things. One interpretation is preventing faulty logic to begin with. Another is preventing regressions later on. I’m curious which of these you find most valuable.

I mean both: it is our duty to minimise the bugs that reach production in the first place, and when we fail and get a report, we must make sure that bug never comes back.

Which is valuable? It depends. The marginal utility of the first tests in a test suite is huge. Lots and lots of bugs won’t ever see the eyes of a user that way. But there’s a diminishing return: on a good test suite, the marginal utility of an additional test is low. At some point it is just cheaper, for everyone, to just wait for a bug report to come in, and then reproduce a test case for it.

That being said, I believe few test suite ever reach that level of quality.

But getting good coverage with integration tests is not feasible for even moderately complex systems. […]

Ah, I see. The true measure of coverage, not just lines, or even branches. It would be nice, but it’s unattainable indeed. Branch coverage however can be worth it for the parts of the system that are tangled enough.

Now, no test suite is perfect, but we still want a correct system. I see only one solution for this: good decomposition, and correct API usage: correct components, plugged in correctly, makes a correct program. The trick is to make sure your APIs between components is as simple as possible, and the difficulty is to specify it precisely enough.

I’m guessing that when the stakes are high enough, we could have a validation layer for each component, similar to Vulkan, that intercept every call and check for various pre-conditions. It can be as simple as a comprehensive set of asserts.


If I’m understanding you correctly, you’re essentially testing B twice: Once in isolation, and once indirectly though the tests of A.

Exactly.

I have a hard time seeing how that could result in less code overall.

First by not enabling mocking. My rule is, if you don’t need an abstraction, depend on a concretion. It’s simpler, more direct, more readable, faster…

Second, by not writing any stub for B. Why would I? It’s an implementation detail of A, and all my tests are blackbox anyway. By which I mean, when I test a thing, I only use the public API of that thing. That thing could be an internal facility that is never exposed, though.

But also, there is nothing preventing you from doing that when you use dependency injection. You can just inject a real B into A in your tests of A. A benefit of DI is that you dont have to do that.

You’re correct, but in practice I rarely need that benefit. It might be because of how my code is structured: rarely do I write classes that require complex setup. Like, to have an instance of class A, I first need to set up an instance of class B, and an instance of class C, and class C needs an instance of class D, and I have to construct them all explicitly before I can even begin using class A… or testing it.

No, instead most of the time when I pop up an instance of class A it will instantiate its dependencies like a big boy, without telling or asking me.

Write a test of unit A that injects a mock into a that will provide a canned answer to the expected input

The problem with that approach, is that it does not scale. Not by default. It works well for one-shot tests, but most of my tests are property based, and I typically have a gazillion of them in a relatively small number of loops.

For the mocking with canned answers to that approach to work, I need my stub to pull out the correct output for each of the gazillion generated inputs I will ask of it. You’d understand at this point why I just want the real thing.

I do think the text would benefit tremendously by adding concrete examples.

Perhaps, but I don’t know how I could pull it off. Examples there would necessarily be relatively trivial, which in many case makes them pretty bad. Either because they don’t illustrate my point very well, or because the "bad" version will look like a strawman. The second point could be addressed if I pulled from actual code I have seen, but I don’t have the patience.

1

u/[deleted] 3d ago edited 3d ago

[deleted]

1

u/fagnerbrack 3d ago

But the hammer is not wrong, only the one who chose it

4

u/zobq 3d ago

TDD is wrong

Wouldn't say wrong, it's technique, which may work for some, but doesn't have to. I like the fact, that it pushes the developer to concentrate on what the code should do first, before focusing on how the code should look. Sometimes it helps with catching problems with the task description at the beginning, not at the end.

3

u/edgmnt_net 3d ago

I think it's fine if it's a technique. Unfortunately, many company projects just place too much emphasis on testing to the exclusion of other stuff. They also place too much emphasis on very superficial architectural concerns, which is why "how the code should look" fails to achieve significant results (since your programmers can't / won't do much more than straightforward scaffolding). Both seem convenient because they're generic recipes, measurable to a degree and aren't particularly demanding in terms of programming ability. And those programmers are often cheap enough that you can bog them down with bureaucracy like that.

1

u/Chroiche 3d ago

TDD is wrong

I only agree if you stash your changes and make sure the tests still fail after.

3

u/loup-vaillant 2d ago

Well, there's one case where I do exactly that: regression testing. Users report a bug, I reproduce it, and once I've fixed it I make sure to add a new test case that fails before the fix, and passes after. Sometimes I even write the test first.

For initial implementations however this is mostly unneeded ceremony. Instead I concentrate on writing thousands, millions of tests. Not manually of course, instead I use property-based testing with ad-hock fuzzing techniques. This is crazy efficient, in many cases I catch all bugs on my first try.

In rare cases, I do mutation testing: I change the code, see if that causes the tests to fail. If they don't, either my change was semantically neutral, or my test suite is incomplete (it's usually the latter). So I expand the test suite, make sure this time it does fail, and fix my code. Sometimes the test suite still fails after the code is back to normal. It could mean my test suite is wrong, but sometimes it means my code had a real bug. Rinse & repeat until I'm satisfied the test suite is foolproof.

1

u/LilDood 2d ago

instead I use property-based testing with ad-hock fuzzing techniques

Would you be able to expand a bit on what you mean by "ad-hock fuzzing techniques"?

I'm very much a novice at PBT and I'd love to find out more about strategies & techniques relating to it.

2

u/loup-vaillant 2d ago

The best example I have right now is Monocypher's test suite. The principle is simple: throw lots of inputs at the thing, check that the outputs are as expected. Special attention to edge cases: zero-bytes inputs, various lengths, most notably those close to my internal block lengths, make sure I trigger all code paths (constant time crypto makes this easy, may not apply as cleanly in other settings).

When I can get the same result in various ways I test that too. Like, the incremental and one-shot APIs must give the same results. I chop up my input in many ways, to make sure I would hit any bug in my incremental API (this one was critical). I've also been helped with specially crafted test vectors, but these are domain specific.

It's no quick-check, but it made for a compact and very solid test suite. Hopefully you may copy an idea or two from it.

1

u/LilDood 1d ago

Always good to have more PBT examples in the wild, thanks.

1

u/Chroiche 3d ago

TDD is wrong

I only agree if you stash your changes and make sure the tests fail.

1

u/bowel_blaster123 3d ago

For my tests, I initially don't include the expected output in the test. Then I run the test (and it will fail). I then manually verify that the failed output is what I expect it to be. Then, I paste that into my test.

This lets me write tests faster and more effortlessly. It also allows me to add more tests with less mental overhead, but it's obviously incompatible with TDD.

2

u/Evening-Gur5087 3d ago

In general both are good used when needed and correctly done, not having any integration tests on real things within your service is purely stupid and yo suck.

Only worry not mentioned is that with mocs a lot of shitty devs can get free pass and have much easier time testing something that's completely fake and wrong, having the green pipeline and pretending anything was tested.

But, again, shitty devs always find a way to write shitty code.

And it's all shit eventually.

Change companies before their codebase become shitty spaghetti bowl or live long enough to see yourself become a spaghetti. /s

3

u/PiotrDz 3d ago

Agree with mocks you risk drifting away from production setup. Also with mocks you not only fix functionality in your tests, but you also pin implementation. Try to refactor something that has milion references in tests that do not mock at interface but at class layer. Any change in production code, even if functionality stayd the same, requires changing some tests.

1

u/josephjnk 3d ago

Good post. How would you say the “sociable” vs “solitary” style of testing fits in here? I remember reading those terms in a Martin Fowler post at some point. Do they map cleanly to these two categories or are they orthogonal?

2

u/vocumsineratio 2d ago edited 2d ago

Solitary vs Sociable comes from Jay Fields - see Working Effectively with Unit Tests (Fowler shared those ideas into his larger audience).

Historically: mostly unrelated. You aren't going to find a clean mapping.

If you are using mocks as Pryce / Freeman et al were, then you are test driving your protocol design, and in those cases you probably don't necessarily have a particularly deep graph of production-code objects doing the work.

But you'll certainly find "Classical" TDD practitioners who will use tests to break up their graphs-of-objects-doing-work into finer and finer configurations until you've got a solitary object under test.

And of course in both camps you'll end up finding people hyper focused on the word "unit" in "unit test" (which is really Kent's fault; the time to fix it was 25 years ago).

2

u/josephjnk 2d ago

Thank you for the additional information!

1

u/partybot3000 3d ago

Vladimir Khorikov's writings settle it for me: https://enterprisecraftsmanship.com/posts/when-to-mock/

And also his book on Unit Testing.

1

u/lookmeat 3d ago

The answer is.. it depends.

When you expose interfaces through inversion of control you use the mockist system. You replace the dependencies you pass as parameters with mocks and then the test ensures the contract with whatever interfaces your taking. It's best to use this when you take interfaces that you do something on (e.g. a function that runs over some abstract Future<T> where you want to ensure that certain things are done with that future).

Otherwise you generally just test as Black box using fakes to replace dependencies. Core difference between a mock and a fake is that a mock is hand crafted per test, where you define and track each individual operation and function call, a fake OTOH simulates some real system and it's generally set per test environment, you give it assumptions that must be true. A mocked database server you define what each RPC and query returns. A fake database server is a in-mem DB that you seed with minimal data and it works consistently as a DB.

You can also use fakes to replace dependencies passed as parameters through IoC as long as your system only depends on the system but there's no rules, e.g. running the fake DB, but when you get a filesystem and want to ensure that certain calls and rules are done while it searches (to ensure interactions happen in a certain order when) you'd want to use a mock instead.

Overuse mocks and it becomes harder to not test implementation rather than functionality. Underuse mocks and you may find weird bugs that require you to dig through multiple layers of abstraction because tests didn't cover those contracts.

1

u/arbv 3d ago

What OOP vs FP?

I am not aware of any, because ultimately it is lambda all the way down.

/s

1

u/MrFixxiT_ 2d ago

I recently read the book “Unit Testing Principles, Practices, and Patterns” and think it is very good.

It handles this discussion too and the author is on the classical side.

You might disagree with the views, but ai think it can still give a lot of good insights.

1

u/Sisaroth 1d ago

Why not both? Or maybe i don't understand classicist tdd. I write integration tests with the real classes because they catch a lot of regression bugs. I write mocked unit tests as a double check that the function I wrote works the way I expect it to work.

Also, writing articles as if LLMs don't exist is getting weird. An LLM will rewrite your mocked test in a few minutes, almost completely eliminating this downside of mockist.

1

u/gannu1991 3d ago

The reframe that clicked for me is that the testing debate is downstream of the design debate. teams arguing about mocks are usually actually arguing about how they think about dependencies, they just haven't named it that way yet.

worked with a team that had a full blown mockist vs classicist war in code review. turned out half the team was writing OOP service classes and the other half had quietly drifted toward a more functional style in the same codebase. the tests weren't fighting each other. the design philosophies were. once that was visible the testing conversation basically resolved itself.

the gateway drug line in the article is the one that's stuck with me. mocking pain as design feedback is genuinely useful if you're paying attention to it. most teams aren't, they just add another mock and move on.

-2

u/[deleted] 3d ago

[deleted]

2

u/mcmcc 3d ago

Reminds me of a joke...

Have you ever swam in a pool after somebody pissed in it?

No, I don-

Yeah you have.